Nginx Slow download over 1Gbps load !!

[Reinis Rozitis]

> Yes, we recently tried downloading file over FTP and encountered the same 
> slow transfer rate.

Then it's not really a nginx issue, it seems you just hit the servers 
(current) network limit.


> zpool iostat is quite stable yet. We're using HBA LSI-9211 , so its not 
> hardware controller as FreeBSD recommends to use HBA in order to directly 
> access all drives for scrubbing and data-integrity purposes.
> Do you recommend Hardware-Raid ?

Well no, exactly opposite - ZFS works best(better) with bare disks than 
hardware raid between.
Just usually theese HBAs come with IR (internal raid) firmware - also you 
mentioned Raid 10 in your setup while in ZFS-world such term isn't often 
used (rather than mirrored pool or raidz(1,2,3..)).


As to how to solve the bandwidth issue - from personal experience I have 
more success (less trouble/expensive etc) with interface bonding on the 
server itself (for Linux with balance-alb which doesn't require any specific 
switch features or port configuration) rather than relying on the switches. 
The drawback is that a single stream/download can't go beyond one physical 
interface speed limit (for 1Gbe = ~100MB/s) but the total bandwidth cap is 
roughly multiplied by the device count bonded together.

Not a BSD guy but I guess the lagg in loadbalance mode would do the same ( 
https://www.freebsd.org/cgi/man.cgi?lagg ).

rr