Optimize nginx for uploading/download huge files..
mdounin at mdounin.ru
Thu Jan 5 13:04:38 UTC 2012
On Wed, Jan 04, 2012 at 07:38:08PM -0500, ronjpark wrote:
> stefancaunter Wrote:
> > On Tue, Jan 3, 2012 at 5:35 PM, ronjpark
> > <nginx-forum at nginx.us> wrote:
> > The bottleneck is neither CPU nor memory. VM disk
> > performance will be
> > bad. Any throughput you get on the virtual disk
> > will degrade as each
> > concurrent upload client triggers reads and
> > writes. This is one case
> > where you need a SAN, ZFS filesystem, or an SSD
> > drive; configure with
> > large block size if files are >1GB. Even server
> > class hard drives will
> > thrash and slow down everything with enough
> > clients contending for the
> > disk. Remember, in relative time, if a cpu
> > operation takes a second,
> > an operation on the disk takes one month, under
> > ideal conditions, and
> > you are giving nginx less than ideal conditions.
> > Stefan Caunter
> > http://scaleengine.com/contact
> Thank you Stefan.
> I also doubted what you said and tested with 1MB file.
> client_body_buffer_size is set to 2M and proxy_buffer_size = 2M also.
> But it doesn't work.
> As my understainding, nginx doesn't make disk I/O for this file
> transaction because of enough big buffer size. Right?
> Please fix me if I'm wrong.
To limit disk I/O for big requests/responses there are following
1. Using larger buffers, notably client_body_buffer_size,
proxy_buffer_size, proxy_buffers. This is basically what you've
2. For responses you may also completely disable disk buffering
using "proxy_max_temp_file_size 0". This implies that nginx won't
be able to read full response from a backend though, and
connection to a backend will be busy for a time needed to client to
download a response.
Directive description may be found here:
More information about the nginx