Does Nginx block on file IO?

[Cliff Wells]

On Sun, 2011-05-01 at 20:26 -0400, Andy wrote:
> Hi.
> 
> I was always under the impression that Nginx is non-blocking for file
> IO. Then I was told it wasn't.
> 
> I'm considering using Nginx to serve static images. Pretty much every
> connection will result in a file IO. If Nginx blocks for file IO, then
> using Nginx here wouldn't be any better than using Apache, right? Every
> connection will lead to a file IO which blocks the entire Nginx process.
> So to serve 500 concurrent connections I'll need 500 Nginx processes.

You cannot take a single measure such as concurrent requests independent
of requests per second and make a good prediction about performance
requirements.  If you have 500 concurrent requests, and each request
takes an average of 1ms, then you could conceivably serve all 500
requests within 0.5 seconds with a single worker, which is quite
reasonable.  People have reported serving 10K requests per second using
Nginx on rather modest hardware (laptops even).  You'll probably find
that your OS is the bottleneck, not Nginx (and you should focus your
performance tuning at that level - TCP buffers and the like).

It's also worth pointing out that if you currently have 500 concurrent
requests on Apache, that number may actually decrease with Nginx,
assuming requests are finished faster (i.e. if Nginx finishes each
request twice as fast as Apache, you'd only see 250 concurrent requests,
while requests per second would double).  Obviously that's simplified,
but hopefully clarifies my point.

In any case, no need to have a worker per request, just start with the
recommendation of 1 worker per core and tune from there.  It's true that
the *concurrent* number of requests is limited by the number of workers
you have (the remaining requests will be served serially as each worker
is freed), but in practice, it becomes academic. Serialization is going
to happen at some level, even if you have a threaded server, so
basically software threads buy you little more than the feeling of
concurrency at the cost of massive amounts of memory.

> Would Nginx work in such a use case? Any tips on how to use Nginx for
> serving static media?

Serving static content is actually where Nginx has been demonstrated to
outperform Apache by a wide margin, in no small part because it leaves
lots of memory for filesystem caching, and also because it causes less
context-switches than a threaded server.

At the end of the day, you should do some performance testing, since
performance is going to depend a lot on factors that will only be
revealed on your particular setup and your particular data (number and
size of files, cache misses, etc).

Regards,
Cliff