nginx serving large files - performance issues with more than ~800-1000 connections

[Maxim Dounin]

Hello!

On Sat, Jul 07, 2012 at 10:54:14AM +0800, Tomasz Chmielewski wrote:

> On 07/07/2012 01:17 AM, Tomasz Chmielewski wrote:
> >On 07/07/2012 12:17 AM, Maxim Dounin wrote:
> >
> >>On the other hand, the fact that second nginx instance running on
> >>different port doesn't have problems (as indicated in your
> >>original message) suggests there is some blocking which makes
> >>first instance slow.
> >>
> >>Try looking into what nginx processes do when this happens.
> >>Something like
> >>
> >>ps -eopid,user,args,wchan | grep nginx
> >>
> >>should show where it blocks.
> >
> >OK, it's the time of the day when the traffic drops (Asia based
> >deployment), below ~600 connections, so I no longer see this "blocking".
> >Will follow up tomorrow.
> 
> OK, the traffic is back, I'm finally able to run "ps
> -eopid,user,args,wchan". I see 16 nginx processes in
> "wait_answer_interruptible" state (which matches the
> worker_processes setting on this server).

Trivial grep though Linux sources suggests this is blocking on 
fuse filesystem accesses.  So after all it's disk blocking, at 
least from nginx point of view.

All basic optimization steps still apply, though correct 
optimization of your filesystem probably much more important here.  

(Just in case, correct optimization for network filesystems like 
nfs is to avoid their use with nginx and use proxying instead.)

Maxim Dounin