nginx serving large files - performance issues with more than ~800-1000 connections

[Tomasz Chmielewski]

On 07/07/2012 01:17 AM, Tomasz Chmielewski wrote:
> On 07/07/2012 12:17 AM, Maxim Dounin wrote:
>
>> On the other hand, the fact that second nginx instance running on
>> different port doesn't have problems (as indicated in your
>> original message) suggests there is some blocking which makes
>> first instance slow.
>>
>> Try looking into what nginx processes do when this happens.
>> Something like
>>
>> ps -eopid,user,args,wchan | grep nginx
>>
>> should show where it blocks.
>
> OK, it's the time of the day when the traffic drops (Asia based
> deployment), below ~600 connections, so I no longer see this "blocking".
> Will follow up tomorrow.

OK, the traffic is back, I'm finally able to run "ps 
-eopid,user,args,wchan". I see 16 nginx processes in 
"wait_answer_interruptible" state (which matches the worker_processes 
setting on this server).


root at ca2:~# ps aux|grep D
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      1900  0.0  0.0   7608   844 pts/1    S+   02:48   0:00 grep D

root at ca2:~# ps aux|grep D
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      1902  0.0  0.0   7608   848 pts/1    S+   02:48   0:00 grep D

root at ca2:~# netstat -tpna | grep -c ESTABLI
637

root at ca2:~# ps aux|grep D
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      1906  0.0  0.0   7608   844 pts/1    S+   02:48   0:00 grep D

root at ca2:~# ps -eopid,user,args,wchan | grep nginx
  1908 root     grep nginx                  pipe_wait
25101 root     nginx: master process /usr/ rt_sigsuspend
29353 www-data nginx: worker process       wait_answer_interruptible
29354 www-data nginx: worker process       wait_answer_interruptible
29355 www-data nginx: worker process       wait_answer_interruptible
29356 www-data nginx: worker process       wait_answer_interruptible
29357 www-data nginx: worker process       wait_answer_interruptible
29358 www-data nginx: worker process       wait_answer_interruptible
29359 www-data nginx: worker process       wait_answer_interruptible
29360 www-data nginx: worker process       wait_answer_interruptible
29361 www-data nginx: worker process       wait_answer_interruptible
29362 www-data nginx: worker process       wait_answer_interruptible
29363 www-data nginx: worker process       wait_answer_interruptible
29364 www-data nginx: worker process       wait_answer_interruptible
29365 www-data nginx: worker process       wait_answer_interruptible
29366 www-data nginx: worker process       wait_answer_interruptible
29367 www-data nginx: worker process       wait_answer_interruptible
29368 www-data nginx: worker process       wait_answer_interruptible




-- 
Tomasz Chmielewski
http://www.ptravever.com