Nginx TCP Delays

Maxim Dounin mdounin at mdounin.ru
Mon Sep 21 12:46:07 MSD 2009


Hello!

On Sun, Sep 20, 2009 at 10:47:21PM -0700, Khalid Shaikh wrote:

> We are using Nginx on this web server.  Look at the # of reading & writing
> requests.  a curl http://localhost/test.html can take up to 45 seconds.
> Can anyone help?
> 
> Using telnet I can see that basically the Nginx server is taking time to get
> to the TCP connection that I initiated.
> 
> This happens only during peak times of the web site.

Looks like either nginx has no CPU time due to fastcgi sitting on 
the same machine, or nginx workers are blocked on disk.  You may find 
out actual reason by examining state of nginx worker processes in 
top.

If the problem is CPU time you shuld just add some to nginx, e.g. 
by using worker_priority directive.  Reducing load on fastcgi 
(e.g. by using fastcgi_cache) may help too.

If the problem is disk load, you may try the following:

1. Set sendfile_max_chunk to something like 128k or 256k.  This 
should prevent fast clients from blocking nginx in sendfile() for 
too long.

2. Don't use sendfile at all and use small number of big 
output_buffers (e.g. output_buffers 1 256k;).  This should reduce 
number of disk accesses (and seeks) at cost of not using 
sendfile() - e.g. some additional CPU usage. 

3. If fastcgi returns big responses that are buffered to disk, try 
raising fastcgi_buffers and/or setting fastcgi_max_temp_file_size 
0;.  This should prevent nginx from going to disk at cost of not 
releasing fastcgi connection as early as possible.  Also you 
should consider finding the source of such responses and using 
X-Accel-Redirect instead if it's possible.

4. Consider using aio.  Though aio on Linux isn't something 
generally usable, but it may help anyway.  Note that some bugs may 
be still here, it's expiremental feature.

Maxim Dounin

> 
> Active connections: 8467
> server accepts handled requests
>  380771 380771 835836
> Reading: 75 Writing: 1497 Waiting: 6895
> http://67.159.60.59/status      45.03 seconds
> 
> I've attached the nginx.conf
> 
> user www-data www-data;
> worker_processes 32;
> 
> error_log  /var/www/log/nginx_error.log;
> pid        /var/run/nginx.pid;
> 
> events {
>   worker_connections  10024;
> }
> 
> http {
>   include       /etc/nginx/mime.types;
>   default_type  application/octet-stream;
> 
>   sendfile         on;
>   tcp_nopush       on;
>   tcp_nodelay      on;
> 
>   server {
>     listen       80;
>     server_name  xs.to;
>     root   /var/www/xs;
>     error_page  404 index.php;
>     error_page   500 502 503 504  index.php;
>     access_log off;
> 
>     location / {
>       root   /var/www/xs;
>       index  index.php default.php;
>       rewrite  ^/albums/(.*)$  /albums/showalbum.php?$1?  last;
>       rewrite  ^/community/(.*)$  /community.php?$1?  last;
>       error_page  404 index.php;
>       error_page   500 502 503 504  index.php;
>     }
> 
>     location /status {
>       stub_status on;
>       access_log off;
>     }
> 
>     location ~ .php$ {
>       fastcgi_pass 127.0.0.1:8888;
>       fastcgi_index index.php;
>       fastcgi_intercept_errors on;
>       error_page  404 index.php;
>       error_page   500 502 503 504  index.php;
>       fastcgi_param  SCRIPT_FILENAME    /var/www/xs/$fastcgi_script_name;
>       include /etc/nginx/fastcgi_params;
>     }
> 
>     location ~* ^.+.(jpg|jpeg|gif|tiff|png|bmp|ico|fav|html)$ {
>       access_log        off;
>       expires           30d;
>     }
>   }





More information about the nginx mailing list