Nginx TCP Delays

[Khalid Shaikh]

Maxim,
Great advice.  Give me 24 hours to test it out.

The specific requests I am doing /test.html and /status do not hit php-fpm
and I verified that by turning it off and hitting those URLs for responses.

I do not think CPU is the issue, but I still value the advice and will go
through the variables and see if I get improvements.

Best,
Khalid

On Mon, Sep 21, 2009 at 1:46 AM, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Sun, Sep 20, 2009 at 10:47:21PM -0700, Khalid Shaikh wrote:
>
> > We are using Nginx on this web server.  Look at the # of reading &
> writing
> > requests.  a curl http://localhost/test.html can take up to 45 seconds.
> > Can anyone help?
> >
> > Using telnet I can see that basically the Nginx server is taking time to
> get
> > to the TCP connection that I initiated.
> >
> > This happens only during peak times of the web site.
>
> Looks like either nginx has no CPU time due to fastcgi sitting on
> the same machine, or nginx workers are blocked on disk.  You may find
> out actual reason by examining state of nginx worker processes in
> top.
>
> If the problem is CPU time you shuld just add some to nginx, e.g.
> by using worker_priority directive.  Reducing load on fastcgi
> (e.g. by using fastcgi_cache) may help too.
>
> If the problem is disk load, you may try the following:
>
> 1. Set sendfile_max_chunk to something like 128k or 256k.  This
> should prevent fast clients from blocking nginx in sendfile() for
> too long.
>
> 2. Don't use sendfile at all and use small number of big
> output_buffers (e.g. output_buffers 1 256k;).  This should reduce
> number of disk accesses (and seeks) at cost of not using
> sendfile() - e.g. some additional CPU usage.
>
> 3. If fastcgi returns big responses that are buffered to disk, try
> raising fastcgi_buffers and/or setting fastcgi_max_temp_file_size
> 0;.  This should prevent nginx from going to disk at cost of not
> releasing fastcgi connection as early as possible.  Also you
> should consider finding the source of such responses and using
> X-Accel-Redirect instead if it's possible.
>
> 4. Consider using aio.  Though aio on Linux isn't something
> generally usable, but it may help anyway.  Note that some bugs may
> be still here, it's expiremental feature.
>
> Maxim Dounin
>
> >
> > Active connections: 8467
> > server accepts handled requests
> >  380771 380771 835836
> > Reading: 75 Writing: 1497 Waiting: 6895
> > http://67.159.60.59/status      45.03 seconds
> >
> > I've attached the nginx.conf
> >
> > user www-data www-data;
> > worker_processes 32;
> >
> > error_log  /var/www/log/nginx_error.log;
> > pid        /var/run/nginx.pid;
> >
> > events {
> >   worker_connections  10024;
> > }
> >
> > http {
> >   include       /etc/nginx/mime.types;
> >   default_type  application/octet-stream;
> >
> >   sendfile         on;
> >   tcp_nopush       on;
> >   tcp_nodelay      on;
> >
> >   server {
> >     listen       80;
> >     server_name  xs.to;
> >     root   /var/www/xs;
> >     error_page  404 index.php;
> >     error_page   500 502 503 504  index.php;
> >     access_log off;
> >
> >     location / {
> >       root   /var/www/xs;
> >       index  index.php default.php;
> >       rewrite  ^/albums/(.*)$  /albums/showalbum.php?$1?  last;
> >       rewrite  ^/community/(.*)$  /community.php?$1?  last;
> >       error_page  404 index.php;
> >       error_page   500 502 503 504  index.php;
> >     }
> >
> >     location /status {
> >       stub_status on;
> >       access_log off;
> >     }
> >
> >     location ~ .php$ {
> >       fastcgi_pass 127.0.0.1:8888;
> >       fastcgi_index index.php;
> >       fastcgi_intercept_errors on;
> >       error_page  404 index.php;
> >       error_page   500 502 503 504  index.php;
> >       fastcgi_param  SCRIPT_FILENAME    /var/www/xs/$fastcgi_script_name;
> >       include /etc/nginx/fastcgi_params;
> >     }
> >
> >     location ~* ^.+.(jpg|jpeg|gif|tiff|png|bmp|ico|fav|html)$ {
> >       access_log        off;
> >       expires           30d;
> >     }
> >   }
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nginx.org/pipermail/nginx/attachments/20090921/051ad3c8/attachment.html>