Nginx Slowdown on Solaris

[Maxim Dounin]

Hello!

On Thu, Apr 28, 2011 at 08:12:59AM -0400, darckos wrote:

> Hello, 
> 
> I use nginx 0.8.53 on Solaris 10 and I feel like I reach a kind of limit
> but I'm not abble to find it.
> I use Nginx to serve static image and sometimes to get one image I need
> to wait 10 or more seconds.

Most likely nginx actually disk bound and worker processes are 
blocked on disk, and that's what causes such delays.

You may want to examine iostat output to see if it really happens.

> nginx is launched with a ulimit -n 65536, and here is my config:
> 
> worker_processes  40;
> error_log  logs/error.log;
> events {
>         worker_connections  8192;
>         devpoll_events  1;
> }
> http {
>     include       mime.types;
>     default_type  application/octet-stream;
> 
>     sendfile        on;

[...]

> Do you have any idea to solve this problem ?

Try the following (in no particular order, you may want actually 
to start with (3)):

1. Tune worker_processes to add more (or less) concurency to make 
sure disks are reasonably loaded but not overloaded (i.e. disk r/w 
times are small enough).

2. Try switching accept_mutex off to get better request 
distribution between worker processes.

    events {
        accept_mutex off;
        ...
    }

3. Switch off sendfile and tune output_buffers to use 1-2 big 
enough buffers to make sure requests to disks will be big (and 
disks aren't trashed with large number of small requests).  
Something like

    output_buffers 2 256k;

will do the trick (using bigger buffers may help better if you 
have enough memory and your images are big enough).

Ideally aio should be used in such situation, but it's not 
available with nginx under Solaris.

Maxim Dounin