accpet_mutex cause nginx worker balance problem

[Maxim Dounin]

Hello!

On Sun, Aug 03, 2014 at 09:29:19AM -0400, xinghua_hi wrote:

> hello,  
> 
>       I use ab to test performance. but when i turn on accept_mutex,  I
> found the num of connection for every nginx worker is not balance. for
> example, 
> I have 4 core and start 4 nginx worker.
> 
> ab -n 300000 -c 1000 -k "http://XXX/"
> 
> the establish connection of nginx worker 
> 
> (netstat -antp|grep ESTABLISHED|awk -F" " '{print $7}'|sort|uniq -c)
> 
>  644      24619/nginx:
>   1         24620/nginx:
>   53       24621/nginx:
>  302      24622/nginx:
> 
> but when I turn off accept_mutex
> 
>  255      24660/nginx:
>  358      24661/nginx:
>  232      24662/nginx:
>  155      24663/nginx:
> 
> so if my test app is cpu bound,  I will find one or two core of my 4 cpu are
> very busy but other cores are very idle
> 
> I kown accept_mutex can control accept by turn and realize worker balance by
> ngx_accept_disabled, but why " accpet_mutex off" seems control worker
> balance more well ? when accept_mutex on ,  one or two worker can accept
> connection much more than other.

With accept mutex enabled, nginx only tries to accept new 
connections in one worker process (the one which was first to 
become idle).  This is expected to cause disbalance in tests with 
small number of connections.

The ngx_accept_disabled variable is mostly unrelated and only used when 
worker_connections are exhausted.

-- 
Maxim Dounin
http://nginx.org/