mail proxy (IMAP/POP3): balancing between workers

[Roland Rosenfeld]

Hi,

I run NginX as mail proxy (IMAP/POP3) and have a setup with 

worker_processes 8;
worker_rlimit_nofile 32768;
events {
    worker_connections  4096;
    multi_accept on;
}


I upgraded this setup from Linux 3.16 and NginX 1.10 to Linux 4.9 and
NginX 1.14.
After this upgrade I run into trouble, since after reaching a maximum
of approximately 2600 proxied connections I run into the following
error messages:
 4096 worker_connections are not enough
or
 4096 worker_connections are not enough while in http auth state


I found out, that nearly all connections were proxied by the first
worker process, while nearly all other worker processes seem to be
mostly inactive.

But I want to balance the connections between the workers, otherwise
the worker_rlimit_nofile and the worker_connections are too low.


As a first workaround I defined
 accept_mutex on;
which changed it's defaults in 0.11.3 from "on" to "off".
This seems to mitigate the issue for me (at least all worker processes
now use the CPU again according to ps output).  I'm not sure, whether
the balancing is as good as with the old setup, but it looks much
better than before the workaround.


But what's the correct way to tell NginX, that it should balance the
connections between all worker processes? According to manual
"accept_mutex on" isend needed with EPOLLEXCLUSIVE which should be
active on my system with Linux 4.9 and glibc 2.24.

Greetings
Roland