I couldn't find anything in the mailing list about this issue, surely we are not the only one?
When activating reuseport I am seeing all requests be served from a single nginx process. All others are just idling (SIGALARM interruption of epoll_wait / epoll_wait timeout according to strace).
Process 442 attached - interrupt to quit epoll_wait(60, 8225010, 512, 4294967295) = -1 EINTR (Interrupted system call) --- SIGALRM (Alarm clock) @ 0 (0) --- rt_sigreturn(0xe) = -1 EINTR (Interrupted system call) epoll_wait(60, 8225010, 512, 4294967295) = -1 EINTR (Interrupted system call) --- SIGALRM (Alarm clock) @ 0 (0) ---
This only occurs with reuseport, as soon as it is disabled the load is correctly distributed again.
Configuration: worker_processes 12; # 2x8 cores on server multiple server blocks on different IP's and ports with reuseaddr. Linux kernel: 3.18.20
Server nic has interrupts over all cores:
# sudo ethtool -S eth0 |grep rx | grep pack rx_packets: 11244443305 rx_queue_0_packets: 1381842455 rx_queue_1_packets: 1373383493 rx_queue_2_packets: 1490287703 rx_queue_3_packets: 1440591930 rx_queue_4_packets: 1378550073 rx_queue_5_packets: 1373473609 rx_queue_6_packets: 1437806438
We have also experimented with disabling iptables and anything else on the server that could be interfering. I have also loaded it onto three other fresh servers with the same kernel (same OS image), but with different nic cards (with and without multiple rx queues) with no changes.
This has me stumped. Ideas?