SO_REUSEPORT

[SplitIce]

Issue found.

If worker_processes is set at the start of the config file the feature
works fine, if it is set at the end of the config file it does not.

On Wed, Nov 25, 2015 at 9:38 AM, SplitIce <mat999 at gmail.com> wrote:

> Ok,
>
> I have found it to be a configuration bug. With a fresh configuration
> (default configuration from package, with reuseport added) and reuseport
> enabled the feature works (same 3.18 kernel, same nginx binary).
>
> Now I just need to identify which line of our production configuration
> creates this bug.
>
> I will update when I know more.
>
> On Wed, Nov 25, 2015 at 12:42 AM, Valentin V. Bartenev <vbart at nginx.com>
> wrote:
>
>> On Wednesday 25 November 2015 00:25:19 SplitIce wrote:
>> > Hi all,
>> >
>> > I couldn't find anything in the mailing list about this issue, surely we
>> > are not the only one?
>> >
>> > When activating reuseport I am seeing all requests be served from a
>> single
>> > nginx process. All others are just idling (SIGALARM interruption of
>> > epoll_wait / epoll_wait timeout according to strace).
>> >
>> > Process 442 attached - interrupt to quit
>> > epoll_wait(60, 8225010, 512, 4294967295) = -1 EINTR (Interrupted system
>> > call)
>> > --- SIGALRM (Alarm clock) @ 0 (0) ---
>> > rt_sigreturn(0xe)                       = -1 EINTR (Interrupted system
>> call)
>> > epoll_wait(60, 8225010, 512, 4294967295) = -1 EINTR (Interrupted system
>> > call)
>> > --- SIGALRM (Alarm clock) @ 0 (0) ---
>> >
>> >
>> >
>> > This only occurs with reuseport, as soon as it is disabled the load is
>> > correctly distributed again.
>> >
>> > Configuration:
>> > worker_processes 12; # 2x8 cores on server
>> > multiple server blocks on different IP's and ports with reuseaddr.
>> > Linux kernel: 3.18.20
>> >
>> > Server nic has interrupts over all cores:
>> >
>> > # sudo ethtool -S eth0 |grep rx | grep pack
>> >      rx_packets: 11244443305
>> >      rx_queue_0_packets: 1381842455
>> >      rx_queue_1_packets: 1373383493
>> >      rx_queue_2_packets: 1490287703
>> >      rx_queue_3_packets: 1440591930
>> >      rx_queue_4_packets: 1378550073
>> >      rx_queue_5_packets: 1373473609
>> >      rx_queue_6_packets: 1437806438
>> >
>> >
>> > We have also experimented with disabling iptables and anything else on
>> the
>> > server that could be interfering. I have also loaded it onto three other
>> > fresh servers with the same kernel (same OS image), but with different
>> nic
>> > cards (with and without multiple rx queues) with no changes.
>> >
>> > This has me stumped. Ideas?
>> >
>>
>> You should try another kernel.
>>
>>   wbr, Valentin V. Bartenev
>>
>> _______________________________________________
>> nginx-devel mailing list
>> nginx-devel at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20151125/0c107be2/attachment.html>