[Patch] SO_REUSEPORT support from master process

[Sepherosa Ziehau]

Heh, I never made that patch for Linux and I don't use Linux ;).  And
since Linux does not inherit sockets, once SO_REUSEPORT listen socket
is closed you should have seen the strange timeout as you have
described (as I had told you linux missing the socket inheritance
feature).  On DFLY, we could max out all CPUs w/o problem and have no
strange timeout or connection drops when doing binary upgrading.

Well, at least your test result suggests for SO_REUSEPORT, different
OSes may need OS-specific implementation eventually to achieve better
performance or keep binary upgrading works.  You probably could keep
your patch as Linux specific patch as what we do for the dports.

Best Regards,
sephe


On Fri, Oct 31, 2014 at 6:24 AM, Lu, Yingqi <yingqi.lu at intel.com> wrote:
> Hi All,
>
> We tested the dragonfly approach on Linux (RHEL 6.5 with kernel 3.13.9). We used the same testing environment for both our patch and the dragonfly patch. Here is what we found:
>
> 1. Our patch has 36% better performance (operations/sec) comparing to dragonfly patch.
> 2. Our patch has 53% lower response time comparing to dragonfly approach even at 36% higher throughput level.
> 3. Our patch can scale the CPU utilization and frequency to the max capacity while dragonfly patch cannot.
> 4. Our patch does not have any issues with "upgrade binary on the fly". However, dragonfly patch creates a spike in the response time during the upgrade. It also has lots of connection timeouts/losses with high load.
>
> Above findings are based on Linux OS.
>
> Thanks,
> Yingqi
>
> -----Original Message-----
> From: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Lu, Yingqi
> Sent: Wednesday, October 08, 2014 11:24 AM
> To: nginx-devel at nginx.org
> Subject: RE: [Patch] SO_REUSEPORT support from master process
>
> One more comment from me: duplicate listen sockets in kernel is not a trivia thing to do and it may take long time before people can see it. Addressing it Nginx may not be as ideal as in kernel, but at least user can see the performance improvement sooner. In fact, we see up to 48% performance improvement on modern Intel system. Just my two cents.
>
> Again, thanks very much for everyone for helping us review this.
>
> Thanks,
> Yingqi
>
> -----Original Message-----
> From: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Lu, Yingqi
> Sent: Wednesday, October 08, 2014 10:05 AM
> To: nginx-devel at nginx.org
> Subject: RE: [Patch] SO_REUSEPORT support from master process
>
> Hi Maxim,
>
> Thanks for letting us know.
>
> Our updated patch is located at http://forum.nginx.org/read.php?29,253446,253446#msg-253446
>
> It supposes to address all the style issues and fixes the restart and binary upgrade issues. This is just a FYI in case you are not aware of.
>
> Thanks,
> Yingqi
>
> -----Original Message-----
> From: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Maxim Dounin
> Sent: Wednesday, October 08, 2014 5:59 AM
> To: nginx-devel at nginx.org
> Subject: Re: [Patch] SO_REUSEPORT support from master process
>
> Hello!
>
> On Tue, Oct 07, 2014 at 07:32:08PM +0000, Lu, Yingqi wrote:
>
>> Dear All,
>>
>> It has been quiet for a while on this patch. I am checking to see if
>> there is any questions/feedbacks/concerns we need to address?
>>
>> Please let me know. Thanks very much for your help!
>
> Apart from style/coding issues, I disagree with the whole approach.
>
> As far as I understand the patch idea, it tries to introduce multiple listening sockets to avoid in-kernel lock contention.
> This is something that can be done completely in kernel though, and I see no reason to introduce any changes to nginx here.
>
> The approach previously discussed with Sepherosa Ziehau looks much more interesting.
>
> --
> Maxim Dounin
> http://nginx.org/
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel



-- 
Tomorrow Will Never Die