[Patch] SO_REUSEPORT support from master process

[Sepherosa Ziehau]

Does Linux handle un'accepted(2)' sockets inheritance between
SO_REUSEPORT sockets, if one of the SO_REUSEPORT socket is closed?
It's one of the concern that nginx folks have about SO_REUSEPORT.  I
am not following Linux, so I am not sure about it at all.  That's also
why I wanted to keep one SO_REUSEPORT socket opened.

BTW, DragonFly had my patch enabled by default in the dports system
for a long time.

And many systems have the symbol of SO_REUSEPORT and the setsockopt
works too, e.g. FreeBSD and probably other BSDs, but on kernel side,
SO_REUSEPORT does not work in the fashion like Linux or DragonFly.  So
even simple run-time test would not be able detect the "new"
SO_REUSEPORT support.

Best Regards,
sephe


On Sat, Aug 23, 2014 at 12:55 AM, Lu, Yingqi <yingqi.lu at intel.com> wrote:
> Dear All,
>
>
>
> The “SO_REUSEPORT support for listen sockets support” patches submitted by
> Sepherosa Ziehau are posted and discussed in [1] and [2]. Last update on the
> threads was 09/05/2013 and the patch is not included in the current Nginx
> code. Reading from the discussion, my understanding is that his patch makes
> a dedicated listen socket for each of the child process. In order to make
> sure at any given time there is always a listen socket available, the patch
> makes the first worker process different/special than the rest.
>
>
>
> Here, I am proposing a simpler way to enable the SO_REUSEPORT support. It is
> just to create and configure certain number of listen sockets in the master
> process with SO_REUSEPORT enabled. All the children processes can inherit.
> In this case, we do not need to worry about ensuring 1 available listen
> socket at the run time. The number of the listen sockets to be created is
> calculated based on the number of active CPU threads. With big system that
> has more CPU threads (where we have the scalability issue), there are more
> duplicated listen sockets created to improve the throughput and scalability.
> With system that has only 8 or less CPU threads, there will be only 1 listen
> socket. This makes sure duplicated listen sockets only being created when
> necessary. In case that SO_REUSEPORT is not supported by the OS, it will
> fall back to the default/original behavior (this is tested on Linux kernel
> 3.8.8 where SO_REUSEPORT is not supported).
>
>
>
> This prototype patch has been tested on an Intel modern dual socket platform
> with a three tier open source web server workload
> (PHP+Nginx/memcached/MySQL). The web server has 2 IP network interfaces
> configured for testing. The Linux kernel used for testing is 3.13.9. Data
> show:
>
>
>
> Case 1: with single listen statement (Listen 80) specified in the
> configuration file, there is 46.3% throughout increase.
>
> Case 2: with dual listen statements (for example, Listen 192.168.1.1:80 and
> Listen 192.168.1.2:80), there is 10% throughput increase.
>
>
>
> Both testing cases keep everything the same except the patch itself to get
> above result.
>
>
>
> The reason that Case1 has bigger performance gains is that Case1 by default
> only has 1 listen socket while Case2 by default already has 2.
>
>
>
> Please review it and let me know your questions and comments.
>
>
>
> [1] http://forum.nginx.org/read.php?29,241283,241283
>
> [2] http://forum.nginx.org/read.php?29,241470,241470
>
>
>
> 1. Software and workloads used in performance tests may have been optimized
> for performance only on Intel microprocessors. Performance tests, such as
> SYSmark and MobileMark, are measured using specific computer systems,
> components, software, operations and functions. Any change to any of those
> factors may cause the results to vary. You should consult other information
> and performance tests to assist you in fully evaluating your contemplated
> purchases, including the performance of that product when combined with
> other products.
>
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel



-- 
Tomorrow Will Never Die