[PATCH] SO_REUSEPORT support for listen sockets
sepherosa at gmail.com
Tue Jul 30 08:22:33 UTC 2013
On Mon, Jul 29, 2013 at 10:57 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> On Sun, Jul 28, 2013 at 09:11:26PM +0800, Sepherosa Ziehau wrote:
>> > 2) this feature should be disabled on DragonFly versions prior to the
>> > 740d1d9 commit, because it clearly wouldn't do any good there,
>> On DragonFlyBSD, I could use a sysctl node to detect this feature.
>> However, this obviously is OS specific, I am not quite sure about
>> where to put that code. Any hint on this?
> DragonFly is currently handled as a variant of FreeBSD (see
> auto/os/conf, src/core/ngx_config.h, src/os/unix/ngx_freebsd_config.h,
> If you want to do sysctl runtime checks, correct place would be in
> src/os/unix/ngx_freebsd_init.c, ngx_os_specific_init(). I'm not
> sure it worth the effort though, probably just assuming user knows
> better will be ok.
Agree. We will just trust users.
>> I am not quite sure about how nginx handles upgrade. But if
>> so_reuseport is enabled and worker process exits (assuming new worker
>> is not forked by old worker), any pending sockets on listen socket's
>> completion queue but not yet accept(2)'ed will be dropped (at least
>> this is the case in DragonFlyBSD).
> By "pending sockets completion queue" you mean exactly one
> socket's queue, as created by a worker process? I.e., there is no
> mechanism to pass unaccepted connections from one socket to other
> sockets listening on the same address if the socket is closed
> (e.g. due to process exit)?
> This sounds bad, as it will result in connections being lost not
> only on binary upgrades but also on normal configuration reloads.
> Just for reference, upgrade process works as follows:
> - old master fork()'s and then uses execve() to start a new master
> process, with listenings sockets descriptors enumerated in the
> - new master process parses configuration (using inherited
> listening sockets fds it got from old master) and fork()'s new
> worker processes
> - old master asks old worker processes to exit gracefully (to stop
> accepting new connections and to exit once processing of
> currently active requests is complete)
> Configuration reload works as follows:
> - master process loads new configuration
> - master process fork()'s new worker processes (with a new configuration)
> - master proces asks old worker processes to exit gracefully
> Some documentation is here:
Thank you very much for the hint! The patch needs some changes to
handle this, as well as DragonFly's kernel.
After I have done the DragonFly kernel part, I would do the following
changes to the current patch:
If so_reuseport is enabled, the master will open the listen sockets
and after forking the first worker, the master closes all opened
listen sockets (they are inherited by the first worker); so the kernel
would have at least one listen socket to migrate the
completed-but-not-yet-accepted sockets of the to-be-closed listen
Tomorrow Will Never Die
More information about the nginx-devel