Nginx + fair load balancer patch looping

[Alexander Staubo]

On 3/14/08, Grzegorz Nosek <grzegorz.nosek at gmail.com> wrote:
> If it doesn't kill your I/O, it might be useful (as an insight into WTF
>  the fair balancer is choosing). I think that the most useful would be
>  capturing debug_http logs (only those with [upstream_fair] should be
>  enough), but that generates a truly massive amount of data.

Well, it happened again. What is interesting -- and I am pretty sure
this is what happened before -- is that the errant process is actually
a worker shutting down:

root      4746  0.0  0.0  18316  1356 ?        Ss   Mar25   0:00
nginx: master process /usr/sbin/nginx
www-data  4301 56.6 34.4 6026976 2818648 ?     R    Mar27 1127:20  \_
nginx: worker process is shutting down
www-data 17604  0.7  0.0  18984  2132 ?        S    06:27   4:10  \_
nginx: worker process
www-data 17605  0.7  0.0  19048  2228 ?        S    06:27   4:12  \_
nginx: worker process
www-data 17606  0.7  0.0  18824  2012 ?        S    06:27   3:58  \_
nginx: worker process
www-data 17607  0.7  0.0  18788  1960 ?        S    06:27   4:10  \_
nginx: worker process

I don't know why it's shutting down, though. It could be the log
rotation job that has poked it.

Running strace -e connect on this process yields an infinite sequence
of the following two lines:

connect(3, {sa_family=AF_INET, sin_port=htons(11003),
sin_addr=inet_addr("...")}, 16) = -1 EINPROGRESS (Operation now in
progress)
connect(4, {sa_family=AF_INET, sin_port=htons(11003),
sin_addr=inet_addr("...")}, 16) = -1 EINPROGRESS (Operation now in
progress)

where the address being connected to is one of the back ends. I also
ran a full strace, and I can send you the output privately if you
like.

I have not tried the latest snapshot yet. We are still running the one
from February 12th or so.

Alexander.