limit_zone and nginx worker crashing

[Maxim Dounin]

Hello!

On Tue, May 24, 2011 at 05:24:53PM -0700, Matthieu Tourne wrote:

> Hi,
> 
> I've noticed an issue when limit_zone (and probably limit_req) is used.
> If an nginx worker crashes, the cleanup functions for that request will not
> get called and the number of connections will not be decremented as it
> should be in ngx_http_limit_zone_cleanup().
> 
> Since it's a shared memory segment when a new worker is started, the number
> of connection is still at the same value.
> So if a worker crashes multiple times, we might end up always returning '503
> Service Temporary Unavailable'.
> 
> I'm not sure there would be a way to call the cleanup functions if a worker
> has crashed, but maybe the rbtree for limit_zone should be re-initialized on
> a respawn ?

Shared memory contents may be cleared with binary upgrade 
procedure, see

http://wiki.nginx.org/CommandLine#Upgrading_To_a_New_Binary_On_The_Fly

Note that worker crash isn't something normal, and the only really 
available option is to find out crash reason and fix it.

Maxim Dounin