Nginx crash - possibly keepalives

[Maxim Dounin]

Hello!

On Wed, Oct 12, 2011 at 09:32:41AM -0700, Matthieu Tourne wrote:

> On Wed, Oct 12, 2011 at 5:20 AM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> > Hello!
> >
> > On Tue, Oct 11, 2011 at 07:53:36PM -0700, Matthieu Tourne wrote:
> >
> > > Hi all,
> > >
> > > After turning on keepalives, we've been seeing one crash pretty
> > > consistently.
> > > We're running nginx 1.1.5 and here is the backtrace :

[...]

> (gdb) fr 2
> #2  0x000000000041c534 in ngx_event_pipe_write_chain_to_temp_file
> (p=0x173d048) at src/event/ngx_event_pipe.c:771
> 771 src/event/ngx_event_pipe.c: No such file or directory.
>  in src/event/ngx_event_pipe.c
> (gdb) p *p
> $1 = { ..., free_raw_bufs = 0x0, in = 0x0, ..., out = 0x0, ...,
> busy = 0x17b9ec0, ..., input_filter = 0x45c873 <ngx_http_proxy_chunked_filter>,
> ..., cacheable = 1, ... allocated = 9, bufs = {num = 9, size = 
> 8192}, ..., busy_size = 8192, ...,
>   length = 168, ... }

Ok, it looks like I'm right and all buffers are in busy chain.  
Likely this happens due to upstream sending response in many small 
chunks.

I'll try to reproduce it here and provide proper fix.

[...]

> > I think I see what's going on here, just to confirm.  The
> > underlying issue seems to be one identified in commit message
> > here:
> >
> > http://trac.nginx.org/nginx/changeset/4136/nginx
> >
> > (all available buffers in the p->out chain)
> >
> > Setting proxy_busy_buffers to something lower compared to
> > proxy_buffers may help as a workaround.
> >
> 
> We'll try that and see if that has a positive effect, thank you.
> Also, would having u->buffering == 0 (proxy_buffering off) help in this
> scenario ?

Yes, switching off proxy_buffering should help.  Switching off 
disk buffering (proxy_cache off; proxy_max_temp_file_size 0;) 
should help as well.

Maxim Dounin