nginx 1.4.1 - slow transfers / connection resets

[Maxim Dounin]

Hello!

On Tue, Aug 20, 2013 at 03:14:20PM +0200, Philip Hofstetter wrote:

> Hello!
> 
> On Tue, Aug 20, 2013 at 1:26 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> > 2013/08/20 09:34:31 [debug] 1692#0: *1101651 http upstream process non buffered downstream
> > 2013/08/20 09:34:31 [info] 1692#0: *1101651 client timed out (110: Connection timed out) while sending to client, client: 80.219.149.116, server: , request: "POST /index.php/winclient/gnegg HTTP/1.0", upstream: "http://127.0.0.1:8081/index.php/winclient/gnegg", host: "REDACTED.popscan.ch"
> > 2013/08/20 09:34:31 [debug] 1692#0: *1101651 finalize http upstream request: 408
> >
> > After a 60 seconds timer was fired and client connection was
> > closed as timed out.
> 
> Yeah. That's what I feared. But the connection was definitely still
> open and data was being transferred.
> 
> 
> > Unfortunately, with location-level debug logs it's not possible to
> > see event handling details (and that's why it's generally
> > recommended to activate debug log at global level, BTW).
> 
> any idea how to do this on a system that's under load (60 requests per
> second)? As I said before: When I do the same request on a system
> that's not under load, the problem doesn't appear.

60 requests per second is low enough, just switching on debug log 
should work.

> > But I would suppose everything is fine there as well, and the problem is
> > actually a result of kernel's behaviour.
> 
> I started suspecting as much. Any pointers how I could work around/fix
> the issue on the kernel level?

No exact recommendation, but likely it's related to buffering at 
some point.  First of all I would recommend to look at what 
actually happens on the wire with tcpdump/wireshark.  If there is 
indeed transfer stall for 60+ seconds - you should look at the 
client's side of the TCP connection, i.e. either client's kernel 
or curl.  If there is continous flow of packets - it's likely 
something to do with sending part (just in case, aren't you using 
send_lowat? if set too high it may cause such symptoms).

-- 
Maxim Dounin
http://nginx.org/en/donation.html