nginx 1.4.1 - slow transfers / connection resets
mdounin at mdounin.ru
Tue Aug 20 13:50:57 UTC 2013
On Tue, Aug 20, 2013 at 03:14:20PM +0200, Philip Hofstetter wrote:
> On Tue, Aug 20, 2013 at 1:26 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> > 2013/08/20 09:34:31 [debug] 1692#0: *1101651 http upstream process non buffered downstream
> > 2013/08/20 09:34:31 [info] 1692#0: *1101651 client timed out (110: Connection timed out) while sending to client, client: 18.104.22.168, server: , request: "POST /index.php/winclient/gnegg HTTP/1.0", upstream: "http://127.0.0.1:8081/index.php/winclient/gnegg", host: "REDACTED.popscan.ch"
> > 2013/08/20 09:34:31 [debug] 1692#0: *1101651 finalize http upstream request: 408
> > After a 60 seconds timer was fired and client connection was
> > closed as timed out.
> Yeah. That's what I feared. But the connection was definitely still
> open and data was being transferred.
> > Unfortunately, with location-level debug logs it's not possible to
> > see event handling details (and that's why it's generally
> > recommended to activate debug log at global level, BTW).
> any idea how to do this on a system that's under load (60 requests per
> second)? As I said before: When I do the same request on a system
> that's not under load, the problem doesn't appear.
60 requests per second is low enough, just switching on debug log
> > But I would suppose everything is fine there as well, and the problem is
> > actually a result of kernel's behaviour.
> I started suspecting as much. Any pointers how I could work around/fix
> the issue on the kernel level?
No exact recommendation, but likely it's related to buffering at
some point. First of all I would recommend to look at what
actually happens on the wire with tcpdump/wireshark. If there is
indeed transfer stall for 60+ seconds - you should look at the
client's side of the TCP connection, i.e. either client's kernel
or curl. If there is continous flow of packets - it's likely
something to do with sending part (just in case, aren't you using
send_lowat? if set too high it may cause such symptoms).
More information about the nginx