nginx 1.4.1 - slow transfers / connection resets

[Philip Hofstetter]

Ok. I have three debug logs now:

http://www.gnegg.ch/debug-cancel.log
is the first log I created where I quit curl once nginx has logged a
200 status with a truncated length to the access log (how can it log
success while it's still transmitting data?)

http://www.gnegg.ch/debug-full.log
is the same request, but this time waiting for curl to complain about
the connection reset. Again, nginx logs a 200 with truncated length
(way before curl bails out)

http://www.gnegg.ch/debug-staticfile.log
Is me downloading a static file from one of the backend servers. This
shows the same behavior as the dynamically generated response and
helps ruling out fastcgi issues.

To add a further note: The machine which shows this issue is under
considerable load. When I run the tests against and identical machine
which is not under load, the download runs correctly (until I do put
it under load at which point it fails the same way).

The fact that nginx logs the request as successful (but truncated)
while it's still ongoing does kinda point to a kernel issue, but I'm
really just guessing at this point.

Philip

On Tue, Aug 20, 2013 at 9:23 AM, Philip Hofstetter
<phofstetter at sensational.ch> wrote:
> The last debug log I sent is not showing the full picture. In this
> case, I was aborting the curl command once nginx has logged an
> incomplete response (status=200 but too short length) to access.log,
> but while it was still transferring data (how's that even possible)?
>
> Hence the "connection reset by peer" in the log.
>
> I'm now making a second log, this time waiting it out. I will also
> produce a third log transferring a static file from the backend nginx
> in order to rule out fastcgi issues.
>
> Philip
>
> On Tue, Aug 20, 2013 at 9:14 AM, Philip Hofstetter
> <phofstetter at sensational.ch> wrote:
>> Hi,
>>
>>
>> On Mon, Aug 19, 2013 at 7:05 PM, Lukas Tribus <luky-37 at hotmail.com> wrote:
>>
>>> Looks like there is some timeout at 600 seconds (Time Spent: 0:09:58)? Any match
>>> in the haproxy or nginx configurations?
>>
>> That's consistent with what nginx is logging to the error log. But it
>> doesn't make sense as there is data being transmitted.
>>
>>>> I have a nginx (stock ubuntu config) as a reverse proxy in front of  a
>>>> haproxy in front of 5 more nginx machines which use fastcgi to talk to
>>>> php-fpm.
>>>
>>> Since you can reproduce it with curl, why not track the issue down to a
>>> specific part of your infrastructure (try on the nginx backends first,
>>> then on the haproxy box, and then on the frontent nginx box).
>>
>> That's what I've done before coming here. No issues on either haproxy
>> or the backend.
>>
>> Here's the debug log of just the failing request (thanks, nginx, for
>> making error_log a directive that can be used in a location block):
>>
>> http://www.gnegg.ch/debug.log
>>
>> Philip
>>
>> --
>> Sensational AG
>> Giesshübelstrasse 62c, Postfach 1966, 8021 Zürich
>> Tel. +41 43 544 09 60, Mobile  +41 79 341 01 99
>> info at sensational.ch, http://www.sensational.ch
>
>
>
> --
> Sensational AG
> Giesshübelstrasse 62c, Postfach 1966, 8021 Zürich
> Tel. +41 43 544 09 60, Mobile  +41 79 341 01 99
> info at sensational.ch, http://www.sensational.ch



-- 
Sensational AG
Giesshübelstrasse 62c, Postfach 1966, 8021 Zürich
Tel. +41 43 544 09 60, Mobile  +41 79 341 01 99
info at sensational.ch, http://www.sensational.ch