truncated output with gzip, large files and slow rate

[Maxim Dounin]

Hello!

On Mon, Apr 16, 2012 at 09:37:27AM +0100, James Lee wrote:

> Hello,
> 
> nginx is truncating output with the following:
>  + gzip on
>  + compressed size greater than 100k (approx)
>  + client downloads at a slow rate
> 
> 
> *** Steps to reproduce ***
> 
> nginx.conf:
> 
> events {
> }
> http {
>     types {
>         text/plain txt;
>     }
>     gzip on;
>     gzip_types text/plain;
>     server {
>         listen 80 default_server;
>         server_name _;
>         root /path/to/root;
>     }
> }
> 
> 
> Create a test file that is larger than 100k compressed:
> 
> $ head -1000 /dev/random > /path/to/root/test.txt
> 
> 
> Request exhibiting failure:
> 
> $ curl -H "Accept-Encoding: gzip" --limit-rate 50k
> localhost/test.txt | gzcat | cksum
>   % Total    % Received % Xferd  Average Speed   Time    Time
> Time  Current
>                                  Dload  Upload   Total   Spent
> Left  Speed
> 100  180k    0  180k    0     0   2966      0 --:--:--  0:01:02
> --:--:--     0
> curl: (18) transfer closed with outstanding read data remaining
> 
> gzcat: stdin: unexpected end of file
> 649550918       184260
> 
> 
> and with wget:
> 
> $ wget --header "Accept-Encoding: gzip" --limit-rate=50k -O
> /dev/stdout localhost/test.txt | gzcat | cksum
> 
> --2012-04-15 11:27:04--  http://localhost/test.txt
> Resolving localhost (localhost)... 127.0.0.1, ::1
> Connecting to localhost (localhost)|127.0.0.1|:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: unspecified [text/plain]
> Saving to: `/dev/stdout'
> 
>     [                                <=>     ] 184,330     --.-K/s   in 61s
> 
> 2012-04-15 11:28:06 (2.96 KB/s) - `/dev/stdout' saved [184330]
> 
> 
> gzcat: stdin: unexpected end of file
> 649550918       184260
> 
> 
> 
> 
> Some variants showing success (correct size and cksum) by avoiding
> one of compression or limit-rate:
> 
> $ curl -H "Accept-Encoding: gzip" localhost/test.txt | gzcat | cksum
>   % Total    % Received % Xferd  Average Speed   Time    Time
> Time  Current
>                                  Dload  Upload   Total   Spent
> Left  Speed
> 100  235k    0  235k    0     0  5970k      0 --:--:-- --:--:--
> --:--:-- 6370k
> 2871747365      241269
> 
> 
> $ curl --limit-rate 50k localhost/test.txt | cksum
>   % Total    % Received % Xferd  Average Speed   Time    Time
> Time  Current
>                                  Dload  Upload   Total   Spent
> Left  Speed
> 100  235k  100  235k    0     0  47700      0  0:00:05  0:00:05
> --:--:-- 55650
> 2871747365      241269
> 
> 
> 
> There are also failures with major web browsers including Firefox
> 11.0, MSIE 9.0 and Safari.
> 
> No error is reported by nginx.  The http return code and logged code
> is 200.  The only clue in the server is the sent size reported is
> smaller than the expected compressed size.
> 
> 
> 
> *** Other information ***
> 
> nginx 1.1.19
> Solaris 10 u10

Could you please provide debug log and nginx -V output?
See http://wiki.nginx.org/Debugging for details.

Maxim Dounin