[BUG] Gunzip module may cause requests to fail

[Aviram Cohen]

Thank you, Maxim. I think we pretty much agree.
The following is the suggested patch. Didn't include a patch for a chunked response, as you are right, that can evolve from an application error.
Feel free to change the error message.

diff -r cebc9a2c2144 src/http/modules/ngx_http_gunzip_filter_module.c
--- a/src/http/modules/ngx_http_gunzip_filter_module.c	Tue Apr 21 17:11:58 2015 +0300
+++ b/src/http/modules/ngx_http_gunzip_filter_module.c	Fri Dec 04 01:54:14 2015 +0200
@@ -140,6 +140,12 @@
 
     r->gzip_vary = 1;
 
+    if (r->headers_out.content_length_n == 0) {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "gunzip filter: zero length data to decompress");
+        return ngx_http_next_header_filter(r);
+    }
+
     if (!r->gzip_tested) {
         if (ngx_http_gzip_ok(r) == NGX_OK) {
             return ngx_http_next_header_filter(r);


-----Original Message-----
From: nginx-devel [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Maxim Dounin
Sent: יום ד 02 דצמבר 2015 20:26
To: nginx-devel at nginx.org
Subject: Re: [BUG] Gunzip module may cause requests to fail

Hello!

On Wed, Dec 02, 2015 at 07:39:25AM +0000, Aviram Cohen wrote:

> Maxim, I want to set up a reverse proxy the gunzips responses of a 
> backend server I cannot control (for example, a cache server for a 
> development infrastructure).
> The response is marked with Content-Length set to zero, so Valentin's 
> comment regarding why the response is empty is irrelevant. It is also 
> irrelevant when the server sends a chunked response with an initial 0 
> sized chunk. I've seen servers that do both.
> The HTTP level is okay besides the extra gzip header, and so this is a 
> "little bit pregnant" response ;)

Both cases described indicate that the response was corrupted earlier, and then an attempt to recover the corruption was done, likely unintended.  And, as previously said, nobody knows how the response is expected to look like if it wasn't malformed.

Though in the case when Content-Length is known in advance it is possible to improve current behaviour by logging an error earlier and passing the request unmodified.  Patches are welcome.

> HTTP clients such as web browsers, wget and curl do handle this 
> response, but my reverse proxy doesn't.
> What do you suggest that I'd do?

The problem is that most programs fail to check return codes properly.  This is not a plus though, but rather bugs in relevant programs.  And these bugs make it very easy to confuse valid responses with corrupted ones, as well as to incorrectly gunzip valid responses.

Trying to make nginx bug-to-bug compatible with browsers isn't likely to be beneficial though.  By trying to "fix" nginx to be bug-to-bug compatible with browsers and produce the same results as careless browsers do will make it impossible for proper clients to detect errors.  And this is what we are trying to avoid.

As for what to do, an obvious solution would be, as already suggested, to disable gunzipping of responses from sources you can't control and are known to return malformed responses.

> Regarding the error description itself, a patch that can applied to 
> better describe the error seems as easy for me as a fix, so to me it 
> seems irrelevant as long as you refuse to fix the issue.
> Obviously I can provide a patch for the issue itself, which I still 
> consider as an Nginx bug :)

If you think you know how to improve things - feel free to provide patches.  Though as we already tried to explain several times, it's not a bug.  At most, it's a suboptimal error handling.

--
Maxim Dounin
https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fnginx.org%2f&data=01%7c01%7cavcohe%40064d.mgd.microsoft.com%7cdaa80652f928400f4da508d2fb45fe6e%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=bPCynOkotayJWM7HJqvxms1mQ7D7YdHTek35BtYzPGc%3d

_______________________________________________
nginx-devel mailing list
nginx-devel at nginx.org
https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fmailman.nginx.org%2fmailman%2flistinfo%2fnginx-devel&data=01%7c01%7cavcohe%40064d.mgd.microsoft.com%7cdaa80652f928400f4da508d2fb45fe6e%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=z7Xc6UVwQCDGjS6iSbmzchq9yjmBp1p77k89F2XcDAY%3d