[BUG] Gunzip module may cause requests to fail

[Maxim Dounin]

Hello!

On Wed, Dec 02, 2015 at 07:39:25AM +0000, Aviram Cohen wrote:

> Maxim, I want to set up a reverse proxy the gunzips responses of 
> a backend server I cannot control (for example, a cache server 
> for a development infrastructure).
> The response is marked with Content-Length set to zero, so 
> Valentin's comment regarding why the response is empty is 
> irrelevant. It is also irrelevant when the server sends a 
> chunked response with an initial 0 sized chunk. I've seen 
> servers that do both.
> The HTTP level is okay besides the extra gzip header, and so 
> this is a "little bit pregnant" response ;)

Both cases described indicate that the response was corrupted 
earlier, and then an attempt to recover the corruption was done, 
likely unintended.  And, as previously said, nobody knows how the 
response is expected to look like if it wasn't malformed.

Though in the case when Content-Length is known in advance it is 
possible to improve current behaviour by logging an error earlier 
and passing the request unmodified.  Patches are welcome.

> HTTP clients such as web browsers, wget and curl do handle this 
> response, but my reverse proxy doesn't.
> What do you suggest that I'd do?

The problem is that most programs fail to check return codes 
properly.  This is not a plus though, but rather bugs in relevant 
programs.  And these bugs make it very easy to confuse valid 
responses with corrupted ones, as well as to incorrectly gunzip 
valid responses.

Trying to make nginx bug-to-bug compatible with browsers isn't 
likely to be beneficial though.  By trying to "fix" nginx to be 
bug-to-bug compatible with browsers and produce the same results 
as careless browsers do will make it impossible for proper clients 
to detect errors.  And this is what we are trying to avoid.

As for what to do, an obvious solution would be, as already 
suggested, to disable gunzipping of responses from sources you 
can't control and are known to return malformed responses.

> Regarding the error description itself, a patch that can applied 
> to better describe the error seems as easy for me as a fix, so 
> to me it seems irrelevant as long as you refuse to fix the 
> issue.
> Obviously I can provide a patch for the issue itself, which I 
> still consider as an Nginx bug :)

If you think you know how to improve things - feel free to provide 
patches.  Though as we already tried to explain several times, 
it's not a bug.  At most, it's a suboptimal error handling.

-- 
Maxim Dounin
http://nginx.org/