Occasional misplaced gzip headers?

[Frank Farmer]

We're still seeing this on nginx 0.7.65, although the users who've been 
reporting these issues claim  they're less frequent since the upgrade.

Again, here's the first few bytes of the raw response as received by the 
client, downloaded, and forwarded to us:

$ od -c postupgrade.dat | head
0000000 037 213  \b  \0  \0  \0  \0  \0  \0 003 002  \0  \0  \0 377 377
0000020 003  \0  \0  \0  \0  \0  \0  \0  \0  \0   H   T   T   P   /   1
0000040   .   1       2   0   0       O   K  \r  \n   S   e   r   v   e
0000060   r   :       n   g   i   n   x   /   0   .   7   .   6   5  \r
0000100  \n   D   a   t   e   :       M   o   n   ,       0   5       A


Frank Farmer wrote:
> Hi all,
>
> I've been seeing rare malformed responses from nginx with a strange 26 
> byte header prepended to them.
>
> $ od -c file.part | head
> 0000000 037 213  \b  \0  \0  \0  \0  \0  \0 003 002  \0  \0  \0 377 377
> 0000020 003  \0  \0  \0  \0  \0  \0  \0  \0  \0   H   T   T   P   /   1
> 0000040   .   1       2   0   0       O   K  \r  \n   S   e   r   v   e
>
> The first two bytes look suspiciously like a gzip header.
>
> I saw a problem like this back in April '09 when I first deployed 
> nginx as a reverse proxy in front of Apache:
>
> http://stackoverflow.com/questions/736499/strange-http-gzip-issue
>
> I upgraded versions at the time to 0.6.35, and the problem persisted.  
> Turns out, I had gzip on in apache, as well as in nginx.  Disabling 
> gzip in apache seemed to correct the issue.  We ran with that 
> configuration, with nginx on port 80, and apache behind it, on the 
> same box for a year without further issue.
>
> Recently, we moved nginx to a separate box on the same LAN, keeping 
> the same version (0.6.35 -- I know, it's old, but it's been stable).  
> Suddenly, we're seeing very similar symptoms again.  It doesn't happen 
> reliably, and it doesn't happen often.  I had one user report 3 
> instances over the course of 15 minutes.  After receiving a garbled 
> response, a second request for the same URL seems to succeed.
> We still have one site going through nginx 0.6.35 on the same server 
> as apache without issue.  Only the "remote" nginx installation is 
> malfunctioning.
>
> I'm not seeing any errors in the apache or nginx error logs for these 
> requests, and I'm confident that apache is not gzipping in this 
> context, only nginx (the nginx access log shows a response size about 
> 25% that of apache, suggesting that apache's bodies are infact 
> uncompressed)
>
> We've upgraded to 0.7.65, but I'm not really expecting that to resolve 
> the issue -- if it does, I'll certainly report back.  Our sysop 
> crawled the nginx changelogs and much of the mailing list archives, 
> and didn't find anything that sounded relevant.
>
> So, has anyone seen anything like this before?  Any leads would be 
> welcome.
>
>
> Thanks!
> Frank Farmer