Potential Bug: Gzipping preventing HTTP range requests

Sat Jun 25 08:43:26 MSD 2011

On Sat, Jun 25, 2011 at 08:22:21AM +0400, Igor Sysoev wrote:
> On Fri, Jun 24, 2011 at 07:04:04PM -0400, Ensiferous wrote:
> > Yeah it's a weird situation. As a user I would probably expect that the
> > range applied to the actual content served, before it was compressed. So
> > that if I request 100 bytes when everything is transferred and
> > decompressed I have 100 bytes worth of content.
> 
> A "Content-Length" header of a gzipped response corresponds to length
> of the gzipped data. I'm not sure if this specified in RFC, but this
> is de facto behaviour. So since range is associated with the
> "Content-Length" header it should work with already gzipped body,
> so Apache 2.3.8 does it right:
> 
> ---------
> $ nc httpd.apache.org 80
> GET / HTTP/1.0
> Host: httpd.apache.org
> Range: bytes=10-151
> Accept-Encoding: gzip
> 
> HTTP/1.1 206 Partial Content
> Date: Sat, 25 Jun 2011 04:17:51 GMT
> Server: Apache/2.3.8 (Unix) mod_ssl/2.3.8 OpenSSL/1.0.0c
> Last-Modified: Sun, 22 May 2011 17:04:34 GMT
> ETag: "b96c29-247b-4a3e0595b4c80-gzip"
> Accept-Ranges: bytes
> Vary: Accept-Encoding
> Content-Encoding: gzip
> Content-Range: bytes 10-151/2631
> Content-Length: 142
> Connection: close
> Content-Type: text/html
> 
> ?Zms?:??_  [ ...  gzipped data ]
> 
> ---------
> $ nc httpd.apache.org 80
> GET / HTTP/1.0
> Host: httpd.apache.org
> Range: bytes=10-20, 30-50
> Accept-Encoding: gzip
> 
> HTTP/1.1 206 Partial Content
> Date: Sat, 25 Jun 2011 04:19:14 GMT
> Server: Apache/2.3.8 (Unix) mod_ssl/2.3.8 OpenSSL/1.0.0c
> Last-Modified: Sun, 22 May 2011 17:04:34 GMT
> ETag: "b96c29-247b-4a3e0595b4c80-gzip"
> Accept-Ranges: bytes
> Vary: Accept-Encoding
> Content-Encoding: gzip
> Content-Length: 226
> Connection: close
> Content-Type: multipart/byteranges; boundary=4a6819ef865b815212
> 
> 
> --4a6819ef865b815212
> Content-type: text/html
> Content-range: bytes 10-20/2631
> 
> ?Zms?:??_?
> --4a6819ef865b815212
> Content-type: text/html
> Content-range: bytes 30-50/2631
> 
> oo	????;yas?SF?Vc[
> --4a6819ef865b815212--
> --------
> 
> However, curl does not understand it:
> 
> --------
> $ curl -r 10-151 httpd.apache.org/
> html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
>                "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
> <html>
>  <head>
> $ curl -r 10-151 --compressed httpd.apache.org/
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
>                "http://www.w3.org/TR/
> --------
> 
> Note also that it's impossible to ungzip a response part if you have not
> preceding parts from the very start.

It seems I was wrong: curl tries to ungzip received body using zlib,
so since the first 10 bytes is gzip header, curl successfully decompresses
body, but this example does not work:

$ curl -r 11-151 --compressed httpd.apache.org/
curl: (61) Error while processing content unencoding: invalid distance code


-- 
Igor Sysoev