Transfer-encoding chunked hex markers appearing in output?

[Maxim Dounin]

Hello!

On Mon, Mar 05, 2012 at 11:22:52AM +0800, Henry wrote:

> I've written a very simple script which acts as a specific proxy for
> XML feeds, reading from a source URL and passing the content
> straight through. It's trivial and it's working fine when run as a
> standalone HTTP server (via node.js).
> 
> However when I put nginx in front of it I get what appear to be the
> chunk hex markers in the output - which obviously renders the XML
> invalid. Here is an example of the output, note the "f32" at line 1,
> and the repeated "1000" on lines 53 & 85, the "58a" on lines 125 &
> 137 and so on.
> 
> https://gist.github.com/1973377
> 
> Here is my (reduced) nginx conf:
> 
>   server {
>       listen 80;
>       server_name feeder;
>       location / {
>           proxy_pass http://127.0.0.1:8888;
>       }
>   }
> 
> I've confirmed that the chunks themselves don't contain the spurious
> data (by dumping them to a file and checking with dhex), and apart
> from the expected Server: change the HTTP headers are the same with
> and without nginx:
> 
>   HTTP/1.1 200
>   Server: nginx/1.0.12
>   Date: Mon, 05 Mar 2012 02:02:38 GMT
>   Content-Type: text/xml; charset=UTF-8
>   Connection: keep-alive
>   etag: rM1jGkNUA9AzVj2iMQN9KWwLgcQ
>   last-modified: Mon, 05 Mar 2012 02:07:53 GMT
>   expires: Mon, 05 Mar 2012 02:08:03 GMT
>   cache-control: private, max-age=0
>   x-content-type-options: nosniff
>   x-xss-protection: 1; mode=block
>   transfer-encoding: chunked
> 
> I'm at a loss how to proceed and I'm hoping someone on the list
> recognises the problem.

According to RFC2616, "A server MUST NOT send transfer-codings 
to an HTTP/1.0 client."  That is, correct solution is to fix your 
script - it must not return chunked data to nginx (as nginx uses 
HTTP/1.0 in requests to backends).

Alternatively, you may upgrade to 1.1.x which has support for 
HTTP/1.1 to backends and is able to recognize and remove chunked 
encoding in such situations.

Maxim Dounin