Matching & Acting upon Headers received from Upstream with proxy_pass

[Maxim Dounin]

Hello!

On Thu, Jan 31, 2019 at 09:06:10PM +0000, Alec Muffett wrote:

> I'm running a reverse proxy and I want to trap when upstream is sending me:
> 
>   Content-Encoding: gzip
> 
> ...and on those occasions return (probably) 406 downstream to the client;
> the reason for this is that I am always using:
> 
>   proxy_set_header Accept-Encoding "identity";
> 
> ...so the upstream should *never* send me gzip/etc; but sometimes it does
> so because of errors with CDN configuration and "Vary:" headers, and that
> kind of thing. I would like to make the situation more obvious and easier
> to detect.

The 406 looks wrong to me.  It tells the client that the response 
is not acceptable as per accept headers in the request.  In your 
case it is more like 500, as the problem is that your backend 
returned a response it shouldn't, and your frontend thinks it's a 
fatal error.

Note well that HTTP servers are allowed to return gzipped 
responses even to requests with "Accept-Encoding: identify".  
Quoting RFC 2616 (https://tools.ietf.org/html/rfc2616#section-10.4.7):

      Note: HTTP/1.1 servers are allowed to return responses which are
      not acceptable according to the accept headers sent in the
      request. In some cases, this may even be preferable to sending a
      406 response. User agents are encouraged to inspect the headers of
      an incoming response to determine if it is acceptable. 

For example, this is something that can happen when only gzipped 
variants of files are available on the server.  Apache can be 
configured to add encoding based on the extension (so ".gz" files 
will be returned with "Content-Encoding: gz", see 
https://httpd.apache.org/docs/2.4/mod/mod_mime.html#addencoding), 
and nginx can return gzipped variants regardless of client's 
support with "gzip_static always;" (see 
http://nginx.org/r/gzip_static).

> I have been trying solutions like:
> 
>   if ( $upstream_http_content_encoding ~ /gzip/ ) { return 406; }
> 
> and:
> 
>   map $upstream_http_content_encoding $badness {
>   br 1;
>   compress 1;
>   deflate 1;
>   gzip 1;
>   identity 0;
>   default 0;
>   }
>   ...
>   server { ...
>   if ($badness) { return 406; }
> 
> ...but nothing is working like I had hoped, I suspect because I do not know
> if/where to place the if-statement such that the
> upstream_http_content_encoding is both set and valid during an appropriate
> processing phase.

This is not going to work as rewrite module directives are 
processed while selecting a configuration to work with (see 
http://nginx.org/en/docs/http/ngx_http_rewrite_module.html).  
Obviously enough this happens before the request is passed to the 
upstream, and before the response is received.

> The most annoying thing is that I can see that the
> upstream_http_content_encoding variable is set to "gzip", because if I do:
> 
>   more_set_headers "Foo: /$upstream_http_content_encoding/";
> 
> ...then I can see the "Foo: /gzip/" value on the client; but that does not
> help me do what I want.
> 
> Can anyone suggest a route forward, please?

You should be able to do what you want by writing a header filter 
which will check headers returned and will return an error if 
these headers don't match your expectations.

-- 
Maxim Dounin
http://mdounin.ru/