Revisiting the long-overdue "TODO always gunzip" in ngx_http_gunzip_filter_module.c

[Maxim Dounin]

Hello!

On Sun, Oct 20, 2019 at 12:00:07PM -0500, J.R. wrote:

> Recently I was looking into having my upstream server gzip content
> that is sent to nginx (which is acting as a reverse proxy) to reduce
> local bandwidth. However, I needed to decompress the response so nginx
> could do some manipulation, then obviously it would get re-compressed
> (typically with brotli) before sending out to the client.
> 
> After finding numerous posts saying that "that feature doesn't exist",
> then looking at the source of the gunzip module and finding the "TODO"
> saying the exact thing I was looking for, I managed to find this old
> post from 2013 where someone made a patch to do the exact thing
> needed:
> 
> http://mailman.nginx.org/pipermail/nginx-devel/2013-January/003276.html
> 
> It's only about 11 lines of code, attached is a newer patch I did
> against 1.17.4 since the gunzip source changed a bit. Basically the
> patch adds the option "gunzip_force on;"
> 
> I do agree with the TODO comment in the source that having a way for
> other modules to request decompression would be great, but also having
> a configuration option (like this patch) is also a good feature.
> 
> After implementing this patch, and configuring the upstream server to
> use gzip level 1 (figured maximize speed and minimize cpu usage), my
> local bandwidth usage between the two immediately dropped 80-85%, and
> I swear even the load levels dropped a hair, probably due to less
> networking overhead, despite the compressing & decompressing of
> content.
> 
> I realize there is probably a mile-long TODO list for nginx features,
> but something so trivial like this patch has such a huge impact on
> network usage and I'm sure a better implementation would still only be
> a minimal code change. Just trying to give this a nudge so it's not
> forgotten, as I did come across quite a few posts online into how to
> do such a thing, so the demand is out there.

Just in case, my position hasn't changed since then:

http://mailman.nginx.org/pipermail/nginx-devel/2013-January/003284.html

Also note that if you really need to force gunziping for some reason, 
you can do so out of the box by using an additional local proxying 
layer with appropriate "proxy_set_header Accept-Encoding".

-- 
Maxim Dounin
http://mdounin.ru/