caching issues with nginx as reverse-proxy

[Maxim Dounin]

Hello!

On Thu, Jun 30, 2016 at 12:02:29AM +0200, dm at whocaresabout.de wrote:

> currently I am serving files with a size about 1,5G (static without dynamic
> content) using a hand full of nodes and nginx in reverse proxy setup.
> Caching works, ..but not as expected. During the requests nginx creates a
> lot of temp caching files that grow up to the size of the origin file
> delivered from backend server. It's kinda weird that even if the inactive
> caching time out is not reached, the file is randomly downloaded again from
> backend.

Multiple temporary files can be created when the response is not 
yet cached and there are multiple requests to the resource in 
question.  Once the response is cached it will be returned from 
cache to subsequent requests, and no additional temporary files 
will appear.

To mininize parallel caching by multiple simultaneous requests 
there is the "proxy_cache_lock" directive.  Though for 1.5G files 
default timeouts it uses may not be enough, consider tuning 
proxy_cache_lock_age and proxy_cache_lock_timeout:

http://nginx.org/r/proxy_cache_lock_age
http://nginx.org/r/proxy_cache_lock_timeout

Additionally, consider using the slice module.  It was specially 
designed to improve caching of large files, see here:

http://nginx.org/en/docs/http/ngx_http_slice_module.html

[...]

> proxy_cache_path /var/tmp/nginx-cache/rproxy levels=1:2
> keys_zone=rproxy:260m max_size=5g inactive=260m use_temp_path=off;

Note well: with "max_size" set to 5g the cache in question can 
hold about 3 1.5G files.  It may not be what you want for 
effective caching. 

[...]

-- 
Maxim Dounin
http://nginx.org/