Nginx proxy cache purge process does not clean up items fast enough for new elements
mdounin at mdounin.ru
Wed Oct 16 16:48:38 UTC 2019
On Wed, Oct 16, 2019 at 09:44:08AM -0400, sachin.shetty at gmail.com wrote:
> Thankyou Maxim, is there anyway I can make the cache manager a bit more
> aggressive in prune and purge? We already leave 20% of space free on the
> disks, but the concurrent request rate for large files can be huge and we
> still run in to this issue.
Most likely you are hitting space limits due to temporary files, which
aren't managed by cache manager at all. My recommendation would
be to consider proxy_cache_lock and/or "proxy_cache_use_stale
updating" to reduce number of concurrent requests trying to cache
the same files, see here:
Also you may want to tune things like proxy_cache_min_uses to
reduce unneeded caching on sporadic requests.
> What are your thoughts about disabling buffering on such issues? This is not
> a fatal error, so we should stop buffering and switch to streaming mode and
> let the request succeed with a error log line in error.log.
Yes, in theory. But the question is how complex and error prone
the resulting code would be. Hence the current choice is to treat
such errors as fatal.
More information about the nginx