nginx cache growing well above max_size threshold

[Lucas Rolff]

Okay cool, I'll give it a try

In our case we do not run http2 on the machines since haproxy runs in front as well (which doesn't support http2)

I'll also try enable a bit more verbose logging on one of the machines to see what the logs say

Thanks a lot Maxim!

Best regards,
Lucas Rolff

Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: nginx <nginx-bounces at nginx.org> on behalf of Maxim Dounin <mdounin at mdounin.ru>
Sent: Thursday, September 14, 2017 6:55:57 PM
To: nginx at nginx.org
Subject: Re: nginx cache growing well above max_size threshold

Hello!

On Thu, Sep 14, 2017 at 04:34:09PM +0000, Lucas Rolff wrote:

> I have a minor question, so I have an nginx box using
> proxy_cache, it has a key zone of 40 gigabyte (so it can cache
> 320 million files), a max_size of 1500 gigabyte for the cache
> and the inactive set to 30 days.
>
> However we experience that nginx goes well above the defined
> limit - in our case the max size is 1500 gigabyte, but the cache
> directory takes goes well above 1700 gigabyte.
>
> There's a total of 42.000.000 files currently on the system,
> meaning the average filesize is about 43 kilobyte.
>
> Normally I know that nginx can go slightly above the limit,
> until the cache manager purges the files, but it stays at about
> 1700 gigabyte constantly unless we manually clear out the size.
>
> I see there's a change in 1.13.1 that ignores long locked cache
> entries, is it possible that this bugfix actually fixes above
> issue?
>
> Upgrading is rather time consuming and we have to ensure nginx
> versions across the platform, so I wonder if anyone has some
> pointers if the above bugfix would maybe solve our issue.
> (currently the custom nginx version is based on nginx 1.10.3).

https://trac.nginx.org/nginx/ticket/1163

TL;DR:

This behaviour indicate there is a problem somewhere, likely
socket leaks or process crashes.  Reports suggests it might be
related to HTTP/2.  The change in 1.13.1 don't fix the root cause,
but will allow nginx to keep cache under max_size regardless of
the problem.

--
Maxim Dounin
http://nginx.org/
_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170914/ed092baa/attachment-0001.html>