Multiple Cache Manager Processes or Threads

[Maxim Dounin]

Hello!

On Thu, Nov 30, 2017 at 12:20:19PM -0500, traquila wrote:

> I have an issue with the cache manager and the way I use it.
> When I configure 2 different caches zones, one very huge and one very fast,
> the cache manager can't delete files quickly enough and lead to a partition
> full.
> 
> For example:
> proxy_cache_path /mnt/hdd/cache levels=1:2:2 keys_zone=cache_hdd:40g
> max_size=40000g inactive=5d;
> proxy_cache_path /mnt/ram/cache levels=1:2   keys_zone=cache_ram:300m
> max_size=300g inactive=1h;
> 
> On the beginning, ram cache is correctly purge around 40GB (+/- Input
> bandwidth*10sec) , but when the hdd cache begins to fill up, ram cache
> growing over 50GB. I think the cache manager is stuck by the slowness of the
> filesystem / hardware.
> 
> I can fix this by using 2 nginx on the same machine, one configured as ram
> cache, the other as hdd cache; but I wonder if it would be possible to
> create a cache manager process for each proxy_cache_path directive.

Which nginx version you are using?

With nginx 1.11.5+, there are manager_files / manager_sleep / 
manager_threshold parameters you may want to play with, see 
http://nginx.org/r/proxy_cache_path.  These parameters allows 
limiting cache manager's work on a particular cache to some finite 
time, and therefore help to better maintain specified max_size of 
other caches.

If you are using an older version, an upgrade to the recent 
version might help even without further tuning, as older versions 
do not limit cache manager's work on a particular cache at all.

-- 
Maxim Dounin
http://mdounin.ru/