proxy_cache purge details

[Maxim Dounin]

Hello!

On Thu, Jan 16, 2014 at 01:33:17PM +0100, David Watzke wrote:

> Hello,
> 
> we've got a problem with the proxy_cache feature in nginx. To be more
> precise, the problem occurs when the cache loader kicks in and starts
> deleting the expired files that are stored on a LVM-striped (non-raid) ext4
> partition across six huge SSD disks. The purge (sometimes?) takes ages and
> it completely kills the reads from the partition.
> 
> That's hardly an nginx issue, but it is why we would like to know if there's
> a posibility to force the cache purge so that small amounts of files get
> deleted more often rather than a lot of files at once getting deleted less
> often.
> 
> Also, it would help us to know just exactly how (and where) does the nginx
> store the last-access-time information for each file (for the 'inactive'
> feature in proxy_cache_path directive), if the atime feature is off for
> performance reasons. I'm guessing that it needs to store this so that it
> knows when to delete the files. It's quite difficult for us to find it in
> the sources, so if you could point us in the right direction, it would be
> awesome!

Last access information, as used to track cache items inactivity, 
is stored in cache memory zone only.  Or, more precisely, derived 
"expire" value is stored in nodes.  If cache is reloaded from 
disk, last access is assumed to be at the time of loading cache 
item by the loader.

It is strange that removing inactive cache items causes problems 
(there shouldn't be many at any given moment, event after reload, 
because loading is limited by loader_files, loader_sleep, 
loader_threshold parameters of the proxy_cache_path directive).  
But if it is, it's mostly the ngx_http_file_cache_expire() 
function you should look into.

-- 
Maxim Dounin
http://nginx.org/