Cache questions

Maxim Dounin mdounin at
Tue Jul 14 00:50:54 MSD 2009


On Mon, Jul 13, 2009 at 04:02:06PM -0400, Jim Ohlstein wrote:

> Igor Sysoev wrote:
>> On Mon, Jul 13, 2009 at 03:33:21PM +0400, Maxim Dounin wrote:
>>> Hello!
>>> On Fri, Jul 10, 2009 at 07:39:46PM -0400, Jim Ohlstein wrote:
>>>> I'm using the fastcgi cache for static files (images, 
>>>> javascript,css)  and just found multiple lines in the error log 
>>>> like this one:
>>>> 2009/07/10 10:22:54 [crit] 22476#0: ngx_slab_alloc() failed: no 
>>>> memory  in cache keys zone "one"
>>>> So I increased the memory available for the zone and reloaded 
>>>> nginx. It  took over five hours to go through the cache but these 
>>>> are the relevant  entries:
>>>> 2009/07/10 12:11:03 [notice] 21038#0: start cache manager process 32730
>>>> 2009/07/10 12:11:04 [notice] 21038#0: cache manager process 22480 
>>>> exited  with code 0
>>>> and finally
>>>> 2009/07/10 17:43:27 [notice] 32730#0: http file cache:   
>>>> /usr/local/nginx/cache 11638.289M, bsize: 4096
>>>> My questions are:
>>>> Is that simply the total (11638.289MB or 11.4GB) of all of the file 
>>>>  sizes, or is that the actual disk space consumed taking into 
>>>> account  total number of blocks used multiplied by the block size? 
>>>> The number        
>>> It's size on disk (i.e. number of blocks * block size), but for  
>>> files only (it doesn't take directories into account).
>> Just note: nginx rounds a file size to the bsize.
>> bsize is f_bsize from statfs() or f_frsize from statvfs().
>> I'm not sure that bsize matches always a filesystem allocation unit.
> OK, Thanks.
> In trying to tune this, if I set fastcgi_cache_min_uses to 2, does that  
> mean that a file will only be written to the cache the second time that  
> it is requested? Google translate did not give me a clear answer to this  
> from the Russian documentation. I think that I could improve efficiency  
> greatly if I didn't cache files on the first request.

Yes, file will be written only after second request.  Note that 
shared memory will be allocated anyway.

> I would know  
> better if I could get some statistics. I know the last time I asked the  
> answer was "not yet". Do you have any idea when this might be  
> implemented even on a rudimentary basis?

Cache status was implemented in 0.8.3:

    *) Feature: the $upstream_cache_status variable.

You may log it to access_log and collect stats you want.

>>>> could be quite different given what I estimate are nearly one 
>>>> million  mostly small files in the cache at this point.
>>>> When I next upgrade nginx (I'm running 0.8.4), and I attempt a 
>>>> "graceful  upgrade" will it have to go through this entire process 
>>>> again?
>>> Yes.  Binary upgrade doesn't preserve shared memory zones, so cache 
>>> will be rescanned again.
>>> Note that nginx uses cache even before it was completely scanned, so 
>>> it shouldn't be a problem.
> Yes, but it uses a fair bit of system resources scanning a large cache.  
> Nothing much to be done about it though.

Actually, nginx tries to be nice and pauses scanning for 200ms 
if takes more than 200ms to scan 100 files.

Maxim Dounin

More information about the nginx mailing list