Cache questions

[Jim Ohlstein]

Igor Sysoev wrote:
> On Mon, Jul 13, 2009 at 03:33:21PM +0400, Maxim Dounin wrote:
>
>   
>> Hello!
>>
>> On Fri, Jul 10, 2009 at 07:39:46PM -0400, Jim Ohlstein wrote:
>>
>>     
>>> I'm using the fastcgi cache for static files (images, javascript,css)  
>>> and just found multiple lines in the error log like this one:
>>>
>>> 2009/07/10 10:22:54 [crit] 22476#0: ngx_slab_alloc() failed: no memory  
>>> in cache keys zone "one"
>>>
>>> So I increased the memory available for the zone and reloaded nginx. It  
>>> took over five hours to go through the cache but these are the relevant  
>>> entries:
>>>
>>> 2009/07/10 12:11:03 [notice] 21038#0: start cache manager process 32730
>>> 2009/07/10 12:11:04 [notice] 21038#0: cache manager process 22480 exited  
>>> with code 0
>>>
>>> and finally
>>>
>>> 2009/07/10 17:43:27 [notice] 32730#0: http file cache:  
>>> /usr/local/nginx/cache 11638.289M, bsize: 4096
>>>
>>> My questions are:
>>>
>>> Is that simply the total (11638.289MB or 11.4GB) of all of the file  
>>> sizes, or is that the actual disk space consumed taking into account  
>>> total number of blocks used multiplied by the block size? The number  
>>>       
>> It's size on disk (i.e. number of blocks * block size), but for 
>> files only (it doesn't take directories into account).
>>     
>
> Just note: nginx rounds a file size to the bsize.
> bsize is f_bsize from statfs() or f_frsize from statvfs().
> I'm not sure that bsize matches always a filesystem allocation unit.
>   

OK, Thanks.

In trying to tune this, if I set fastcgi_cache_min_uses to 2, does that 
mean that a file will only be written to the cache the second time that 
it is requested? Google translate did not give me a clear answer to this 
from the Russian documentation. I think that I could improve efficiency 
greatly if I didn't cache files on the first request. I would know 
better if I could get some statistics. I know the last time I asked the 
answer was "not yet". Do you have any idea when this might be 
implemented even on a rudimentary basis?

>   
>>> could be quite different given what I estimate are nearly one million  
>>> mostly small files in the cache at this point.
>>>
>>> When I next upgrade nginx (I'm running 0.8.4), and I attempt a "graceful  
>>> upgrade" will it have to go through this entire process again?
>>>       
>> Yes.  Binary upgrade doesn't preserve shared memory zones, so 
>> cache will be rescanned again.
>>
>> Note that nginx uses cache even before it was completely scanned, 
>> so it shouldn't be a problem.
>>     

Yes, but it uses a fair bit of system resources scanning a large cache. 
Nothing much to be done about it though.

>> Maxim Dounin
>>     
>
>
>