Cache questions

[Maxim Dounin]

Hello!

On Mon, Jul 13, 2009 at 08:45:03PM -0400, Jim Ohlstein wrote:

>
>
> Igor Sysoev wrote:
>> On Mon, Jul 13, 2009 at 04:02:06PM -0400, Jim Ohlstein wrote:
>>
>>   
>>> Igor Sysoev wrote:
>>>     
>>>> On Mon, Jul 13, 2009 at 03:33:21PM +0400, Maxim Dounin wrote:
>>>>
>>>>        
>>>>> Hello!
>>>>>
>>>>> On Fri, Jul 10, 2009 at 07:39:46PM -0400, Jim Ohlstein wrote:
>>>>>
>>>>>            
>>>>>> I'm using the fastcgi cache for static files (images, 
>>>>>> javascript,css)  and just found multiple lines in the error log 
>>>>>> like this one:
>>>>>>
>>>>>> 2009/07/10 10:22:54 [crit] 22476#0: ngx_slab_alloc() failed: no 
>>>>>> memory  in cache keys zone "one"
>>>>>>
>>>>>> So I increased the memory available for the zone and reloaded 
>>>>>> nginx. It  took over five hours to go through the cache but 
>>>>>> these are the relevant  entries:
>>>>>>
>>>>>> 2009/07/10 12:11:03 [notice] 21038#0: start cache manager process 32730
>>>>>> 2009/07/10 12:11:04 [notice] 21038#0: cache manager process 
>>>>>> 22480 exited with code 0
>>>>>>
>>>>>> and finally
>>>>>>
>>>>>> 2009/07/10 17:43:27 [notice] 32730#0: http file cache:   
>>>>>> /usr/local/nginx/cache 11638.289M, bsize: 4096
>>>>>>
>>>>>> My questions are:
>>>>>>
>>>>>> Is that simply the total (11638.289MB or 11.4GB) of all of the 
>>>>>> file  sizes, or is that the actual disk space consumed taking 
>>>>>> into account  total number of blocks used multiplied by the 
>>>>>> block size? The number                 
>>>>> It's size on disk (i.e. number of blocks * block size), but for  
>>>>> files only (it doesn't take directories into account).
>>>>>            
>>>> Just note: nginx rounds a file size to the bsize.
>>>> bsize is f_bsize from statfs() or f_frsize from statvfs().
>>>> I'm not sure that bsize matches always a filesystem allocation unit.
>>>>        
>>> OK, Thanks.
>>>
>>> In trying to tune this, if I set fastcgi_cache_min_uses to 2, does 
>>> that mean that a file will only be written to the cache the second 
>>> time that it is requested? Google translate did not give me a clear 
>>> answer to this from the Russian documentation. I think that I could 
>>> improve efficiency greatly if I didn't cache files on the first 
>>> request.
>>>     
>>
>> Yes, a file will be cached only on the second request made during period
>> set in by "inactive" option of fastcgi_cache_path directive.
>>
>>   
>>> I would know better if I could get some statistics. I know the last 
>>> time I asked the answer was "not yet". Do you have any idea when this 
>>> might be implemented even on a rudimentary basis?
>>>     
>>
>> You may log $upstream_cache_status.
>>   
>
> Can you give me an example of how to do this?
>
> I have tried various permutations of:
>
> location ~ (jpg|jpeg|png|gif|ico|js|css)$ {
>    fastcgi_pass unix:/tmp/my.sock;
>    fastcgi_cache one;
>    fastcgi_cache_key unix:/tmp/my.sock.1$request_uri;
>    fastcgi_ignore_headers  Cache-Control  Expires;
>    fastcgi_cache_valid 200 302 1d;
>    fastcgi_cache_valid 301 7d;
>    fastcgi_cache_valid any 10m;
>    fastcgi_cache_min_uses 2;
>    include /usr/local/nginx/conf/fastcgi_params;
>    fastcgi_param  SCRIPT_NAME myscript;
>    fastcgi_buffers 64 8k;
>    access_log  logs/my-cache.access.log;
>    log_format '$remote_addr - $remote_user [$time_local] '
>                          '"$request" $status $body_bytes_sent '
>                          '"$http_referer" "$http_user_agent"' '  
> $upstream_cache_status ';
> }
>
> The logs give me all above with exception of "$upstream_cache_status".  
> Every entry ends with the user agent.

log_format  cachestatus  '$remote_addr - $remote_user [$time_local] '
                         '"$request" $status $body_bytes_sent '
                         '"$http_referer" "$http_user_agent" '
                         '$upstream_cache_status';

access_log  /path/to/log  cachestatus;

Note that log_format's first argument is format name, and second 
argument of access_log is format name too (optional, defaults 
to 'combined' predefined format).

See http://wiki.nginx.org/NginxHttpLogModule for details.

Maxim Dounin