"No space left on device" for temp cache - v1.7.4

[arraisgabriel]

Thank you very much for the quick response.  It looks like that the cache
now stores many small files because of the revalidation feature, and reached
the inode storage limit.


Maxim Dounin Wrote:
-------------------------------------------------------
> Hello!
> 
> On Thu, Sep 11, 2014 at 10:15:43AM -0400, arraisgabriel wrote:
> 
> > Hi, recently we noticed that the version 1.7.3 added a feature
> important to
> > our infrastructure: "cache revalidation now uses If-None-Match
> header if
> > possible.".
> > So we changed part of our cache to the 1.7.4 version, but something
> strange
> > started to happen, at certain point of disk usage nginx started to
> return
> > 500 to all requests with this kind of message in the error log:
> > 
> > [crit] 12908#0: *7209656 open() "/cache/nginx_tmp/0002938835" failed
> (28: No
> > space left on device) while reading upstream, client:
> xxx.xxx.xxx.xxx,
> > server: , request: "GET http://xxxxxxxx.net/ HTTP/1.1", upstream:
> > "http://xxx.xx.xxx.xxx:80/", host: "xxxxxxxx.net"
> > 
> > It looked like that there wasn't enough space in the temporary cache
> > directory. But running df -h (and after sync && df -h) the result
> was:
> > 
> > [user at nginx ~]$ df -h
> > Filesystem      Size  Used Avail Use% Mounted on
> > /dev/xvda1      7.8G  2.3G  5.4G  30% /
> > devtmpfs        3.7G   20K  3.7G   1% /dev
> > tmpfs              3.7G     0     3.7G   0% /dev/shm
> > /dev/xvdb        40G   12G   27G  31%  24% /cache      
> > ---------------------------------------- used for cache
> 
> ENOSPC from open() likely means you've run out of inodes, not disk 
> space.  Try looking into "df -i", it may be helpful.
> 
> -- 
> Maxim Dounin
> http://nginx.org/
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,253185,253192#msg-253192