How to disable creating tmpfile when using nginx as a cache

[张强（qiangzhang）]

Hi community,

I am using nginx as L1 Cache for small static files(10~100k) and use xfs as low level file system.

450G SSD is used and when files number stored to 25000000 and disk usage up to 85%, system load is very high (>15, on 32 CPUs ).

After basic debug/detect it using perf, I located the hot code at creating tmpfiles while receiving upstream data, showed as below:

[cid:image002.jpg at 01D035A2.9EF5B020]

Can any can guide me how to disable creating tmpfile for small (<200k) files and write to target cache dir while finished directly?

I have tried some configure item like: proxy_max_temp_file_size, from official document for proxy_max_temp_file_size I found:
“The zero value disables buffering of responses to temporary files.”

But why tmpfile still be created?


It seems that meta operation (inode alloc/free) is the bottleneck while using nginx to cache small files, could anyone give some way to optimize it?
For example:
- switch to other filesystem like ext4 ?
- adjust some keys_zone polices?
- others?

Thanks
Qiang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20150121/bd2e3301/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 77740 bytes
Desc: image002.jpg
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20150121/bd2e3301/attachment-0001.jpg>