Nginx record length during disk IO

[Maxim Dounin]

Hello!

On Mon, Jan 16, 2017 at 07:10:41AM -0500, nicktgr15 wrote:

> Thanks for the useful information Maxim! 
> 
> We ended up using strace to monitor the system calls and it looks like that
> with our current setup (i.e. default buffer size) the record length is 65536
> bytes.
> 
> read(17, "\355\247=^\256\36\361\235~\356z"..., 65536) = 65536
> write(18, "\355\247=^\256\36\361\235~\356z"..., 65536) = 65536
> 
> At the moment we are setting use_temp_path=on using tmpfs as our temporary
> location which triggers a failed rename system call and then the above
> read/write calls happen.
> 
> rename("/var/lib/nginx/tmp/proxy/1/00/0000001",
> "/dev/shm/cache/a/f8/0c725bfea12c2f361c37") = -1 EXDEV (Invalid cross-device
> link)

This is exactly what proxy_cache_path documentation talks about 
(http://nginx.org/r/proxy_cache_path):

: Starting from version 0.8.9, temporary files 
: and the cache can be put on different file systems. However, be 
: aware that in this case a file is copied across two file systems 
: instead of the cheap renaming operation. It is thus recommended 
: that for any given location both cache and a directory holding 
: temporary files are put on the same file system.

Copying in question uses an internal 64k buffer.

> Do you think that having use_temp_path=on in combination with tmpfs has any
> advantages or we should set use_temp_path=off as suggested in the docs?

Copying files across file systems is something usually better to 
avoid, even if one of the file systems is tmpfs.  And almost 
always there are better ways to use memory then for temporary 
files on tmpfs.

Note well that "use_temp_path=off" is mostly identical to 
"use_temp_path=on" and proxy_temp_path configured to use the same 
filesystem.  It was introduced for complex configurations where 
different caches can be used, thus making it non-trivial to always 
configure proxy_temp_path on the same filesystem.

-- 
Maxim Dounin
http://nginx.org/