Temp disk space usage

[Maxim Dounin]

Hello!

On Wed, 17 Oct 2007, Andrew Deason wrote:

> I'm running nginx 0.5.26 on OpenBSD 4.0 as a load-balancing reverse
> proxy to two backend web servers. Recently we tried serving several
> moderately large static files (around 100-200MB each), and had numerous
> concurrent connections downloading them at a time.
>
> When this started, the disk space usage on the machine running nginx
> began climbing rapidly, and soon ran out of space. There's only a few
> GB that nginx can use; we did not anticipate this server to need a lot
> of disk space, as it's not intended to be a caching proxy. Anyway, when
> the disk is full, nginx understandably started throwing "writev() failed
> (28: No space left on device) while reading upstream" errors, but when
> nginx was restarted, everything was fine (and we gained several GB of
> space back).
>
> The only nginx.conf directive that we have that has anything to do with
> buffering/caching/temp files is `large_client_header_buffers 4 8K;`,
> which doesn't have anything to do with this. Everything else related to
> buffers and temp files should be set to the default.
>
> Is nginx buffering the entire file from the backend server on disk
> while serving it to the client? Are there any options to tune this, or
> limit disk usage by temporary files?

By default, nginx will buffer reply from upstream to memory buffers 
(specified by proxy_buffers/proxy_buffers) and then to disk into directory 
specified by proxy_temp_path.

You may avoid disk buffering by setting proxy_buffering to off.

Detailed docs in English can be found here:

http://wiki.codemongers.com/NginxHttpProxyModule#proxy_buffering

Note:
It's really good idea to serve static files directly from nginx, not from 
backend servers.

Maxim Dounin


>
> -- 
> Andrew Deason
> zeroguy at verizon.net
>
>