Client body upload fully occupies worker

[Maxim Dounin]

Hello!

On Fri, May 25, 2018 at 09:58:37AM -0400, python wrote:

> While client uploads content into permanent temp file ngx worker doesn't
> process any requests until its finished.
> 
> Its an issue in case of large files (GBs) uploading since number of parallel
> uploads are limited by number of workers.

This can happen if the client upload bandwidth is larger than the 
available disk bandwidth.  Normally this does not happens with 
typical clients, but may happen in various unbalanced 
configurations.  In particular, often such problems are reported 
when testing uploads on a localhost.

> https://www.nginx.com/blog/thread-pools-boost-performance-9x/ advises to use
> aio_write:
> "At the moment, offloading to thread pools is implemented only for three
> essential operations: the read() syscall on most operating systems,
> sendfile() on Linux, and aio_write() on Linux which is used when writing
> some temporary files such as those for the cache".
> 
> I tried to configure aio_write, but looks that approach won't work for
> client body upload.
> 
> 1. How to configure thread pool usage for client body upload?

As of now, thread pools cannot be used for client body upload.

> 2. Could you please suggest any other approaches for non-blocking client
> body upload?

See https://trac.nginx.org/nginx/ticket/1431#comment:1 for a 
detailed explanation on how things are expected to work and how 
nginx worker can be monopolized, as well as some suggestsions on 
how to mitigate the issue.

In the particular case of writing client body to a temporary file, 
consider tuning "client_body_buffer_size" to a large enough value 
(http://nginx.org/r/client_body_buffer_size), as well as using 
smaller receive buffers.  You can tune receive buffers using the 
"rcvbuf=<n>" parameter of the "listen" directive, see 
http://nginx.org/r/listen.

For large uploads you may also consider using 
"proxy_request_buffering off", see 
http://nginx.org/r/proxy_request_buffering.

-- 
Maxim Dounin
http://mdounin.ru/