Reading large request body using ngx_http_read_client_request_body

[Maxim Dounin]

Hello!

On Mon, May 20, 2019 at 03:04:05PM -0400, NginxNewbee wrote:

> Thanks for your comments, Maxim. I truly appreciate it.
> 
> For first comment, the reason I chose to do request processing on thread is
> so it wouldn't block nginx. We launch one thread per request (from content
> handler). There will never will be multiple threads working on a request. r
> is passed onto a thread callback. Inside that callback, we extract headers,
> request body etc. from r and store it in our own objects. Then we do some
> compute (business logic) and then generate a response and write to output
> buffers of r. Then thread returns and its completion handler is called by
> nginx. In the completion handler we finalize request (with NGX_DONE). 
> Assumption here is that nginx wouldn't be messing with object r as long as
> our thread is executing (since we haven't finalized request yet). With this
> context, If you still think this isn't correct (and because of thread, I am
> seeing issue in reading body for large content size), I'll be go ahead and
> change my code and not use nginx  functions on thread.

Your assumption is not correct.  At any time nginx may have 
reasons to do something with r - e.g., when client closes a 
connection or cancels an HTTP/2 stream, or when something happens 
in a subrequest.

Moreover, functions such as ngx_http_read_client_request_body() 
work with various global objects, such as the list of all timers, 
various event-related things, and so on.  It is simply not 
expected to be called from other threads.

If you want to do your own processing in a thread, you have to 
extract all the needed information (including reading the request 
body) in the main thread context, and pass everything to your 
thread in your own tread-safe structures.

> For second comment - I may have not it explained correctly in my first post,
> but we do not call finalize request until thread finishes.

>From your explanation I assume that you return from the thread as 
long as ngx_http_read_client_request_body() returns NGX_AGAIN, so 
basically the request is finalized as long as 
ngx_http_read_client_request_body() returns NGX_AGAIN.

On the other hand, this is not really important as it is not going 
to work anyway - due to the above problem with trying to call 
ngx_http_read_client_request_body() in a thread.

-- 
Maxim Dounin
http://mdounin.ru/