Regarding HTTP chunked Body being stored in temp_file

[Maxim Dounin]

Hello!

On Fri, Sep 16, 2022 at 07:31:08AM +0000, Devashi Tandon wrote:

> Hi,
> 
> In our module code, we are processing the HTTP request body when 
> it is not stored in r->request_body->temp_file.
> 
> When I send a 9381 bytes body, NGINX doesn't store the body in 
> temp_file but in the internal buffers. Hence we are able to 
> process the body.
> 
> However, when I enable chunked encoding, the same 9381 bytes 
> body, gets stored in the r->request_body->temp_file.
> 
> To avoid getting stored in temp_file, I have to increase the 
> client_body_buffer_size to a larger value than the default. In 
> that case, chunked encoded http body is NOT stored in temp_file 
> and we are able to process it.
> 
> Is there any reason why the behaviour of client_body_buffer_size 
> is different in case of regular HTTP traffic v/s chunked encoded 
> HTTP traffic? Why do we need a larger buffer size to ensure 
> chunked encoded traffic doesn't get stored in temp_file?

The client_body_buffer_size defaults to 8192 bytes, and this means 
that nginx can store up to 8192 bytes of raw data without using 
disk buffering.  If you are lucky enough, some request body bytes 
can also happen to be in the last client header buffer (and 9381 
you are seeing suggests it's the case).

With chunked encoding you'll likely get less space in the header 
buffer due to "Transfer-Encoding: chunked" additional header.  
Further, some space in the client body buffer will be spent on the 
chunked encoding framing.  Hence when using chunked encoding 
you'll need a larger buffer to keep the body of the same size in 
memory.

-- 
Maxim Dounin
http://mdounin.ru/