Buffering issues with nginx

[Valentin V. Bartenev]

On Friday 21 July 2017 07:02:07 Dan34 wrote:
[..]
> I run nginx on 8080, for testing, since it's not suitable for live use on 80
> in my case and I'm trying to figure out how to fix it.
> And here's why I believe that there is a bug.
> 
> In my case, I wrote test code on node side that serves some binary content.
> I can control speed at what node serves this content. On receiving end (on
> the other side of the planet) I use wget with --limite-rate. In the test
> that I'm trying to fix I send 5MB from nodejs at 20KB/s speed, client that
> requests that binary data reads it at 10KB/s. Obviously overall speed has to
> be 10KB/s as it's limited by the client that requests the data.
> 
> What happens is that entire connection from nginx to node is closed after
> node sends all data to nginx. Basically in my test 5MB will take
> approximately 500s to deliver, but node gets tcp connection closed 255 s
> from start (when there is still 250 more seconds to go and 2.5MB is still
> stuck on nginx side). So, no matter what I do nginx totally breaks my
> scenario, it does not obey any configs and still buffers 2.5MB
> 
[..]

No buffering doesn't mean no buffers used at all.

In your scenario there are at least 5 buffers involved, and 4 of them
are in OS kernel.

Here's the list:

1. Write socket buffer in kernel on node.js side where node.js
   writes data.

2. Read socket buffer in kernel for node.js connection from what nginx
   reads data.

3. Heap memory buffer to that nginx reads data from kernel socket buffer
   (controlled by proxy_buffers and proxy_buffer_size directives).

   No buffering here means that nginx doesn't keep that data in buffers
   for some time, but writes it immediately to write socket buffer in kernel
   for client connection.

4. Write socket buffer in kernel for client connection where nginx
   writes data.

5. Read socket buffer in kernel for client connection from what wget
   reads data.


   wbr, Valentin V. Bartenev