Proxy buffering and slow clients related issues

Maxim Dounin mdounin at
Tue Oct 3 13:11:23 UTC 2017


On Mon, Oct 02, 2017 at 11:51:33PM -0400, rnmx18 wrote:

> Hi,
> We are trying to use NGINX for caching service in low bandwidth, high
> latency mobile networks. The service is to stream 10-sec video segments of
> different types ranging from 2MB to 50MB.
> NGINX proxy_buffering configuration is as follows:
>     proxy_buffering on;
>     proxy_buffer_size          4k;
>     proxy_buffers              64 4k;
>     proxy_busy_buffers_size    128k;
>     proxy_temp_file_write_size 64k;
> The slow clients results in NGINX buffering of the response and writing data
> to disk as part of temporary buffering.  The disk IO is causing higher TTFB
> and higher load time for video download.
> We have tried to configure the proxy_max_temp_file_size to 0 to disable the
> buffering.  This change results in interrupts not being balanced as shown
> below in the top command output  - core0 and core15 is using 100%.
> top - 13:53:04 up 6 days,  2:31,  5 users,  load average: 6.42, 4.35, 3.84
> Tasks: 370 total,   6 running, 364 sleeping,   0 stopped,   0 zombie
> %Cpu0  :  0.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,100.0 si,  0.0
> st


> Couple of queries:
> a) Why do we get unbalanced interrupts when buffering is disabled?

Try looking on what runs on these CPUs.

Given that it's "100.0 si", I would suggest that it is your NIC 
interrupt threads trying to cope with load.

> b) How to configure NGINX to throttle the upstream read and avoid temp
> buffering?

With proxy_max_temp_file_size set to 0 nginx won't buffer anything 
to disk, and will read from upstream up to available 
proxy_buffers.  As long as configured buffers are full, nginx will 
stop reading from the upstream server till at least one buffer is 

That is, nginx will read from the upstream at a rate 
controlled by bandwidth of connected clients.  You can use normal 
client limiting mechanisms such as limit_rate and limit_conn if 
the rate observed is too high.

Additionly, the proxy_limit_rate directive can be used to control 
rate limiting of connections to upstream servers, see  Though this is primary 
useful when you don't disable disk buffering but rather have to 
keep it enabled, for example, when using cache.

Maxim Dounin

More information about the nginx mailing list