Proxy buffering and slow clients related issues

[Maxim Dounin]

Hello!

On Mon, Oct 02, 2017 at 11:51:33PM -0400, rnmx18 wrote:

> Hi,
> 
> We are trying to use NGINX for caching service in low bandwidth, high
> latency mobile networks. The service is to stream 10-sec video segments of
> different types ranging from 2MB to 50MB.
> 
> NGINX proxy_buffering configuration is as follows:
> 
>     proxy_buffering on;
>     proxy_buffer_size          4k;
>     proxy_buffers              64 4k;
>     proxy_busy_buffers_size    128k;
>     proxy_temp_file_write_size 64k;
> 
> The slow clients results in NGINX buffering of the response and writing data
> to disk as part of temporary buffering.  The disk IO is causing higher TTFB
> and higher load time for video download.
> 
> We have tried to configure the proxy_max_temp_file_size to 0 to disable the
> buffering.  This change results in interrupts not being balanced as shown
> below in the top command output  - core0 and core15 is using 100%.
> 
> top - 13:53:04 up 6 days,  2:31,  5 users,  load average: 6.42, 4.35, 3.84
> Tasks: 370 total,   6 running, 364 sleeping,   0 stopped,   0 zombie
> %Cpu0  :  0.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,100.0 si,  0.0
> st

[...]

> Couple of queries:
> a) Why do we get unbalanced interrupts when buffering is disabled?

Try looking on what runs on these CPUs.

Given that it's "100.0 si", I would suggest that it is your NIC 
interrupt threads trying to cope with load.

> b) How to configure NGINX to throttle the upstream read and avoid temp
> buffering?

With proxy_max_temp_file_size set to 0 nginx won't buffer anything 
to disk, and will read from upstream up to available 
proxy_buffers.  As long as configured buffers are full, nginx will 
stop reading from the upstream server till at least one buffer is 
free.

That is, nginx will read from the upstream at a rate 
controlled by bandwidth of connected clients.  You can use normal 
client limiting mechanisms such as limit_rate and limit_conn if 
the rate observed is too high.

Additionly, the proxy_limit_rate directive can be used to control 
rate limiting of connections to upstream servers, see 
http://nginx.org/r/proxy_limit_rate.  Though this is primary 
useful when you don't disable disk buffering but rather have to 
keep it enabled, for example, when using cache.

-- 
Maxim Dounin
http://nginx.org/