nginx proxy being slow

[Maxim Dounin]

Hello!

On Mon, Oct 06, 2014 at 09:06:19PM -0400, imran_k wrote:

> We are trying to act as a proxy for a site within the same DMZ. Things seem
> to work fine, except when there is quite a heavy load. There are many CSS
> assets that just hang upon retrieval. Sometimes the full page comes through;
> sometimes just spins forever. 
> 
> Server: nginx 1.6.1 running on Linux. 
> Memory: 18Gb
> 
> proxy_buffering on;
> proxy_buffers 256 8k;
> proxy_busy_buffers_size 64;

Just a side note: using 64 bytes for proxy_busy_buffers_size looks 
like a bad idea.  Additionally, it will be rejected by nginx as 
long as you use 8k proxy buffers.

> proxy_temp_file_write_size 64;

Same here.  64 bytes is way too low.

> Under heavy loads, about 1500 requests a second, a page is not completely
> sent back to the browser as some of the CSS resources taking anywhere from 2
> - 10 seconds to return. It will just spin until eventually it gets sent
> back. CPU and memory usage is not dramatically high. Smaller sites return
> without any issue at all. 
> 
> Do I have the buffering wrong or is there something else at play?

First of all, you may want to find out what causes problems you 
observe.  From your description I suspect you are actually 
debugging listen queue overflows.  When using Linux with 
net.ipv4.tcp_abort_on_overflow set to 0 (which is the default) it 
is not trivial to debug unless you are looking closesly into 
tcpdump and/or network stats (try looking into queue sizes in "ss 
-nlt").

-- 
Maxim Dounin
http://nginx.org/