Handling 500k concurrent connections on Linux

[Andrew Alexeev]

John,

On Oct 9, 2012, at 11:10 PM, John Watson wrote:

> I was wondering if anyone had some tips/guidelines for scaling Nginx on
> Linux to >500k concurrent connections. Playing with the
> nginx_http_push_stream module in streaming mode. Noticing periodic slow
> accept and/or response headers. I've scoured the Internet
> looking/learning ways to tune Nginx/Linux but I think I've exhausted my
> abilities.
> 
> Any help would be appreciated.
> 
> Hardware
> Dual Nehalem 5520
> 24G RAM
> Intel 82576 (igb)
> Ubuntu 12.04.1 (3.2.0-31-generic x86_64)
> 
> Thank You,
> 
> John W

I'd assume you've already checked/fixed the following, right?

1) Error logs - anything wrong seen in there?

2) http://nginx.org/en/docs/ngx_core_module.html#multi_accept and http://nginx.org/en/docs/ngx_core_module.html#accept_mutex - did you try it on/off?

3) file descriptors limits (cat /proc/sys/fs/file-max, sudo - nginx && ulimit, worker_rlimit_nofile)

4) sysctl net.ipv4.ip_local_port_range (if you're aiming at proxying all those connections to upstreams)

Additional information about what's happening in all those 500k connections might be helpful, as well as the relevant configuration section :)

Hope this helps


-- 
AA @ nginx
http://nginx.com/support.html