Performance plateau when keepalives enabled
nginx-forum at nginx.us
Mon Mar 8 19:30:52 MSK 2010
httperf is running from two separate client machines both within 10ms ping from the webserver. While httperf is running, I am also browsing the site on my workstation (200ms from the webserver) to see how it feels for a user.
When I run the benchmark with keepalive_timeout 0, nginx is very very fast and accepts new connections instantly. Concurrent connections never goes above 5 since the benchmark clients are so close to the server. However,
When I run the benchmark with keepalive_timeout 65, nginx is very slow (5-20 seconds delay) to accept new connections as soon as it reaches 1500 concurrent connections. It plateaus here: it completes all the requests but it never exceeds approximately 1500 concurrent connections.
Is this normal? Everywhere I read tells me to leave keepalive on and it shouldn't make any difference in nginx. So is this a keepalive problem, or a problem handling >1500 concurrent connections? What should I look at to fix it?
The box is 97% idle and it is not waiting on disk.
OS is Linux 2.6 with fs.file-max=65535 also before starting nginx, ulimit -n 65535 is called (from /etc/init.d/nginx)
There are no errors at all in the nginx error.log. Not for file descriptors, or anything. I watched error.log at "info" logging level and the only messages at all are the keepalive connection closing messages, which I believe are normal.
The nginx.conf has (amongst other things):
worker_processes 4; # it's quad core machine
Posted at Nginx Forum: http://forum.nginx.org/read.php?2,61482,61482#msg-61482
More information about the nginx