Nginx proxy overhead with / out keep alive requests

[Maxim Dounin]

Hello!

On Tue, Jun 12, 2012 at 04:30:51PM -0400, paphillon wrote:

> I have a test setup to measure the nginx overhead when plugged in front
> of a Jboss tomcat server.
> In tomcat I have deployed a test.jsp and use ab to measure the
> performance with the following scenarios 
> 
> With out keep alive ab option
> ab --> <jboss tomcat URL> (ab -n 5000 -c 5
> http://jboss.tomcat.url/test.jsp)
> ab --> <nginx URL> --> <jboss tomcat URL>  (ab -n 5000 -c 5
> http://nginx.proxy.url/test.jsp)
> 
> With keep alive ab option (-k)
> ab  --> <jboss tomcat URL>  (ab -n 5000 -c 5 -k
> http://jboss.tomcat.url/test.jsp)
> ab  --> <nginx URL> --> <jboss tomcat URL>  (ab -n 5000 -c 5 -k
> http://nginx.proxy.url/test.jsp)
> 
> The performance numbers WITHOUT keep alive is almost same BUT WITH keep
> alive option the performance numbers are very different and nginx takes
> about 5 secs more than the page accessed directly via tomcat. Tomcat
> takes only 0.503430 on an average 
> 
> Why should there be so much of deviation with Keep alive? Is there
> anything I am missing?

Unless you've configured keepalive to upstreams (see 
http://nginx.org/r/keepalive) nginx will not use keepalive 
connections to tomcat.  And the expected result is: nginx is in 
par with tomcat used directly without keepalive.

Nnumbers extracted from your data for clarity:

tomcat + keepalive:   9931.87 r/s
tomcat w/o keepalive: 1073.32 r/s
nginx  + keepalive:    919.03 r/s
nginx  w/o keepalvie: 1016.89 r/s

All nginx numbers are about tomcat's one without keepalive, as 
expected (see above).

The limiting factor is clearly tomcat's connection establishment 
cost, which drops performance from 10k r/s to 1k r/s.  You may 
want to configure upstream keepalive to cope with it if it's really 
matters in you case (i.e. if real requests are as fast as test one 
you've used; usually real request processing takes much more than 
connection establishment).  See the link above for details.

Maxim Dounin