Query about keepalive

[Fasih]

Hi All,

Few days back I was trying to evaluate the performance of upstream
keepalive feature for a website when I noticed a rather unexpected
behaviour. It would be help me understand what's going on in the test.

Here's what I did:
1. Setup httperf to run a session load. This basically means that a text
file with different urls is supplied to httperf. httperf sends all the
requests in bursts spaced by a sec towards nginx.

2. Tcpdump is run on the machine

3. Before the tests begin, cache is cleared and nginx restarted

4. Test is repeated with httperf "replaying" the requests 1 time first, and
re-run with repeat count 4 to account for setup of connections/cache
priming etc

5. All the steps are repeated once without keepalive and with keepalive 512;

Results:
SessionKeep aliveConns upstreamConn TimeUnique upstream hostsReqs upstreamAvg
time to 1st byteMax upstream conn reuseClient conns (1)Client reqs (1)Client
replies (1)Testdur (1)Client connsClient reqsClient repliesTestdurmy-site148
8.30858191920.15262331213013031.219852052078.064my-site019220.716919192
0.1679461213013025.680852052071.781

Analysis:
* First row with keepalive, second row without keepalive.
* With keepalive, number of connections upstream (as seen in tcpdump) is
48. Note that my-site has multiple (19 - unique upstream hosts) subdomains,
each of which is individually configured. Without keepalive 192 connections
are made
* Total time spent establishing connections is 8.3 vs 20.7
* Latency is ~0.15secs
All these are as expected, test duration however goes from 78 -> 71 secs.
An *increase* in the time for test to complete.

As the number of unique upstream increases, the time increases further.
This wasnt something that I could explain. Please help me understand, is it
a bug in the system?

Configuration:
nginx version: nginx/1.1.18
built by gcc 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5)
TLS SNI support enabled
configure arguments: --without-http_ssi_module --without-http_geo_module
--without-http_uwsgi_module --without-http_scgi_module
--without-http_memcached_module --without-mail_pop3_module
--without-mail_imap_module --without-mail_smtp_module
--with-http_ssl_module --with-debug --without-http_rewrite_module
--prefix=/home/faskiri/Downloads/nginx-1.1.18/install

Cache config:
    proxy_cache_path  /tmp/cache levels=1:2 keys_zone=my-cache:8m
max_size=1000m inactive=600m;

Highly appreciate any help on this.

Regards
+Fasih
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20120424/8acb5386/attachment-0001.html>