reverse ssl proxy - speed & jitter

[newnovice]

I am setting up a nginx reverse ssl proxy - I have a machine I can use with
2 E5-2650 CPU's and lots of RAM. I have nginx-1.6.0 + openssl-1.0.1h
installed. I have taken into consideration most optimization suggestions out
there and incorporated them. I will attach a copy of my config file. 

(optimizing first connection experience is good) With my testing just for
handshake + connection setup with 2K cert it is taking 3.5ms on average. I
see spikes in this time every 40 or so handshakes. I would like the 90+
percentile of the handshakes to not have any jitter/variance. 

testing method: 
time 
for i in {1..1000}; 
do 
    httperf --hog --server localhost --port 443 --ssl --uri /nginx_ping
--ssl-no-reuse --num-calls 1  --num-conns 1 --rate 1 | egrep "Connection
time \[ms\]\: |Reply time \[ms\]\: " | awk {'print $5'} | xargs | tr -s " "
", " >> test.log; 
done; 

-if you think this methodology is not right - do let me know. I have looked
at the tcpdumps and made sure a full handshake is happening and then a GET
request is issued

gives me: request-time, connect_time, response_time 
request_time = connect_time(ssl handshake + connection setup) +
response_time. 


1. I want to debug why there is jitter in the handshake time - i want the
90th, 95th, 99th, 99.9th percentiles to also be around 3.5ms. 
2. I want to see if i can make nginx any faster to do handshake. what is the
fastest you guys think this can happen 
3. how can i profile nginx and proceed to make this faster

all comments are welcome!

thanks!


not sure how to attach config: 
config details: 
5 workers, worker_priority -10, timer_resolution 200ms, worker_cpu_affinity
to separates cores on cpu2, error_log to dev/null, use epoll, worker_conns
2000, multi_accept on, accept_mutex off, sendfile on, tcp_nopush on,
tcp_nodelay on, file caches, keepalive_timeout  5000, keepalive_requests
100000, reset_timedout_connection on, client_body_timeout 10, send_timeout
2, gzip, server_tokens off, postpone_output 0. upstream: keep alive 180,
proxy_buffering off, client_body_buffer_size 512K,
large_client_header_buffers 4 64k, client_max_body_size 0. server: listen
443 ssl, access_log off, ssl_buffer_size  8k, ssl_session_timeout  10m,
ssl_protocols  SSLv3 TLSv1, ssl_ciphers RC4-MD5, ssl_prefer_server_ciphers  
on, ssl_session_cache   shared:SSL:10m. location /nginx_ping - return 200.

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,252002,252002#msg-252002