nginx SSL proxy

[Igor Sysoev]

On Fri, Feb 13, 2009 at 02:40:25AM +0200, Pavel Ivashkov wrote:

>   I have nginx to reverse proxy SSL site to end users. (And yes, I
>   need SSL on the back-end as well as on the front-end.)
> 
>   The problem is - it is too slow (~200ms overhead per each request).
> 
>   I found this comment from Dave Cheney:
> > The the dog slowness you are seeing is probably nginx renegitiation SSL on
> > every backend request. At the moment nginx will issue a connection close
> > after each request.
> 
>   So my question is whether it's still true for the nginx-0.7.34 ?
>   (assuming that client to nginx keeps single connection alive for all
>   requests)
> 
>   In the log file I can see several entries like:
>   =====================
> 2009/02/13 02:35:33 [debug] 26638#0: *1 SSL handshake handler: 0
> 2009/02/13 02:35:33 [debug] 26638#0: *1 SSL_do_handshake: 1
> 2009/02/13 02:35:33 [debug] 26638#0: *1 SSL: TLSv1, cipher: "RC4-MD5 SSLv3 Kx=RS
> A Au=RSA Enc=RC4(128) Mac=MD5"
> 2009/02/13 02:35:33 [debug] 26638#0: *1 save session: 08975D40:2
> 2009/02/13 02:35:33 [debug] 26638#0: *1 http upstream send request  
>   =====================
> 
>   Does it actually mean the SSL connection initialization to backend?

Yes, as nginx still does not use persisent connection to backend, it
has to do SSL renegitiation. However, it tries to reuse previously
saved SSL session, this eliminates RSA:

proxy_ssl_session_reuse on|off;  # default is on

Changes with nginx 0.7.11                                        18 Aug 2008

    *) Feature: the "proxy_ssl_session_reuse" directive.

If a session was reused you will lines:
"SSL: TLSv1, cipher: "RC4-MD5 SSLv3 Kx=RS A Au=RSA Enc=RC4(128) Mac=MD5"
"SSL reused session"

However, I can not say what may be a cause of 200ms delay.
Probably tcpdump may help.

>   Turning on google_perftools_profiles gave me the following list but
>   I'm not sure how to interpret the results and whether is shows the
>   full picture:
> 
> # pprof --text /usr/local/nginx/sbin/nginx profile.24433
> Total: 51 samples
>       11  21.6%  21.6%       11  21.6% _x86_AES_encrypt
>        5   9.8%  31.4%        5   9.8% sha1_block_asm_data_order
>        4   7.8%  39.2%        4   7.8% AES_cbc_encrypt
>        4   7.8%  47.1%        4   7.8% memcpy
>        4   7.8%  54.9%        4   7.8% __epoll_wait_nocancel
>        3   5.9%  60.8%        3   5.9% md5_block_asm_host_order
>        3   5.9%  66.7%        3   5.9% __read_nocancel
>        2   3.9%  70.6%        2   3.9% RC4
>        2   3.9%  74.5%        2   3.9% __write_nocancel
>        1   2.0%  76.5%        1   2.0% bn_sub_part_words
>        1   2.0%  78.4%        1   2.0% __gettimeofday_internal
>        1   2.0%  80.4%       25  49.0% ngx_worker_process_cycle
>        1   2.0%  82.4%        1   2.0% CRYPTO_lock
>        1   2.0%  84.3%        4   7.8% ngx_ssl_send_chain
>        1   2.0%  86.3%        1   2.0% lh_doall_arg
>        1   2.0%  88.2%        1   2.0% OPENSSL_cleanse
>        1   2.0%  90.2%        1   2.0% ERR_clear_error
>        1   2.0%  92.2%        1   2.0% tls1_mac
>        1   2.0%  94.1%        1   2.0% BN_from_montgomery
>        1   2.0%  96.1%        1   2.0% ngx_palloc
>        1   2.0%  98.0%        6  11.8% ngx_http_write_filter
>        1   2.0% 100.0%        1   2.0% RC4_set_key
>        0   0.0% 100.0%        1   2.0% DH_OpenSSL
>        0   0.0% 100.0%        6  11.8% ngx_http_postpone_filter
>        0   0.0% 100.0%        1   2.0% ERR_add_error_data
>        0   0.0% 100.0%        2   3.9% ngx_http_upstream_finalize_request
>        0   0.0% 100.0%        3   5.9% ssl3_write
>        0   0.0% 100.0%       12  23.5% ngx_http_upstream_process_upstream
>        0   0.0% 100.0%       10  19.6% ngx_event_pipe  

As I understand here main CPU hogs are AES/SHA cipher, but not RSA.

Here are RSA related functions:

        1   2.0%  76.5%        1   2.0% bn_sub_part_words
        1   2.0%  94.1%        1   2.0% BN_from_montgomery


-- 
Igor Sysoev
http://sysoev.ru/en/