SSL_read error on multiple simultaneous upstream SSL downloads

[Agent Coulson]

I am able to reproduce the following error when I have nginx configured
with an upstream https connection.  I have tweaked various settings all to
no avail (proxy_buffer_size, proxy_buffers, proxy_ssl_session_reuse).

2013/10/18 17:17:31 [debug] 15644#0: *39 SSL_read: -1, SSL_pending: 16384
2013/10/18 17:17:31 [debug] 15644#0: *39 SSL_get_error: 1
2013/10/18 17:17:31 [error] 15644#0: *39 SSL_read() failed (SSL:
error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed or bad record
mac) while sending to client, client: 127.0.0.1, server: -, request: "GET
/test-1 HTTP/1.1", upstream: "https://x.x.x.x:443/test-1", host:
"localhost:1182"

I've applied the following patch to log the SSL_pending bytes after an
SSL_read.

--- dist/nginx-1.4.3/src/event/ngx_event_openssl.c      2013-10-08
12:07:14.000000000 +0000
+++ new/nginx-1.4.3/src/event/ngx_event_openssl.c       2013-10-18
17:37:15.059940303 +0000
@@ -952,7 +952,9 @@ ngx_ssl_recv(ngx_connection_t *c, u_char

         n = SSL_read(c->ssl->connection, buf, size);

-        ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL_read: %d", n);
+        ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0,
+                       "SSL_read: %d, SSL_pending: %d",
+                       n, SSL_pending(c->ssl->connection));

         if (n > 0) {
             bytes += n;

I've seen a bug report on this too (http://trac.nginx.org/nginx/ticket/215),
so thought i would send this here to see if anyone else is actively working
on the issue.

Here are my configure settings:

./configure --prefix=/var/nginx --with-debug --with-http_ssl_module
--without-http_auth_basic_module --without-http_autoindex_module
--without-http_browser_module --without-http-cache
--without-http_charset_module --without-http_empty_gif_module
--without-http_fastcgi_module --without-http_geo_module
--without-http_gzip_module --without-http_limit_conn_module
--without-http_map_module --without-http_memcached_module
--without-http_referer_module --without-http_rewrite_module
--without-http_scgi_module --without-http_split_clients_module
--without-http_ssi_module --without-http_upstream_ip_hash_module
--without-http_userid_module --without-http_uwsgi_module
--without-mail_imap_module --without-mail_pop3_module
--without-mail_smtp_module

Here is my configuration:

### Begin nginx.conf ###

worker_processes 1;

error_log logs/error.log debug;

pid logs/nginx.pid;

events {

 worker_connections 1024;

 }

http {

 include mime.types;
default_type application/octet-stream;

 access_log logs/access.log;

 keepalive_timeout 60;

 upstream http {

 server upstream.srv:443;
keepalive 512;

 }

 server {

 listen 1182 default_server;

  server_name -;

  ssl_protocols SSLv3 TLSv1;
ssl_ciphers RC4:HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;

  location / {

 proxy_pass https://http;

   proxy_redirect off;
proxy_read_timeout 10s;
proxy_connect_timeout 6s;


   proxy_buffering off;
proxy_buffer_size 64k;
proxy_buffers 6 16k;
proxy_busy_buffers_size 80k;

   proxy_pass_header Server;
proxy_pass_header Date;
proxy_pass_header X-Pad;

   proxy_set_header Connection "Keep-Alive";
proxy_set_header Host "upstream.srv";

 }

 }

 }
 ### End nginx.conf ###
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20131018/db37df66/attachment.html>