SSL_read error on multiple simultaneous upstream SSL downloads

[Piotr Sikora]

Hey,

> Looks like a regression in OpenSSL 1.0.0+.  I'm able to reporduce
> the problem with OpenSSL 1.0.0 and more recent versions, including
> recent git snapshot, but everything is fine with OpenSSL 0.9.8y
> and previous versions.
>
> Bisection on OpenSSL 1.0.0 branch may be a helpful to trace the
> exact cause.

I've looked a bit into this over the weekend and it seems that it's
being triggered by use of both: reading ahead and releasing buffers
(introduced in OpenSSL-1.0.0, hence the regression) on the client side
with upstream buffering off (I wasn't able to reproduce it with
upstream buffering on, but that might be just because it's harder to
trigger, as OpenSSL code path is effectively the same in both cases).

I don't think that we're affected on the server side (which would
actually suggest nginx bug), so the work-around for the issue (at
least for the time being) is to stop releasing buffers when nginx acts
as a client. I'm a bit tempted to do it only for the case with
buffering turned off, but from looking at the code I can't tell why it
would make a difference.

Patch attached.

Best regards,
Piotr Sikora


# HG changeset patch
# User Piotr Sikora <piotr at cloudflare.com>
# Date 1382381412 25200
#      Mon Oct 21 11:50:12 2013 -0700
# Node ID 353b7360da138542933e01686f359ed72e7930d1
# Parent  e6a1623f87bc96d5ec62b6d77356aa47dbc60756
SSL: release OpenSSL buffers only when acting as a server.

Signed-off-by: Piotr Sikora <piotr at cloudflare.com>

diff -r e6a1623f87bc -r 353b7360da13 src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c Mon Oct 21 18:20:32 2013 +0800
+++ b/src/event/ngx_event_openssl.c Mon Oct 21 11:50:12 2013 -0700
@@ -237,10 +237,6 @@ ngx_ssl_create(ngx_ssl_t *ssl, ngx_uint_
     SSL_CTX_set_options(ssl->ctx, SSL_OP_NO_COMPRESSION);
 #endif

-#ifdef SSL_MODE_RELEASE_BUFFERS
-    SSL_CTX_set_mode(ssl->ctx, SSL_MODE_RELEASE_BUFFERS);
-#endif
-
     SSL_CTX_set_read_ahead(ssl->ctx, 1);

     SSL_CTX_set_info_callback(ssl->ctx, ngx_ssl_info_callback);
diff -r e6a1623f87bc -r 353b7360da13 src/http/modules/ngx_http_ssl_module.c
--- a/src/http/modules/ngx_http_ssl_module.c Mon Oct 21 18:20:32 2013 +0800
+++ b/src/http/modules/ngx_http_ssl_module.c Mon Oct 21 11:50:12 2013 -0700
@@ -528,6 +528,10 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t *
         return NGX_CONF_ERROR;
     }

+#ifdef SSL_MODE_RELEASE_BUFFERS
+    SSL_CTX_set_mode(conf->ssl.ctx, SSL_MODE_RELEASE_BUFFERS);
+#endif
+
 #ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME

     if (SSL_CTX_set_tlsext_servername_callback(conf->ssl.ctx,
diff -r e6a1623f87bc -r 353b7360da13 src/mail/ngx_mail_ssl_module.c
--- a/src/mail/ngx_mail_ssl_module.c Mon Oct 21 18:20:32 2013 +0800
+++ b/src/mail/ngx_mail_ssl_module.c Mon Oct 21 11:50:12 2013 -0700
@@ -285,6 +285,10 @@ ngx_mail_ssl_merge_conf(ngx_conf_t *cf,
         return NGX_CONF_ERROR;
     }

+#ifdef SSL_MODE_RELEASE_BUFFERS
+    SSL_CTX_set_mode(conf->ssl.ctx, SSL_MODE_RELEASE_BUFFERS);
+#endif
+
     cln = ngx_pool_cleanup_add(cf->pool, 0);
     if (cln == NULL) {
         return NGX_CONF_ERROR;