<div dir="ltr"><div><div>Yes, I am able to reproduce this talking to the same nginx as an upstream, here is my new config. To reproduce, create a file in the root which is several Mb, i used 20Mb, and issus multiple simultaneous curl's to the object, i found rate-limiting my curl is the best way to repro. This suggests there is some problem when we have to buffer. I'm skeptical that this is an openssl issue as I have used multiple different openssl versions and still run into this. However for completeness, I've reprod with openssl sources from <a href="http://openssl.org">openssl.org</a> (openssl-1.0.1e) as you suggested.<br>
<br></div>Updated conf:<br><br></div>### Begin ngxin.conf ###<br><div><br>worker_processes 1;<br><br>error_log logs/error.log debug;<br><br>pid logs/nginx.pid;<br><br>events {<br> worker_connections 1024;<br>
}<br><br><br>http {<br> include mime.types;<br> default_type application/octet-stream;<br><br> access_log logs/access.log;<br><br> keepalive_timeout 60;<br><br> upstream http {<br> server <a href="http://127.0.0.1:1183">127.0.0.1:1183</a>;<br>
keepalive 512;<br> }<br><br> server {<br> listen 1182 default_server;<br><br> server_name -;<br><br> ssl_protocols SSLv3 TLSv1;<br> ssl_ciphers RC4:HIGH:!aNULL:!MD5;<br> ssl_prefer_server_ciphers on;<br>
<br> location / {<br> proxy_pass <a href="https://http">https://http</a>;<br><br> proxy_redirect off;<br> proxy_read_timeout 10s;<br> proxy_connect_timeout 6s;<br> <br>
proxy_buffering off;<br> proxy_buffer_size 64k;<br> proxy_buffers 6 16k;<br> proxy_busy_buffers_size 80k;<br><br> proxy_pass_header Server;<br> proxy_pass_header Date;<br>
proxy_pass_header X-Pad;<br><br> proxy_set_header Connection "Keep-Alive";<br> proxy_set_header Host "upstream.srv";<br> }<br> }<br><br> server {<br>
listen 1183 ssl;<br> server_name upstream.srv;<br><br> ssl_certificate /var/nginx/conf/upstream.srv.pem;<br> ssl_certificate_key /var/nginx/conf/upstream.srv.key;<br><br> ssl_protocols SSLv3 TLSv1;<br>
ssl_ciphers RC4:HIGH:!aNULL:!MD5;<br> ssl_prefer_server_ciphers on;<br><br> location / {<br> root /var/nginx/html;<br> }<br> }<br>}<br><br>### End ngxin.conf ###<br><br></div><div>
configure flags for nginx 1.4.3:<br><br>./configure --prefix=/var/nginx --with-debug --with-http_ssl_module --without-http_auth_basic_module --without-http_autoindex_module --without-http_browser_module --without-http-cache --without-http_charset_module --without-http_empty_gif_module --without-http_fastcgi_module --without-http_geo_module --without-http_gzip_module --without-http_limit_conn_module --without-http_map_module --without-http_memcached_module --without-http_referer_module --without-http_rewrite_module --without-http_scgi_module --without-http_split_clients_module --without-http_ssi_module --without-http_upstream_ip_hash_module --without-http_userid_module --without-http_uwsgi_module --without-mail_imap_module --without-mail_pop3_module --without-mail_smtp_module --with-openssl=/tmp/openssl-1.0.1e<br>
<br></div><div></div><div>I start nginx and then issue 3 simultaneous curl's from the local box, rate-limited. This should be sufficient for anyone else to repro the issue.<br><br>curl --limit-rate 800k -v -o /dev/null <a href="http://localhost:1182/20m.txt&">http://localhost:1182/20m.txt&</a><br>
curl --limit-rate 800k -v -o /dev/null <a href="http://localhost:1182/20m.txt&">http://localhost:1182/20m.txt&</a><br>curl --limit-rate 800k -v -o /dev/null <a href="http://localhost:1182/20m.txt&">http://localhost:1182/20m.txt&</a><br>
<br></div><div>At least one will fail with bytes remaining, and you will see the error in the error.log.<br><br>2013/10/18 19:56:50 [error] 14667#0: *4 SSL_read() failed (SSL: error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac) while sending to client, client: 127.0.0.1, server: -, request: "GET /20m.bin HTTP/1.1", upstream: "<a href="https://127.0.0.1:1183/20m.bin">https://127.0.0.1:1183/20m.bin</a>", host: "localhost:1182"<br>
<br>thanks for your attention.<br></div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Oct 18, 2013 at 7:06 PM, Maxim Dounin <span dir="ltr"><<a href="mailto:mdounin@mdounin.ru" target="_blank">mdounin@mdounin.ru</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello!<br>
<div class="im"><br>
On Fri, Oct 18, 2013 at 06:01:14PM +0000, Agent Coulson wrote:<br>
<br>
> I am able to reproduce the following error when I have nginx configured<br>
> with an upstream https connection. I have tweaked various settings all to<br>
> no avail (proxy_buffer_size, proxy_buffers, proxy_ssl_session_reuse).<br>
><br>
> 2013/10/18 17:17:31 [debug] 15644#0: *39 SSL_read: -1, SSL_pending: 16384<br>
> 2013/10/18 17:17:31 [debug] 15644#0: *39 SSL_get_error: 1<br>
> 2013/10/18 17:17:31 [error] 15644#0: *39 SSL_read() failed (SSL:<br>
> error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed or bad record<br>
> mac) while sending to client, client: 127.0.0.1, server: -, request: "GET<br>
> /test-1 HTTP/1.1", upstream: "<a href="https://x.x.x.x:443/test-1" target="_blank">https://x.x.x.x:443/test-1</a>", host:<br>
> "localhost:1182"<br>
<br>
</div>I tend to think it's highly unlikely it's a problem in nginx.<br>
Most likely, it's a problem either in OpenSSL library used on<br>
nginx side, or in SSL implementation used on a backend.<br>
<br>
First thing I would recommend to test is to make sure you are able<br>
to reporoduce the problem:<br>
<br>
1. Using nginx statically compiled with a known version of the<br>
OpenSSL library (--with-openssl=..., with sources from<br>
<a href="http://openssl.org" target="_blank">openssl.org</a>).<br>
<br>
2. Using the same nginx as a backend.<br>
<br>
[...]<br>
<div class="im"><br>
> I've seen a bug report on this too (<a href="http://trac.nginx.org/nginx/ticket/215" target="_blank">http://trac.nginx.org/nginx/ticket/215</a>),<br>
> so thought i would send this here to see if anyone else is actively working<br>
> on the issue.<br>
<br>
</div>As of now, no one provided enough steps to reproduce the problem.<br>
And, see above, most likely the problem is not in nginx.<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Maxim Dounin<br>
<a href="http://nginx.org/en/donation.html" target="_blank">http://nginx.org/en/donation.html</a><br>
<br>
_______________________________________________<br>
nginx-devel mailing list<br>
<a href="mailto:nginx-devel@nginx.org">nginx-devel@nginx.org</a><br>
<a href="http://mailman.nginx.org/mailman/listinfo/nginx-devel" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx-devel</a><br>
</font></span></blockquote></div><br></div>