https + large-file sending, sometimes fails

[Igor Sysoev]

On Tue, Dec 18, 2007 at 10:15:03AM +0100, G??bor Farkas wrote:

> Igor Sysoev wrote:
> >On Mon, Dec 17, 2007 at 03:56:39PM +0100, G??bor Farkas wrote:
> >
> >>Igor Sysoev wrote:
> >>>On Mon, Dec 17, 2007 at 03:12:33PM +0100, G?bor Farkas wrote:
> >>>
> >>>>i am sending large (400mb) csv files using nginx, using https.
> >>>>
> >>>>sometimes not the whole file is served by nginx.
> >>>>it simply closes the connection before the whole file is sent.
> >>>>
> >>>>
> >>>>when such problems happen, the error-log contains this:
> >>>>
> >>>>2007/12/17 01:02:03 [crit] 21821#0: *864836 SSL_write() failed (SSL: 
> >>>>error:1409F07F:SSL routines:SSL3_WRITE_PENDING:bad write retry) while 
> >>>>sending response to client, client: 1.2.3.4, server: www.example.com, 
> >>>>URL: "/some/url/to/a.csv", upstream: 
> >>>>"http://internal-ip/some/auth/url/a.csv", host: "www.example.com"
> >>>>
> >>>>debian lenny (it has nginx 0.5.30-1)
> 
> maybe i am completely wrong here, but:
> 
> (on ubuntu gutsy and hardy):
> 
> simply create a minimal https-serving nginx-config, serve a 200mb file,
> and try to fetch it from a different computer, using a lot of
> concurrent-requests (something like "ab -n 1000 -c 100"),
> and you will get the mentioned error.
> 
> so, basically, any https-serving is broken.
> 
> i find this very hard to believe, but i do not know what should i change
> in the test.
> 
> (the packages:nginx-0.5.33, openssl-0.9.8g)
> 
> any ideas why this happens?

In the debug log I have not seen any invalid things from nginx side.
Then I have looked OpenSSL sources and now I suspect the bug in OpenSSL.
The attached patch may fix OpenSSL.

Could you build patched OpenSSL version and link it statically with nginx:

      tar zxf openssl-0.9.8g.tar.gz
      patch -d openssl-0.9.8g < bad_write_retry.txt
      tar zxf nginx-0.5.33.tar.gz
      cd nginx-0.5.33
      ./configure --with-openssl=../openssl-0.9.8g ...


-- 
Igor Sysoev
http://sysoev.ru/en/
-------------- next part --------------
--- ssl/s3_pkt.c	2007-12-19 00:06:58.000000000 +0300
+++ ssl/s3_pkt.c	2007-12-19 00:07:24.000000000 +0300
@@ -757,6 +757,7 @@
 			return(i);
 		s->s3->wbuf.offset+=i;
 		s->s3->wbuf.left-=i;
+		s->s3->wpend_tot-=i;
 		}
 	}