SSL_read error on multiple simultaneous upstream SSL downloads

[Maxim Dounin]

Hello!

On Wed, Oct 23, 2013 at 02:26:41PM -0700, Piotr Sikora wrote:

> Hey Maxim,
> 
> > While I tend to think that the problem is indeed related to
> > SSL_MODE_RELEASE_BUFFERS I don't see any reasons why the server
> > side shouldn't be affected.  Could you please point out why you
> > think so?
> 
> Well, I don't see this from the code, so it's just a hunch, but:
> - I wasn't able to reproduce it on the server side with big uploads,
> - I wasn't able to reproduce it on the client side with buffering on,
> - I was able to consistently reproduce it on the client side with buffering off,
> - I did a fast scan on some of our production logs and I see those
> errors only for content that would be transferred with proxy buffering
> off,
> - I think that we would see much more complains if this was happening
> on the server side or on the client side with default settings
> (buffering on).
> 
> I know this isn't very scientific, but those are the facts.
> 
> Note: I didn't play around with WebSockets... They are effectively
> unbuffered, so they might be triggering this issue on the server side.

As far as I understand, the problem happens if for some reason 
nginx isn't able to read all the data OpenSSL read from a socket, 
i.e. if some data are left in the OpenSSL read buffers.  (And of 
course it only happens if OpenSSL uses the same buffers for 
multiple connections.)

This is not something impossible on the server side - but likely 
much less common than with proxy_buffering set to off.  It can 
happen e.g. with pipelined requests, or if a request with a body 
is delayed with limit_req.

-- 
Maxim Dounin
http://nginx.org/en/donation.html