Peer closed connection in SSL handshake marking upstream as failed

[Maxim Dounin]

Hello!

On Wed, Jun 21, 2017 at 04:51:55PM +0100, Steven Hartland wrote:

> We're seeing an 502 bad gateway responses to client on an nginx load 
> balanced upstream due to "no live upstreams".
> 
> The upstream in question has 2 servers defined with default settings 
> running over https (proxy_pass https://myupstream).
> 
> When this happens we see "no live upstreams while connecting to 
> upstream" in the nginx error log and just prior to this:
> "peer closed connection in SSL handshake (54: Connection reset by peer) 
> while SSL handshaking to upstream".
> 
> We currently believe that the client closing the connection is causing 
> the upstream to have a failure counted against it.
> 
> With the defaults of max_fails=1 and fail_timeout=10 it only takes two 
> such closes within a 10 second  window to take down all upstream nodes 
> resulting in the "no live upstreams" and hence all subsequent 
> connections for the next 10 seconds fail instantly with 502 bad gateway.
> 
> Does this explanation seem plausible, is this a bug in nginx?

Unlikely.

-- 
Maxim Dounin
http://nginx.org/