Peer closed connection in SSL handshake marking upstream as failed

[Steven Hartland]

We're seeing an 502 bad gateway responses to client on an nginx load 
balanced upstream due to "no live upstreams".

The upstream in question has 2 servers defined with default settings 
running over https (proxy_pass https://myupstream).

When this happens we see "no live upstreams while connecting to 
upstream" in the nginx error log and just prior to this:
"peer closed connection in SSL handshake (54: Connection reset by peer) 
while SSL handshaking to upstream".

We currently believe that the client closing the connection is causing 
the upstream to have a failure counted against it.

With the defaults of max_fails=1 and fail_timeout=10 it only takes two 
such closes within a 10 second  window to take down all upstream nodes 
resulting in the "no live upstreams" and hence all subsequent 
connections for the next 10 seconds fail instantly with 502 bad gateway.

Does this explanation seem plausible, is this a bug in nginx?

We're currently testing with max_fails=10 as a potential workaround.

     Regards
     Steve





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170621/4b9b014a/attachment-0001.html>