upstream (tcp stream mode) doesn't detect connecton failure

[Maxim Dounin]

Hello!

On Mon, Jan 08, 2018 at 12:37:41PM +0000, Cecile, Adam wrote:

> Hello,
> 
> 
> I'm using this quite complicated setup involving SNI routing and proxy_protocol but I'm stuck on something.
> 
> 
> Here is the configuration file:
> 
> http://paste.debian.net/hidden/62e13f9c/
> 
> 
> Routing, proxy_protocol, logging stuff is working just fine, the only (quite critical issue) is that the "mag" upstream doesn't see connection failures and does not switch to the second server.
> 
> 
> In the mag.log file I just see:
> 
> 98.98.98.98 [08/Jan/2018:10:56:10 +0100] proxying to "mag":10.0.0.1:443 TCP 500 0 239 1.01
> 
> 
> But instead of blacklisting this server and moving to 10.0.0.2 I receive a connection closed error on the client.

As far as I understand your configuration, you have two stream 
proxy layers:

1. The first one uses ssl_preread to obtain SNI name and tries to 
   do some routing based on it.  This layer also adds to the PROXY 
   protocol to backend connections.

2. The second one strips PROXY protocol header.

The problem with "upstream doesn't see connection failures" is 
because connection failures are only seen at the second layer (the 
log line above belongs to the second layer).  The first layer will 
only see a connection close, and it won't know if there was an 
error or not.

Also note:

- You use $proxy_protocol_addr in the "upstream mag {...}" block, 
  but the upstream block is used only in the first layer, where 
  $proxy_protocol_addr won't be available according to your 
  configuration.

- You use $name in the logs of the second layer.  It will always 
  point to "map", as there is no ssl_preread in the second layer, 
  hence $ssl_preread_server_name will be not available.

Depending on what you actually want to achieve, the most 
straightforward solution might be to actually remove the second 
proxy layer.

-- 
Maxim Dounin
http://mdounin.ru/