load balancer the best way to solve

[Maxim Dounin]

Hello!

On Thu, Jan 12, 2023 at 09:30:12PM -0600, Rick Gutierrez wrote:

> Hi list, I have a situation where I am looking for the best way to
> solve it, I have a load balancer with nginx 1.22.1 and behind it I
> have three backend servers:
> 
>                                /  -> App1
>                               /
> load balancer.  ----/ --> App2
>                             /
>                            /---> App3
> 
> if shutdown app1, the load balancer keeps sending traffic to app1 ,
> and the clients are in a lag waiting for app1 to respond, I think the
> load balancer should send all those clients to app2 and app3, but it
> doesn't.
> 
> it put me in research mode :)  and the nginx version can't do that,
> it's only in the plus version, correct me if I'm wrong, but "Voilà"
> https://docs.nginx.com/nginx/admin-guide/load-balancer/http-health-check/#hc_active
> , It gave me hope when reading that this option could help me, that by
> adding it to the nginx configuration it would not be able to continue
> sending requests, but it is not like that.

Certainly nginx can do that.  By default, all user requests are 
used by nginx to detect any upstream server failures, and re-route 
requests to other available servers.

Active health checks, which are indeed only available in the 
commercial version, are only different that they also use requests 
generated periodically by nginx-plus itself.  This might improve 
service to some real clients in some specific cases, but not 
generally required.

> logs:
> 
> 2023/01/12 18:07:38 [error] 26895#26895: *834 no live upstreams while
> connecting to upstream, client: 44.210.106.130, server: demo.app.com,
> request: "GET /aaaaaaaaaaaaaaaaaaaaaaaaaqr HTTP/1.1", upstream:
> "http://paginaweb/aaaaaaaaaaaaaaaaaaaaaaaaaqr", host: "demo.app.com",
> referrer: "http://173.255.X.X:80/aaaaaaaaaaaaaaaaaaaaaaaaaqr"
> 2023/01/12 18:07:38 [error] 26895#26895: *832 no live upstreams while
> connecting to upstream, client: 44.210.106.130, server: demo.app.com,
> request: "GET /99vt HTTP/1.1", upstream: "http://paginaweb/99vt",
> host: "demo.app.com", referrer: "http://173.255.X.X:80/99vt"

The errors indicate that all your upstream servers were not 
responding properly, and were either all tried for the particular 
request, or were disabled based on "fail_timeout=30s max_fails=3" 
in your configuration.

Usually looking into other errors in the logs makes it immediately 
obvious what actually happened.  Alternatively, you may want to 
further dig into what happened with the requests by logging the 
$upstream_addr and $upstream_status variables (see 
https://nginx.org/r/$upstream_addr and 
https://nginx.org/r/$upstream_status for details).

[...]

-- 
Maxim Dounin
http://mdounin.ru/