load balancer the best way to solve

[Rick Gutierrez]

Hi list, I have a situation where I am looking for the best way to
solve it, I have a load balancer with nginx 1.22.1 and behind it I
have three backend servers:

                               /  -> App1
                              /
load balancer.  ----/ --> App2
                            /
                           /---> App3

if shutdown app1, the load balancer keeps sending traffic to app1 ,
and the clients are in a lag waiting for app1 to respond, I think the
load balancer should send all those clients to app2 and app3, but it
doesn't.

it put me in research mode :)  and the nginx version can't do that,
it's only in the plus version, correct me if I'm wrong, but "Voilà"
https://docs.nginx.com/nginx/admin-guide/load-balancer/http-health-check/#hc_active
, It gave me hope when reading that this option could help me, that by
adding it to the nginx configuration it would not be able to continue
sending requests, but it is not like that.

logs:

2023/01/12 18:07:38 [error] 26895#26895: *834 no live upstreams while
connecting to upstream, client: 44.210.106.130, server: demo.app.com,
request: "GET /aaaaaaaaaaaaaaaaaaaaaaaaaqr HTTP/1.1", upstream:
"http://paginaweb/aaaaaaaaaaaaaaaaaaaaaaaaaqr", host: "demo.app.com",
referrer: "http://173.255.X.X:80/aaaaaaaaaaaaaaaaaaaaaaaaaqr"
2023/01/12 18:07:38 [error] 26895#26895: *832 no live upstreams while
connecting to upstream, client: 44.210.106.130, server: demo.app.com,
request: "GET /99vt HTTP/1.1", upstream: "http://paginaweb/99vt",
host: "demo.app.com", referrer: "http://173.255.X.X:80/99vt"
2023/01/12 18:07:38 [error] 26895#26895: *838 no live upstreams while
connecting to upstream, client: 44.210.106.130, server: demo.app.com,
request: "GET /99vu HTTP/1.1", upstream: "http://paginaweb/99vu",
host: "173.255.X.X:443"
2023/01/12 18:07:39 [error] 26895#26895: *841 no live upstreams while
connecting to upstream, client: 44.210.106.130, server: demo.app.com,
request: "GET /99vu HTTP/1.1", upstream: "http://paginaweb/99vu",
host: "demo.app.com", referrer: "http://173.255.X.X:80/99vu"
2023/01/12 20:24:16 [error] 26895#26895: *6206 upstream prematurely
closed connection while reading response header from upstream, client:
167.94.138.62, server: demo.app.com, request: "GET / HTTP/1.1",
upstream: "http://104.237.138.27:80/", host: "demo.app.com"
2023/01/12 20:24:16 [error] 26895#26895: *6206 no live upstreams while
connecting to upstream, client: 167.94.138.62, server: demo.app.com,
request: "GET / HTTP/1.1", upstream: "http://paginaweb/", host:
"demo.app.com"
2023/01/12 20:24:26 [error] 26895#26895: *6220 no live upstreams while
connecting to upstream, client: 167.248.133.46, server: demo.app.com,
request: "GET / HTTP/1.1", upstream: "http://paginaweb/", host:
"demo.app.com"
2023/01/12 21:14:27 [error] 26895#26895: *7559 no live upstreams while
connecting to upstream, client: 162.221.192.26, server: demo.app.com,
request: "GET / HTTP/1.1", upstream: "http://paginaweb/", host:
"173.255.X.X"


#### CONFIG LB ####

upstream paginaweb {
        ip_hash;
        server 96.X.X.X:80 weight=1 fail_timeout=30s max_fails=3;
        server 104.X.X.X:80 weight=1 fail_timeout=30s max_fails=3;
        server 190.X.X.X:80 weight=1 fail_timeout=30s max_fails=3;
        keepalive 5;
    }

server {

        server_tokens off;
        listen 80;
server_name demo.app.com;
pagespeed unplugged;
        return 301 https://$server_name$request_uri;
}


server {
    listen 443 ssl http2;
    server_name demo.app.com;
    access_log /var/www/vhost/demo.app.com/logs/access.log specialLog;
    error_log /var/www/vhost/demo.app.com/logs/error.log;
    add_header Cache-Control "max-age=86400,  public";
    ssl_certificate /etc/letsencrypt/live/demo.app.com/fullchain.pem;
# managed by Certbot
    ssl_certificate_key
/etc/letsencrypt/live/demo.app.com/privkey.pem; # managed by Certbot
    ssl_trusted_certificate /etc/letsencrypt/live/demo.app.com/chain.pem;
#    ssl_certificate /etc/nginx/ssl/nginx-selfsigned.crt;
#    ssl_certificate_key /etc/nginx/ssl/nginx-selfsigned.key;
    ssl_protocols  TLSv1.3 TLSv1.2;
    ssl_ciphers
ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-A
ES256-SHA384;
    ssl_prefer_server_ciphers on;
    ssl_stapling on;
    ssl_stapling_verify on;
    add_header Strict-Transport-Security "max-age=31557600; includeSubDomains";
    add_header X-Xss-Protection "1; mode=block" always;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    keepalive_requests 1000;
    keepalive_timeout  75 75;

    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout  30m;

    gzip            on;
    gzip_types text/plain  text/css text/xml text/javascript;
    gzip_proxied any;
    gzip_vary on;

        client_max_body_size 5120m;
        location / {
   proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
   proxy_set_header X-Real_IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_set_header X-WAf-Proxy true;
   real_ip_header X-Real-IP;
   proxy_http_version 1.1;
            proxy_pass http://paginaweb;
   proxy_redirect off;
#            health_check;
        }
    }

any idea?
-- 
rickygm

http://gnuforever.homelinux.com