Nginx manage multiple https website with keepalived

[linbo liao]

I already submit an issue in keepalived github issue page
<https://github.com/acassen/keepalived/issues/731> and stackoverflow. Paste
again for more people's help.

I want to use Nginx to manager multiple https website, refer to nginx
document <http://nginx.org/en/docs/http/configuring_https_servers.html>
(Name-based
HTTPS servers section), one method is to assign a separate IP for every
HTTPS servers. And in our environment, this is the only method.

Due to single-point issue, I want to use keepalived to manage master-backup
Nginx node. The logic is:

1. Setup master/backup nginx node

2. Master nginx will assign multiple vip via keepalived

3. Master nginx will be up, backup nginx is down. (due to backup nginx has
no vip, start will fail)

4. If master nginx is down, vip transfer to backup node, backup nginx start.

I test in Centos 7 with keepalived v1.3.5, but meet some issue.
Configurationmaster node

global_defs {
   router_id LVS_DEVEL}

vrrp_script chk_nginx {
    script "/usr/sbin/pidof nginx"
    interval 3
    !weight -5
    rise 1
    fall 2}

vrrp_instance VI_1 {
    state MASTER
    interface eth1
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.2.16
        192.168.2.17
    }
    track_script {
        chk_nginx
    }

    notify /etc/keepalived/notify_keepalived.sh
    notify_stop "systemctl stop nginx"}

backup node

global_defs {
   router_id LVS_DEVEL}

vrrp_script chk_nginx {
    script "/usr/sbin/pidof  nginx"
    interval 3
    !weight -5
    rise 1
    fall 2}

vrrp_instance VI_1 {
    state BACKUP
    interface eth1
    virtual_router_id 51
    priority 96
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.2.16
        192.168.2.17
    }
    track_script {
        chk_nginx
    }

    notify /etc/keepalived/notify_keepalived.sh
    notify_stop "systemctl stop nginx"}

check script:

$ cat /etc/keepalived/notify_keepalived.sh#!/bin/bash
TYPE=$1
NAME=$2
STATE=$3

echo $STATE > /tmp/k.log:case $STATE in
        "MASTER") systemctl start nginx
                  exit 0
                  ;;
        "BACKUP") systemctl stop nginx
                  exit 0
                  ;;
        "FAULT")  systemctl stop nginx
                  exit 0
                  ;;
        *)        echo "ipsec unknown state"
                  exit 1
                  ;;esac

method 1

If unset weight, keepalived startup will check nginx pid immediately, even
I set interval and fall parameter. Master nginx won't enter master state,
all node will enter fault state. No master will elect and no active nginx
will come up.

Dec 30 04:59:00 localhost systemd: Starting LVS and VRRP High Availability
Monitor... Dec 30 04:59:00 localhost Keepalived[20039]: Starting Keepalived
v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2 Dec 30 04:59:00 localhost
Keepalived[20039]: Unable to resolve default script username
'keepalived_script' - ignoring Dec 30 04:59:00 localhost Keepalived[20039]:
Opening file '/etc/keepalived/keepalived.conf'. Dec 30 04:59:00 localhost
systemd: PID file /var/run/keepalived.pid not readable (yet?) after start.
Dec 30 04:59:00 localhost Keepalived[20040]: Starting Healthcheck child
process, pid=20041 Dec 30 04:59:00 localhost Keepalived[20040]: Starting
VRRP child process, pid=20042 Dec 30 04:59:00 localhost systemd: Started
LVS and VRRP High Availability Monitor. Dec 30 04:59:00 localhost
Keepalived_healthcheckers[20041]: Opening file
'/etc/keepalived/keepalived.conf'. Dec 30 04:59:00 localhost
Keepalived_vrrp[20042]: Registering Kernel netlink reflector Dec 30
04:59:00 localhost Keepalived_vrrp[20042]: Registering Kernel netlink
command channel Dec 30 04:59:00 localhost Keepalived_vrrp[20042]:
Registering gratuitous ARP shared channel Dec 30 04:59:00 localhost
Keepalived_vrrp[20042]: Opening file '/etc/keepalived/keepalived.conf'. Dec
30 04:59:00 localhost Keepalived_vrrp[20042]: VRRP_Instance(VI_1) removing
protocol VIPs. Dec 30 04:59:00 localhost Keepalived_vrrp[20042]: WARNING -
script `systemctl` resolved by path search to `/usr/bin/systemctl`. Please
specify full path. Dec 30 04:59:00 localhost Keepalived_vrrp[20042]:
SECURITY VIOLATION - scripts are being executed but script_security not
enabled. Dec 30 04:59:00 localhost Keepalived_vrrp[20042]: Using LinkWatch
kernel netlink reflector... Dec 30 04:59:00 localhost
Keepalived_vrrp[20042]: VRRP sockpool: [ifindex(3), proto(112), unicast(0),
fd(10,11)] Dec 30 04:59:00 localhost Keepalived_vrrp[20042]:
/usr/sbin/pidof nginx exited with status 1 Dec 30 04:59:01 localhost
Keepalived_vrrp[20042]: VRRP_Instance(VI_1) Now in FAULT state Dec 30
04:59:03 localhost Keepalived_vrrp[20042]: /usr/sbin/pidof nginx exited
with status 1 Dec 30 04:59:06 localhost Keepalived_vrrp[20042]:
/usr/sbin/pidof nginx exited with status 1
method 2

If uncomment weigth, startup works fine. Master node assign vip and master
nginx startup. Backup nginx is down.

However, when I shutdown master nginx, master node priority (100-5) >
backup node (96-5). Although master nginx is down, but vip still be in
master node.
method 3

set master weight -5, set backup weigth 2.

   1. Startup keepalived, master node get vip, master nginx start. Backup
   nginx is down.
   2. Shutdown master nginx, master node priority 95 < backup node 96,
   backup node get vip, backup nginx start.
   3. Shutdown backup nginx, master node priority 95 < backup node
   96(98-2), backup still hold vip, no active nginx come up.

For this scenario, program startup depends on vip, how to manage HA?

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20171231/19657d76/attachment-0001.html>