Dual load balancer (one failsafe)?

[Alexander Staubo]

On Tue, Jun 10, 2008 at 12:33 PM, Floren Munteanu <nginx at yqed.com> wrote:
>> We use Nginx on six boxes in conjunction with Heartbeat2. The main web
>> server IP is automatically reassigned if the current web server box
>> goes down.
>
> I presume Heartbeat does that for you?

Yes. That's the point of Heartbeat. You have two boxes that monitor
each other; when one goes down, the other node uses ARP to negotiate
to take over the IP. (ARP is the underlying protocol that LANs use to
assign IPs to physical machines.) One of the boxes always has two IPs.

Box A
10.0.0.1
10.0.1.100   # magic IP

Box B
10.0.0.2

In the event of a failure on box A:

Box A    # now dead
10.0.0.1

Box B
10.0.0.2
10.0.1.100   # magic IP

You set your DNS (www.example.com) to point to 10.0.1.100. When the IP
is reassigned, the requests go to box B.

> I was hoping we can do all the above simply by using the nginx configuration file.
> I wonder if Igor can comment on that, when he is back from vacation. I would be curious to see if we can do the dual balancer setup, using only native nginx software.

Not possible if you you really mean "failsafe" in the sense that both
boxes are redundant in the event of a total failure -- such as a
kernel panic, hardware crash or power disruption. For such a scenario,
the failure mechanism has to exist *outside* the failing box.

You could add another load-balancer to balance between box A and B,
but that merely moves the single point of failure (SPF) to another
box. It might lower the probability of total failure, but you can't
reasonably keep adding load-balancers to resolve an SPF.

Heartbeat solves the SPF issue by relegating the routing to the
network hardware by using ARP -- the non-failing box uses ARP to tell
the router that it should have the new IP. Thus your router becomes
the SPF, but that's okay because if your router dies, there's no
network connectivity.

You will probably want redundant routers, of course. If you have
redundant routers, your Internet connection becomes a SPF. So you will
want two connections. Then your hosting facility becomes the SPF. :-)

Alexander.