Upstream Consistent Hash Inconsistency

[Maxim Dounin]

Hello!

On Wed, Feb 10, 2021 at 07:03:30PM +0200, Ari Croock wrote:

> That makes sense to me, except for the fact that "hash" (without
> "consistent") doesn't seem to be doing round-robin load balancing.
> 
> Is there a reason that regular "hash" keeps returning a consistent IP? I
> could understand if both directives resulted in the behaviour you
> described, but it seems strange that only one would.

That's because the algorithms used are quite different.

Simple hashing algorithm doesn't care about being consistent, and 
uses the Nth peer, where N is calculated as a hash modulo 
total number of peers.  This works good as long as the list of 
servers is not changed.  If the list is changed, however, new hashing 
will result in completely different peer being used for requests 
with the same hash key.  Note that "the list is changed" also 
implies changes in number or order of IP addresses if you use 
names in the configuration.

Consistent hashing works differently: instead, it tries to 
preserve mapping from hash to a particular upstream server.  To do 
so, it relies on names used in the configuration, so changes in 
the configuration, such as order of servers, or even changes of IP 
addresses used by particular servers, do not affect distribution 
of the requests between servers.  As a result, if a name in the 
configuration maps to more than one IP address, these addresses 
are equal to the algorithm, since they use the same name.

It should be possible to implement consistent hashing differently, 
for example, using IP addresses of the particular peers instead of 
names from the configuration.  This approach will probably match 
what you are trying to do somewhat better.  This is not how it is 
currently implemented in nginx though.

-- 
Maxim Dounin
http://mdounin.ru/