Upstream Consistent Hash Inconsistency

[Ari Croock]

Hi all,

Not sure if this belongs on the devel mailing list, I thought I would try
to post here first.


*Summary*
The "hash" and "hash ... consistent" upstream directives behave differently
when an upstream hostname resolves to multiple IP addresses.


*My Use-case:*

I am using nginx as a reverse proxy for load balancing inside a
docker-compose network.
In docker-compose, you can setup a service with multiple replicas and with
a common hostname. A DNS lookup returns the IP addresses for each replica
(e.g. if I have containers "dash_1" and "dash_2", "dash" could resolve to
the IP addresses for both).

The application we are hosting requires sticky sessions. We have
implemented this using the "hash ... consistent" upstream directive.
So an example upstream could look like:
upstream test_upstream {
hash $token consistent;
server dashboard_test:3838;
}
where "$token" is the unique token for hashing.
Note that "dashboard_test" can resolve to multiple IP addresses as
explained above.

As I understand it, when nginx starts up it will do a DNS lookup for
"dashboard_test" which resolves to multiple IP addresses, effectively
resulting in an upstream group with multiple "servers".


*My Problem:*

It seems that the "hash" and "hash consistent" directives behave
differently.

I had a quick look at the source (ngx_http_upstream_hash_module.c) and it
looks like the "regular" hash method determines an actual IP address from
the hash, but the "chash" *only determines the entry in the upstream group*.

Empirically, this does seem to be the case. With the "hash" directive nginx
always proxies to the same upstream server. With the "hash consistent"
directive I get a different upstream server on every request.

Can anyone comment on whether this is intended behaviour? From my point of
view this seems to be a bug. But I can imagine that changing this behaviour
might break someone else's use case.

Kind regards
-- 
Ari Croock | Isazi | Software Engineer | BSc (Eng) - Electrical

38 Homestead Road, Rivonia, Johannesburg, South Africa, 2128 |(: +27 11 027
6996

*:  acroock at isazi.ai | www.isazi.ai
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20210210/37450956/attachment.htm>