[PATCH 2 of 2] The "sort=" parameter of the "resolver" directive
Sergey Kandaurov
pluknet at nginx.com
Thu Jul 7 15:49:51 UTC 2022
> On 6 Jul 2022, at 05:23, Maxim Dounin <mdounin at mdounin.ru> wrote:
>
> Hello!
>
> On Tue, Jun 28, 2022 at 08:25:36PM +0400, Sergey Kandaurov wrote:
>
>> # HG changeset patch
>> # User Ruslan Ermilov <ru at nginx.com>
>> # Date 1645589387 -10800
>> # Wed Feb 23 07:09:47 2022 +0300
>> # Node ID e80adbf788f6796c6bdf415938abb19b7aa43e3e
>> # Parent 04e314eb6b4d20a48c5d7bab0609e1b03b51b406
>> The "sort=" parameter of the "resolver" directive.
>
> As already noted, should be "prefer=".
>
Fixed, thanks.
>>
>> diff -r 04e314eb6b4d -r e80adbf788f6 src/core/ngx_resolver.c
>> --- a/src/core/ngx_resolver.c Wed Feb 23 07:08:37 2022 +0300
>> +++ b/src/core/ngx_resolver.c Wed Feb 23 07:09:47 2022 +0300
>> @@ -227,6 +227,7 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_
>> }
>>
>> #if (NGX_HAVE_INET6)
>> +
>> if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) {
>>
>> if (ngx_strcmp(&names[i].data[5], "on") == 0) {
>> @@ -260,6 +261,24 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_
>>
>> continue;
>> }
>> +
>> + if (ngx_strncmp(names[i].data, "prefer=", 7) == 0) {
>> +
>> + if (ngx_strcmp(&names[i].data[7], "ipv4") == 0) {
>> + r->prefer = NGX_RESOLVE_PREFER_A;
>> +
>> + } else if (ngx_strcmp(&names[i].data[7], "ipv6") == 0) {
>> + r->prefer = NGX_RESOLVE_PREFER_AAAA;
>> +
>> + } else {
>> + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
>> + "invalid parameter: %V", &names[i]);
>> + return NULL;
>> + }
>> +
>> + continue;
>> + }
>> +
>> #endif
>>
>> ngx_memzero(&u, sizeof(ngx_url_t));
>> @@ -4250,7 +4269,27 @@ ngx_resolver_export(ngx_resolver_t *r, n
>> }
>>
>> i = 0;
>> - d = rotate ? ngx_random() % n : 0;
>> +
>> + switch (r->prefer) {
>> +
>> +#if (NGX_HAVE_INET6)
>> + case NGX_RESOLVE_PREFER_A:
>> + d = 0;
>> + break;
>> +
>> + case NGX_RESOLVE_PREFER_AAAA:
>> + d = rn->naddrs6;
>> +
>> + if (d == n) {
>> + d = 0;
>> + }
>> +
>> + break;
>> +#endif
>> +
>> + default:
>> + d = rotate ? ngx_random() % n : 0;
>> + }
>
> With this code, a configuration like this:
>
> resolver ... prefer=ipv4;
> set $foo "";
> proxy_pass http://example.com$foo;
>
> will result in only IPv4 addresses being used assuming successful
> connections, and IPv6 addresses being used only as a backup. This
> looks quite different from the current behaviour, as well as from
> what we do with
>
> proxy_pass http://example.com;
>
> when using system resolver.
Can you please elaborate, what specific concerns are you referring to?
The prefer option implements exactly the expected behaviour:
first, a flat array is populated with preferred addresses
(IPv4 for "prefer=ipv4", if any), then - with the rest, such as IPv6.
The API user iterates though them until she gets a "successful" address.
If the name is in the resolver cache, then rotation is also applied.
The default nginx resolver behaviour is to rotate resolved addresses
regardless of address families. Unlike this, in case of "prefer=ipv4",
addresses are rotated within address families, that is, AFs are sorted:
ipv4_x, ipv4_y, ipv4_z; ipv6_x, ipv6_y, ipv6_z
This is close to how system resolver is used with getaddrinfo(), which
depends on a preference and, if applicable, AF/address reachability.
In the latter, I refer to AI_ADDRCONFIG and to getaddrinfo() implementation
that uses connect(2)/getsockname(2) to get address family for reordering.
E.g., even if a network interface has IPv6 addresses, they might still
not be respected and will be inserted to the tail, regardless of policy
(FreeBSD's libc hacked a bit for demo purposes):
getaddrinfo:
aio[0] flags=400 family=28 socktype=1 proto=6
connect failed
aio[1] flags=400 family=28 socktype=1 proto=6
connect failed
aio[2] flags=400 family=2 socktype=1 proto=6
srcsa 2
aio[3] flags=400 family=2 socktype=1 proto=6
srcsa 2
qsort(comp_dst)
aio[0] flags=400 family=2 socktype=1 proto=6
aio[1] flags=400 family=2 socktype=1 proto=6
aio[2] flags=400 family=28 socktype=1 proto=6
aio[3] flags=400 family=28 socktype=1 proto=6
ngx_inet_resolve_host:
family 2
family 2
family 28
family 28
But in general (e.g., for "localhost"), this works well:
prefer_ipv4:
family 2
family 28
prefer_ipv6:
family 28
family 2
>
> Not sure we want to introduce such behaviour. While it might be
> closer to what RFC 6724 recommends for clients, it is clearly not
> in line with how we handle multiple upstream addresses in general,
> and certainly will confuse users. If we want to introduce this,
> it probably should be at least consistent within resolver vs.
> system resolver cases.
If you refer to how we balance though multiple addresses in upstream
implicitly defined with proxy_pass vs. proxy_pass with variable, then
I tend to agree with you. In implicitly defined upstream, addresses
are selected with rr balancer, which eventually makes them tried all.
OTOH, the prefer option used for proxy_pass with variable, factually
moves the unprefer addresses to backup, and since the upstream group
isn't preserved across requests, this makes them almost never tried.
But this is how proxy_pass with variable is used to work.
>
>>
>> if (rn->naddrs) {
>> j = rotate ? ngx_random() % rn->naddrs : 0;
>> diff -r 04e314eb6b4d -r e80adbf788f6 src/core/ngx_resolver.h
>> --- a/src/core/ngx_resolver.h Wed Feb 23 07:08:37 2022 +0300
>> +++ b/src/core/ngx_resolver.h Wed Feb 23 07:09:47 2022 +0300
>> @@ -36,6 +36,9 @@
>>
>> #define NGX_RESOLVER_MAX_RECURSION 50
>>
>> +#define NGX_RESOLVE_PREFER_A 1
>> +#define NGX_RESOLVE_PREFER_AAAA 2
>> +
>>
>> typedef struct ngx_resolver_s ngx_resolver_t;
>>
>> @@ -175,6 +178,8 @@ struct ngx_resolver_s {
>> ngx_queue_t srv_expire_queue;
>> ngx_queue_t addr_expire_queue;
>>
>> + unsigned prefer:2;
>> +
>> unsigned ipv4:1;
>>
>> #if (NGX_HAVE_INET6)
>
--
Sergey Kandaurov
More information about the nginx-devel
mailing list