[PATCH] Resolver: relax validation of response flags to allow AD and CD

[Aleksei Bavshin]

A correct nameserver implementation should not be sending any of these flags to a non-security aware client. CD must be copied from a query to the corresponding response, and AD must only be set if all the RRsets are authentic and DO or AD were present in the query. The behavior was allowed in the early DNSSEC standards, though, and the prohibition in the RFC is not strict. The change is, in fact, motivated by a report about some DNS server passing CD in a response to the nginx resolver.

There's a valid concern that CD may mean that the upstream nameserver did not check the response signatures and passed the answer as is. There's nothing we can do with such nameserver from the nginx side, though - we can't prevent it from sending unexpected or unchecked responses. And there's already an error message if an RRSIG record makes it to the answer we receive:
    2022/04/28 17:34:10 [error] 27847#27847: unexpected RR type 46 in DNS response

# HG changeset patch
# User Aleksei Bavshin <a.bavshin at f5.com>
# Date 1651176054 25200
#      Thu Apr 28 13:00:54 2022 -0700
# Node ID 5a570e610b375d1f3442a5b0fc1844be3909d103
# Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
Resolver: relax validation of response flags to allow AD and CD

The check introduced in aebdca7e8f8f was using reserved must-be-zero bits
definition from RFC1035. Later RFCs for DNS Security Extensions (4035 & 6840)
allocated AD and CD from these reserved bits.

It's said that AD and CD SHOULD only appear in the reply for a query from
a security aware resolver, but that is not a strict prohibition, and it was
not included in the early set of DNSSEC RFCs. We may encounter these bits in
a reply when the upstream nameserver implements pre-RFC4035 (with RFC6840
clarifications) standard or uses overly aggressive caching.

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -1746,7 +1746,7 @@ ngx_resolver_process_response(ngx_resolv
                    (response->nar_hi << 8) + response->nar_lo);
 
     /* response to a standard query */
-    if ((flags & 0xf870) != 0x8000 || (trunc && tcp)) {
+    if ((flags & 0xf840) != 0x8000 || (trunc && tcp)) {
         ngx_log_error(r->log_level, r->log, 0,
                       "invalid %s DNS response %ui fl:%04Xi",
                       tcp ? "TCP" : "UDP", ident, flags);