From pluknet at nginx.com  Thu Jun  6 14:32:21 2024
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Thu, 6 Jun 2024 18:32:21 +0400
Subject: [PATCH 2 of 2] Stream: do not reallocate a parsed SNI host
In-Reply-To: <20240530164807.m3sz6xaxuemggsmv@N00W24XTQX>
References: <e82a7318ed48fdbc1273.1716805303@enoparse.local>
 <88fa18a0f05f7dead38a.1716805304@enoparse.local>
 <20240530164807.m3sz6xaxuemggsmv@N00W24XTQX>
Message-ID: <dpbx7pxzloespcm76u2xrjygy2i3q5fq37mmuodzozvz22m436@k35hbysjpmat>

On Thu, May 30, 2024 at 08:48:07PM +0400, Roman Arutyunyan wrote:
> Hi,
> 
> On Mon, May 27, 2024 at 02:21:44PM +0400, Sergey Kandaurov wrote:
> > # HG changeset patch
> > # User Sergey Kandaurov <pluknet at nginx.com>
> > # Date 1716805288 -14400
> > #      Mon May 27 14:21:28 2024 +0400
> > # Node ID 88fa18a0f05f7dead38a127bb24e5cf861f3d66d
> > # Parent  e82a7318ed48fdbc1273771bc96357e9dc232975
> > Stream: do not reallocate a parsed SNI host.
> > 
> > Unlike in http SNI callback, it doesn't need to be saved here.
> > 
> > diff --git a/src/stream/ngx_stream_ssl_module.c b/src/stream/ngx_stream_ssl_module.c
> > --- a/src/stream/ngx_stream_ssl_module.c
> > +++ b/src/stream/ngx_stream_ssl_module.c
> > @@ -501,7 +501,7 @@ ngx_stream_ssl_servername(ngx_ssl_conn_t
> >  
> >      host.data = (u_char *) servername;
> >  
> > -    rc = ngx_stream_validate_host(&host, c->pool, 1);
> > +    rc = ngx_stream_validate_host(&host, c->pool, 0);
> >  
> >      if (rc == NGX_ERROR) {
> >          goto error;
> 
> While it's true the host variable is only used within this function, its
> content can be referenced by session after exiting it.  And that's why we
> should keep the reallocation here.
> 
> ngx_stream_find_virtual_server() calls ngx_stream_regex_exec() for server
> name regex, which can save references to parts of the host in capture variables.
> 

Indeed, this created unreferenced memory access.  Even though the passed
pointer appears to be valid until SSL shutdown, which follows stream log
handler, it's best to avoid.

# HG changeset patch
# User Sergey Kandaurov <pluknet at nginx.com>
# Date 1717681574 -14400
#      Thu Jun 06 17:46:14 2024 +0400
# Node ID 8a7d0ff7b75c664ddef2f985899612ef79643a59
# Parent  89277aea013e649cb2dfcb9621ce06e544ab1c31
Stream ssl_preread: do not reallocate a parsed SNI host.

We own this memory from the session pool.

diff --git a/src/stream/ngx_stream_ssl_preread_module.c b/src/stream/ngx_stream_ssl_preread_module.c
--- a/src/stream/ngx_stream_ssl_preread_module.c
+++ b/src/stream/ngx_stream_ssl_preread_module.c
@@ -519,7 +519,7 @@ ngx_stream_ssl_preread_servername(ngx_st
 
     host = *servername;
 
-    rc = ngx_stream_validate_host(&host, c->pool, 1);
+    rc = ngx_stream_validate_host(&host, c->pool, 0);
 
     if (rc == NGX_ERROR) {
         return NGX_ERROR;

From arut at nginx.com  Fri Jun  7 15:47:41 2024
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Fri, 07 Jun 2024 19:47:41 +0400
Subject: [PATCH 0 of 2] Stream preread fixes
Message-ID: <patchbomb.1717775261@arut-laptop>

Stream preread fixes.

From arut at nginx.com  Fri Jun  7 15:47:42 2024
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Fri, 07 Jun 2024 19:47:42 +0400
Subject: [PATCH 1 of 2] Stream: removed empty input buffer after peek preread
In-Reply-To: <patchbomb.1717775261@arut-laptop>
References: <patchbomb.1717775261@arut-laptop>
Message-ID: <e2f6f5d01ff6f0dd2e3f.1717775262@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1717774411 -14400
#      Fri Jun 07 19:33:31 2024 +0400
# Node ID e2f6f5d01ff6f0dd2e3f0c9328e794af52e65881
# Parent  02e9411009b987f408214ab4a8b6b6093f843bcd
Stream: removed empty input buffer after peek preread.

Since peek preread was introduced in cf890df37bb6 (1.25.5), an empty input
buffer was prepended to client input by ngx_stream_proxy_module after peek
preread, since c->buffer was empty.  An empty c->buffer indicates an empty
datagram for SOCK_DGRAM (see d127837c714f), but has no effect on SOCK_STREAM
connections.  The change eliminates it for SOCK_STREAM.

diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c
--- a/src/stream/ngx_stream_proxy_module.c
+++ b/src/stream/ngx_stream_proxy_module.c
@@ -863,7 +863,9 @@ ngx_stream_proxy_init_upstream(ngx_strea
         u->upstream_buf.last = p;
     }
 
-    if (c->buffer && c->buffer->pos <= c->buffer->last) {
+    if (c->buffer
+        && (c->buffer->pos < c->buffer->last || c->type == SOCK_DGRAM))
+    {
         ngx_log_debug1(NGX_LOG_DEBUG_STREAM, c->log, 0,
                        "stream proxy add preread buffer: %uz",
                        c->buffer->last - c->buffer->pos);

From arut at nginx.com  Fri Jun  7 15:47:43 2024
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Fri, 07 Jun 2024 19:47:43 +0400
Subject: [PATCH 2 of 2] Stream: limit SOCK_DGRAM preread to a single datagram
In-Reply-To: <patchbomb.1717775261@arut-laptop>
References: <patchbomb.1717775261@arut-laptop>
Message-ID: <231701a85ca1943113f3.1717775263@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1717774526 -14400
#      Fri Jun 07 19:35:26 2024 +0400
# Node ID 231701a85ca1943113f3a3cd9174bd9c9dea1b2f
# Parent  e2f6f5d01ff6f0dd2e3f0c9328e794af52e65881
Stream: limit SOCK_DGRAM preread to a single datagram.

Previously, returning NGX_AGAIN from a preread handler for a datagram resulted
in an attempt to read another datagram from the socket.  This attempt could
fail or result in a datagram unrelated to the current client session.  Now
an error is triggered if bytes beyond the first datagram are requested by a
preread handler.

The only preread module available in nginx is ngx_stream_ssl_preread_module,
which does not support SOCK_DGRAM.  However a SOCK_DGRAM preread handler can
be implemented in njs or a third-party module.

diff --git a/src/stream/ngx_stream_core_module.c b/src/stream/ngx_stream_core_module.c
--- a/src/stream/ngx_stream_core_module.c
+++ b/src/stream/ngx_stream_core_module.c
@@ -254,6 +254,12 @@ ngx_stream_core_preread_phase(ngx_stream
         }
     }
 
+    if (c->type == SOCK_DGRAM) {
+        ngx_log_error(NGX_LOG_ERR, c->log, 0, "datagram preread failed");
+        rc = NGX_STREAM_BAD_REQUEST;
+        goto done;
+    }
+
     if (c->buffer == NULL) {
         c->buffer = ngx_create_temp_buf(c->pool, cscf->preread_buffer_size);
         if (c->buffer == NULL) {

From pluknet at nginx.com  Fri Jun  7 16:48:23 2024
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Fri, 7 Jun 2024 20:48:23 +0400
Subject: [PATCH 1 of 2] Rewritten host header validation to follow
 generic parsing rules
In-Reply-To: <LO0P123MB6319FF781672708FBF0C61CD9EF12@LO0P123MB6319.GBRP123.PROD.OUTLOOK.COM>
References: <e82a7318ed48fdbc1273.1716805303@enoparse.local>
 <LO0P123MB6319FF781672708FBF0C61CD9EF12@LO0P123MB6319.GBRP123.PROD.OUTLOOK.COM>
Message-ID: <tfnlaj4oafo7xpqf2gu45gxkbpo5zusdbc55u6b3m32wn6lnbi@w6bkdippzokq>

On Tue, May 28, 2024 at 12:53:46PM +0100, J Carter wrote:
> Hello Sergey,
> 
> On Mon, 27 May 2024 14:21:43 +0400
> Sergey Kandaurov <pluknet at nginx.com> wrote:
> 
> > # HG changeset patch
> > # User Sergey Kandaurov <pluknet at nginx.com>
> > # Date 1716805272 -14400
> > #      Mon May 27 14:21:12 2024 +0400
> > # Node ID e82a7318ed48fdbc1273771bc96357e9dc232975
> > # Parent  f58b6f6362387eeace46043a6fc0bceb56a6786a
> > Rewritten host header validation to follow generic parsing rules.
> > 
> > It now uses a generic model of state-based machine, with more strict
> > parsing rules borrowed from ngx_http_validate_host(),
> 
> I think you mean "borrowed from ngx_http_parse_request_line()".
> 

Sure, tnx.

The problem is that both functions make a subset of parsing
and validation, and these sets just partially intersect.
In particular, ngx_http_parse_request_line() currently detects
invalid characters in a port subcomponent of authority, and
ngx_http_validate_host() handles a trailing dot.
So I think it makes sense to make them unifined, this will also
remove the need to validate host in absolute URIs.
Below is an updated version with both parsers further polished
(stream part is excluded for now).

Also, it may have sense to rename ngx_http_validate_host()
to something like ngx_http_parse_host(), similar to
ngx_http_parse_uri(), out of this series.

# HG changeset patch
# User Sergey Kandaurov <pluknet at nginx.com>
# Date 1717777582 -14400
#      Fri Jun 07 20:26:22 2024 +0400
# Node ID 0cba4301e4980871de7aceb46acddf8f2b5a7318
# Parent  02e9411009b987f408214ab4a8b6b6093f843bcd
Improved parsing of host in absolute URIs.

When the request line is in the absolute-URI form, a host identified
by a registered name (reg-name) is now restricted to start with an
alphanumeric character (see RFC 1123, RFC 3986).  Previously, empty
domain labels or host starting with a hyphen were accepted.

Additionally, host with a trailing dot is taken into account.

diff --git a/src/http/ngx_http_parse.c b/src/http/ngx_http_parse.c
--- a/src/http/ngx_http_parse.c
+++ b/src/http/ngx_http_parse.c
@@ -113,8 +113,10 @@ ngx_http_parse_request_line(ngx_http_req
         sw_schema_slash_slash,
         sw_host_start,
         sw_host,
+        sw_host_dot,
         sw_host_end,
         sw_host_ip_literal,
+        sw_host_ip_literal_dot,
         sw_port,
         sw_after_slash_in_uri,
         sw_check_uri,
@@ -354,27 +356,50 @@ ngx_http_parse_request_line(ngx_http_req
                 break;
             }
 
-            state = sw_host;
+            if (ch == '.' || ch == '-') {
+                return NGX_HTTP_PARSE_INVALID_REQUEST;
+            }
 
             /* fall through */
 
         case sw_host:
+        case sw_host_dot:
+
+            if (ch == '.') {
+                if (state == sw_host_dot) {
+                    return NGX_HTTP_PARSE_INVALID_REQUEST;
+                }
+
+                state = sw_host_dot;
+                break;
+            }
 
             c = (u_char) (ch | 0x20);
             if (c >= 'a' && c <= 'z') {
+                state = sw_host;
+                break;
+            }
+
+            if ((ch >= '0' && ch <= '9') || ch == '-') {
+                state = sw_host;
                 break;
             }
 
-            if ((ch >= '0' && ch <= '9') || ch == '.' || ch == '-') {
-                break;
+            if (state == sw_host_start) {
+                return NGX_HTTP_PARSE_INVALID_REQUEST;
+            }
+
+            if (state == sw_host_dot) {
+                r->host_end = p - 1;
+
+            } else {
+                r->host_end = p;
             }
 
             /* fall through */
 
         case sw_host_end:
 
-            r->host_end = p;
-
             switch (ch) {
             case ':':
                 state = sw_port;
@@ -404,6 +429,18 @@ ngx_http_parse_request_line(ngx_http_req
             break;
 
         case sw_host_ip_literal:
+        case sw_host_ip_literal_dot:
+
+            if (ch == '.') {
+                if (state == sw_host_ip_literal_dot) {
+                    return NGX_HTTP_PARSE_INVALID_REQUEST;
+                }
+
+                state = sw_host_ip_literal_dot;
+                break;
+            }
+
+            state = sw_host_ip_literal;
 
             if (ch >= '0' && ch <= '9') {
                 break;
@@ -418,10 +455,10 @@ ngx_http_parse_request_line(ngx_http_req
             case ':':
                 break;
             case ']':
+                r->host_end = p + 1;
                 state = sw_host_end;
                 break;
             case '-':
-            case '.':
             case '_':
             case '~':
                 /* unreserved */
# HG changeset patch
# User Sergey Kandaurov <pluknet at nginx.com>
# Date 1717777646 -14400
#      Fri Jun 07 20:27:26 2024 +0400
# Node ID 3f7ac1d90a6d4eceabaa5ce45dabf53efd99ed67
# Parent  0cba4301e4980871de7aceb46acddf8f2b5a7318
Rewritten host validation to match host parsing in absolute URIs.

It is reimplemented based on ngx_http_parse_request_line() state machine.
This introduces several changes, in particular:
- host name with underscores is rejected;
- a port subcomponent is restricted to digits;
- for IP literals, a missing closing bracket and trailing dot are detected.

diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c
+++ b/src/http/ngx_http_request.c
@@ -2145,72 +2145,157 @@ ngx_int_t
 ngx_http_validate_host(ngx_str_t *host, ngx_pool_t *pool, ngx_uint_t alloc)
 {
     u_char  *h, ch;
-    size_t   i, dot_pos, host_len;
+    size_t   i, host_len;
 
     enum {
-        sw_usual = 0,
-        sw_literal,
-        sw_rest
+        sw_host_start = 0,
+        sw_host,
+        sw_host_dot,
+        sw_host_end,
+        sw_host_ip_literal,
+        sw_host_ip_literal_dot,
+        sw_port,
     } state;
 
-    dot_pos = host->len;
     host_len = host->len;
 
     h = host->data;
 
-    state = sw_usual;
+    state = sw_host_start;
 
     for (i = 0; i < host->len; i++) {
         ch = h[i];
 
-        switch (ch) {
-
-        case '.':
-            if (dot_pos == i - 1) {
+        switch (state) {
+
+        case sw_host_start:
+
+            if (ch == '[') {
+                state = sw_host_ip_literal;
+                break;
+            }
+
+            if (ch == '.' || ch == '-') {
                 return NGX_DECLINED;
             }
-            dot_pos = i;
-            break;
-
-        case ':':
-            if (state == sw_usual) {
-                host_len = i;
-                state = sw_rest;
-            }
-            break;
-
-        case '[':
-            if (i == 0) {
-                state = sw_literal;
-            }
-            break;
-
-        case ']':
-            if (state == sw_literal) {
-                host_len = i + 1;
-                state = sw_rest;
-            }
-            break;
-
-        default:
-
-            if (ngx_path_separator(ch)) {
-                return NGX_DECLINED;
-            }
-
-            if (ch <= 0x20 || ch == 0x7f) {
-                return NGX_DECLINED;
+
+            /* fall through */
+
+        case sw_host:
+        case sw_host_dot:
+
+            if (ch == '.') {
+                if (state == sw_host_dot) {
+                    return NGX_DECLINED;
+                }
+
+                state = sw_host_dot;
+                break;
             }
 
             if (ch >= 'A' && ch <= 'Z') {
                 alloc = 1;
+                state = sw_host;
+                break;
             }
 
+            if (ch >= 'a' && ch <= 'z') {
+                state = sw_host;
+                break;
+            }
+
+            if ((ch >= '0' && ch <= '9') || ch == '-') {
+                state = sw_host;
+                break;
+            }
+
+            if (state == sw_host_dot) {
+                host_len = i - 1;
+
+            } else {
+                host_len = i;
+            }
+
+            /* fall through */
+
+        case sw_host_end:
+
+            if (ch == ':') {
+                state = sw_port;
+                break;
+            }
+            return NGX_DECLINED;
+
+        case sw_host_ip_literal:
+        case sw_host_ip_literal_dot:
+
+            if (ch == '.') {
+                if (state == sw_host_ip_literal_dot) {
+                    return NGX_DECLINED;
+                }
+
+                state = sw_host_ip_literal_dot;
+                break;
+            }
+
+            state = sw_host_ip_literal;
+
+            if (ch >= 'A' && ch <= 'Z') {
+                alloc = 1;
+                break;
+            }
+
+            if (ch >= 'a' && ch <= 'z') {
+                break;
+            }
+
+            if (ch >= '0' && ch <= '9') {
+                break;
+            }
+
+            switch (ch) {
+            case ':':
+                break;
+            case ']':
+                host_len = i + 1;
+                state = sw_host_end;
+                break;
+            case '-':
+            case '_':
+            case '~':
+                /* unreserved */
+                break;
+            case '!':
+            case '$':
+            case '&':
+            case '\'':
+            case '(':
+            case ')':
+            case '*':
+            case '+':
+            case ',':
+            case ';':
+            case '=':
+                /* sub-delims */
+                break;
+            default:
+                return NGX_DECLINED;
+            }
             break;
+
+        case sw_port:
+            if (ch >= '0' && ch <= '9') {
+                break;
+            }
+            return NGX_DECLINED;
         }
     }
 
-    if (dot_pos == host_len - 1) {
+    if (state == sw_host_ip_literal) {
+        return NGX_DECLINED;
+    }
+
+    if (h[host_len - 1] == '.') {
         host_len--;
     }
 
# HG changeset patch
# User Sergey Kandaurov <pluknet at nginx.com>
# Date 1717777737 -14400
#      Fri Jun 07 20:28:57 2024 +0400
# Node ID 722bcffe3d3c9ff4314a2813227c47dc5eff660e
# Parent  3f7ac1d90a6d4eceabaa5ce45dabf53efd99ed67
Skip host validation in absolute URIs.

Now that parsing of host in the absolute-URI form and of the host header
were made the same in previous changes, it makes no sense to validate
host once again.  Only host case normalization to lowercase is applied
(RFC 3986, 6.2.2.1) after parsing absolute URI as this is out of scope.

No functional changes intended.

diff --git a/src/http/ngx_http_parse.c b/src/http/ngx_http_parse.c
--- a/src/http/ngx_http_parse.c
+++ b/src/http/ngx_http_parse.c
@@ -374,8 +374,13 @@ ngx_http_parse_request_line(ngx_http_req
                 break;
             }
 
-            c = (u_char) (ch | 0x20);
-            if (c >= 'a' && c <= 'z') {
+            if (ch >= 'A' && ch <= 'Z') {
+                r->host_normalize = 1;
+                state = sw_host;
+                break;
+            }
+
+            if (ch >= 'a' && ch <= 'z') {
                 state = sw_host;
                 break;
             }
@@ -446,8 +451,12 @@ ngx_http_parse_request_line(ngx_http_req
                 break;
             }
 
-            c = (u_char) (ch | 0x20);
-            if (c >= 'a' && c <= 'z') {
+            if (ch >= 'A' && ch <= 'Z') {
+                r->host_normalize = 1;
+                break;
+            }
+
+            if (ch >= 'a' && ch <= 'z') {
                 break;
             }
 
diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c
+++ b/src/http/ngx_http_request.c
@@ -1147,18 +1147,15 @@ ngx_http_process_request_line(ngx_event_
                 host.len = r->host_end - r->host_start;
                 host.data = r->host_start;
 
-                rc = ngx_http_validate_host(&host, r->pool, 0);
-
-                if (rc == NGX_DECLINED) {
-                    ngx_log_error(NGX_LOG_INFO, c->log, 0,
-                                  "client sent invalid host in request line");
-                    ngx_http_finalize_request(r, NGX_HTTP_BAD_REQUEST);
-                    break;
-                }
-
-                if (rc == NGX_ERROR) {
-                    ngx_http_close_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
-                    break;
+                if (r->host_normalize) {
+                    host.data = ngx_pnalloc(r->pool, host.len);
+                    if (host.data == NULL) {
+                        ngx_http_close_request(r,
+                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
+                        break;
+                    }
+
+                    ngx_strlow(host.data, r->host_start, host.len);
                 }
 
                 if (ngx_http_set_virtual_server(r, &host) == NGX_ERROR) {
diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -466,6 +466,9 @@ struct ngx_http_request_s {
 
     unsigned                          http_state:4;
 
+    /* host with upper case */
+    unsigned                          host_normalize:1;
+
     /* URI with "/." and on Win32 with "//" */
     unsigned                          complex_uri:1;
 

From jordanc.carter at outlook.com  Sat Jun  8 23:03:09 2024
From: jordanc.carter at outlook.com (=?iso-8859-1?q?J_Carter?=)
Date: Sun, 09 Jun 2024 00:03:09 +0100
Subject: [PATCH] Slice filter: proxy_cache_background_update support (ticket
 #1348)
Message-ID: <LO0P123MB63194601EC85E555B42EE37F9EC42@LO0P123MB6319.GBRP123.PROD.OUTLOOK.COM>

# HG changeset patch
# User J Carter <jordanc.carter at outlook.com>
# Date 1717886685 -3600
#      Sat Jun 08 23:44:45 2024 +0100
# Node ID 1b8a60f7640be4a900ac77d8022b7d8cc6944186
# Parent  02e9411009b987f408214ab4a8b6b6093f843bcd
Slice filter: proxy_cache_background_update support (ticket #1348).

Previously, subrequests of a slice subrequest would have an empty
$slice_range variable value. This prevented
proxy_cache_background_update and friends from successfully
fetching and populating correctly.

This occurred for two reasons:
- Firstly, a single context was reused for all slice subrequests,
where each $slice_range value was overwritten by subsequent slice
subrequests.
- Secondly, subrequests not initiated by slice filter were unable to
access $slice_range in a parent subrequest.

Each slice subrequests now retains $slice_range and subrequests of
slice subrequests now utilize the parent slice subrequest's
$slice_range if available.

diff --git a/src/http/modules/ngx_http_slice_filter_module.c b/src/http/modules/ngx_http_slice_filter_module.c
--- a/src/http/modules/ngx_http_slice_filter_module.c
+++ b/src/http/modules/ngx_http_slice_filter_module.c
@@ -18,11 +18,16 @@ typedef struct {
 typedef struct {
     off_t                start;
     off_t                end;
-    ngx_str_t            range;
     ngx_str_t            etag;
     unsigned             last:1;
     unsigned             active:1;
     ngx_http_request_t  *sr;
+} ngx_http_slice_shctx_t;
+
+
+typedef struct {
+    ngx_str_t range;
+    ngx_http_slice_shctx_t *sh;
 } ngx_http_slice_ctx_t;
 
 
@@ -105,6 +110,7 @@ ngx_http_slice_header_filter(ngx_http_re
     ngx_int_t                        rc;
     ngx_table_elt_t                 *h;
     ngx_http_slice_ctx_t            *ctx;
+    ngx_http_slice_shctx_t          *sh;
     ngx_http_slice_loc_conf_t       *slcf;
     ngx_http_slice_content_range_t   cr;
 
@@ -113,6 +119,8 @@ ngx_http_slice_header_filter(ngx_http_re
         return ngx_http_next_header_filter(r);
     }
 
+    sh = ctx->sh;
+
     if (r->headers_out.status != NGX_HTTP_PARTIAL_CONTENT) {
         if (r == r->main) {
             ngx_http_set_ctx(r, NULL, ngx_http_slice_filter_module);
@@ -127,10 +135,10 @@ ngx_http_slice_header_filter(ngx_http_re
 
     h = r->headers_out.etag;
 
-    if (ctx->etag.len) {
+    if (sh->etag.len) {
         if (h == NULL
-            || h->value.len != ctx->etag.len
-            || ngx_strncmp(h->value.data, ctx->etag.data, ctx->etag.len)
+            || h->value.len != sh->etag.len
+            || ngx_strncmp(h->value.data, sh->etag.data, sh->etag.len)
                != 0)
         {
             ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
@@ -140,7 +148,7 @@ ngx_http_slice_header_filter(ngx_http_re
     }
 
     if (h) {
-        ctx->etag = h->value;
+        sh->etag = h->value;
     }
 
     if (ngx_http_slice_parse_content_range(r, &cr) != NGX_OK) {
@@ -163,15 +171,15 @@ ngx_http_slice_header_filter(ngx_http_re
 
     end = ngx_min(cr.start + (off_t) slcf->size, cr.complete_length);
 
-    if (cr.start != ctx->start || cr.end != end) {
+    if (cr.start != sh->start || cr.end != end) {
         ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                       "unexpected range in slice response: %O-%O",
                       cr.start, cr.end);
         return NGX_ERROR;
     }
 
-    ctx->start = end;
-    ctx->active = 1;
+    sh->start = end;
+    sh->active = 1;
 
     r->headers_out.status = NGX_HTTP_OK;
     r->headers_out.status_line.len = 0;
@@ -198,16 +206,16 @@ ngx_http_slice_header_filter(ngx_http_re
     r->preserve_body = 1;
 
     if (r->headers_out.status == NGX_HTTP_PARTIAL_CONTENT) {
-        if (ctx->start + (off_t) slcf->size <= r->headers_out.content_offset) {
-            ctx->start = slcf->size
+        if (sh->start + (off_t) slcf->size <= r->headers_out.content_offset) {
+            sh->start = slcf->size
                          * (r->headers_out.content_offset / slcf->size);
         }
 
-        ctx->end = r->headers_out.content_offset
+        sh->end = r->headers_out.content_offset
                    + r->headers_out.content_length_n;
 
     } else {
-        ctx->end = cr.complete_length;
+        sh->end = cr.complete_length;
     }
 
     return rc;
@@ -217,9 +225,11 @@ ngx_http_slice_header_filter(ngx_http_re
 static ngx_int_t
 ngx_http_slice_body_filter(ngx_http_request_t *r, ngx_chain_t *in)
 {
+    u_char                     *p;
     ngx_int_t                   rc;
     ngx_chain_t                *cl;
-    ngx_http_slice_ctx_t       *ctx;
+    ngx_http_slice_ctx_t       *ctx, *sr_ctx;
+    ngx_http_slice_shctx_t     *sh;
     ngx_http_slice_loc_conf_t  *slcf;
 
     ctx = ngx_http_get_module_ctx(r, ngx_http_slice_filter_module);
@@ -228,32 +238,34 @@ ngx_http_slice_body_filter(ngx_http_requ
         return ngx_http_next_body_filter(r, in);
     }
 
+    sh = ctx->sh;
+
     for (cl = in; cl; cl = cl->next) {
         if (cl->buf->last_buf) {
             cl->buf->last_buf = 0;
             cl->buf->last_in_chain = 1;
             cl->buf->sync = 1;
-            ctx->last = 1;
+            sh->last = 1;
         }
     }
 
     rc = ngx_http_next_body_filter(r, in);
 
-    if (rc == NGX_ERROR || !ctx->last) {
+    if (rc == NGX_ERROR || !sh->last) {
         return rc;
     }
 
-    if (ctx->sr && !ctx->sr->done) {
+    if (sh->sr && !sh->sr->done) {
         return rc;
     }
 
-    if (!ctx->active) {
+    if (!sh->active) {
         ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                       "missing slice response");
         return NGX_ERROR;
     }
 
-    if (ctx->start >= ctx->end) {
+    if (sh->start >= sh->end) {
         ngx_http_set_ctx(r, NULL, ngx_http_slice_filter_module);
         ngx_http_send_special(r, NGX_HTTP_LAST);
         return rc;
@@ -263,25 +275,36 @@ ngx_http_slice_body_filter(ngx_http_requ
         return rc;
     }
 
-    if (ngx_http_subrequest(r, &r->uri, &r->args, &ctx->sr, NULL,
+    if (ngx_http_subrequest(r, &r->uri, &r->args, &sh->sr, NULL,
                             NGX_HTTP_SUBREQUEST_CLONE)
         != NGX_OK)
     {
         return NGX_ERROR;
     }
 
-    ngx_http_set_ctx(ctx->sr, ctx, ngx_http_slice_filter_module);
+    sr_ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_slice_ctx_t));
+    if (sr_ctx == NULL) {
+        return NGX_ERROR;
+    }
+
+    ngx_http_set_ctx(sh->sr, sr_ctx, ngx_http_slice_filter_module);
 
     slcf = ngx_http_get_module_loc_conf(r, ngx_http_slice_filter_module);
 
-    ctx->range.len = ngx_sprintf(ctx->range.data, "bytes=%O-%O", ctx->start,
-                                 ctx->start + (off_t) slcf->size - 1)
-                     - ctx->range.data;
+    p = ngx_pnalloc(r->pool, sizeof("bytes=-") - 1 + 2 * NGX_OFF_T_LEN);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
 
-    ctx->active = 0;
+    sr_ctx->range.data = p;
+    sr_ctx->range.len = ngx_sprintf(p, "bytes=%O-%O", sh->start,
+                                    sh->start + (off_t) slcf->size - 1) - p;
+
+    sh->active = 0;
+    sr_ctx->sh = ctx->sh;
 
     ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
-                   "http slice subrequest: \"%V\"", &ctx->range);
+                   "http slice subrequest: \"%V\"", &sr_ctx->range);
 
     return rc;
 }
@@ -397,12 +420,25 @@ ngx_http_slice_range_variable(ngx_http_r
 {
     u_char                     *p;
     ngx_http_slice_ctx_t       *ctx;
+    ngx_http_slice_shctx_t     *sh;
     ngx_http_slice_loc_conf_t  *slcf;
 
     ctx = ngx_http_get_module_ctx(r, ngx_http_slice_filter_module);
 
     if (ctx == NULL) {
-        if (r != r->main || r->headers_out.status) {
+        if (r != r->main) {
+            if (r->headers_out.status) {
+                v->not_found = 1;
+                return NGX_OK;
+            }
+
+            /* cache_background_update handling */
+            ctx = ngx_http_get_module_ctx(r->parent,
+                                          ngx_http_slice_filter_module);
+            if (ctx) {
+                goto found;
+            }
+
             v->not_found = 1;
             return NGX_OK;
         }
@@ -421,19 +457,27 @@ ngx_http_slice_range_variable(ngx_http_r
 
         ngx_http_set_ctx(r, ctx, ngx_http_slice_filter_module);
 
+        sh = ngx_pcalloc(r->pool, sizeof(ngx_http_slice_shctx_t));
+        if (sh == NULL) {
+            return NGX_ERROR;
+        }
+
+        ctx->sh = sh;
+
         p = ngx_pnalloc(r->pool, sizeof("bytes=-") - 1 + 2 * NGX_OFF_T_LEN);
         if (p == NULL) {
             return NGX_ERROR;
         }
 
-        ctx->start = slcf->size * (ngx_http_slice_get_start(r) / slcf->size);
+        sh->start = slcf->size * (ngx_http_slice_get_start(r) / slcf->size);
 
         ctx->range.data = p;
-        ctx->range.len = ngx_sprintf(p, "bytes=%O-%O", ctx->start,
-                                     ctx->start + (off_t) slcf->size - 1)
-                         - p;
+        ctx->range.len = ngx_sprintf(p, "bytes=%O-%O", sh->start,
+                                     sh->start + (off_t) slcf->size - 1) - p;
     }
 
+found:
+
     v->data = ctx->range.data;
     v->valid = 1;
     v->not_found = 0;

From noreply at nginx.com  Mon Jun 10 21:50:04 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Mon, 10 Jun 2024 21:50:04 +0000
Subject: [njs] Added reference to nginx community page at SUPPORT.md.
Message-ID: <hg.993e28c8eb84.1718056204.-5540478789349328979@dev.nginx>

details:   https://hg.nginx.org/njs/rev/993e28c8eb84
branches:  
changeset: 2351:993e28c8eb84
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Jun 05 18:23:18 2024 -0700
description:
Added reference to nginx community page at SUPPORT.md.

diffstat:

 SUPPORT.md |  20 +-------------------
 1 files changed, 1 insertions(+), 19 deletions(-)

diffs (30 lines):

diff -r 5e18d7e5f632 -r 993e28c8eb84 SUPPORT.md
--- a/SUPPORT.md	Thu May 30 23:24:30 2024 -0700
+++ b/SUPPORT.md	Wed Jun 05 18:23:18 2024 -0700
@@ -8,25 +8,7 @@ Don't know how something in this project
 
 ## NGINX Specific Questions and/or Issues
 
-This isn't the right place to get support for NGINX specific questions, but the following resources are available below. Thanks for your understanding!
-
-### Community Slack
-
-We have a community [Slack](https://nginxcommunity.slack.com/)!
-
-If you are not a member, click [here](https://community.nginx.org/joinslack) to sign up (and let us know if the link does not seem to be working!)
-
-Once you join, check out the `#beginner-questions` and `nginx-users` channels :)
-
-### Documentation
-
-For a comprehensive list of all NGINX directives, check out <https://nginx.org/en/docs/dirindex.html>.
-
-For a comprehensive list of admin and deployment guides for all NGINX products, check out <https://docs.nginx.com>.
-
-### Mailing List
-
-Want to get in touch with the NGINX development team directly? Try using the relevant mailing list found at <https://mailman.nginx.org/mailman3/lists/>!
+See the [nginx community page](https://nginx.org/en/community.html) for more information on how to get help with NGINX.
 
 ## Contributing
 

From noreply at nginx.com  Mon Jun 10 21:50:06 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Mon, 10 Jun 2024 21:50:06 +0000
Subject: [njs] Fixed open byte overread in decodeURI() and
 decodeURIComponent().
Message-ID: <hg.d67e5b627677.1718056206.-5540478789349328979@dev.nginx>

details:   https://hg.nginx.org/njs/rev/d67e5b627677
branches:  
changeset: 2352:d67e5b627677
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Thu Jun 06 23:10:12 2024 -0700
description:
Fixed open byte overread in decodeURI() and decodeURIComponent().

Found by OSS-Fuzz and MemorySanitizer.

diffstat:

 src/njs_string.c         |  2 +-
 src/test/njs_unit_test.c |  4 ++++
 2 files changed, 5 insertions(+), 1 deletions(-)

diffs (33 lines):

diff -r 993e28c8eb84 -r d67e5b627677 src/njs_string.c
--- a/src/njs_string.c	Wed Jun 05 18:23:18 2024 -0700
+++ b/src/njs_string.c	Thu Jun 06 23:10:12 2024 -0700
@@ -4074,7 +4074,7 @@ njs_string_decode_uri(njs_vm_t *vm, njs_
             n++;
         } while (((cp << n) & 0x80));
 
-        if (njs_slow_path(n > 4)) {
+        if (njs_slow_path(n > 4 || src + njs_length("%00") * (n - 1)  > end)) {
             goto uri_error;
         }
 
diff -r 993e28c8eb84 -r d67e5b627677 src/test/njs_unit_test.c
--- a/src/test/njs_unit_test.c	Wed Jun 05 18:23:18 2024 -0700
+++ b/src/test/njs_unit_test.c	Thu Jun 06 23:10:12 2024 -0700
@@ -10016,13 +10016,17 @@ static njs_unit_test_t  njs_test[] =
               " '%',"
               " '%0',"
               " '%QQ',"
+              " '%C0%' + '0',"
               " '%C0%10',"
+              " '%C0%80',"
               " '%DC%C7',"
               " '%80%81%82',"
               " '%EF%5C%A0',"
               " '%EF%A0%5E',"
+              " '%E0%EF%' + '0',"
               " '%E0%EF%A0',"
               " '%E0%A0%EF',"
+              " '%F0%A2%95%' + '0',"
               " '%FF%A2%95%BB',"
               "].every(v=>{try { decodeURI(v)} catch(e) {return e.name == 'URIError'}})"),
       njs_str("true")},

From noreply at nginx.com  Mon Jun 10 22:15:04 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Mon, 10 Jun 2024 22:15:04 +0000
Subject: [njs] Tests: compatibility with "openssl" app from OpenSSL 3.2.0.
Message-ID: <hg.23538ab89e41.1718057704.-920153629825991941@dev.nginx>

details:   https://hg.nginx.org/njs/rev/23538ab89e41
branches:  
changeset: 2353:23538ab89e41
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Thu Jun 06 14:54:45 2024 -0700
description:
Tests: compatibility with "openssl" app from OpenSSL 3.2.0.

diffstat:

 nginx/t/js_fetch_https.t        |  3 +++
 nginx/t/stream_js_fetch_https.t |  3 +++
 2 files changed, 6 insertions(+), 0 deletions(-)

diffs (28 lines):

diff -r d67e5b627677 -r 23538ab89e41 nginx/t/js_fetch_https.t
--- a/nginx/t/js_fetch_https.t	Thu Jun 06 23:10:12 2024 -0700
+++ b/nginx/t/js_fetch_https.t	Thu Jun 06 14:54:45 2024 -0700
@@ -126,7 +126,10 @@ my $d = $t->testdir();
 default_bits = 2048
 encrypt_key = no
 distinguished_name = req_distinguished_name
+x509_extensions = myca_extensions
 [ req_distinguished_name ]
+[ myca_extensions ]
+basicConstraints = critical,CA:TRUE
 EOF
 
 $t->write_file('myca.conf', <<EOF);
diff -r d67e5b627677 -r 23538ab89e41 nginx/t/stream_js_fetch_https.t
--- a/nginx/t/stream_js_fetch_https.t	Thu Jun 06 23:10:12 2024 -0700
+++ b/nginx/t/stream_js_fetch_https.t	Thu Jun 06 14:54:45 2024 -0700
@@ -203,7 +203,10 @@ my $d = $t->testdir();
 default_bits = 2048
 encrypt_key = no
 distinguished_name = req_distinguished_name
+x509_extensions = myca_extensions
 [ req_distinguished_name ]
+[ myca_extensions ]
+basicConstraints = critical,CA:TRUE
 EOF
 
 $t->write_file('myca.conf', <<EOF);

From noreply at nginx.com  Mon Jun 10 22:21:03 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Mon, 10 Jun 2024 22:21:03 +0000
Subject: [njs] Fixed tracking of argument scope.
Message-ID: <hg.81ff15b57343.1718058063.-4590301072485942426@dev.nginx>

details:   https://hg.nginx.org/njs/rev/81ff15b57343
branches:  
changeset: 2354:81ff15b57343
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Jun 07 21:46:30 2024 -0700
description:
Fixed tracking of argument scope.

This properly catches unsupported "await" in arguments.

This fixes #730 issue on Github.

diffstat:

 src/njs_parser.c         |   6 +++---
 src/njs_parser.h         |   2 +-
 src/test/njs_unit_test.c |  12 ++++++------
 3 files changed, 10 insertions(+), 10 deletions(-)

diffs (71 lines):

diff -r 23538ab89e41 -r 81ff15b57343 src/njs_parser.c
--- a/src/njs_parser.c	Thu Jun 06 14:54:45 2024 -0700
+++ b/src/njs_parser.c	Fri Jun 07 21:46:30 2024 -0700
@@ -2827,7 +2827,7 @@ njs_parser_arguments(njs_parser_t *parse
         return njs_parser_stack_pop(parser);
     }
 
-    parser->scope->in_args = 1;
+    parser->scope->in_args++;
 
     njs_parser_next(parser, njs_parser_argument_list);
 
@@ -2840,7 +2840,7 @@ static njs_int_t
 njs_parser_parenthesis_or_comma(njs_parser_t *parser, njs_lexer_token_t *token,
     njs_queue_link_t *current)
 {
-    parser->scope->in_args = 0;
+    parser->scope->in_args--;
 
     if (token->type == NJS_TOKEN_CLOSE_PARENTHESIS) {
         njs_lexer_consume_token(parser->lexer, 1);
@@ -3575,7 +3575,7 @@ njs_parser_await(njs_parser_t *parser, n
         return NJS_ERROR;
     }
 
-    if (parser->scope->in_args) {
+    if (parser->scope->in_args > 0) {
         njs_parser_syntax_error(parser, "await in arguments not supported");
         return NJS_ERROR;
     }
diff -r 23538ab89e41 -r 81ff15b57343 src/njs_parser.h
--- a/src/njs_parser.h	Thu Jun 06 14:54:45 2024 -0700
+++ b/src/njs_parser.h	Fri Jun 07 21:46:30 2024 -0700
@@ -26,7 +26,7 @@ struct njs_parser_scope_s {
     uint8_t                         arrow_function;
     uint8_t                         dest_disable;
     uint8_t                         async;
-    uint8_t                         in_args;
+    uint32_t                        in_args;
 };
 
 
diff -r 23538ab89e41 -r 81ff15b57343 src/test/njs_unit_test.c
--- a/src/test/njs_unit_test.c	Thu Jun 06 14:54:45 2024 -0700
+++ b/src/test/njs_unit_test.c	Fri Jun 07 21:46:30 2024 -0700
@@ -19938,19 +19938,19 @@ static njs_unit_test_t  njs_test[] =
     { njs_str("(async function() {console.log('Number: ' + await 111)})"),
       njs_str("SyntaxError: await in arguments not supported in 1") },
 
-    { njs_str("function f(a) {}"
-              "(async function() {f(await 111)})"),
+    { njs_str("(async function() {f(await 111)})"),
+      njs_str("SyntaxError: await in arguments not supported in 1") },
+
+    { njs_str("(async function() {f(f(1), await 111)})"),
       njs_str("SyntaxError: await in arguments not supported in 1") },
 
     { njs_str("async () => [await x(1)(),]; async () => [await x(1)()]"),
       njs_str("[object AsyncFunction]") },
 
-    { njs_str("function f(a, b, c) {}"
-              "(async function() {f(1, 'a', await 111)})"),
+    { njs_str("(async function() {f(1, 'a', await 111)})"),
       njs_str("SyntaxError: await in arguments not supported in 1") },
 
-    { njs_str("function f(a) {}"
-              "(async function() {f('Number: ' + await 111)})"),
+    { njs_str("(async function() {f('Number: ' + await 111)})"),
       njs_str("SyntaxError: await in arguments not supported in 1") },
 
     { njs_str("async function f1() {try {f(await f1)} catch(e) {}}"),

From noreply at nginx.com  Mon Jun 10 22:57:04 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Mon, 10 Jun 2024 22:57:04 +0000
Subject: [njs] Fixed integer overflow in Date.parse().
Message-ID: <hg.ae4f50f7b7b3.1718060224.2864227356777792175@dev.nginx>

details:   https://hg.nginx.org/njs/rev/ae4f50f7b7b3
branches:  
changeset: 2355:ae4f50f7b7b3
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Jun 07 22:58:53 2024 -0700
description:
Fixed integer overflow in Date.parse().

Found by OSS-Fuzz and UndefinedSanitizer.

diffstat:

 src/njs_date.c           |  13 ++++++-------
 src/test/njs_unit_test.c |   6 ++++++
 2 files changed, 12 insertions(+), 7 deletions(-)

diffs (47 lines):

diff -r 81ff15b57343 -r ae4f50f7b7b3 src/njs_date.c
--- a/src/njs_date.c	Fri Jun 07 21:46:30 2024 -0700
+++ b/src/njs_date.c	Fri Jun 07 22:58:53 2024 -0700
@@ -676,8 +676,10 @@ njs_date_string_parse(njs_value_t *date)
             }
         }
 
-        p = njs_date_number_parse(&tm[NJS_DATE_MSEC], p, end, ms_length);
-        if (njs_slow_path(p == NULL)) {
+        if (njs_slow_path(njs_date_number_parse(&tm[NJS_DATE_MSEC], p, end,
+                                                njs_min(ms_length, 3))
+                          == NULL))
+        {
             return NAN;
         }
 
@@ -686,12 +688,9 @@ njs_date_string_parse(njs_value_t *date)
 
         } else if (ms_length == 2) {
             tm[NJS_DATE_MSEC] *= 10;
+        }
 
-        } else if (ms_length >= 4) {
-            for (ms_length -= 3; ms_length > 0; ms_length--) {
-                tm[NJS_DATE_MSEC] /= 10;
-            }
-        }
+        p += ms_length;
 
         if (p < end) {
             utc_off = njs_date_utc_offset_parse(p, end);
diff -r 81ff15b57343 -r ae4f50f7b7b3 src/test/njs_unit_test.c
--- a/src/test/njs_unit_test.c	Fri Jun 07 21:46:30 2024 -0700
+++ b/src/test/njs_unit_test.c	Fri Jun 07 22:58:53 2024 -0700
@@ -16285,6 +16285,12 @@ static njs_unit_test_t  njs_test[] =
     { njs_str("Date.parse('2011-06-24T06:01:02.6255555Z')"),
       njs_str("1308895262625") },
 
+    { njs_str("Date.parse('2011-06-24T06:01:02.625555555Z')"),
+      njs_str("1308895262625") },
+
+    { njs_str("Date.parse('2011-06-24T06:01:02.62555555599999Z')"),
+      njs_str("1308895262625") },
+
     { njs_str("Date.parse('2011-06-24T06:01:02.625555Z5')"),
       njs_str("NaN") },
 

From noreply at nginx.com  Mon Jun 10 23:05:04 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Mon, 10 Jun 2024 23:05:04 +0000
Subject: [njs] Modules: shared dictionary add, set,
 incr methods timeout argument.
Message-ID: <hg.a5da85a3d309.1718060704.3534336317638814986@dev.nginx>

details:   https://hg.nginx.org/njs/rev/a5da85a3d309
branches:  
changeset: 2356:a5da85a3d309
user:      jo-carter <104033676+jo-carter at users.noreply.github.com>
date:      Sun Jun 02 09:11:13 2024 +0100
description:
Modules: shared dictionary add, set, incr methods timeout argument.

The optional timeout argument overrides the timeout specified with
the shared_dict_zone directive for the effected key and operation
only. The timeout is specified in milliseconds.

This is useful when the majority of keys are expected to require
unique timeouts.

diffstat:

 nginx/ngx_js_shared_dict.c |  85 ++++++++++++++++++++++++++++++++++++---------
 nginx/t/js_shared_dict.t   |  55 ++++++++++++++++++++++++++---
 2 files changed, 116 insertions(+), 24 deletions(-)

diffs (298 lines):

diff -r ae4f50f7b7b3 -r a5da85a3d309 nginx/ngx_js_shared_dict.c
--- a/nginx/ngx_js_shared_dict.c	Fri Jun 07 22:58:53 2024 -0700
+++ b/nginx/ngx_js_shared_dict.c	Sun Jun 02 09:11:13 2024 +0100
@@ -85,15 +85,16 @@ static ngx_js_dict_node_t *ngx_js_dict_l
 #define NGX_JS_DICT_FLAG_MUST_NOT_EXIST   2
 
 static ngx_int_t ngx_js_dict_set(njs_vm_t *vm, ngx_js_dict_t *dict,
-    njs_str_t *key, njs_value_t *value, unsigned flags);
+    njs_str_t *key, njs_value_t *value, ngx_msec_t timeout, unsigned flags);
 static ngx_int_t ngx_js_dict_add(ngx_js_dict_t *dict, njs_str_t *key,
-    njs_value_t *value, ngx_msec_t now);
+    njs_value_t *value, ngx_msec_t timeout, ngx_msec_t now);
 static ngx_int_t ngx_js_dict_update(ngx_js_dict_t *dict,
-    ngx_js_dict_node_t *node, njs_value_t *value, ngx_msec_t now);
+    ngx_js_dict_node_t *node, njs_value_t *value, ngx_msec_t timeout,
+    ngx_msec_t now);
 static ngx_int_t ngx_js_dict_get(njs_vm_t *vm, ngx_js_dict_t *dict,
     njs_str_t *key, njs_value_t *retval);
 static ngx_int_t ngx_js_dict_incr(ngx_js_dict_t *dict, njs_str_t *key,
-    njs_value_t *delta, njs_value_t *init, double *value);
+    njs_value_t *delta, njs_value_t *init, double *value, ngx_msec_t timeout);
 static ngx_int_t ngx_js_dict_delete(njs_vm_t *vm, ngx_js_dict_t *dict,
     njs_str_t *key, njs_value_t *retval);
 static ngx_int_t ngx_js_dict_copy_value_locked(njs_vm_t *vm,
@@ -726,7 +727,8 @@ njs_js_ext_shared_dict_incr(njs_vm_t *vm
     double               value;
     ngx_int_t            rc;
     njs_str_t            key;
-    njs_value_t         *delta, *init;
+    ngx_msec_t           timeout;
+    njs_value_t         *delta, *init, *timeo;
     ngx_js_dict_t       *dict;
     ngx_shm_zone_t      *shm_zone;
     njs_opaque_value_t   lvalue;
@@ -765,7 +767,30 @@ njs_js_ext_shared_dict_incr(njs_vm_t *vm
         njs_value_number_set(init, 0);
     }
 
-    rc = ngx_js_dict_incr(shm_zone->data, &key, delta, init, &value);
+    timeo = njs_arg(args, nargs, 4);
+    if (!njs_value_is_undefined(timeo)) {
+        if (!njs_value_is_number(timeo)) {
+            njs_vm_type_error(vm, "timeout is not a number");
+            return NJS_ERROR;
+        }
+
+        if (!dict->timeout) {
+            njs_vm_type_error(vm, "shared dict must be declared with timeout");
+            return NJS_ERROR;
+        }
+
+        timeout = (ngx_msec_t) njs_value_number(timeo);
+
+        if (timeout < 1) {
+            njs_vm_type_error(vm, "timeout must be greater than or equal to 1");
+            return NJS_ERROR;
+        }
+
+    } else {
+        timeout = dict->timeout;
+    }
+
+    rc = ngx_js_dict_incr(shm_zone->data, &key, delta, init, &value, timeout);
     if (rc == NGX_ERROR) {
         njs_vm_error(vm, "failed to increment value in shared dict");
         return NJS_ERROR;
@@ -936,7 +961,8 @@ njs_js_ext_shared_dict_set(njs_vm_t *vm,
 {
     njs_str_t        key;
     ngx_int_t        rc;
-    njs_value_t     *value;
+    ngx_msec_t       timeout;
+    njs_value_t     *value, *timeo;
     ngx_js_dict_t   *dict;
     ngx_shm_zone_t  *shm_zone;
 
@@ -967,7 +993,30 @@ njs_js_ext_shared_dict_set(njs_vm_t *vm,
         }
     }
 
-    rc = ngx_js_dict_set(vm, shm_zone->data, &key, value, flags);
+    timeo = njs_arg(args, nargs, 3);
+    if (!njs_value_is_undefined(timeo)) {
+        if (!njs_value_is_number(timeo)) {
+            njs_vm_type_error(vm, "timeout is not a number");
+            return NJS_ERROR;
+        }
+
+        if (!dict->timeout) {
+            njs_vm_type_error(vm, "shared dict must be declared with timeout");
+            return NJS_ERROR;
+        }
+
+        timeout = (ngx_msec_t) njs_value_number(timeo);
+
+        if (timeout < 1) {
+            njs_vm_type_error(vm, "timeout must be greater than or equal to 1");
+            return NJS_ERROR;
+        }
+
+    } else {
+        timeout = dict->timeout;
+    }
+
+    rc = ngx_js_dict_set(vm, shm_zone->data, &key, value, timeout, flags);
     if (rc == NGX_ERROR) {
         return NJS_ERROR;
     }
@@ -1118,7 +1167,7 @@ ngx_js_dict_node_free(ngx_js_dict_t *dic
 
 static ngx_int_t
 ngx_js_dict_set(njs_vm_t *vm, ngx_js_dict_t *dict, njs_str_t *key,
-    njs_value_t *value, unsigned flags)
+    njs_value_t *value, ngx_msec_t timeout, unsigned flags)
 {
     ngx_msec_t           now;
     ngx_time_t          *tp;
@@ -1137,7 +1186,7 @@ ngx_js_dict_set(njs_vm_t *vm, ngx_js_dic
             return NGX_DECLINED;
         }
 
-        if (ngx_js_dict_add(dict, key, value, now) != NGX_OK) {
+        if (ngx_js_dict_add(dict, key, value, timeout, now) != NGX_OK) {
             goto memory_error;
         }
 
@@ -1149,7 +1198,7 @@ ngx_js_dict_set(njs_vm_t *vm, ngx_js_dic
             }
         }
 
-        if (ngx_js_dict_update(dict, node, value, now) != NGX_OK) {
+        if (ngx_js_dict_update(dict, node, value, timeout, now) != NGX_OK) {
             goto memory_error;
         }
     }
@@ -1170,7 +1219,7 @@ memory_error:
 
 static ngx_int_t
 ngx_js_dict_add(ngx_js_dict_t *dict, njs_str_t *key, njs_value_t *value,
-    ngx_msec_t now)
+    ngx_msec_t timeout, ngx_msec_t now)
 {
     size_t               n;
     uint32_t             hash;
@@ -1214,7 +1263,7 @@ ngx_js_dict_add(ngx_js_dict_t *dict, njs
     ngx_rbtree_insert(&dict->sh->rbtree, &node->sn.node);
 
     if (dict->timeout) {
-        node->expire.key = now + dict->timeout;
+        node->expire.key = now + timeout;
         ngx_rbtree_insert(&dict->sh->rbtree_expire, &node->expire);
     }
 
@@ -1224,7 +1273,7 @@ ngx_js_dict_add(ngx_js_dict_t *dict, njs
 
 static ngx_int_t
 ngx_js_dict_update(ngx_js_dict_t *dict, ngx_js_dict_node_t *node,
-    njs_value_t *value, ngx_msec_t now)
+    njs_value_t *value, ngx_msec_t timeout, ngx_msec_t now)
 {
     u_char     *p;
     njs_str_t   string;
@@ -1249,7 +1298,7 @@ ngx_js_dict_update(ngx_js_dict_t *dict, 
 
     if (dict->timeout) {
         ngx_rbtree_delete(&dict->sh->rbtree_expire, &node->expire);
-        node->expire.key = now + dict->timeout;
+        node->expire.key = now + timeout;
         ngx_rbtree_insert(&dict->sh->rbtree_expire, &node->expire);
     }
 
@@ -1306,7 +1355,7 @@ ngx_js_dict_delete(njs_vm_t *vm, ngx_js_
 
 static ngx_int_t
 ngx_js_dict_incr(ngx_js_dict_t *dict, njs_str_t *key, njs_value_t *delta,
-    njs_value_t *init, double *value)
+    njs_value_t *init, double *value, ngx_msec_t timeout)
 {
     ngx_msec_t           now;
     ngx_time_t          *tp;
@@ -1322,7 +1371,7 @@ ngx_js_dict_incr(ngx_js_dict_t *dict, nj
     if (node == NULL) {
         njs_value_number_set(init, njs_value_number(init)
                                    + njs_value_number(delta));
-        if (ngx_js_dict_add(dict, key, init, now) != NGX_OK) {
+        if (ngx_js_dict_add(dict, key, init, timeout, now) != NGX_OK) {
             ngx_rwlock_unlock(&dict->sh->rwlock);
             return NGX_ERROR;
         }
@@ -1335,7 +1384,7 @@ ngx_js_dict_incr(ngx_js_dict_t *dict, nj
 
         if (dict->timeout) {
             ngx_rbtree_delete(&dict->sh->rbtree_expire, &node->expire);
-            node->expire.key = now + dict->timeout;
+            node->expire.key = now + timeout;
             ngx_rbtree_insert(&dict->sh->rbtree_expire, &node->expire);
         }
     }
diff -r ae4f50f7b7b3 -r a5da85a3d309 nginx/t/js_shared_dict.t
--- a/nginx/t/js_shared_dict.t	Fri Jun 07 22:58:53 2024 -0700
+++ b/nginx/t/js_shared_dict.t	Sun Jun 02 09:11:13 2024 +0100
@@ -40,7 +40,7 @@ http {
 
     js_shared_dict_zone zone=foo:32k timeout=2s evict;
     js_shared_dict_zone zone=bar:64k type=string;
-    js_shared_dict_zone zone=waka:32k type=number;
+    js_shared_dict_zone zone=waka:32k timeout=1000s type=number;
     js_shared_dict_zone zone=no_timeout:32k;
 
     server {
@@ -146,7 +146,14 @@ EOF
     function add(r) {
         var dict = ngx.shared[r.args.dict];
         var value = convertToValue(dict, r.args.value);
-        r.return(200, dict.add(r.args.key, value));
+
+        if (r.args.timeout) {
+            var timeout = Number(r.args.timeout);
+            r.return(200, dict.add(r.args.key, value, timeout));
+
+        } else {
+            r.return(200, dict.add(r.args.key, value));
+        }
     }
 
     function capacity(r) {
@@ -200,8 +207,16 @@ EOF
     function incr(r) {
         var dict = ngx.shared[r.args.dict];
         var def = r.args.def ? parseInt(r.args.def) : 0;
-        var val = dict.incr(r.args.key, parseInt(r.args.by), def);
-        r.return(200, val);
+
+        if (r.args.timeout) {
+            var timeout = Number(r.args.timeout);
+            var val = dict.incr(r.args.key, parseInt(r.args.by), def, timeout);
+            r.return(200, val);
+
+        } else {
+            var val = dict.incr(r.args.key, parseInt(r.args.by), def);
+            r.return(200, val);
+        }
     }
 
     function keys(r) {
@@ -256,7 +271,14 @@ EOF
     function set(r) {
         var dict = ngx.shared[r.args.dict];
         var value = convertToValue(dict, r.args.value);
-        r.return(200, dict.set(r.args.key, value) === dict);
+
+        if (r.args.timeout) {
+            var timeout = Number(r.args.timeout);
+            r.return(200, dict.set(r.args.key, value, timeout) === dict);
+
+        } else {
+            r.return(200, dict.set(r.args.key, value) === dict);
+        }
     }
 
     function size(r) {
@@ -283,7 +305,7 @@ EOF
                      set_clear, size, zones };
 EOF
 
-$t->try_run('no js_shared_dict_zone')->plan(44);
+$t->try_run('no js_shared_dict_zone')->plan(51);
 
 ###############################################################################
 
@@ -350,6 +372,27 @@ like(http_get('/items?dict=waka'),
 
 }
 
+TODO: {
+local $TODO = 'not yet' unless has_version('0.8.5');
+
+http_get('/clear?dict=waka');
+like(http_get('/set?dict=waka&key=BAR&value=1&timeout=1'), qr/true/,
+	'set waka.BAR');
+like(http_get('/add?dict=waka&key=BAR2&value=1&timeout=1'), qr/true/,
+	'add waka.BAR2');
+like(http_get('/incr?dict=waka&key=BAR3&by=42&timeout=1'), qr/42/,
+	'incr waka.BAR3');
+like(http_get('/set?dict=waka&key=FOO&value=42&timeout=1000'), qr/true/,
+	'set waka.FOO');
+like(http_get('/add?dict=waka&key=FOO2&value=42&timeout=1000'), qr/true/,
+	'add waka.FOO2');
+like(http_get('/incr?dict=waka&key=FOO3&by=42&timeout=1000'), qr/42/,
+	'incr waka.FOO3');
+
+like(http_get('/keys?dict=waka'), qr/\[FOO\,FOO2\,FOO3]/, 'waka keys');
+
+}
+
 like(http_get('/pop?dict=bar&key=FOO'), qr/zzz/, 'pop bar.FOO');
 like(http_get('/pop?dict=bar&key=FOO'), qr/undefined/, 'pop deleted bar.FOO');
 http_get('/set?dict=foo&key=BAR&value=xxx');

From jordanc.carter at outlook.com  Tue Jun 11 01:46:13 2024
From: jordanc.carter at outlook.com (J Carter)
Date: Tue, 11 Jun 2024 02:46:13 +0100
Subject: [PATCH] Slice filter: proxy_cache_background_update support
 (ticket #1348)
In-Reply-To: <LO0P123MB63194601EC85E555B42EE37F9EC42@LO0P123MB6319.GBRP123.PROD.OUTLOOK.COM>
References: <LO0P123MB63194601EC85E555B42EE37F9EC42@LO0P123MB6319.GBRP123.PROD.OUTLOOK.COM>
Message-ID: <LO0P123MB631942A5583F0DDF390BD2A79EC72@LO0P123MB6319.GBRP123.PROD.OUTLOOK.COM>

Style cleanup of previous patch. 

Also removed 'status already
returned' guard in slice range variable handler, as it is no longer
needed given other changes in patch.

# HG changeset patch
# User J Carter <jordanc.carter at outlook.com>
# Date 1718069043 -3600
#      Tue Jun 11 02:24:03 2024 +0100
# Node ID bc3e20f3f1f6f86da2ad44bb5b4742bced210b97
# Parent  02e9411009b987f408214ab4a8b6b6093f843bcd
Slice filter: proxy_cache_background_update support (ticket #1348).

Previously, subrequests of a slice subrequest would have an empty
$slice_range variable value. This prevented
proxy_cache_background_update and friends from successfully
fetching and populating correctly.

This occurred for two reasons:
- Firstly, a single context was reused for all slice subrequests,
where each $slice_range value was overwritten by subsequent slice
subrequests.
- Secondly, subrequests not initiated by slice filter were unable to
access $slice_range in a parent subrequest.

Each slice subrequests now retains $slice_range and subrequests of
slice subrequests now utilize the parent slice subrequest's
$slice_range if available.

diff --git a/src/http/modules/ngx_http_slice_filter_module.c b/src/http/modules/ngx_http_slice_filter_module.c
--- a/src/http/modules/ngx_http_slice_filter_module.c
+++ b/src/http/modules/ngx_http_slice_filter_module.c
@@ -18,11 +18,16 @@ typedef struct {
 typedef struct {
     off_t                start;
     off_t                end;
-    ngx_str_t            range;
     ngx_str_t            etag;
     unsigned             last:1;
     unsigned             active:1;
     ngx_http_request_t  *sr;
+} ngx_http_slice_shctx_t;
+
+
+typedef struct {
+    ngx_str_t range;
+    ngx_http_slice_shctx_t *sh;
 } ngx_http_slice_ctx_t;
 
 
@@ -105,6 +110,7 @@ ngx_http_slice_header_filter(ngx_http_re
     ngx_int_t                        rc;
     ngx_table_elt_t                 *h;
     ngx_http_slice_ctx_t            *ctx;
+    ngx_http_slice_shctx_t          *sh;
     ngx_http_slice_loc_conf_t       *slcf;
     ngx_http_slice_content_range_t   cr;
 
@@ -113,6 +119,8 @@ ngx_http_slice_header_filter(ngx_http_re
         return ngx_http_next_header_filter(r);
     }
 
+    sh = ctx->sh;
+
     if (r->headers_out.status != NGX_HTTP_PARTIAL_CONTENT) {
         if (r == r->main) {
             ngx_http_set_ctx(r, NULL, ngx_http_slice_filter_module);
@@ -127,10 +135,10 @@ ngx_http_slice_header_filter(ngx_http_re
 
     h = r->headers_out.etag;
 
-    if (ctx->etag.len) {
+    if (sh->etag.len) {
         if (h == NULL
-            || h->value.len != ctx->etag.len
-            || ngx_strncmp(h->value.data, ctx->etag.data, ctx->etag.len)
+            || h->value.len != sh->etag.len
+            || ngx_strncmp(h->value.data, sh->etag.data, sh->etag.len)
                != 0)
         {
             ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
@@ -140,7 +148,7 @@ ngx_http_slice_header_filter(ngx_http_re
     }
 
     if (h) {
-        ctx->etag = h->value;
+        sh->etag = h->value;
     }
 
     if (ngx_http_slice_parse_content_range(r, &cr) != NGX_OK) {
@@ -163,15 +171,15 @@ ngx_http_slice_header_filter(ngx_http_re
 
     end = ngx_min(cr.start + (off_t) slcf->size, cr.complete_length);
 
-    if (cr.start != ctx->start || cr.end != end) {
+    if (cr.start != sh->start || cr.end != end) {
         ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                       "unexpected range in slice response: %O-%O",
                       cr.start, cr.end);
         return NGX_ERROR;
     }
 
-    ctx->start = end;
-    ctx->active = 1;
+    sh->start = end;
+    sh->active = 1;
 
     r->headers_out.status = NGX_HTTP_OK;
     r->headers_out.status_line.len = 0;
@@ -198,16 +206,16 @@ ngx_http_slice_header_filter(ngx_http_re
     r->preserve_body = 1;
 
     if (r->headers_out.status == NGX_HTTP_PARTIAL_CONTENT) {
-        if (ctx->start + (off_t) slcf->size <= r->headers_out.content_offset) {
-            ctx->start = slcf->size
-                         * (r->headers_out.content_offset / slcf->size);
+        if (sh->start + (off_t) slcf->size <= r->headers_out.content_offset) {
+            sh->start = slcf->size
+                        * (r->headers_out.content_offset / slcf->size);
         }
 
-        ctx->end = r->headers_out.content_offset
-                   + r->headers_out.content_length_n;
+        sh->end = r->headers_out.content_offset
+                  + r->headers_out.content_length_n;
 
     } else {
-        ctx->end = cr.complete_length;
+        sh->end = cr.complete_length;
     }
 
     return rc;
@@ -217,9 +225,11 @@ ngx_http_slice_header_filter(ngx_http_re
 static ngx_int_t
 ngx_http_slice_body_filter(ngx_http_request_t *r, ngx_chain_t *in)
 {
+    u_char                     *p;
     ngx_int_t                   rc;
     ngx_chain_t                *cl;
-    ngx_http_slice_ctx_t       *ctx;
+    ngx_http_slice_ctx_t       *ctx, *sr_ctx;
+    ngx_http_slice_shctx_t     *sh;
     ngx_http_slice_loc_conf_t  *slcf;
 
     ctx = ngx_http_get_module_ctx(r, ngx_http_slice_filter_module);
@@ -228,32 +238,34 @@ ngx_http_slice_body_filter(ngx_http_requ
         return ngx_http_next_body_filter(r, in);
     }
 
+    sh = ctx->sh;
+
     for (cl = in; cl; cl = cl->next) {
         if (cl->buf->last_buf) {
             cl->buf->last_buf = 0;
             cl->buf->last_in_chain = 1;
             cl->buf->sync = 1;
-            ctx->last = 1;
+            sh->last = 1;
         }
     }
 
     rc = ngx_http_next_body_filter(r, in);
 
-    if (rc == NGX_ERROR || !ctx->last) {
+    if (rc == NGX_ERROR || !sh->last) {
         return rc;
     }
 
-    if (ctx->sr && !ctx->sr->done) {
+    if (sh->sr && !sh->sr->done) {
         return rc;
     }
 
-    if (!ctx->active) {
+    if (!sh->active) {
         ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                       "missing slice response");
         return NGX_ERROR;
     }
 
-    if (ctx->start >= ctx->end) {
+    if (sh->start >= sh->end) {
         ngx_http_set_ctx(r, NULL, ngx_http_slice_filter_module);
         ngx_http_send_special(r, NGX_HTTP_LAST);
         return rc;
@@ -263,25 +275,36 @@ ngx_http_slice_body_filter(ngx_http_requ
         return rc;
     }
 
-    if (ngx_http_subrequest(r, &r->uri, &r->args, &ctx->sr, NULL,
+    if (ngx_http_subrequest(r, &r->uri, &r->args, &sh->sr, NULL,
                             NGX_HTTP_SUBREQUEST_CLONE)
         != NGX_OK)
     {
         return NGX_ERROR;
     }
 
-    ngx_http_set_ctx(ctx->sr, ctx, ngx_http_slice_filter_module);
+    sr_ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_slice_ctx_t));
+    if (sr_ctx == NULL) {
+        return NGX_ERROR;
+    }
+
+    sr_ctx->sh = ctx->sh;
+
+    ngx_http_set_ctx(sh->sr, sr_ctx, ngx_http_slice_filter_module);
 
     slcf = ngx_http_get_module_loc_conf(r, ngx_http_slice_filter_module);
 
-    ctx->range.len = ngx_sprintf(ctx->range.data, "bytes=%O-%O", ctx->start,
-                                 ctx->start + (off_t) slcf->size - 1)
-                     - ctx->range.data;
+    p = ngx_pnalloc(r->pool, sizeof("bytes=-") - 1 + 2 * NGX_OFF_T_LEN);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
 
-    ctx->active = 0;
+    sh->active = 0;
+    sr_ctx->range.data = p;
+    sr_ctx->range.len = ngx_sprintf(p, "bytes=%O-%O", sh->start,
+                                    sh->start + (off_t) slcf->size - 1) - p;
 
     ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
-                   "http slice subrequest: \"%V\"", &ctx->range);
+                   "http slice subrequest: \"%V\"", &sr_ctx->range);
 
     return rc;
 }
@@ -397,12 +420,20 @@ ngx_http_slice_range_variable(ngx_http_r
 {
     u_char                     *p;
     ngx_http_slice_ctx_t       *ctx;
+    ngx_http_slice_shctx_t     *sh;
     ngx_http_slice_loc_conf_t  *slcf;
 
     ctx = ngx_http_get_module_ctx(r, ngx_http_slice_filter_module);
 
     if (ctx == NULL) {
-        if (r != r->main || r->headers_out.status) {
+        if (r != r->main) {
+            /* cache_background_update handling */
+            ctx = ngx_http_get_module_ctx(r->parent,
+                                          ngx_http_slice_filter_module);
+            if (ctx) {
+                goto found;
+            }
+
             v->not_found = 1;
             return NGX_OK;
         }
@@ -421,19 +452,27 @@ ngx_http_slice_range_variable(ngx_http_r
 
         ngx_http_set_ctx(r, ctx, ngx_http_slice_filter_module);
 
+        sh = ngx_pcalloc(r->pool, sizeof(ngx_http_slice_shctx_t));
+        if (sh == NULL) {
+            return NGX_ERROR;
+        }
+
+        ctx->sh = sh;
+
         p = ngx_pnalloc(r->pool, sizeof("bytes=-") - 1 + 2 * NGX_OFF_T_LEN);
         if (p == NULL) {
             return NGX_ERROR;
         }
 
-        ctx->start = slcf->size * (ngx_http_slice_get_start(r) / slcf->size);
+        sh->start = slcf->size * (ngx_http_slice_get_start(r) / slcf->size);
 
         ctx->range.data = p;
-        ctx->range.len = ngx_sprintf(p, "bytes=%O-%O", ctx->start,
-                                     ctx->start + (off_t) slcf->size - 1)
-                         - p;
+        ctx->range.len = ngx_sprintf(p, "bytes=%O-%O", sh->start,
+                                     sh->start + (off_t) slcf->size - 1) - p;
     }
 
+found:
+
     v->data = ctx->range.data;
     v->valid = 1;
     v->not_found = 0;


From jordanc.carter at outlook.com  Wed Jun 12 21:04:56 2024
From: jordanc.carter at outlook.com (J Carter)
Date: Wed, 12 Jun 2024 22:04:56 +0100
Subject: [PATCH] Slice filter: proxy_cache_background_update support
 (ticket #1348)
In-Reply-To: <LO0P123MB631942A5583F0DDF390BD2A79EC72@LO0P123MB6319.GBRP123.PROD.OUTLOOK.COM>
References: <LO0P123MB63194601EC85E555B42EE37F9EC42@LO0P123MB6319.GBRP123.PROD.OUTLOOK.COM>
 <LO0P123MB631942A5583F0DDF390BD2A79EC72@LO0P123MB6319.GBRP123.PROD.OUTLOOK.COM>
Message-ID: <LO0P123MB63195B9341D8FC2E4CFF4E469EC02@LO0P123MB6319.GBRP123.PROD.OUTLOOK.COM>

On Tue, 11 Jun 2024 02:46:13 +0100
J Carter <jordanc.carter at outlook.com> wrote:

> Style cleanup of previous patch. 
> 
> Also removed 'status already
> returned' guard in slice range variable handler, as it is no longer
> needed given other changes in patch.
>

Additional fixes.

# HG changeset patch
# User J Carter <jordanc.carter at outlook.com>
# Date 1718225771 -3600
#      Wed Jun 12 21:56:11 2024 +0100
# Node ID 8edd891af4d6474ea139490e3662241212926244
# Parent  02e9411009b987f408214ab4a8b6b6093f843bcd
Slice filter: proxy_cache_background_update support (ticket #1348).

Previously, subrequests of a slice subrequest would have an empty
$slice_range variable value. This prevented
proxy_cache_background_update and friends from successfully
fetching and populating correctly.

This occurred for two reasons:
- Firstly, a single context was reused for all slice subrequests,
where each $slice_range value was overwritten by subsequent slice
subrequests.
- Secondly, subrequests not initiated by slice filter were unable to
access $slice_range in a parent subrequest.

Each slice subrequests now retains $slice_range and subrequests of
slice subrequests now utilize the parent slice subrequest's
$slice_range if available.

diff --git a/src/http/modules/ngx_http_slice_filter_module.c b/src/http/modules/ngx_http_slice_filter_module.c
--- a/src/http/modules/ngx_http_slice_filter_module.c
+++ b/src/http/modules/ngx_http_slice_filter_module.c
@@ -18,11 +18,16 @@ typedef struct {
 typedef struct {
     off_t                start;
     off_t                end;
-    ngx_str_t            range;
     ngx_str_t            etag;
     unsigned             last:1;
     unsigned             active:1;
     ngx_http_request_t  *sr;
+} ngx_http_slice_shctx_t;
+
+
+typedef struct {
+    ngx_str_t range;
+    ngx_http_slice_shctx_t *sh;
 } ngx_http_slice_ctx_t;
 
 
@@ -105,6 +110,7 @@ ngx_http_slice_header_filter(ngx_http_re
     ngx_int_t                        rc;
     ngx_table_elt_t                 *h;
     ngx_http_slice_ctx_t            *ctx;
+    ngx_http_slice_shctx_t          *sh;
     ngx_http_slice_loc_conf_t       *slcf;
     ngx_http_slice_content_range_t   cr;
 
@@ -113,6 +119,8 @@ ngx_http_slice_header_filter(ngx_http_re
         return ngx_http_next_header_filter(r);
     }
 
+    sh = ctx->sh;
+
     if (r->headers_out.status != NGX_HTTP_PARTIAL_CONTENT) {
         if (r == r->main) {
             ngx_http_set_ctx(r, NULL, ngx_http_slice_filter_module);
@@ -127,10 +135,10 @@ ngx_http_slice_header_filter(ngx_http_re
 
     h = r->headers_out.etag;
 
-    if (ctx->etag.len) {
+    if (sh->etag.len) {
         if (h == NULL
-            || h->value.len != ctx->etag.len
-            || ngx_strncmp(h->value.data, ctx->etag.data, ctx->etag.len)
+            || h->value.len != sh->etag.len
+            || ngx_strncmp(h->value.data, sh->etag.data, sh->etag.len)
                != 0)
         {
             ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
@@ -140,7 +148,7 @@ ngx_http_slice_header_filter(ngx_http_re
     }
 
     if (h) {
-        ctx->etag = h->value;
+        sh->etag = h->value;
     }
 
     if (ngx_http_slice_parse_content_range(r, &cr) != NGX_OK) {
@@ -163,15 +171,15 @@ ngx_http_slice_header_filter(ngx_http_re
 
     end = ngx_min(cr.start + (off_t) slcf->size, cr.complete_length);
 
-    if (cr.start != ctx->start || cr.end != end) {
+    if (cr.start != sh->start || cr.end != end) {
         ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                       "unexpected range in slice response: %O-%O",
                       cr.start, cr.end);
         return NGX_ERROR;
     }
 
-    ctx->start = end;
-    ctx->active = 1;
+    sh->start = end;
+    sh->active = 1;
 
     r->headers_out.status = NGX_HTTP_OK;
     r->headers_out.status_line.len = 0;
@@ -198,16 +206,16 @@ ngx_http_slice_header_filter(ngx_http_re
     r->preserve_body = 1;
 
     if (r->headers_out.status == NGX_HTTP_PARTIAL_CONTENT) {
-        if (ctx->start + (off_t) slcf->size <= r->headers_out.content_offset) {
-            ctx->start = slcf->size
-                         * (r->headers_out.content_offset / slcf->size);
+        if (sh->start + (off_t) slcf->size <= r->headers_out.content_offset) {
+            sh->start = slcf->size
+                        * (r->headers_out.content_offset / slcf->size);
         }
 
-        ctx->end = r->headers_out.content_offset
-                   + r->headers_out.content_length_n;
+        sh->end = r->headers_out.content_offset
+                  + r->headers_out.content_length_n;
 
     } else {
-        ctx->end = cr.complete_length;
+        sh->end = cr.complete_length;
     }
 
     return rc;
@@ -217,9 +225,11 @@ ngx_http_slice_header_filter(ngx_http_re
 static ngx_int_t
 ngx_http_slice_body_filter(ngx_http_request_t *r, ngx_chain_t *in)
 {
+    u_char                     *p;
     ngx_int_t                   rc;
     ngx_chain_t                *cl;
-    ngx_http_slice_ctx_t       *ctx;
+    ngx_http_slice_ctx_t       *ctx, *sr_ctx;
+    ngx_http_slice_shctx_t     *sh;
     ngx_http_slice_loc_conf_t  *slcf;
 
     ctx = ngx_http_get_module_ctx(r, ngx_http_slice_filter_module);
@@ -228,32 +238,34 @@ ngx_http_slice_body_filter(ngx_http_requ
         return ngx_http_next_body_filter(r, in);
     }
 
+    sh = ctx->sh;
+
     for (cl = in; cl; cl = cl->next) {
         if (cl->buf->last_buf) {
             cl->buf->last_buf = 0;
             cl->buf->last_in_chain = 1;
             cl->buf->sync = 1;
-            ctx->last = 1;
+            sh->last = 1;
         }
     }
 
     rc = ngx_http_next_body_filter(r, in);
 
-    if (rc == NGX_ERROR || !ctx->last) {
+    if (rc == NGX_ERROR || !sh->last) {
         return rc;
     }
 
-    if (ctx->sr && !ctx->sr->done) {
+    if (sh->sr && !sh->sr->done) {
         return rc;
     }
 
-    if (!ctx->active) {
+    if (!sh->active) {
         ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                       "missing slice response");
         return NGX_ERROR;
     }
 
-    if (ctx->start >= ctx->end) {
+    if (sh->start >= sh->end) {
         ngx_http_set_ctx(r, NULL, ngx_http_slice_filter_module);
         ngx_http_send_special(r, NGX_HTTP_LAST);
         return rc;
@@ -263,25 +275,36 @@ ngx_http_slice_body_filter(ngx_http_requ
         return rc;
     }
 
-    if (ngx_http_subrequest(r, &r->uri, &r->args, &ctx->sr, NULL,
+    if (ngx_http_subrequest(r, &r->uri, &r->args, &sh->sr, NULL,
                             NGX_HTTP_SUBREQUEST_CLONE)
         != NGX_OK)
     {
         return NGX_ERROR;
     }
 
-    ngx_http_set_ctx(ctx->sr, ctx, ngx_http_slice_filter_module);
+    sr_ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_slice_ctx_t));
+    if (sr_ctx == NULL) {
+        return NGX_ERROR;
+    }
+
+    sr_ctx->sh = ctx->sh;
+
+    ngx_http_set_ctx(sh->sr, sr_ctx, ngx_http_slice_filter_module);
 
     slcf = ngx_http_get_module_loc_conf(r, ngx_http_slice_filter_module);
 
-    ctx->range.len = ngx_sprintf(ctx->range.data, "bytes=%O-%O", ctx->start,
-                                 ctx->start + (off_t) slcf->size - 1)
-                     - ctx->range.data;
+    p = ngx_pnalloc(r->pool, sizeof("bytes=-") - 1 + 2 * NGX_OFF_T_LEN);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
 
-    ctx->active = 0;
+    sh->active = 0;
+    sr_ctx->range.data = p;
+    sr_ctx->range.len = ngx_sprintf(p, "bytes=%O-%O", sh->start,
+                                    sh->start + (off_t) slcf->size - 1) - p;
 
     ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
-                   "http slice subrequest: \"%V\"", &ctx->range);
+                   "http slice subrequest: \"%V\"", &sr_ctx->range);
 
     return rc;
 }
@@ -397,12 +420,25 @@ ngx_http_slice_range_variable(ngx_http_r
 {
     u_char                     *p;
     ngx_http_slice_ctx_t       *ctx;
+    ngx_http_slice_shctx_t     *sh;
     ngx_http_slice_loc_conf_t  *slcf;
 
     ctx = ngx_http_get_module_ctx(r, ngx_http_slice_filter_module);
 
     if (ctx == NULL) {
-        if (r != r->main || r->headers_out.status) {
+        if (r->headers_out.status) {
+            v->not_found = 1;
+            return NGX_OK;
+        }
+
+        if (r != r->main) {
+            /* cache_background_update handling */
+            ctx = ngx_http_get_module_ctx(r->parent,
+                                          ngx_http_slice_filter_module);
+            if (ctx) {
+                goto found;
+            }
+
             v->not_found = 1;
             return NGX_OK;
         }
@@ -419,6 +455,13 @@ ngx_http_slice_range_variable(ngx_http_r
             return NGX_ERROR;
         }
 
+        sh = ngx_pcalloc(r->pool, sizeof(ngx_http_slice_shctx_t));
+        if (sh == NULL) {
+            return NGX_ERROR;
+        }
+
+        ctx->sh = sh;
+
         ngx_http_set_ctx(r, ctx, ngx_http_slice_filter_module);
 
         p = ngx_pnalloc(r->pool, sizeof("bytes=-") - 1 + 2 * NGX_OFF_T_LEN);
@@ -426,14 +469,15 @@ ngx_http_slice_range_variable(ngx_http_r
             return NGX_ERROR;
         }
 
-        ctx->start = slcf->size * (ngx_http_slice_get_start(r) / slcf->size);
+        sh->start = slcf->size * (ngx_http_slice_get_start(r) / slcf->size);
 
         ctx->range.data = p;
-        ctx->range.len = ngx_sprintf(p, "bytes=%O-%O", ctx->start,
-                                     ctx->start + (off_t) slcf->size - 1)
-                         - p;
+        ctx->range.len = ngx_sprintf(p, "bytes=%O-%O", sh->start,
+                                     sh->start + (off_t) slcf->size - 1) - p;
     }
 
+found:
+
     v->data = ctx->range.data;
     v->valid = 1;
     v->not_found = 0;

From noreply at nginx.com  Wed Jun 12 21:54:04 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Wed, 12 Jun 2024 21:54:04 +0000
Subject: [njs] Modules: removed extra copying in shared dictionary.
Message-ID: <hg.95bb22b30a0d.1718229244.-7388357313336745782@dev.nginx>

details:   https://hg.nginx.org/njs/rev/95bb22b30a0d
branches:  
changeset: 2357:95bb22b30a0d
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Jun 10 23:06:26 2024 -0700
description:
Modules: removed extra copying in shared dictionary.

 After 5730d5f pool copy is not needed because
 njs_vm_value_string_create() always create a copy.

diffstat:

 nginx/ngx_js_shared_dict.c |  14 ++------------
 1 files changed, 2 insertions(+), 12 deletions(-)

diffs (29 lines):

diff -r a5da85a3d309 -r 95bb22b30a0d nginx/ngx_js_shared_dict.c
--- a/nginx/ngx_js_shared_dict.c	Sun Jun 02 09:11:13 2024 +0100
+++ b/nginx/ngx_js_shared_dict.c	Mon Jun 10 23:06:26 2024 -0700
@@ -1440,23 +1440,13 @@ ngx_js_dict_copy_value_locked(njs_vm_t *
     ngx_js_dict_node_t *node, njs_value_t *retval)
 {
     njs_int_t   ret;
-    njs_str_t   string;
     ngx_uint_t  type;
-    ngx_pool_t  *pool;
 
     type = dict->type;
 
     if (type == NGX_JS_DICT_TYPE_STRING) {
-        pool = ngx_external_pool(vm, njs_vm_external_ptr(vm));
-
-        string.length = node->u.value.len;
-        string.start = ngx_pstrdup(pool, &node->u.value);
-        if (string.start == NULL) {
-            return NGX_ERROR;
-        }
-
-        ret = njs_vm_value_string_create(vm, retval, string.start,
-                                         string.length);
+        ret = njs_vm_value_string_create(vm, retval, node->u.value.data,
+                                         node->u.value.len);
         if (ret != NJS_OK) {
             return NGX_ERROR;
         }

From noreply at nginx.com  Wed Jun 12 22:10:03 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Wed, 12 Jun 2024 22:10:03 +0000
Subject: [njs] HTTP: fixed r.subrequest() error handling.
Message-ID: <hg.3ba1433803ee.1718230203.3813808961216002220@dev.nginx>

details:   https://hg.nginx.org/njs/rev/3ba1433803ee
branches:  
changeset: 2358:3ba1433803ee
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Thu May 30 22:22:48 2024 -0700
description:
HTTP: fixed r.subrequest() error handling.

Previously, when at least 2 subrequests were scheduled they both
succeed, but the callback for the second threw an exception
heap-use-after-free happened: a nested chain of
ngx_http_run_posted_requests() calls and terminating request in the
inner call left outer calls with already freed request pointer.

The issue was introduced in 0.8.1 (4cb8e873e8c6).

diffstat:

 nginx/ngx_http_js_module.c |  31 ++++++++++++++++++++-----------
 nginx/t/js_subrequests.t   |  19 ++++++++++++++++++-
 2 files changed, 38 insertions(+), 12 deletions(-)

diffs (129 lines):

diff -r 95bb22b30a0d -r 3ba1433803ee nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c	Mon Jun 10 23:06:26 2024 -0700
+++ b/nginx/ngx_http_js_module.c	Thu May 30 22:22:48 2024 -0700
@@ -276,8 +276,8 @@ static void ngx_http_js_event_finalize(n
 static ngx_js_ctx_t *ngx_http_js_ctx(njs_vm_t *vm, ngx_http_request_t *r);
 
 static void ngx_http_js_periodic_handler(ngx_event_t *ev);
-static void ngx_http_js_periodic_write_event_handler(ngx_http_request_t *r);
 static void ngx_http_js_periodic_shutdown_handler(ngx_event_t *ev);
+static void ngx_http_js_periodic_write_handler(ngx_event_t *ev);
 static void ngx_http_js_periodic_finalize(ngx_http_request_t *r, ngx_int_t rc);
 static void ngx_http_js_periodic_destroy(ngx_http_request_t *r,
     ngx_js_periodic_t *periodic);
@@ -4220,7 +4220,10 @@ ngx_http_js_periodic_handler(ngx_event_t
     c->data = r;
     c->destroyed = 0;
     c->pool = r->pool;
+    c->read->log = &periodic->log;
     c->read->handler = ngx_http_js_periodic_shutdown_handler;
+    c->write->log = &periodic->log;
+    c->write->handler = ngx_http_js_periodic_write_handler;
 
     periodic->connection = c;
     periodic->log_ctx.request = r;
@@ -4234,7 +4237,6 @@ ngx_http_js_periodic_handler(ngx_event_t
     r->valid_unparsed_uri = 1;
 
     r->health_check = 1;
-    r->write_event_handler = ngx_http_js_periodic_write_event_handler;
 
     rc = ngx_http_js_init_vm(r, ngx_http_js_periodic_session_proto_id);
 
@@ -4263,12 +4265,17 @@ ngx_http_js_periodic_handler(ngx_event_t
 
 
 static void
-ngx_http_js_periodic_write_event_handler(ngx_http_request_t *r)
+ngx_http_js_periodic_write_handler(ngx_event_t *ev)
 {
-    ngx_http_js_ctx_t  *ctx;
-
-    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
-                   "http js periodic write event handler");
+    ngx_connection_t    *c;
+    ngx_http_js_ctx_t   *ctx;
+    ngx_http_request_t  *r;
+
+    c = ev->data;
+    r = c->data;
+
+    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0,
+                   "http js periodic write handler");
 
     ctx = ngx_http_get_module_ctx(r, ngx_http_js_module);
 
@@ -4340,6 +4347,10 @@ ngx_http_js_periodic_destroy(ngx_http_re
     c->fd = (ngx_socket_t) -1;
     c->pool = NULL;
     c->destroyed = 1;
+
+    if (c->write->posted) {
+        ngx_delete_posted_event(c->write);
+    }
 }
 
 
@@ -4451,10 +4462,8 @@ ngx_http_js_event_finalize(ngx_http_requ
     }
 
     if (rc == NGX_OK) {
-        ngx_http_post_request(r, NULL);
-    }
-
-    ngx_http_run_posted_requests(r->connection);
+        ngx_post_event(r->connection->write, &ngx_posted_events);
+    }
 }
 
 
diff -r 95bb22b30a0d -r 3ba1433803ee nginx/t/js_subrequests.t
--- a/nginx/t/js_subrequests.t	Mon Jun 10 23:06:26 2024 -0700
+++ b/nginx/t/js_subrequests.t	Thu May 30 22:22:48 2024 -0700
@@ -180,6 +180,10 @@ http {
             js_content test.sr_in_sr_callback;
         }
 
+        location /sr_error_in_callback {
+            js_content test.sr_error_in_callback;
+        }
+
         location /sr_uri_except {
             js_content test.sr_uri_except;
         }
@@ -417,6 +421,12 @@ EOF
         .then(body_fwd_cb);
     }
 
+    function sr_error_in_callback(r) {
+        r.subrequest("/sub1", () => {});
+        r.subrequest("/sub1", () => { throw "Oops!"; });
+        r.return(200);
+    }
+
     function sr_in_sr_callback(r) {
         r.subrequest('/return', function (reply) {
                 try {
@@ -508,7 +518,7 @@ EOF
                     sr_js_in_subrequest, sr_js_in_subrequest_pr, js_sub,
                     sr_in_sr_callback, sr_out_of_order, sr_except_not_a_func,
                     sr_uri_except, sr_except_failed_to_convert_options_arg,
-                    sr_unsafe};
+                    sr_unsafe, sr_error_in_callback};
 
 EOF
 
@@ -575,6 +585,13 @@ like(http_get('/sr_unsafe'), qr/500/s, '
 
 }
 
+TODO: {
+local $TODO = 'not yet' unless has_version('0.8.5');
+
+http_get('/sr_error_in_callback');
+
+}
+
 $t->stop();
 
 ok(index($t->read_file('error.log'), 'callback is not a function') > 0,

From a.bavshin at nginx.com  Thu Jun 13 22:28:55 2024
From: a.bavshin at nginx.com (=?iso-8859-1?q?Aleksei_Bavshin?=)
Date: Thu, 13 Jun 2024 15:28:55 -0700
Subject: [PATCH 0 of 9] Upstream: re-resolvable servers.
Message-ID: <patchbomb.1718317735@fedora-wsl.local>

The series is a compilation of patches with the upstream re-resolve
feature from the Nginx Plus.  The original commits were rebased on top
of the current OSS code, grouped by features introduced and squashed.
Some formatting quirks and other minor oddities could be attributed to a
conscious effort to reduce divergence with the source branch.


This is a resubmission of the series from Feb 2023[1]. Main differences:

- The code is now identical to the NGINX Plus (sans features we're not
  cleared to publish at the moment and minor cosmetic differences).

  The implementation might be not perfect architecturally, but it's
  been heavily tested and has been running in many production
  environments for many years.  It's also already familiar to the team,
  simplifying the review, and serves as a good starting point for
  further improvements.

- The configuration directives are now explicitly disabled on Windows
  platform.  See the corresponding commit for the rationale.

- Tests are now ported to use the features available in the NGINX OSS
  and added to the series.


Known limitations:

- All the periodic resolve tasks are handled in the first worker
  process.

- The functionality requires a shared zone of a sufficient size to be
  configured in the upstream block.  A rough estimation is 2k for a
  configured server entry + 2k for each resolved address.
  The zone requirement could be lifted with local allocation of the
  resolved peer data, but implementing that was out of scope.

[1]: https://mailman.nginx.org/pipermail/nginx-devel/2023-February/4MCLSVRK7EX6DNKHFZN6CA4SKZUSA3GA.html

From a.bavshin at nginx.com  Thu Jun 13 22:28:56 2024
From: a.bavshin at nginx.com (=?iso-8859-1?q?Aleksei_Bavshin?=)
Date: Thu, 13 Jun 2024 15:28:56 -0700
Subject: [PATCH 1 of 9] Upstream: re-resolvable servers
In-Reply-To: <patchbomb.1718317735@fedora-wsl.local>
References: <patchbomb.1718317735@fedora-wsl.local>
Message-ID: <56aeae9355df8a2ee07e.1718317736@fedora-wsl.local>

# HG changeset patch
# User Ruslan Ermilov <ru at nginx.com>
# Date 1392462754 -14400
#      Sat Feb 15 15:12:34 2014 +0400
# Node ID 56aeae9355df8a2ee07e21b65b6869747dd9ee45
# Parent  02e9411009b987f408214ab4a8b6b6093f843bcd
Upstream: re-resolvable servers.

Specifying the upstream server by a hostname together with the
"resolve" parameter will make the hostname to be periodically
resolved, and upstream servers added/removed as necessary.

This requires a "resolver" at the "http" configuration block.

The "resolver_timeout" parameter also affects when the failed
DNS requests will be attempted again.  Responses with NXDOMAIN
will be attempted again in 10 seconds.

Upstream has a configuration generation number that is incremented each
time servers are added/removed to the primary/backup list.  This number
is remembered by the peer.init method, and if peer.get detects a change
in configuration, it returns NGX_BUSY.

Each server has a reference counter.  It is incremented by peer.get and
decremented by peer.free.  When a server is removed, it is removed from
the list of servers and is marked as "zombie".  The memory allocated by
a zombie peer is freed only when its reference count becomes zero.

Re-resolvable servers utilize timers that also hold a reference.  A
reference is also held while upstream keepalive caches an idle
connection.

Co-authored-by: Roman Arutyunyan <arut at nginx.com>
Co-authored-by: Sergey Kandaurov <pluknet at nginx.com>
Co-authored-by: Vladimir Homutov <vl at nginx.com>

diff --git a/src/http/modules/ngx_http_upstream_hash_module.c b/src/http/modules/ngx_http_upstream_hash_module.c
--- a/src/http/modules/ngx_http_upstream_hash_module.c
+++ b/src/http/modules/ngx_http_upstream_hash_module.c
@@ -24,6 +24,9 @@ typedef struct {
 
 typedef struct {
     ngx_http_complex_value_t            key;
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    ngx_uint_t                          config;
+#endif
     ngx_http_upstream_chash_points_t   *points;
 } ngx_http_upstream_hash_srv_conf_t;
 
@@ -49,6 +52,8 @@ static ngx_int_t ngx_http_upstream_get_h
 
 static ngx_int_t ngx_http_upstream_init_chash(ngx_conf_t *cf,
     ngx_http_upstream_srv_conf_t *us);
+static ngx_int_t ngx_http_upstream_update_chash(ngx_pool_t *pool,
+    ngx_http_upstream_srv_conf_t *us);
 static int ngx_libc_cdecl
     ngx_http_upstream_chash_cmp_points(const void *one, const void *two);
 static ngx_uint_t ngx_http_upstream_find_chash_point(
@@ -178,11 +183,18 @@ ngx_http_upstream_get_hash_peer(ngx_peer
 
     ngx_http_upstream_rr_peers_rlock(hp->rrp.peers);
 
-    if (hp->tries > 20 || hp->rrp.peers->single || hp->key.len == 0) {
+    if (hp->tries > 20 || hp->rrp.peers->number < 2 || hp->key.len == 0) {
         ngx_http_upstream_rr_peers_unlock(hp->rrp.peers);
         return hp->get_rr_peer(pc, &hp->rrp);
     }
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    if (hp->rrp.peers->config && hp->rrp.config != *hp->rrp.peers->config) {
+        ngx_http_upstream_rr_peers_unlock(hp->rrp.peers);
+        return hp->get_rr_peer(pc, &hp->rrp);
+    }
+#endif
+
     now = ngx_time();
 
     pc->cached = 0;
@@ -262,6 +274,7 @@ ngx_http_upstream_get_hash_peer(ngx_peer
     }
 
     hp->rrp.current = peer;
+    ngx_http_upstream_rr_peer_ref(hp->rrp.peers, peer);
 
     pc->sockaddr = peer->sockaddr;
     pc->socklen = peer->socklen;
@@ -285,6 +298,26 @@ ngx_http_upstream_get_hash_peer(ngx_peer
 static ngx_int_t
 ngx_http_upstream_init_chash(ngx_conf_t *cf, ngx_http_upstream_srv_conf_t *us)
 {
+    if (ngx_http_upstream_init_round_robin(cf, us) != NGX_OK) {
+        return NGX_ERROR;
+    }
+
+    us->peer.init = ngx_http_upstream_init_chash_peer;
+
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    if (us->shm_zone) {
+        return NGX_OK;
+    }
+#endif
+
+    return ngx_http_upstream_update_chash(cf->pool, us);
+}
+
+
+static ngx_int_t
+ngx_http_upstream_update_chash(ngx_pool_t *pool,
+    ngx_http_upstream_srv_conf_t *us)
+{
     u_char                             *host, *port, c;
     size_t                              host_len, port_len, size;
     uint32_t                            hash, base_hash;
@@ -299,25 +332,32 @@ ngx_http_upstream_init_chash(ngx_conf_t 
         u_char                          byte[4];
     } prev_hash;
 
-    if (ngx_http_upstream_init_round_robin(cf, us) != NGX_OK) {
-        return NGX_ERROR;
+    hcf = ngx_http_conf_upstream_srv_conf(us, ngx_http_upstream_hash_module);
+
+    if (hcf->points) {
+        ngx_free(hcf->points);
+        hcf->points = NULL;
     }
 
-    us->peer.init = ngx_http_upstream_init_chash_peer;
-
     peers = us->peer.data;
     npoints = peers->total_weight * 160;
 
     size = sizeof(ngx_http_upstream_chash_points_t)
-           + sizeof(ngx_http_upstream_chash_point_t) * (npoints - 1);
+           - sizeof(ngx_http_upstream_chash_point_t)
+           + sizeof(ngx_http_upstream_chash_point_t) * npoints;
 
-    points = ngx_palloc(cf->pool, size);
+    points = pool ? ngx_palloc(pool, size) : ngx_alloc(size, ngx_cycle->log);
     if (points == NULL) {
         return NGX_ERROR;
     }
 
     points->number = 0;
 
+    if (npoints == 0) {
+        hcf->points = points;
+        return NGX_OK;
+    }
+
     for (peer = peers->peer; peer; peer = peer->next) {
         server = &peer->server;
 
@@ -401,7 +441,6 @@ ngx_http_upstream_init_chash(ngx_conf_t 
 
     points->number = i + 1;
 
-    hcf = ngx_http_conf_upstream_srv_conf(us, ngx_http_upstream_hash_module);
     hcf->points = points;
 
     return NGX_OK;
@@ -481,7 +520,22 @@ ngx_http_upstream_init_chash_peer(ngx_ht
 
     ngx_http_upstream_rr_peers_rlock(hp->rrp.peers);
 
-    hp->hash = ngx_http_upstream_find_chash_point(hcf->points, hash);
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    if (hp->rrp.peers->config
+        && (hcf->points == NULL || hcf->config != *hp->rrp.peers->config))
+    {
+        if (ngx_http_upstream_update_chash(NULL, us) != NGX_OK) {
+            ngx_http_upstream_rr_peers_unlock(hp->rrp.peers);
+            return NGX_ERROR;
+        }
+
+        hcf->config = *hp->rrp.peers->config;
+    }
+#endif
+
+    if (hcf->points->number) {
+        hp->hash = ngx_http_upstream_find_chash_point(hcf->points, hash);
+    }
 
     ngx_http_upstream_rr_peers_unlock(hp->rrp.peers);
 
@@ -517,6 +571,20 @@ ngx_http_upstream_get_chash_peer(ngx_pee
     pc->cached = 0;
     pc->connection = NULL;
 
+    if (hp->rrp.peers->number == 0) {
+        pc->name = hp->rrp.peers->name;
+        ngx_http_upstream_rr_peers_unlock(hp->rrp.peers);
+        return NGX_BUSY;
+    }
+
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    if (hp->rrp.peers->config && hp->rrp.config != *hp->rrp.peers->config) {
+        pc->name = hp->rrp.peers->name;
+        ngx_http_upstream_rr_peers_unlock(hp->rrp.peers);
+        return NGX_BUSY;
+    }
+#endif
+
     now = ngx_time();
     hcf = hp->conf;
 
@@ -597,6 +665,7 @@ ngx_http_upstream_get_chash_peer(ngx_pee
 found:
 
     hp->rrp.current = best;
+    ngx_http_upstream_rr_peer_ref(hp->rrp.peers, best);
 
     pc->sockaddr = best->sockaddr;
     pc->socklen = best->socklen;
@@ -664,6 +733,7 @@ ngx_http_upstream_hash(ngx_conf_t *cf, n
     }
 
     uscf->flags = NGX_HTTP_UPSTREAM_CREATE
+                  |NGX_HTTP_UPSTREAM_MODIFY
                   |NGX_HTTP_UPSTREAM_WEIGHT
                   |NGX_HTTP_UPSTREAM_MAX_CONNS
                   |NGX_HTTP_UPSTREAM_MAX_FAILS
diff --git a/src/http/modules/ngx_http_upstream_ip_hash_module.c b/src/http/modules/ngx_http_upstream_ip_hash_module.c
--- a/src/http/modules/ngx_http_upstream_ip_hash_module.c
+++ b/src/http/modules/ngx_http_upstream_ip_hash_module.c
@@ -163,11 +163,19 @@ ngx_http_upstream_get_ip_hash_peer(ngx_p
 
     ngx_http_upstream_rr_peers_rlock(iphp->rrp.peers);
 
-    if (iphp->tries > 20 || iphp->rrp.peers->single) {
+    if (iphp->tries > 20 || iphp->rrp.peers->number < 2) {
         ngx_http_upstream_rr_peers_unlock(iphp->rrp.peers);
         return iphp->get_rr_peer(pc, &iphp->rrp);
     }
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    if (iphp->rrp.peers->config && iphp->rrp.config != *iphp->rrp.peers->config)
+    {
+        ngx_http_upstream_rr_peers_unlock(iphp->rrp.peers);
+        return iphp->get_rr_peer(pc, &iphp->rrp);
+    }
+#endif
+
     now = ngx_time();
 
     pc->cached = 0;
@@ -232,6 +240,7 @@ ngx_http_upstream_get_ip_hash_peer(ngx_p
     }
 
     iphp->rrp.current = peer;
+    ngx_http_upstream_rr_peer_ref(iphp->rrp.peers, peer);
 
     pc->sockaddr = peer->sockaddr;
     pc->socklen = peer->socklen;
@@ -268,6 +277,7 @@ ngx_http_upstream_ip_hash(ngx_conf_t *cf
     uscf->peer.init_upstream = ngx_http_upstream_init_ip_hash;
 
     uscf->flags = NGX_HTTP_UPSTREAM_CREATE
+                  |NGX_HTTP_UPSTREAM_MODIFY
                   |NGX_HTTP_UPSTREAM_WEIGHT
                   |NGX_HTTP_UPSTREAM_MAX_CONNS
                   |NGX_HTTP_UPSTREAM_MAX_FAILS
diff --git a/src/http/modules/ngx_http_upstream_least_conn_module.c b/src/http/modules/ngx_http_upstream_least_conn_module.c
--- a/src/http/modules/ngx_http_upstream_least_conn_module.c
+++ b/src/http/modules/ngx_http_upstream_least_conn_module.c
@@ -124,6 +124,12 @@ ngx_http_upstream_get_least_conn_peer(ng
 
     ngx_http_upstream_rr_peers_wlock(peers);
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    if (peers->config && rrp->config != *peers->config) {
+        goto busy;
+    }
+#endif
+
     best = NULL;
     total = 0;
 
@@ -244,6 +250,7 @@ ngx_http_upstream_get_least_conn_peer(ng
     best->conns++;
 
     rrp->current = best;
+    ngx_http_upstream_rr_peer_ref(peers, best);
 
     n = p / (8 * sizeof(uintptr_t));
     m = (uintptr_t) 1 << p % (8 * sizeof(uintptr_t));
@@ -278,8 +285,18 @@ failed:
         }
 
         ngx_http_upstream_rr_peers_wlock(peers);
+
+#if (NGX_HTTP_UPSTREAM_ZONE)
+        if (peers->config && rrp->config != *peers->config) {
+            goto busy;
+        }
+#endif
     }
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+busy:
+#endif
+
     ngx_http_upstream_rr_peers_unlock(peers);
 
     pc->name = peers->name;
@@ -303,6 +320,7 @@ ngx_http_upstream_least_conn(ngx_conf_t 
     uscf->peer.init_upstream = ngx_http_upstream_init_least_conn;
 
     uscf->flags = NGX_HTTP_UPSTREAM_CREATE
+                  |NGX_HTTP_UPSTREAM_MODIFY
                   |NGX_HTTP_UPSTREAM_WEIGHT
                   |NGX_HTTP_UPSTREAM_MAX_CONNS
                   |NGX_HTTP_UPSTREAM_MAX_FAILS
diff --git a/src/http/modules/ngx_http_upstream_random_module.c b/src/http/modules/ngx_http_upstream_random_module.c
--- a/src/http/modules/ngx_http_upstream_random_module.c
+++ b/src/http/modules/ngx_http_upstream_random_module.c
@@ -17,6 +17,9 @@ typedef struct {
 
 typedef struct {
     ngx_uint_t                            two;
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    ngx_uint_t                            config;
+#endif
     ngx_http_upstream_random_range_t     *ranges;
 } ngx_http_upstream_random_srv_conf_t;
 
@@ -127,6 +130,11 @@ ngx_http_upstream_update_random(ngx_pool
 
     rcf = ngx_http_conf_upstream_srv_conf(us, ngx_http_upstream_random_module);
 
+    if (rcf->ranges) {
+        ngx_free(rcf->ranges);
+        rcf->ranges = NULL;
+    }
+
     peers = us->peer.data;
 
     size = peers->number * sizeof(ngx_http_upstream_random_range_t);
@@ -186,11 +194,15 @@ ngx_http_upstream_init_random_peer(ngx_h
     ngx_http_upstream_rr_peers_rlock(rp->rrp.peers);
 
 #if (NGX_HTTP_UPSTREAM_ZONE)
-    if (rp->rrp.peers->shpool && rcf->ranges == NULL) {
+    if (rp->rrp.peers->config
+        && (rcf->ranges == NULL || rcf->config != *rp->rrp.peers->config))
+    {
         if (ngx_http_upstream_update_random(NULL, us) != NGX_OK) {
             ngx_http_upstream_rr_peers_unlock(rp->rrp.peers);
             return NGX_ERROR;
         }
+
+        rcf->config = *rp->rrp.peers->config;
     }
 #endif
 
@@ -220,11 +232,18 @@ ngx_http_upstream_get_random_peer(ngx_pe
 
     ngx_http_upstream_rr_peers_rlock(peers);
 
-    if (rp->tries > 20 || peers->single) {
+    if (rp->tries > 20 || peers->number < 2) {
         ngx_http_upstream_rr_peers_unlock(peers);
         return ngx_http_upstream_get_round_robin_peer(pc, rrp);
     }
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    if (peers->config && rrp->config != *peers->config) {
+        ngx_http_upstream_rr_peers_unlock(peers);
+        return ngx_http_upstream_get_round_robin_peer(pc, rrp);
+    }
+#endif
+
     pc->cached = 0;
     pc->connection = NULL;
 
@@ -274,6 +293,7 @@ ngx_http_upstream_get_random_peer(ngx_pe
     }
 
     rrp->current = peer;
+    ngx_http_upstream_rr_peer_ref(peers, peer);
 
     if (now - peer->checked > peer->fail_timeout) {
         peer->checked = now;
@@ -314,11 +334,18 @@ ngx_http_upstream_get_random2_peer(ngx_p
 
     ngx_http_upstream_rr_peers_wlock(peers);
 
-    if (rp->tries > 20 || peers->single) {
+    if (rp->tries > 20 || peers->number < 2) {
         ngx_http_upstream_rr_peers_unlock(peers);
         return ngx_http_upstream_get_round_robin_peer(pc, rrp);
     }
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    if (peers->config && rrp->config != *peers->config) {
+        ngx_http_upstream_rr_peers_unlock(peers);
+        return ngx_http_upstream_get_round_robin_peer(pc, rrp);
+    }
+#endif
+
     pc->cached = 0;
     pc->connection = NULL;
 
@@ -384,6 +411,7 @@ ngx_http_upstream_get_random2_peer(ngx_p
     }
 
     rrp->current = peer;
+    ngx_http_upstream_rr_peer_ref(peers, peer);
 
     if (now - peer->checked > peer->fail_timeout) {
         peer->checked = now;
@@ -467,6 +495,7 @@ ngx_http_upstream_random(ngx_conf_t *cf,
     uscf->peer.init_upstream = ngx_http_upstream_init_random;
 
     uscf->flags = NGX_HTTP_UPSTREAM_CREATE
+                  |NGX_HTTP_UPSTREAM_MODIFY
                   |NGX_HTTP_UPSTREAM_WEIGHT
                   |NGX_HTTP_UPSTREAM_MAX_CONNS
                   |NGX_HTTP_UPSTREAM_MAX_FAILS
diff --git a/src/http/modules/ngx_http_upstream_zone_module.c b/src/http/modules/ngx_http_upstream_zone_module.c
--- a/src/http/modules/ngx_http_upstream_zone_module.c
+++ b/src/http/modules/ngx_http_upstream_zone_module.c
@@ -18,6 +18,10 @@ static ngx_http_upstream_rr_peers_t *ngx
     ngx_slab_pool_t *shpool, ngx_http_upstream_srv_conf_t *uscf);
 static ngx_http_upstream_rr_peer_t *ngx_http_upstream_zone_copy_peer(
     ngx_http_upstream_rr_peers_t *peers, ngx_http_upstream_rr_peer_t *src);
+static void ngx_http_upstream_zone_set_single(
+    ngx_http_upstream_srv_conf_t *uscf);
+static void ngx_http_upstream_zone_remove_peer_locked(
+    ngx_http_upstream_rr_peers_t *peers, ngx_http_upstream_rr_peer_t *peer);
 
 
 static ngx_command_t  ngx_http_upstream_zone_commands[] = {
@@ -33,6 +37,11 @@ static ngx_command_t  ngx_http_upstream_
 };
 
 
+static ngx_int_t ngx_http_upstream_zone_init_worker(ngx_cycle_t *cycle);
+static void ngx_http_upstream_zone_resolve_timer(ngx_event_t *event);
+static void ngx_http_upstream_zone_resolve_handler(ngx_resolver_ctx_t *ctx);
+
+
 static ngx_http_module_t  ngx_http_upstream_zone_module_ctx = {
     NULL,                                  /* preconfiguration */
     NULL,                                  /* postconfiguration */
@@ -55,7 +64,7 @@ ngx_module_t  ngx_http_upstream_zone_mod
     NGX_HTTP_MODULE,                       /* module type */
     NULL,                                  /* init master */
     NULL,                                  /* init module */
-    NULL,                                  /* init process */
+    ngx_http_upstream_zone_init_worker,    /* init process */
     NULL,                                  /* init thread */
     NULL,                                  /* exit thread */
     NULL,                                  /* exit process */
@@ -188,9 +197,15 @@ ngx_http_upstream_zone_copy_peers(ngx_sl
     ngx_http_upstream_srv_conf_t *uscf)
 {
     ngx_str_t                     *name;
+    ngx_uint_t                    *config;
     ngx_http_upstream_rr_peer_t   *peer, **peerp;
     ngx_http_upstream_rr_peers_t  *peers, *backup;
 
+    config = ngx_slab_calloc(shpool, sizeof(ngx_uint_t));
+    if (config == NULL) {
+        return NULL;
+    }
+
     peers = ngx_slab_alloc(shpool, sizeof(ngx_http_upstream_rr_peers_t));
     if (peers == NULL) {
         return NULL;
@@ -214,6 +229,7 @@ ngx_http_upstream_zone_copy_peers(ngx_sl
     peers->name = name;
 
     peers->shpool = shpool;
+    peers->config = config;
 
     for (peerp = &peers->peer; *peerp; peerp = &peer->next) {
         /* pool is unlocked */
@@ -223,6 +239,17 @@ ngx_http_upstream_zone_copy_peers(ngx_sl
         }
 
         *peerp = peer;
+        peer->id = (*peers->config)++;
+    }
+
+    for (peerp = &peers->resolve; *peerp; peerp = &peer->next) {
+        peer = ngx_http_upstream_zone_copy_peer(peers, *peerp);
+        if (peer == NULL) {
+            return NULL;
+        }
+
+        *peerp = peer;
+        peer->id = (*peers->config)++;
     }
 
     if (peers->next == NULL) {
@@ -239,6 +266,7 @@ ngx_http_upstream_zone_copy_peers(ngx_sl
     backup->name = name;
 
     backup->shpool = shpool;
+    backup->config = config;
 
     for (peerp = &backup->peer; *peerp; peerp = &peer->next) {
         /* pool is unlocked */
@@ -248,6 +276,17 @@ ngx_http_upstream_zone_copy_peers(ngx_sl
         }
 
         *peerp = peer;
+        peer->id = (*backup->config)++;
+    }
+
+    for (peerp = &backup->resolve; *peerp; peerp = &peer->next) {
+        peer = ngx_http_upstream_zone_copy_peer(backup, *peerp);
+        if (peer == NULL) {
+            return NULL;
+        }
+
+        *peerp = peer;
+        peer->id = (*backup->config)++;
     }
 
     peers->next = backup;
@@ -279,6 +318,7 @@ ngx_http_upstream_zone_copy_peer(ngx_htt
         dst->sockaddr = NULL;
         dst->name.data = NULL;
         dst->server.data = NULL;
+        dst->host = NULL;
     }
 
     dst->sockaddr = ngx_slab_calloc_locked(pool, sizeof(ngx_sockaddr_t));
@@ -301,12 +341,37 @@ ngx_http_upstream_zone_copy_peer(ngx_htt
         }
 
         ngx_memcpy(dst->server.data, src->server.data, src->server.len);
+
+        if (src->host) {
+            dst->host = ngx_slab_calloc_locked(pool,
+                                             sizeof(ngx_http_upstream_host_t));
+            if (dst->host == NULL) {
+                goto failed;
+            }
+
+            dst->host->name.data = ngx_slab_alloc_locked(pool,
+                                                         src->host->name.len);
+            if (dst->host->name.data == NULL) {
+                goto failed;
+            }
+
+            dst->host->peers = peers;
+            dst->host->peer = dst;
+
+            dst->host->name.len = src->host->name.len;
+            ngx_memcpy(dst->host->name.data, src->host->name.data,
+                       src->host->name.len);
+        }
     }
 
     return dst;
 
 failed:
 
+    if (dst->host) {
+        ngx_slab_free_locked(pool, dst->host);
+    }
+
     if (dst->server.data) {
         ngx_slab_free_locked(pool, dst->server.data);
     }
@@ -323,3 +388,337 @@ failed:
 
     return NULL;
 }
+
+
+static void
+ngx_http_upstream_zone_set_single(ngx_http_upstream_srv_conf_t *uscf)
+{
+    ngx_http_upstream_rr_peers_t  *peers;
+
+    peers = uscf->peer.data;
+
+    if (peers->number == 1
+        && (peers->next == NULL || peers->next->number == 0))
+    {
+        peers->single = 1;
+
+    } else {
+        peers->single = 0;
+    }
+}
+
+
+static void
+ngx_http_upstream_zone_remove_peer_locked(ngx_http_upstream_rr_peers_t *peers,
+    ngx_http_upstream_rr_peer_t *peer)
+{
+    peers->total_weight -= peer->weight;
+    peers->number--;
+    peers->tries -= (peer->down == 0);
+    (*peers->config)++;
+    peers->weighted = (peers->total_weight != peers->number);
+
+    ngx_http_upstream_rr_peer_free(peers, peer);
+}
+
+
+static ngx_int_t
+ngx_http_upstream_zone_init_worker(ngx_cycle_t *cycle)
+{
+    ngx_uint_t                      i;
+    ngx_event_t                    *event;
+    ngx_http_upstream_rr_peer_t    *peer;
+    ngx_http_upstream_rr_peers_t   *peers;
+    ngx_http_upstream_srv_conf_t   *uscf, **uscfp;
+    ngx_http_upstream_main_conf_t  *umcf;
+
+    if (ngx_process != NGX_PROCESS_WORKER
+        && ngx_process != NGX_PROCESS_SINGLE)
+    {
+        return NGX_OK;
+    }
+
+    umcf = ngx_http_cycle_get_module_main_conf(cycle, ngx_http_upstream_module);
+
+    if (umcf == NULL) {
+        return NGX_OK;
+    }
+
+    uscfp = umcf->upstreams.elts;
+
+    for (i = 0; i < umcf->upstreams.nelts; i++) {
+
+        uscf = uscfp[i];
+
+        if (uscf->shm_zone == NULL) {
+            continue;
+        }
+
+        peers = uscf->peer.data;
+
+        do {
+            ngx_http_upstream_rr_peers_wlock(peers);
+
+            for (peer = peers->resolve; peer; peer = peer->next) {
+
+                if (peer->host->worker != ngx_worker) {
+                    continue;
+                }
+
+                event = &peer->host->event;
+                ngx_memzero(event, sizeof(ngx_event_t));
+
+                event->data = uscf;
+                event->handler = ngx_http_upstream_zone_resolve_timer;
+                event->log = cycle->log;
+                event->cancelable = 1;
+
+                ngx_http_upstream_rr_peer_ref(peers, peer);
+                ngx_add_timer(event, 1);
+            }
+
+            ngx_http_upstream_rr_peers_unlock(peers);
+
+            peers = peers->next;
+
+        } while (peers);
+    }
+
+    return NGX_OK;
+}
+
+
+static void
+ngx_http_upstream_zone_resolve_timer(ngx_event_t *event)
+{
+    ngx_resolver_ctx_t            *ctx;
+    ngx_http_upstream_host_t      *host;
+    ngx_http_upstream_rr_peer_t   *template;
+    ngx_http_upstream_rr_peers_t  *peers;
+    ngx_http_upstream_srv_conf_t  *uscf;
+
+    host = (ngx_http_upstream_host_t *) event;
+    uscf = event->data;
+    peers = host->peers;
+    template = host->peer;
+
+    if (template->zombie) {
+        (void) ngx_http_upstream_rr_peer_unref(peers, template);
+
+        ngx_shmtx_lock(&peers->shpool->mutex);
+
+        if (host->service.len) {
+            ngx_slab_free_locked(peers->shpool, host->service.data);
+        }
+
+        ngx_slab_free_locked(peers->shpool, host->name.data);
+        ngx_slab_free_locked(peers->shpool, host);
+        ngx_shmtx_unlock(&peers->shpool->mutex);
+
+        return;
+    }
+
+    ctx = ngx_resolve_start(uscf->resolver, NULL);
+    if (ctx == NULL) {
+        goto retry;
+    }
+
+    if (ctx == NGX_NO_RESOLVER) {
+        ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                      "no resolver defined to resolve %V", &host->name);
+        return;
+    }
+
+    ctx->name = host->name;
+    ctx->handler = ngx_http_upstream_zone_resolve_handler;
+    ctx->data = host;
+    ctx->timeout = uscf->resolver_timeout;
+    ctx->cancelable = 1;
+
+    if (ngx_resolve_name(ctx) == NGX_OK) {
+        return;
+    }
+
+retry:
+
+    ngx_add_timer(event, ngx_max(uscf->resolver_timeout, 1000));
+}
+
+
+static void
+ngx_http_upstream_zone_resolve_handler(ngx_resolver_ctx_t *ctx)
+{
+    time_t                         now;
+    in_port_t                      port;
+    ngx_msec_t                     timer;
+    ngx_uint_t                     i, j;
+    ngx_event_t                   *event;
+    ngx_resolver_addr_t           *addr;
+    ngx_http_upstream_host_t      *host;
+    ngx_http_upstream_rr_peer_t   *peer, *template, **peerp;
+    ngx_http_upstream_rr_peers_t  *peers;
+    ngx_http_upstream_srv_conf_t  *uscf;
+
+    host = ctx->data;
+    event = &host->event;
+    uscf = event->data;
+    peers = host->peers;
+    template = host->peer;
+
+    ngx_http_upstream_rr_peers_wlock(peers);
+
+    if (template->zombie) {
+        (void) ngx_http_upstream_rr_peer_unref(peers, template);
+
+        ngx_http_upstream_rr_peers_unlock(peers);
+
+        ngx_shmtx_lock(&peers->shpool->mutex);
+        ngx_slab_free_locked(peers->shpool, host->name.data);
+        ngx_slab_free_locked(peers->shpool, host);
+        ngx_shmtx_unlock(&peers->shpool->mutex);
+
+        ngx_resolve_name_done(ctx);
+
+        return;
+    }
+
+    now = ngx_time();
+
+    if (ctx->state) {
+        ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                      "%V could not be resolved (%i: %s)",
+                      &ctx->name, ctx->state,
+                      ngx_resolver_strerror(ctx->state));
+
+        if (ctx->state != NGX_RESOLVE_NXDOMAIN) {
+            ngx_http_upstream_rr_peers_unlock(peers);
+
+            ngx_resolve_name_done(ctx);
+
+            ngx_add_timer(event, ngx_max(uscf->resolver_timeout, 1000));
+            return;
+        }
+
+        /* NGX_RESOLVE_NXDOMAIN */
+
+        ctx->naddrs = 0;
+    }
+
+#if (NGX_DEBUG)
+    {
+    u_char  text[NGX_SOCKADDR_STRLEN];
+    size_t  len;
+
+    for (i = 0; i < ctx->naddrs; i++) {
+        len = ngx_sock_ntop(ctx->addrs[i].sockaddr, ctx->addrs[i].socklen,
+                            text, NGX_SOCKADDR_STRLEN, 0);
+
+        ngx_log_debug3(NGX_LOG_DEBUG_HTTP, event->log, 0,
+                       "name %V was resolved to %*s", &host->name, len, text);
+    }
+    }
+#endif
+
+    for (peerp = &peers->peer; *peerp; /* void */ ) {
+        peer = *peerp;
+
+        if (peer->host != host) {
+            goto next;
+        }
+
+        for (j = 0; j < ctx->naddrs; j++) {
+
+            addr = &ctx->addrs[j];
+
+            if (addr->name.len == 0
+                && ngx_cmp_sockaddr(peer->sockaddr, peer->socklen,
+                                    addr->sockaddr, addr->socklen, 0)
+                   == NGX_OK)
+            {
+                addr->name.len = 1;
+                goto next;
+            }
+        }
+
+        *peerp = peer->next;
+        ngx_http_upstream_zone_remove_peer_locked(peers, peer);
+
+        ngx_http_upstream_zone_set_single(uscf);
+
+        continue;
+
+    next:
+
+        peerp = &peer->next;
+    }
+
+    for (i = 0; i < ctx->naddrs; i++) {
+
+        addr = &ctx->addrs[i];
+
+        if (addr->name.len == 1) {
+            addr->name.len = 0;
+            continue;
+        }
+
+        ngx_shmtx_lock(&peers->shpool->mutex);
+        peer = ngx_http_upstream_zone_copy_peer(peers, NULL);
+        ngx_shmtx_unlock(&peers->shpool->mutex);
+
+        if (peer == NULL) {
+            ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                          "cannot add new server to upstream \"%V\", "
+                          "memory exhausted", peers->name);
+            break;
+        }
+
+        ngx_memcpy(peer->sockaddr, addr->sockaddr, addr->socklen);
+
+        port = ((struct sockaddr_in *) template->sockaddr)->sin_port;
+
+        switch (peer->sockaddr->sa_family) {
+#if (NGX_HAVE_INET6)
+        case AF_INET6:
+            ((struct sockaddr_in6 *) peer->sockaddr)->sin6_port = port;
+            break;
+#endif
+        default: /* AF_INET */
+            ((struct sockaddr_in *) peer->sockaddr)->sin_port = port;
+        }
+
+        peer->socklen = addr->socklen;
+
+        peer->name.len = ngx_sock_ntop(peer->sockaddr, peer->socklen,
+                                       peer->name.data, NGX_SOCKADDR_STRLEN, 1);
+
+        peer->host = template->host;
+        peer->server = template->server;
+
+        peer->weight = template->weight;
+        peer->effective_weight = peer->weight;
+        peer->max_conns = template->max_conns;
+        peer->max_fails = template->max_fails;
+        peer->fail_timeout = template->fail_timeout;
+        peer->down = template->down;
+
+        *peerp = peer;
+        peerp = &peer->next;
+
+        peers->number++;
+        peers->tries += (peer->down == 0);
+        peers->total_weight += peer->weight;
+        peers->weighted = (peers->total_weight != peers->number);
+        peer->id = (*peers->config)++;
+
+        ngx_http_upstream_zone_set_single(uscf);
+    }
+
+    ngx_http_upstream_rr_peers_unlock(peers);
+
+    timer = (ngx_msec_t) 1000 * (ctx->valid > now ? ctx->valid - now + 1 : 1);
+    timer = ngx_min(timer, uscf->resolver_timeout);
+
+    ngx_resolve_name_done(ctx);
+
+    ngx_add_timer(event, timer);
+}
diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -1565,6 +1565,26 @@ ngx_http_upstream_connect(ngx_http_reque
 
     u->state->peer = u->peer.name;
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    if (u->upstream && u->upstream->shm_zone
+        && (u->upstream->flags & NGX_HTTP_UPSTREAM_MODIFY)
+    ) {
+        u->state->peer = ngx_palloc(r->pool,
+                                    sizeof(ngx_str_t) + u->peer.name->len);
+        if (u->state->peer == NULL) {
+            ngx_http_upstream_finalize_request(r, u,
+                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
+            return;
+        }
+
+        u->state->peer->len = u->peer.name->len;
+        u->state->peer->data = (u_char *) (u->state->peer + 1);
+        ngx_memcpy(u->state->peer->data, u->peer.name->data, u->peer.name->len);
+
+        u->peer.name = u->state->peer;
+    }
+#endif
+
     if (rc == NGX_BUSY) {
         ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "no live upstreams");
         ngx_http_upstream_next(r, u, NGX_HTTP_UPSTREAM_FT_NOLIVE);
@@ -6066,6 +6086,7 @@ ngx_http_upstream(ngx_conf_t *cf, ngx_co
     u.no_port = 1;
 
     uscf = ngx_http_upstream_add(cf, &u, NGX_HTTP_UPSTREAM_CREATE
+                                         |NGX_HTTP_UPSTREAM_MODIFY
                                          |NGX_HTTP_UPSTREAM_WEIGHT
                                          |NGX_HTTP_UPSTREAM_MAX_CONNS
                                          |NGX_HTTP_UPSTREAM_MAX_FAILS
@@ -6151,7 +6172,11 @@ ngx_http_upstream(ngx_conf_t *cf, ngx_co
         return rv;
     }
 
-    if (uscf->servers->nelts == 0) {
+    if (uscf->servers->nelts == 0
+#if (NGX_HTTP_UPSTREAM_ZONE)
+        && uscf->shm_zone == NULL
+#endif
+    ) {
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "no servers are inside upstream");
         return NGX_CONF_ERROR;
@@ -6171,6 +6196,9 @@ ngx_http_upstream_server(ngx_conf_t *cf,
     ngx_url_t                    u;
     ngx_int_t                    weight, max_conns, max_fails;
     ngx_uint_t                   i;
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    ngx_uint_t                   resolve;
+#endif
     ngx_http_upstream_server_t  *us;
 
     us = ngx_array_push(uscf->servers);
@@ -6186,6 +6214,9 @@ ngx_http_upstream_server(ngx_conf_t *cf,
     max_conns = 0;
     max_fails = 1;
     fail_timeout = 10;
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    resolve = 0;
+#endif
 
     for (i = 2; i < cf->args->nelts; i++) {
 
@@ -6274,6 +6305,13 @@ ngx_http_upstream_server(ngx_conf_t *cf,
             continue;
         }
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+        if (ngx_strcmp(value[i].data, "resolve") == 0) {
+            resolve = 1;
+            continue;
+        }
+#endif
+
         goto invalid;
     }
 
@@ -6282,6 +6320,13 @@ ngx_http_upstream_server(ngx_conf_t *cf,
     u.url = value[1];
     u.default_port = 80;
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    if (resolve) {
+        /* resolve at run time */
+        u.no_resolve = 1;
+    }
+#endif
+
     if (ngx_parse_url(cf->pool, &u) != NGX_OK) {
         if (u.err) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
@@ -6292,8 +6337,45 @@ ngx_http_upstream_server(ngx_conf_t *cf,
     }
 
     us->name = u.url;
+
+#if (NGX_HTTP_UPSTREAM_ZONE)
+
+    if (resolve && u.naddrs == 0) {
+        ngx_addr_t  *addr;
+
+        /* save port */
+
+        addr = ngx_pcalloc(cf->pool, sizeof(ngx_addr_t));
+        if (addr == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        addr->sockaddr = ngx_palloc(cf->pool, u.socklen);
+        if (addr->sockaddr == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        ngx_memcpy(addr->sockaddr, &u.sockaddr, u.socklen);
+
+        addr->socklen = u.socklen;
+
+        us->addrs = addr;
+        us->naddrs = 1;
+
+        us->host = u.host;
+
+    } else {
+        us->addrs = u.addrs;
+        us->naddrs = u.naddrs;
+    }
+
+#else
+
     us->addrs = u.addrs;
     us->naddrs = u.naddrs;
+
+#endif
+
     us->weight = weight;
     us->max_conns = max_conns;
     us->max_fails = max_fails;
diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h
+++ b/src/http/ngx_http_upstream.h
@@ -104,7 +104,11 @@ typedef struct {
 
     unsigned                         backup:1;
 
-    NGX_COMPAT_BEGIN(6)
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    ngx_str_t                        host;
+#endif
+
+    NGX_COMPAT_BEGIN(4)
     NGX_COMPAT_END
 } ngx_http_upstream_server_t;
 
@@ -115,6 +119,7 @@ typedef struct {
 #define NGX_HTTP_UPSTREAM_FAIL_TIMEOUT  0x0008
 #define NGX_HTTP_UPSTREAM_DOWN          0x0010
 #define NGX_HTTP_UPSTREAM_BACKUP        0x0020
+#define NGX_HTTP_UPSTREAM_MODIFY        0x0040
 #define NGX_HTTP_UPSTREAM_MAX_CONNS     0x0100
 
 
@@ -133,6 +138,8 @@ struct ngx_http_upstream_srv_conf_s {
 
 #if (NGX_HTTP_UPSTREAM_ZONE)
     ngx_shm_zone_t                  *shm_zone;
+    ngx_resolver_t                  *resolver;
+    ngx_msec_t                       resolver_timeout;
 #endif
 };
 
diff --git a/src/http/ngx_http_upstream_round_robin.c b/src/http/ngx_http_upstream_round_robin.c
--- a/src/http/ngx_http_upstream_round_robin.c
+++ b/src/http/ngx_http_upstream_round_robin.c
@@ -32,10 +32,15 @@ ngx_http_upstream_init_round_robin(ngx_c
     ngx_http_upstream_srv_conf_t *us)
 {
     ngx_url_t                      u;
-    ngx_uint_t                     i, j, n, w, t;
+    ngx_uint_t                     i, j, n, r, w, t;
     ngx_http_upstream_server_t    *server;
     ngx_http_upstream_rr_peer_t   *peer, **peerp;
     ngx_http_upstream_rr_peers_t  *peers, *backup;
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    ngx_uint_t                     resolve;
+    ngx_http_core_loc_conf_t      *clcf;
+    ngx_http_upstream_rr_peer_t  **rpeerp;
+#endif
 
     us->peer.init = ngx_http_upstream_init_round_robin_peer;
 
@@ -43,23 +48,99 @@ ngx_http_upstream_init_round_robin(ngx_c
         server = us->servers->elts;
 
         n = 0;
+        r = 0;
         w = 0;
         t = 0;
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+        resolve = 0;
+#endif
+
         for (i = 0; i < us->servers->nelts; i++) {
+
+#if (NGX_HTTP_UPSTREAM_ZONE)
+            if (server[i].host.len) {
+                resolve = 1;
+            }
+#endif
+
             if (server[i].backup) {
                 continue;
             }
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+            if (server[i].host.len) {
+                r++;
+
+            } else {
+                n += server[i].naddrs;
+                w += server[i].naddrs * server[i].weight;
+
+                if (!server[i].down) {
+                    t += server[i].naddrs;
+                }
+            }
+#else
             n += server[i].naddrs;
             w += server[i].naddrs * server[i].weight;
 
             if (!server[i].down) {
                 t += server[i].naddrs;
             }
+#endif
         }
 
-        if (n == 0) {
+#if (NGX_HTTP_UPSTREAM_ZONE)
+        if (us->shm_zone) {
+
+            if (resolve && !(us->flags & NGX_HTTP_UPSTREAM_MODIFY)) {
+                ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
+                              "load balancing method does not support"
+                              " resolving names at run time in"
+                              " upstream \"%V\" in %s:%ui",
+                              &us->host, us->file_name, us->line);
+                return NGX_ERROR;
+            }
+
+            clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
+
+            us->resolver = clcf->resolver;
+            us->resolver_timeout = clcf->resolver_timeout;
+
+            /*
+             * Without "resolver_timeout" in http{}, the value is unset.
+             * Even if we set it in ngx_http_core_merge_loc_conf(), it's
+             * still dependent on the module order and unreliable.
+             */
+            ngx_conf_init_msec_value(us->resolver_timeout, 30000);
+
+            if (resolve
+                && (us->resolver == NULL
+                    || us->resolver->connections.nelts == 0))
+            {
+                ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
+                              "no resolver defined to resolve names"
+                              " at run time in upstream \"%V\" in %s:%ui",
+                              &us->host, us->file_name, us->line);
+                return NGX_ERROR;
+            }
+
+        } else if (resolve) {
+
+            ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
+                          "resolving names at run time requires"
+                          " upstream \"%V\" in %s:%ui"
+                          " to be in shared memory",
+                          &us->host, us->file_name, us->line);
+            return NGX_ERROR;
+        }
+#endif
+
+        if (n == 0
+#if (NGX_HTTP_UPSTREAM_ZONE)
+            && us->shm_zone == NULL
+#endif
+        ) {
             ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                           "no servers in upstream \"%V\" in %s:%ui",
                           &us->host, us->file_name, us->line);
@@ -71,7 +152,8 @@ ngx_http_upstream_init_round_robin(ngx_c
             return NGX_ERROR;
         }
 
-        peer = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peer_t) * n);
+        peer = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peer_t)
+                                     * (n + r));
         if (peer == NULL) {
             return NGX_ERROR;
         }
@@ -86,11 +168,46 @@ ngx_http_upstream_init_round_robin(ngx_c
         n = 0;
         peerp = &peers->peer;
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+        rpeerp = &peers->resolve;
+#endif
+
         for (i = 0; i < us->servers->nelts; i++) {
             if (server[i].backup) {
                 continue;
             }
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+            if (server[i].host.len) {
+
+                peer[n].host = ngx_pcalloc(cf->pool,
+                                           sizeof(ngx_http_upstream_host_t));
+                if (peer[n].host == NULL) {
+                    return NGX_ERROR;
+                }
+
+                peer[n].host->name = server[i].host;
+
+                peer[n].sockaddr = server[i].addrs[0].sockaddr;
+                peer[n].socklen = server[i].addrs[0].socklen;
+                peer[n].name = server[i].addrs[0].name;
+                peer[n].weight = server[i].weight;
+                peer[n].effective_weight = server[i].weight;
+                peer[n].current_weight = 0;
+                peer[n].max_conns = server[i].max_conns;
+                peer[n].max_fails = server[i].max_fails;
+                peer[n].fail_timeout = server[i].fail_timeout;
+                peer[n].down = server[i].down;
+                peer[n].server = server[i].name;
+
+                *rpeerp = &peer[n];
+                rpeerp = &peer[n].next;
+                n++;
+
+                continue;
+            }
+#endif
+
             for (j = 0; j < server[i].naddrs; j++) {
                 peer[n].sockaddr = server[i].addrs[j].sockaddr;
                 peer[n].socklen = server[i].addrs[j].socklen;
@@ -115,6 +232,7 @@ ngx_http_upstream_init_round_robin(ngx_c
         /* backup servers */
 
         n = 0;
+        r = 0;
         w = 0;
         t = 0;
 
@@ -123,15 +241,37 @@ ngx_http_upstream_init_round_robin(ngx_c
                 continue;
             }
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+            if (server[i].host.len) {
+                r++;
+
+            } else {
+                n += server[i].naddrs;
+                w += server[i].naddrs * server[i].weight;
+
+               if (!server[i].down) {
+                   t += server[i].naddrs;
+               }
+            }
+#else
             n += server[i].naddrs;
             w += server[i].naddrs * server[i].weight;
 
             if (!server[i].down) {
                 t += server[i].naddrs;
             }
+#endif
         }
 
-        if (n == 0) {
+        if (n == 0
+#if (NGX_HTTP_UPSTREAM_ZONE)
+            && us->shm_zone == NULL
+#endif
+        ) {
+            return NGX_OK;
+        }
+
+        if (n + r == 0 && !(us->flags & NGX_HTTP_UPSTREAM_BACKUP)) {
             return NGX_OK;
         }
 
@@ -140,12 +280,16 @@ ngx_http_upstream_init_round_robin(ngx_c
             return NGX_ERROR;
         }
 
-        peer = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peer_t) * n);
+        peer = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_rr_peer_t)
+                                     * (n + r));
         if (peer == NULL) {
             return NGX_ERROR;
         }
 
-        peers->single = 0;
+        if (n > 0) {
+            peers->single = 0;
+        }
+
         backup->single = 0;
         backup->number = n;
         backup->weighted = (w != n);
@@ -156,11 +300,46 @@ ngx_http_upstream_init_round_robin(ngx_c
         n = 0;
         peerp = &backup->peer;
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+        rpeerp = &backup->resolve;
+#endif
+
         for (i = 0; i < us->servers->nelts; i++) {
             if (!server[i].backup) {
                 continue;
             }
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+            if (server[i].host.len) {
+
+                peer[n].host = ngx_pcalloc(cf->pool,
+                                           sizeof(ngx_http_upstream_host_t));
+                if (peer[n].host == NULL) {
+                    return NGX_ERROR;
+                }
+
+                peer[n].host->name = server[i].host;
+
+                peer[n].sockaddr = server[i].addrs[0].sockaddr;
+                peer[n].socklen = server[i].addrs[0].socklen;
+                peer[n].name = server[i].addrs[0].name;
+                peer[n].weight = server[i].weight;
+                peer[n].effective_weight = server[i].weight;
+                peer[n].current_weight = 0;
+                peer[n].max_conns = server[i].max_conns;
+                peer[n].max_fails = server[i].max_fails;
+                peer[n].fail_timeout = server[i].fail_timeout;
+                peer[n].down = server[i].down;
+                peer[n].server = server[i].name;
+
+                *rpeerp = &peer[n];
+                rpeerp = &peer[n].next;
+                n++;
+
+                continue;
+            }
+#endif
+
             for (j = 0; j < server[i].naddrs; j++) {
                 peer[n].sockaddr = server[i].addrs[j].sockaddr;
                 peer[n].socklen = server[i].addrs[j].socklen;
@@ -273,7 +452,12 @@ ngx_http_upstream_init_round_robin_peer(
 
     rrp->peers = us->peer.data;
     rrp->current = NULL;
-    rrp->config = 0;
+
+    ngx_http_upstream_rr_peers_rlock(rrp->peers);
+
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    rrp->config = rrp->peers->config ? *rrp->peers->config : 0;
+#endif
 
     n = rrp->peers->number;
 
@@ -281,6 +465,10 @@ ngx_http_upstream_init_round_robin_peer(
         n = rrp->peers->next->number;
     }
 
+    r->upstream->peer.tries = ngx_http_upstream_tries(rrp->peers);
+
+    ngx_http_upstream_rr_peers_unlock(rrp->peers);
+
     if (n <= 8 * sizeof(uintptr_t)) {
         rrp->tried = &rrp->data;
         rrp->data = 0;
@@ -296,7 +484,6 @@ ngx_http_upstream_init_round_robin_peer(
 
     r->upstream->peer.get = ngx_http_upstream_get_round_robin_peer;
     r->upstream->peer.free = ngx_http_upstream_free_round_robin_peer;
-    r->upstream->peer.tries = ngx_http_upstream_tries(rrp->peers);
 #if (NGX_HTTP_SSL)
     r->upstream->peer.set_session =
                                ngx_http_upstream_set_round_robin_peer_session;
@@ -446,6 +633,12 @@ ngx_http_upstream_get_round_robin_peer(n
     peers = rrp->peers;
     ngx_http_upstream_rr_peers_wlock(peers);
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    if (peers->config && rrp->config != *peers->config) {
+        goto busy;
+    }
+#endif
+
     if (peers->single) {
         peer = peers->peer;
 
@@ -458,6 +651,7 @@ ngx_http_upstream_get_round_robin_peer(n
         }
 
         rrp->current = peer;
+        ngx_http_upstream_rr_peer_ref(peers, peer);
 
     } else {
 
@@ -508,8 +702,18 @@ failed:
         }
 
         ngx_http_upstream_rr_peers_wlock(peers);
+
+#if (NGX_HTTP_UPSTREAM_ZONE)
+        if (peers->config && rrp->config != *peers->config) {
+            goto busy;
+        }
+#endif
     }
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+busy:
+#endif
+
     ngx_http_upstream_rr_peers_unlock(peers);
 
     pc->name = peers->name;
@@ -580,6 +784,7 @@ ngx_http_upstream_get_peer(ngx_http_upst
     }
 
     rrp->current = best;
+    ngx_http_upstream_rr_peer_ref(rrp->peers, best);
 
     n = p / (8 * sizeof(uintptr_t));
     m = (uintptr_t) 1 << p % (8 * sizeof(uintptr_t));
@@ -617,9 +822,16 @@ ngx_http_upstream_free_round_robin_peer(
 
     if (rrp->peers->single) {
 
+        if (peer->fails) {
+            peer->fails = 0;
+        }
+
         peer->conns--;
 
-        ngx_http_upstream_rr_peer_unlock(rrp->peers, peer);
+        if (ngx_http_upstream_rr_peer_unref(rrp->peers, peer) == NGX_OK) {
+            ngx_http_upstream_rr_peer_unlock(rrp->peers, peer);
+        }
+
         ngx_http_upstream_rr_peers_unlock(rrp->peers);
 
         pc->tries = 0;
@@ -661,7 +873,10 @@ ngx_http_upstream_free_round_robin_peer(
 
     peer->conns--;
 
-    ngx_http_upstream_rr_peer_unlock(rrp->peers, peer);
+    if (ngx_http_upstream_rr_peer_unref(rrp->peers, peer) == NGX_OK) {
+        ngx_http_upstream_rr_peer_unlock(rrp->peers, peer);
+    }
+
     ngx_http_upstream_rr_peers_unlock(rrp->peers);
 
     if (pc->tries) {
diff --git a/src/http/ngx_http_upstream_round_robin.h b/src/http/ngx_http_upstream_round_robin.h
--- a/src/http/ngx_http_upstream_round_robin.h
+++ b/src/http/ngx_http_upstream_round_robin.h
@@ -14,8 +14,23 @@
 #include <ngx_http.h>
 
 
+typedef struct ngx_http_upstream_rr_peers_s  ngx_http_upstream_rr_peers_t;
 typedef struct ngx_http_upstream_rr_peer_s   ngx_http_upstream_rr_peer_t;
 
+
+#if (NGX_HTTP_UPSTREAM_ZONE)
+
+typedef struct {
+    ngx_event_t                     event;         /* must be first */
+    ngx_uint_t                      worker;
+    ngx_str_t                       name;
+    ngx_http_upstream_rr_peers_t   *peers;
+    ngx_http_upstream_rr_peer_t    *peer;
+} ngx_http_upstream_host_t;
+
+#endif
+
+
 struct ngx_http_upstream_rr_peer_s {
     struct sockaddr                *sockaddr;
     socklen_t                       socklen;
@@ -46,7 +61,12 @@ struct ngx_http_upstream_rr_peer_s {
 #endif
 
 #if (NGX_HTTP_UPSTREAM_ZONE)
+    unsigned                        zombie:1;
+
     ngx_atomic_t                    lock;
+    ngx_uint_t                      id;
+    ngx_uint_t                      refs;
+    ngx_http_upstream_host_t       *host;
 #endif
 
     ngx_http_upstream_rr_peer_t    *next;
@@ -56,8 +76,6 @@ struct ngx_http_upstream_rr_peer_s {
 };
 
 
-typedef struct ngx_http_upstream_rr_peers_s  ngx_http_upstream_rr_peers_t;
-
 struct ngx_http_upstream_rr_peers_s {
     ngx_uint_t                      number;
 
@@ -78,6 +96,12 @@ struct ngx_http_upstream_rr_peers_s {
     ngx_http_upstream_rr_peers_t   *next;
 
     ngx_http_upstream_rr_peer_t    *peer;
+
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    ngx_uint_t                     *config;
+    ngx_http_upstream_rr_peer_t    *resolve;
+    ngx_uint_t                      zombies;
+#endif
 };
 
 
@@ -114,6 +138,67 @@ struct ngx_http_upstream_rr_peers_s {
         ngx_rwlock_unlock(&peer->lock);                                       \
     }
 
+
+#define ngx_http_upstream_rr_peer_ref(peers, peer)                            \
+    (peer)->refs++;
+
+
+static ngx_inline void
+ngx_http_upstream_rr_peer_free_locked(ngx_http_upstream_rr_peers_t *peers,
+    ngx_http_upstream_rr_peer_t *peer)
+{
+    if (peer->refs) {
+        peer->zombie = 1;
+        peers->zombies++;
+        return;
+    }
+
+    ngx_slab_free_locked(peers->shpool, peer->sockaddr);
+    ngx_slab_free_locked(peers->shpool, peer->name.data);
+
+    if (peer->server.data && (peer->host == NULL || peer->host->peer == peer)) {
+        ngx_slab_free_locked(peers->shpool, peer->server.data);
+    }
+
+#if (NGX_HTTP_SSL)
+    if (peer->ssl_session) {
+        ngx_slab_free_locked(peers->shpool, peer->ssl_session);
+    }
+#endif
+
+    ngx_slab_free_locked(peers->shpool, peer);
+}
+
+
+static ngx_inline void
+ngx_http_upstream_rr_peer_free(ngx_http_upstream_rr_peers_t *peers,
+    ngx_http_upstream_rr_peer_t *peer)
+{
+    ngx_shmtx_lock(&peers->shpool->mutex);
+    ngx_http_upstream_rr_peer_free_locked(peers, peer);
+    ngx_shmtx_unlock(&peers->shpool->mutex);
+}
+
+
+static ngx_inline ngx_int_t
+ngx_http_upstream_rr_peer_unref(ngx_http_upstream_rr_peers_t *peers,
+    ngx_http_upstream_rr_peer_t *peer)
+{
+    peer->refs--;
+
+    if (peers->shpool == NULL) {
+        return NGX_OK;
+    }
+
+    if (peer->refs == 0 && peer->zombie) {
+        ngx_http_upstream_rr_peer_free(peers, peer);
+        peers->zombies--;
+        return NGX_DONE;
+    }
+
+    return NGX_OK;
+}
+
 #else
 
 #define ngx_http_upstream_rr_peers_rlock(peers)
@@ -121,6 +206,8 @@ struct ngx_http_upstream_rr_peers_s {
 #define ngx_http_upstream_rr_peers_unlock(peers)
 #define ngx_http_upstream_rr_peer_lock(peers, peer)
 #define ngx_http_upstream_rr_peer_unlock(peers, peer)
+#define ngx_http_upstream_rr_peer_ref(peers, peer)
+#define ngx_http_upstream_rr_peer_unref(peers, peer)  NGX_OK
 
 #endif
 

From a.bavshin at nginx.com  Thu Jun 13 22:28:57 2024
From: a.bavshin at nginx.com (=?iso-8859-1?q?Aleksei_Bavshin?=)
Date: Thu, 13 Jun 2024 15:28:57 -0700
Subject: [PATCH 2 of 9] Stream: re-resolvable servers
In-Reply-To: <patchbomb.1718317735@fedora-wsl.local>
References: <patchbomb.1718317735@fedora-wsl.local>
Message-ID: <a2b1433abda29286189d.1718317737@fedora-wsl.local>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1423128416 -10800
#      Thu Feb 05 12:26:56 2015 +0300
# Node ID a2b1433abda29286189dad01fc3574df235e6aa8
# Parent  56aeae9355df8a2ee07e21b65b6869747dd9ee45
Stream: re-resolvable servers.

Co-authored-by: Ruslan Ermilov <ru at nginx.com>
Co-authored-by: Sergey Kandaurov <pluknet at nginx.com>
Co-authored-by: Vladimir Homutov <vl at nginx.com>

diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c
--- a/src/stream/ngx_stream_proxy_module.c
+++ b/src/stream/ngx_stream_proxy_module.c
@@ -742,6 +742,25 @@ ngx_stream_proxy_connect(ngx_stream_sess
 
     u->state->peer = u->peer.name;
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    if (u->upstream && u->upstream->shm_zone
+        && (u->upstream->flags & NGX_STREAM_UPSTREAM_MODIFY)
+    ) {
+        u->state->peer = ngx_palloc(s->connection->pool,
+                                    sizeof(ngx_str_t) + u->peer.name->len);
+        if (u->state->peer == NULL) {
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+            return;
+        }
+
+        u->state->peer->len = u->peer.name->len;
+        u->state->peer->data = (u_char *) (u->state->peer + 1);
+        ngx_memcpy(u->state->peer->data, u->peer.name->data, u->peer.name->len);
+
+        u->peer.name = u->state->peer;
+    }
+#endif
+
     if (rc == NGX_BUSY) {
         ngx_log_error(NGX_LOG_ERR, c->log, 0, "no live upstreams");
         ngx_stream_proxy_finalize(s, NGX_STREAM_BAD_GATEWAY);
diff --git a/src/stream/ngx_stream_upstream.c b/src/stream/ngx_stream_upstream.c
--- a/src/stream/ngx_stream_upstream.c
+++ b/src/stream/ngx_stream_upstream.c
@@ -319,6 +319,7 @@ ngx_stream_upstream(ngx_conf_t *cf, ngx_
     u.no_port = 1;
 
     uscf = ngx_stream_upstream_add(cf, &u, NGX_STREAM_UPSTREAM_CREATE
+                                           |NGX_STREAM_UPSTREAM_MODIFY
                                            |NGX_STREAM_UPSTREAM_WEIGHT
                                            |NGX_STREAM_UPSTREAM_MAX_CONNS
                                            |NGX_STREAM_UPSTREAM_MAX_FAILS
@@ -388,7 +389,11 @@ ngx_stream_upstream(ngx_conf_t *cf, ngx_
         return rv;
     }
 
-    if (uscf->servers->nelts == 0) {
+    if (uscf->servers->nelts == 0
+#if (NGX_STREAM_UPSTREAM_ZONE)
+        && uscf->shm_zone == NULL
+#endif
+    ) {
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "no servers are inside upstream");
         return NGX_CONF_ERROR;
@@ -408,6 +413,9 @@ ngx_stream_upstream_server(ngx_conf_t *c
     ngx_url_t                      u;
     ngx_int_t                      weight, max_conns, max_fails;
     ngx_uint_t                     i;
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    ngx_uint_t                     resolve;
+#endif
     ngx_stream_upstream_server_t  *us;
 
     us = ngx_array_push(uscf->servers);
@@ -423,6 +431,9 @@ ngx_stream_upstream_server(ngx_conf_t *c
     max_conns = 0;
     max_fails = 1;
     fail_timeout = 10;
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    resolve = 0;
+#endif
 
     for (i = 2; i < cf->args->nelts; i++) {
 
@@ -511,6 +522,13 @@ ngx_stream_upstream_server(ngx_conf_t *c
             continue;
         }
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+        if (ngx_strcmp(value[i].data, "resolve") == 0) {
+            resolve = 1;
+            continue;
+        }
+#endif
+
         goto invalid;
     }
 
@@ -518,6 +536,13 @@ ngx_stream_upstream_server(ngx_conf_t *c
 
     u.url = value[1];
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    if (resolve) {
+        /* resolve at run time */
+        u.no_resolve = 1;
+    }
+#endif
+
     if (ngx_parse_url(cf->pool, &u) != NGX_OK) {
         if (u.err) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
@@ -534,8 +559,45 @@ ngx_stream_upstream_server(ngx_conf_t *c
     }
 
     us->name = u.url;
+
+#if (NGX_STREAM_UPSTREAM_ZONE)
+
+    if (resolve && u.naddrs == 0) {
+        ngx_addr_t  *addr;
+
+        /* save port */
+
+        addr = ngx_pcalloc(cf->pool, sizeof(ngx_addr_t));
+        if (addr == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        addr->sockaddr = ngx_palloc(cf->pool, u.socklen);
+        if (addr->sockaddr == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        ngx_memcpy(addr->sockaddr, &u.sockaddr, u.socklen);
+
+        addr->socklen = u.socklen;
+
+        us->addrs = addr;
+        us->naddrs = 1;
+
+        us->host = u.host;
+
+    } else {
+        us->addrs = u.addrs;
+        us->naddrs = u.naddrs;
+    }
+
+#else
+
     us->addrs = u.addrs;
     us->naddrs = u.naddrs;
+
+#endif
+
     us->weight = weight;
     us->max_conns = max_conns;
     us->max_fails = max_fails;
diff --git a/src/stream/ngx_stream_upstream.h b/src/stream/ngx_stream_upstream.h
--- a/src/stream/ngx_stream_upstream.h
+++ b/src/stream/ngx_stream_upstream.h
@@ -21,6 +21,7 @@
 #define NGX_STREAM_UPSTREAM_FAIL_TIMEOUT  0x0008
 #define NGX_STREAM_UPSTREAM_DOWN          0x0010
 #define NGX_STREAM_UPSTREAM_BACKUP        0x0020
+#define NGX_STREAM_UPSTREAM_MODIFY        0x0040
 #define NGX_STREAM_UPSTREAM_MAX_CONNS     0x0100
 
 
@@ -62,7 +63,11 @@ typedef struct {
 
     unsigned                           backup:1;
 
-    NGX_COMPAT_BEGIN(4)
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    ngx_str_t                          host;
+#endif
+
+    NGX_COMPAT_BEGIN(2)
     NGX_COMPAT_END
 } ngx_stream_upstream_server_t;
 
@@ -83,6 +88,8 @@ struct ngx_stream_upstream_srv_conf_s {
 
 #if (NGX_STREAM_UPSTREAM_ZONE)
     ngx_shm_zone_t                    *shm_zone;
+    ngx_resolver_t                    *resolver;
+    ngx_msec_t                         resolver_timeout;
 #endif
 };
 
diff --git a/src/stream/ngx_stream_upstream_hash_module.c b/src/stream/ngx_stream_upstream_hash_module.c
--- a/src/stream/ngx_stream_upstream_hash_module.c
+++ b/src/stream/ngx_stream_upstream_hash_module.c
@@ -23,6 +23,9 @@ typedef struct {
 
 
 typedef struct {
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    ngx_uint_t                            config;
+#endif
     ngx_stream_complex_value_t            key;
     ngx_stream_upstream_chash_points_t   *points;
 } ngx_stream_upstream_hash_srv_conf_t;
@@ -49,6 +52,8 @@ static ngx_int_t ngx_stream_upstream_get
 
 static ngx_int_t ngx_stream_upstream_init_chash(ngx_conf_t *cf,
     ngx_stream_upstream_srv_conf_t *us);
+static ngx_int_t ngx_stream_upstream_update_chash(ngx_pool_t *pool,
+    ngx_stream_upstream_srv_conf_t *us);
 static int ngx_libc_cdecl
     ngx_stream_upstream_chash_cmp_points(const void *one, const void *two);
 static ngx_uint_t ngx_stream_upstream_find_chash_point(
@@ -178,11 +183,18 @@ ngx_stream_upstream_get_hash_peer(ngx_pe
 
     ngx_stream_upstream_rr_peers_rlock(hp->rrp.peers);
 
-    if (hp->tries > 20 || hp->rrp.peers->single || hp->key.len == 0) {
+    if (hp->tries > 20 || hp->rrp.peers->number < 2 || hp->key.len == 0) {
         ngx_stream_upstream_rr_peers_unlock(hp->rrp.peers);
         return hp->get_rr_peer(pc, &hp->rrp);
     }
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    if (hp->rrp.peers->config && hp->rrp.config != *hp->rrp.peers->config) {
+        ngx_stream_upstream_rr_peers_unlock(hp->rrp.peers);
+        return hp->get_rr_peer(pc, &hp->rrp);
+    }
+#endif
+
     now = ngx_time();
 
     pc->connection = NULL;
@@ -261,6 +273,7 @@ ngx_stream_upstream_get_hash_peer(ngx_pe
     }
 
     hp->rrp.current = peer;
+    ngx_stream_upstream_rr_peer_ref(hp->rrp.peers, peer);
 
     pc->sockaddr = peer->sockaddr;
     pc->socklen = peer->socklen;
@@ -285,6 +298,26 @@ static ngx_int_t
 ngx_stream_upstream_init_chash(ngx_conf_t *cf,
     ngx_stream_upstream_srv_conf_t *us)
 {
+    if (ngx_stream_upstream_init_round_robin(cf, us) != NGX_OK) {
+        return NGX_ERROR;
+    }
+
+    us->peer.init = ngx_stream_upstream_init_chash_peer;
+
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    if (us->shm_zone) {
+        return NGX_OK;
+    }
+#endif
+
+    return ngx_stream_upstream_update_chash(cf->pool, us);
+}
+
+
+static ngx_int_t
+ngx_stream_upstream_update_chash(ngx_pool_t *pool,
+    ngx_stream_upstream_srv_conf_t *us)
+{
     u_char                               *host, *port, c;
     size_t                                host_len, port_len, size;
     uint32_t                              hash, base_hash;
@@ -299,25 +332,33 @@ ngx_stream_upstream_init_chash(ngx_conf_
         u_char                            byte[4];
     } prev_hash;
 
-    if (ngx_stream_upstream_init_round_robin(cf, us) != NGX_OK) {
-        return NGX_ERROR;
+    hcf = ngx_stream_conf_upstream_srv_conf(us,
+                                            ngx_stream_upstream_hash_module);
+
+    if (hcf->points) {
+        ngx_free(hcf->points);
+        hcf->points = NULL;
     }
 
-    us->peer.init = ngx_stream_upstream_init_chash_peer;
-
     peers = us->peer.data;
     npoints = peers->total_weight * 160;
 
     size = sizeof(ngx_stream_upstream_chash_points_t)
-           + sizeof(ngx_stream_upstream_chash_point_t) * (npoints - 1);
+           - sizeof(ngx_stream_upstream_chash_point_t)
+           + sizeof(ngx_stream_upstream_chash_point_t) * npoints;
 
-    points = ngx_palloc(cf->pool, size);
+    points = pool ? ngx_palloc(pool, size) : ngx_alloc(size, ngx_cycle->log);
     if (points == NULL) {
         return NGX_ERROR;
     }
 
     points->number = 0;
 
+    if (npoints == 0) {
+        hcf->points = points;
+        return NGX_OK;
+    }
+
     for (peer = peers->peer; peer; peer = peer->next) {
         server = &peer->server;
 
@@ -401,8 +442,6 @@ ngx_stream_upstream_init_chash(ngx_conf_
 
     points->number = i + 1;
 
-    hcf = ngx_stream_conf_upstream_srv_conf(us,
-                                            ngx_stream_upstream_hash_module);
     hcf->points = points;
 
     return NGX_OK;
@@ -483,7 +522,22 @@ ngx_stream_upstream_init_chash_peer(ngx_
 
     ngx_stream_upstream_rr_peers_rlock(hp->rrp.peers);
 
-    hp->hash = ngx_stream_upstream_find_chash_point(hcf->points, hash);
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    if (hp->rrp.peers->config
+        && (hcf->points == NULL || hcf->config != *hp->rrp.peers->config))
+    {
+        if (ngx_stream_upstream_update_chash(NULL, us) != NGX_OK) {
+            ngx_stream_upstream_rr_peers_unlock(hp->rrp.peers);
+            return NGX_ERROR;
+        }
+
+        hcf->config = *hp->rrp.peers->config;
+    }
+#endif
+
+    if (hcf->points->number) {
+        hp->hash = ngx_stream_upstream_find_chash_point(hcf->points, hash);
+    }
 
     ngx_stream_upstream_rr_peers_unlock(hp->rrp.peers);
 
@@ -518,6 +572,20 @@ ngx_stream_upstream_get_chash_peer(ngx_p
 
     pc->connection = NULL;
 
+    if (hp->rrp.peers->number == 0) {
+        pc->name = hp->rrp.peers->name;
+        ngx_stream_upstream_rr_peers_unlock(hp->rrp.peers);
+        return NGX_BUSY;
+    }
+
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    if (hp->rrp.peers->config && hp->rrp.config != *hp->rrp.peers->config) {
+        pc->name = hp->rrp.peers->name;
+        ngx_stream_upstream_rr_peers_unlock(hp->rrp.peers);
+        return NGX_BUSY;
+    }
+#endif
+
     now = ngx_time();
     hcf = hp->conf;
 
@@ -596,6 +664,7 @@ ngx_stream_upstream_get_chash_peer(ngx_p
     }
 
     hp->rrp.current = best;
+    ngx_stream_upstream_rr_peer_ref(hp->rrp.peers, best);
 
     pc->sockaddr = best->sockaddr;
     pc->socklen = best->socklen;
@@ -663,6 +732,7 @@ ngx_stream_upstream_hash(ngx_conf_t *cf,
     }
 
     uscf->flags = NGX_STREAM_UPSTREAM_CREATE
+                  |NGX_STREAM_UPSTREAM_MODIFY
                   |NGX_STREAM_UPSTREAM_WEIGHT
                   |NGX_STREAM_UPSTREAM_MAX_CONNS
                   |NGX_STREAM_UPSTREAM_MAX_FAILS
diff --git a/src/stream/ngx_stream_upstream_least_conn_module.c b/src/stream/ngx_stream_upstream_least_conn_module.c
--- a/src/stream/ngx_stream_upstream_least_conn_module.c
+++ b/src/stream/ngx_stream_upstream_least_conn_module.c
@@ -120,6 +120,12 @@ ngx_stream_upstream_get_least_conn_peer(
 
     ngx_stream_upstream_rr_peers_wlock(peers);
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    if (peers->config && rrp->config != *peers->config) {
+        goto busy;
+    }
+#endif
+
     best = NULL;
     total = 0;
 
@@ -240,6 +246,7 @@ ngx_stream_upstream_get_least_conn_peer(
     best->conns++;
 
     rrp->current = best;
+    ngx_stream_upstream_rr_peer_ref(peers, best);
 
     n = p / (8 * sizeof(uintptr_t));
     m = (uintptr_t) 1 << p % (8 * sizeof(uintptr_t));
@@ -274,8 +281,18 @@ failed:
         }
 
         ngx_stream_upstream_rr_peers_wlock(peers);
+
+#if (NGX_STREAM_UPSTREAM_ZONE)
+        if (peers->config && rrp->config != *peers->config) {
+            goto busy;
+        }
+#endif
     }
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+busy:
+#endif
+
     ngx_stream_upstream_rr_peers_unlock(peers);
 
     pc->name = peers->name;
@@ -299,6 +316,7 @@ ngx_stream_upstream_least_conn(ngx_conf_
     uscf->peer.init_upstream = ngx_stream_upstream_init_least_conn;
 
     uscf->flags = NGX_STREAM_UPSTREAM_CREATE
+                  |NGX_STREAM_UPSTREAM_MODIFY
                   |NGX_STREAM_UPSTREAM_WEIGHT
                   |NGX_STREAM_UPSTREAM_MAX_CONNS
                   |NGX_STREAM_UPSTREAM_MAX_FAILS
diff --git a/src/stream/ngx_stream_upstream_random_module.c b/src/stream/ngx_stream_upstream_random_module.c
--- a/src/stream/ngx_stream_upstream_random_module.c
+++ b/src/stream/ngx_stream_upstream_random_module.c
@@ -17,6 +17,9 @@ typedef struct {
 
 typedef struct {
     ngx_uint_t                              two;
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    ngx_uint_t                              config;
+#endif
     ngx_stream_upstream_random_range_t     *ranges;
 } ngx_stream_upstream_random_srv_conf_t;
 
@@ -125,6 +128,11 @@ ngx_stream_upstream_update_random(ngx_po
 
     rcf = ngx_stream_conf_upstream_srv_conf(us,
                                             ngx_stream_upstream_random_module);
+    if (rcf->ranges) {
+        ngx_free(rcf->ranges);
+        rcf->ranges = NULL;
+    }
+
     peers = us->peer.data;
 
     size = peers->number * sizeof(ngx_stream_upstream_random_range_t);
@@ -186,11 +194,15 @@ ngx_stream_upstream_init_random_peer(ngx
     ngx_stream_upstream_rr_peers_rlock(rp->rrp.peers);
 
 #if (NGX_STREAM_UPSTREAM_ZONE)
-    if (rp->rrp.peers->shpool && rcf->ranges == NULL) {
+    if (rp->rrp.peers->config
+        && (rcf->ranges == NULL || rcf->config != *rp->rrp.peers->config))
+    {
         if (ngx_stream_upstream_update_random(NULL, us) != NGX_OK) {
             ngx_stream_upstream_rr_peers_unlock(rp->rrp.peers);
             return NGX_ERROR;
         }
+
+        rcf->config = *rp->rrp.peers->config;
     }
 #endif
 
@@ -220,11 +232,18 @@ ngx_stream_upstream_get_random_peer(ngx_
 
     ngx_stream_upstream_rr_peers_rlock(peers);
 
-    if (rp->tries > 20 || peers->single) {
+    if (rp->tries > 20 || peers->number < 2) {
         ngx_stream_upstream_rr_peers_unlock(peers);
         return ngx_stream_upstream_get_round_robin_peer(pc, rrp);
     }
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    if (peers->config && rrp->config != *peers->config) {
+        ngx_stream_upstream_rr_peers_unlock(peers);
+        return ngx_stream_upstream_get_round_robin_peer(pc, rrp);
+    }
+#endif
+
     pc->cached = 0;
     pc->connection = NULL;
 
@@ -274,6 +293,7 @@ ngx_stream_upstream_get_random_peer(ngx_
     }
 
     rrp->current = peer;
+    ngx_stream_upstream_rr_peer_ref(peers, peer);
 
     if (now - peer->checked > peer->fail_timeout) {
         peer->checked = now;
@@ -314,11 +334,18 @@ ngx_stream_upstream_get_random2_peer(ngx
 
     ngx_stream_upstream_rr_peers_wlock(peers);
 
-    if (rp->tries > 20 || peers->single) {
+    if (rp->tries > 20 || peers->number < 2) {
         ngx_stream_upstream_rr_peers_unlock(peers);
         return ngx_stream_upstream_get_round_robin_peer(pc, rrp);
     }
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    if (peers->config && rrp->config != *peers->config) {
+        ngx_stream_upstream_rr_peers_unlock(peers);
+        return ngx_stream_upstream_get_round_robin_peer(pc, rrp);
+    }
+#endif
+
     pc->cached = 0;
     pc->connection = NULL;
 
@@ -384,6 +411,7 @@ ngx_stream_upstream_get_random2_peer(ngx
     }
 
     rrp->current = peer;
+    ngx_stream_upstream_rr_peer_ref(peers, peer);
 
     if (now - peer->checked > peer->fail_timeout) {
         peer->checked = now;
@@ -467,6 +495,7 @@ ngx_stream_upstream_random(ngx_conf_t *c
     uscf->peer.init_upstream = ngx_stream_upstream_init_random;
 
     uscf->flags = NGX_STREAM_UPSTREAM_CREATE
+                  |NGX_STREAM_UPSTREAM_MODIFY
                   |NGX_STREAM_UPSTREAM_WEIGHT
                   |NGX_STREAM_UPSTREAM_MAX_CONNS
                   |NGX_STREAM_UPSTREAM_MAX_FAILS
diff --git a/src/stream/ngx_stream_upstream_round_robin.c b/src/stream/ngx_stream_upstream_round_robin.c
--- a/src/stream/ngx_stream_upstream_round_robin.c
+++ b/src/stream/ngx_stream_upstream_round_robin.c
@@ -38,10 +38,15 @@ ngx_stream_upstream_init_round_robin(ngx
     ngx_stream_upstream_srv_conf_t *us)
 {
     ngx_url_t                        u;
-    ngx_uint_t                       i, j, n, w, t;
+    ngx_uint_t                       i, j, n, r, w, t;
     ngx_stream_upstream_server_t    *server;
     ngx_stream_upstream_rr_peer_t   *peer, **peerp;
     ngx_stream_upstream_rr_peers_t  *peers, *backup;
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    ngx_uint_t                       resolve;
+    ngx_stream_core_srv_conf_t      *cscf;
+    ngx_stream_upstream_rr_peer_t  **rpeerp;
+#endif
 
     us->peer.init = ngx_stream_upstream_init_round_robin_peer;
 
@@ -49,23 +54,100 @@ ngx_stream_upstream_init_round_robin(ngx
         server = us->servers->elts;
 
         n = 0;
+        r = 0;
         w = 0;
         t = 0;
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+        resolve = 0;
+#endif
+
         for (i = 0; i < us->servers->nelts; i++) {
+
+#if (NGX_STREAM_UPSTREAM_ZONE)
+            if (server[i].host.len) {
+                resolve = 1;
+            }
+#endif
+
             if (server[i].backup) {
                 continue;
             }
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+            if (server[i].host.len) {
+                r++;
+
+            } else {
+                n += server[i].naddrs;
+                w += server[i].naddrs * server[i].weight;
+
+                if (!server[i].down) {
+                    t += server[i].naddrs;
+                }
+            }
+#else
             n += server[i].naddrs;
             w += server[i].naddrs * server[i].weight;
 
             if (!server[i].down) {
                 t += server[i].naddrs;
             }
+#endif
         }
 
-        if (n == 0) {
+#if (NGX_STREAM_UPSTREAM_ZONE)
+        if (us->shm_zone) {
+
+            if (resolve && !(us->flags & NGX_STREAM_UPSTREAM_MODIFY)) {
+                ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
+                              "load balancing method does not support"
+                              " resolving names at run time in"
+                              " upstream \"%V\" in %s:%ui",
+                              &us->host, us->file_name, us->line);
+                return NGX_ERROR;
+            }
+
+            cscf = ngx_stream_conf_get_module_srv_conf(cf,
+                                                       ngx_stream_core_module);
+
+            us->resolver = cscf->resolver;
+            us->resolver_timeout = cscf->resolver_timeout;
+
+            /*
+             * Without "resolver_timeout" in stream{}, the value is unset.
+             * Even if we set it in ngx_stream_core_merge_srv_conf(), it's
+             * still dependent on the module order and unreliable.
+             */
+            ngx_conf_init_msec_value(us->resolver_timeout, 30000);
+
+            if (resolve
+                && (us->resolver == NULL
+                    || us->resolver->connections.nelts == 0))
+            {
+                ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
+                              "no resolver defined to resolve names"
+                              " at run time in upstream \"%V\" in %s:%ui",
+                              &us->host, us->file_name, us->line);
+                return NGX_ERROR;
+            }
+
+        } else if (resolve) {
+
+            ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
+                          "resolving names at run time requires"
+                          " upstream \"%V\" in %s:%ui"
+                          " to be in shared memory",
+                          &us->host, us->file_name, us->line);
+            return NGX_ERROR;
+        }
+#endif
+
+        if (n == 0
+#if (NGX_STREAM_UPSTREAM_ZONE)
+            && us->shm_zone == NULL
+#endif
+        ) {
             ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                           "no servers in upstream \"%V\" in %s:%ui",
                           &us->host, us->file_name, us->line);
@@ -77,7 +159,8 @@ ngx_stream_upstream_init_round_robin(ngx
             return NGX_ERROR;
         }
 
-        peer = ngx_pcalloc(cf->pool, sizeof(ngx_stream_upstream_rr_peer_t) * n);
+        peer = ngx_pcalloc(cf->pool, sizeof(ngx_stream_upstream_rr_peer_t)
+                                     * (n + r));
         if (peer == NULL) {
             return NGX_ERROR;
         }
@@ -92,11 +175,45 @@ ngx_stream_upstream_init_round_robin(ngx
         n = 0;
         peerp = &peers->peer;
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+        rpeerp = &peers->resolve;
+#endif
+
         for (i = 0; i < us->servers->nelts; i++) {
             if (server[i].backup) {
                 continue;
             }
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+            if (server[i].host.len) {
+
+                peer[n].host = ngx_pcalloc(cf->pool,
+                                           sizeof(ngx_stream_upstream_host_t));
+                if (peer[n].host == NULL) {
+                    return NGX_ERROR;
+                }
+
+                peer[n].host->name = server[i].host;
+
+                peer[n].sockaddr = server[i].addrs[0].sockaddr;
+                peer[n].socklen = server[i].addrs[0].socklen;
+                peer[n].name = server[i].addrs[0].name;
+                peer[n].weight = server[i].weight;
+                peer[n].effective_weight = server[i].weight;
+                peer[n].current_weight = 0;
+                peer[n].max_conns = server[i].max_conns;
+                peer[n].max_fails = server[i].max_fails;
+                peer[n].fail_timeout = server[i].fail_timeout;
+                peer[n].down = server[i].down;
+                peer[n].server = server[i].name;
+                *rpeerp = &peer[n];
+                rpeerp = &peer[n].next;
+                n++;
+
+                continue;
+            }
+#endif
+
             for (j = 0; j < server[i].naddrs; j++) {
                 peer[n].sockaddr = server[i].addrs[j].sockaddr;
                 peer[n].socklen = server[i].addrs[j].socklen;
@@ -121,6 +238,7 @@ ngx_stream_upstream_init_round_robin(ngx
         /* backup servers */
 
         n = 0;
+        r = 0;
         w = 0;
         t = 0;
 
@@ -129,15 +247,37 @@ ngx_stream_upstream_init_round_robin(ngx
                 continue;
             }
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+            if (server[i].host.len) {
+                r++;
+
+            } else {
+                n += server[i].naddrs;
+                w += server[i].naddrs * server[i].weight;
+
+                if (!server[i].down) {
+                    t += server[i].naddrs;
+                }
+            }
+#else
             n += server[i].naddrs;
             w += server[i].naddrs * server[i].weight;
 
             if (!server[i].down) {
                 t += server[i].naddrs;
             }
+#endif
         }
 
-        if (n == 0) {
+        if (n == 0
+#if (NGX_STREAM_UPSTREAM_ZONE)
+            && us->shm_zone == NULL
+#endif
+        ) {
+            return NGX_OK;
+        }
+
+        if (n + r == 0 && !(us->flags & NGX_STREAM_UPSTREAM_BACKUP)) {
             return NGX_OK;
         }
 
@@ -146,12 +286,16 @@ ngx_stream_upstream_init_round_robin(ngx
             return NGX_ERROR;
         }
 
-        peer = ngx_pcalloc(cf->pool, sizeof(ngx_stream_upstream_rr_peer_t) * n);
+        peer = ngx_pcalloc(cf->pool, sizeof(ngx_stream_upstream_rr_peer_t)
+                                     * (n + r));
         if (peer == NULL) {
             return NGX_ERROR;
         }
 
-        peers->single = 0;
+        if (n > 0) {
+            peers->single = 0;
+        }
+
         backup->single = 0;
         backup->number = n;
         backup->weighted = (w != n);
@@ -162,11 +306,45 @@ ngx_stream_upstream_init_round_robin(ngx
         n = 0;
         peerp = &backup->peer;
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+        rpeerp = &backup->resolve;
+#endif
+
         for (i = 0; i < us->servers->nelts; i++) {
             if (!server[i].backup) {
                 continue;
             }
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+            if (server[i].host.len) {
+
+                peer[n].host = ngx_pcalloc(cf->pool,
+                                           sizeof(ngx_stream_upstream_host_t));
+                if (peer[n].host == NULL) {
+                    return NGX_ERROR;
+                }
+
+                peer[n].host->name = server[i].host;
+
+                peer[n].sockaddr = server[i].addrs[0].sockaddr;
+                peer[n].socklen = server[i].addrs[0].socklen;
+                peer[n].name = server[i].addrs[0].name;
+                peer[n].weight = server[i].weight;
+                peer[n].effective_weight = server[i].weight;
+                peer[n].current_weight = 0;
+                peer[n].max_conns = server[i].max_conns;
+                peer[n].max_fails = server[i].max_fails;
+                peer[n].fail_timeout = server[i].fail_timeout;
+                peer[n].down = server[i].down;
+                peer[n].server = server[i].name;
+                *rpeerp = &peer[n];
+                rpeerp = &peer[n].next;
+                n++;
+
+                continue;
+            }
+#endif
+
             for (j = 0; j < server[i].naddrs; j++) {
                 peer[n].sockaddr = server[i].addrs[j].sockaddr;
                 peer[n].socklen = server[i].addrs[j].socklen;
@@ -280,7 +458,12 @@ ngx_stream_upstream_init_round_robin_pee
 
     rrp->peers = us->peer.data;
     rrp->current = NULL;
-    rrp->config = 0;
+
+    ngx_stream_upstream_rr_peers_rlock(rrp->peers);
+
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    rrp->config = rrp->peers->config ? *rrp->peers->config : 0;
+#endif
 
     n = rrp->peers->number;
 
@@ -288,6 +471,10 @@ ngx_stream_upstream_init_round_robin_pee
         n = rrp->peers->next->number;
     }
 
+    s->upstream->peer.tries = ngx_stream_upstream_tries(rrp->peers);
+
+    ngx_stream_upstream_rr_peers_unlock(rrp->peers);
+
     if (n <= 8 * sizeof(uintptr_t)) {
         rrp->tried = &rrp->data;
         rrp->data = 0;
@@ -304,7 +491,7 @@ ngx_stream_upstream_init_round_robin_pee
     s->upstream->peer.get = ngx_stream_upstream_get_round_robin_peer;
     s->upstream->peer.free = ngx_stream_upstream_free_round_robin_peer;
     s->upstream->peer.notify = ngx_stream_upstream_notify_round_robin_peer;
-    s->upstream->peer.tries = ngx_stream_upstream_tries(rrp->peers);
+
 #if (NGX_STREAM_SSL)
     s->upstream->peer.set_session =
                              ngx_stream_upstream_set_round_robin_peer_session;
@@ -455,6 +642,12 @@ ngx_stream_upstream_get_round_robin_peer
     peers = rrp->peers;
     ngx_stream_upstream_rr_peers_wlock(peers);
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    if (peers->config && rrp->config != *peers->config) {
+        goto busy;
+    }
+#endif
+
     if (peers->single) {
         peer = peers->peer;
 
@@ -467,6 +660,7 @@ ngx_stream_upstream_get_round_robin_peer
         }
 
         rrp->current = peer;
+        ngx_stream_upstream_rr_peer_ref(peers, peer);
 
     } else {
 
@@ -517,8 +711,18 @@ failed:
         }
 
         ngx_stream_upstream_rr_peers_wlock(peers);
+
+#if (NGX_STREAM_UPSTREAM_ZONE)
+        if (peers->config && rrp->config != *peers->config) {
+            goto busy;
+        }
+#endif
     }
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+busy:
+#endif
+
     ngx_stream_upstream_rr_peers_unlock(peers);
 
     pc->name = peers->name;
@@ -589,6 +793,7 @@ ngx_stream_upstream_get_peer(ngx_stream_
     }
 
     rrp->current = best;
+    ngx_stream_upstream_rr_peer_ref(rrp->peers, best);
 
     n = p / (8 * sizeof(uintptr_t));
     m = (uintptr_t) 1 << p % (8 * sizeof(uintptr_t));
@@ -623,9 +828,17 @@ ngx_stream_upstream_free_round_robin_pee
     ngx_stream_upstream_rr_peer_lock(rrp->peers, peer);
 
     if (rrp->peers->single) {
+
+        if (peer->fails) {
+            peer->fails = 0;
+        }
+
         peer->conns--;
 
-        ngx_stream_upstream_rr_peer_unlock(rrp->peers, peer);
+        if (ngx_stream_upstream_rr_peer_unref(rrp->peers, peer) == NGX_OK) {
+            ngx_stream_upstream_rr_peer_unlock(rrp->peers, peer);
+        }
+
         ngx_stream_upstream_rr_peers_unlock(rrp->peers);
 
         pc->tries = 0;
@@ -667,7 +880,10 @@ ngx_stream_upstream_free_round_robin_pee
 
     peer->conns--;
 
-    ngx_stream_upstream_rr_peer_unlock(rrp->peers, peer);
+    if (ngx_stream_upstream_rr_peer_unref(rrp->peers, peer) == NGX_OK) {
+        ngx_stream_upstream_rr_peer_unlock(rrp->peers, peer);
+    }
+
     ngx_stream_upstream_rr_peers_unlock(rrp->peers);
 
     if (pc->tries) {
diff --git a/src/stream/ngx_stream_upstream_round_robin.h b/src/stream/ngx_stream_upstream_round_robin.h
--- a/src/stream/ngx_stream_upstream_round_robin.h
+++ b/src/stream/ngx_stream_upstream_round_robin.h
@@ -14,8 +14,23 @@
 #include <ngx_stream.h>
 
 
+typedef struct ngx_stream_upstream_rr_peers_s  ngx_stream_upstream_rr_peers_t;
 typedef struct ngx_stream_upstream_rr_peer_s   ngx_stream_upstream_rr_peer_t;
 
+
+#if (NGX_STREAM_UPSTREAM_ZONE)
+
+typedef struct {
+    ngx_event_t                      event;         /* must be first */
+    ngx_uint_t                       worker;
+    ngx_str_t                        name;
+    ngx_stream_upstream_rr_peers_t  *peers;
+    ngx_stream_upstream_rr_peer_t   *peer;
+} ngx_stream_upstream_host_t;
+
+#endif
+
+
 struct ngx_stream_upstream_rr_peer_s {
     struct sockaddr                 *sockaddr;
     socklen_t                        socklen;
@@ -44,7 +59,12 @@ struct ngx_stream_upstream_rr_peer_s {
     int                              ssl_session_len;
 
 #if (NGX_STREAM_UPSTREAM_ZONE)
+    unsigned                         zombie:1;
+
     ngx_atomic_t                     lock;
+    ngx_uint_t                       id;
+    ngx_uint_t                       refs;
+    ngx_stream_upstream_host_t      *host;
 #endif
 
     ngx_stream_upstream_rr_peer_t   *next;
@@ -54,8 +74,6 @@ struct ngx_stream_upstream_rr_peer_s {
 };
 
 
-typedef struct ngx_stream_upstream_rr_peers_s  ngx_stream_upstream_rr_peers_t;
-
 struct ngx_stream_upstream_rr_peers_s {
     ngx_uint_t                       number;
 
@@ -76,6 +94,12 @@ struct ngx_stream_upstream_rr_peers_s {
     ngx_stream_upstream_rr_peers_t  *next;
 
     ngx_stream_upstream_rr_peer_t   *peer;
+
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    ngx_uint_t                      *config;
+    ngx_stream_upstream_rr_peer_t   *resolve;
+    ngx_uint_t                       zombies;
+#endif
 };
 
 
@@ -112,6 +136,67 @@ struct ngx_stream_upstream_rr_peers_s {
         ngx_rwlock_unlock(&peer->lock);                                       \
     }
 
+
+#define ngx_stream_upstream_rr_peer_ref(peers, peer)                          \
+    (peer)->refs++;
+
+
+static ngx_inline void
+ngx_stream_upstream_rr_peer_free_locked(ngx_stream_upstream_rr_peers_t *peers,
+    ngx_stream_upstream_rr_peer_t *peer)
+{
+    if (peer->refs) {
+        peer->zombie = 1;
+        peers->zombies++;
+        return;
+    }
+
+    ngx_slab_free_locked(peers->shpool, peer->sockaddr);
+    ngx_slab_free_locked(peers->shpool, peer->name.data);
+
+    if (peer->server.data && (peer->host == NULL || peer->host->peer == peer)) {
+        ngx_slab_free_locked(peers->shpool, peer->server.data);
+    }
+
+#if (NGX_STREAM_SSL)
+    if (peer->ssl_session) {
+        ngx_slab_free_locked(peers->shpool, peer->ssl_session);
+    }
+#endif
+
+    ngx_slab_free_locked(peers->shpool, peer);
+}
+
+
+static ngx_inline void
+ngx_stream_upstream_rr_peer_free(ngx_stream_upstream_rr_peers_t *peers,
+    ngx_stream_upstream_rr_peer_t *peer)
+{
+    ngx_shmtx_lock(&peers->shpool->mutex);
+    ngx_stream_upstream_rr_peer_free_locked(peers, peer);
+    ngx_shmtx_unlock(&peers->shpool->mutex);
+}
+
+
+static ngx_inline ngx_int_t
+ngx_stream_upstream_rr_peer_unref(ngx_stream_upstream_rr_peers_t *peers,
+    ngx_stream_upstream_rr_peer_t *peer)
+{
+    peer->refs--;
+
+    if (peers->shpool == NULL) {
+        return NGX_OK;
+    }
+
+    if (peer->refs == 0 && peer->zombie) {
+        ngx_stream_upstream_rr_peer_free(peers, peer);
+        peers->zombies--;
+        return NGX_DONE;
+    }
+
+    return NGX_OK;
+}
+
 #else
 
 #define ngx_stream_upstream_rr_peers_rlock(peers)
@@ -119,6 +204,8 @@ struct ngx_stream_upstream_rr_peers_s {
 #define ngx_stream_upstream_rr_peers_unlock(peers)
 #define ngx_stream_upstream_rr_peer_lock(peers, peer)
 #define ngx_stream_upstream_rr_peer_unlock(peers, peer)
+#define ngx_stream_upstream_rr_peer_ref(peers, peer)
+#define ngx_stream_upstream_rr_peer_unref(peers, peer)  NGX_OK
 
 #endif
 
diff --git a/src/stream/ngx_stream_upstream_zone_module.c b/src/stream/ngx_stream_upstream_zone_module.c
--- a/src/stream/ngx_stream_upstream_zone_module.c
+++ b/src/stream/ngx_stream_upstream_zone_module.c
@@ -18,6 +18,10 @@ static ngx_stream_upstream_rr_peers_t *n
     ngx_slab_pool_t *shpool, ngx_stream_upstream_srv_conf_t *uscf);
 static ngx_stream_upstream_rr_peer_t *ngx_stream_upstream_zone_copy_peer(
     ngx_stream_upstream_rr_peers_t *peers, ngx_stream_upstream_rr_peer_t *src);
+static void ngx_stream_upstream_zone_set_single(
+    ngx_stream_upstream_srv_conf_t *uscf);
+static void ngx_stream_upstream_zone_remove_peer_locked(
+    ngx_stream_upstream_rr_peers_t *peers, ngx_stream_upstream_rr_peer_t *peer);
 
 
 static ngx_command_t  ngx_stream_upstream_zone_commands[] = {
@@ -33,6 +37,11 @@ static ngx_command_t  ngx_stream_upstrea
 };
 
 
+static ngx_int_t ngx_stream_upstream_zone_init_worker(ngx_cycle_t *cycle);
+static void ngx_stream_upstream_zone_resolve_timer(ngx_event_t *event);
+static void ngx_stream_upstream_zone_resolve_handler(ngx_resolver_ctx_t *ctx);
+
+
 static ngx_stream_module_t  ngx_stream_upstream_zone_module_ctx = {
     NULL,                                  /* preconfiguration */
     NULL,                                  /* postconfiguration */
@@ -52,7 +61,7 @@ ngx_module_t  ngx_stream_upstream_zone_m
     NGX_STREAM_MODULE,                     /* module type */
     NULL,                                  /* init master */
     NULL,                                  /* init module */
-    NULL,                                  /* init process */
+    ngx_stream_upstream_zone_init_worker,  /* init process */
     NULL,                                  /* init thread */
     NULL,                                  /* exit thread */
     NULL,                                  /* exit process */
@@ -185,9 +194,15 @@ ngx_stream_upstream_zone_copy_peers(ngx_
     ngx_stream_upstream_srv_conf_t *uscf)
 {
     ngx_str_t                       *name;
+    ngx_uint_t                      *config;
     ngx_stream_upstream_rr_peer_t   *peer, **peerp;
     ngx_stream_upstream_rr_peers_t  *peers, *backup;
 
+    config = ngx_slab_calloc(shpool, sizeof(ngx_uint_t));
+    if (config == NULL) {
+        return NULL;
+    }
+
     peers = ngx_slab_alloc(shpool, sizeof(ngx_stream_upstream_rr_peers_t));
     if (peers == NULL) {
         return NULL;
@@ -211,6 +226,7 @@ ngx_stream_upstream_zone_copy_peers(ngx_
     peers->name = name;
 
     peers->shpool = shpool;
+    peers->config = config;
 
     for (peerp = &peers->peer; *peerp; peerp = &peer->next) {
         /* pool is unlocked */
@@ -220,6 +236,17 @@ ngx_stream_upstream_zone_copy_peers(ngx_
         }
 
         *peerp = peer;
+        peer->id = (*peers->config)++;
+    }
+
+    for (peerp = &peers->resolve; *peerp; peerp = &peer->next) {
+        peer = ngx_stream_upstream_zone_copy_peer(peers, *peerp);
+        if (peer == NULL) {
+            return NULL;
+        }
+
+        *peerp = peer;
+        peer->id = (*peers->config)++;
     }
 
     if (peers->next == NULL) {
@@ -236,6 +263,7 @@ ngx_stream_upstream_zone_copy_peers(ngx_
     backup->name = name;
 
     backup->shpool = shpool;
+    backup->config = config;
 
     for (peerp = &backup->peer; *peerp; peerp = &peer->next) {
         /* pool is unlocked */
@@ -245,6 +273,17 @@ ngx_stream_upstream_zone_copy_peers(ngx_
         }
 
         *peerp = peer;
+        peer->id = (*backup->config)++;
+    }
+
+    for (peerp = &backup->resolve; *peerp; peerp = &peer->next) {
+        peer = ngx_stream_upstream_zone_copy_peer(backup, *peerp);
+        if (peer == NULL) {
+            return NULL;
+        }
+
+        *peerp = peer;
+        peer->id = (*backup->config)++;
     }
 
     peers->next = backup;
@@ -276,6 +315,7 @@ ngx_stream_upstream_zone_copy_peer(ngx_s
         dst->sockaddr = NULL;
         dst->name.data = NULL;
         dst->server.data = NULL;
+        dst->host = NULL;
     }
 
     dst->sockaddr = ngx_slab_calloc_locked(pool, sizeof(ngx_sockaddr_t));
@@ -298,12 +338,37 @@ ngx_stream_upstream_zone_copy_peer(ngx_s
         }
 
         ngx_memcpy(dst->server.data, src->server.data, src->server.len);
+
+        if (src->host) {
+            dst->host = ngx_slab_calloc_locked(pool,
+                                            sizeof(ngx_stream_upstream_host_t));
+            if (dst->host == NULL) {
+                goto failed;
+            }
+
+            dst->host->name.data = ngx_slab_alloc_locked(pool,
+                                                         src->host->name.len);
+            if (dst->host->name.data == NULL) {
+                goto failed;
+            }
+
+            dst->host->peers = peers;
+            dst->host->peer = dst;
+
+            dst->host->name.len = src->host->name.len;
+            ngx_memcpy(dst->host->name.data, src->host->name.data,
+                       src->host->name.len);
+        }
     }
 
     return dst;
 
 failed:
 
+    if (dst->host) {
+        ngx_slab_free_locked(pool, dst->host);
+    }
+
     if (dst->server.data) {
         ngx_slab_free_locked(pool, dst->server.data);
     }
@@ -320,3 +385,337 @@ failed:
 
     return NULL;
 }
+
+
+static void
+ngx_stream_upstream_zone_set_single(ngx_stream_upstream_srv_conf_t *uscf)
+{
+    ngx_stream_upstream_rr_peers_t  *peers;
+
+    peers = uscf->peer.data;
+
+    if (peers->number == 1
+        && (peers->next == NULL || peers->next->number == 0))
+    {
+        peers->single = 1;
+
+    } else {
+        peers->single = 0;
+    }
+}
+
+
+static void
+ngx_stream_upstream_zone_remove_peer_locked(
+    ngx_stream_upstream_rr_peers_t *peers, ngx_stream_upstream_rr_peer_t *peer)
+{
+    peers->total_weight -= peer->weight;
+    peers->number--;
+    peers->tries -= (peer->down == 0);
+    (*peers->config)++;
+    peers->weighted = (peers->total_weight != peers->number);
+
+    ngx_stream_upstream_rr_peer_free(peers, peer);
+}
+
+
+static ngx_int_t
+ngx_stream_upstream_zone_init_worker(ngx_cycle_t *cycle)
+{
+    ngx_uint_t                        i;
+    ngx_event_t                      *event;
+    ngx_stream_upstream_rr_peer_t    *peer;
+    ngx_stream_upstream_rr_peers_t   *peers;
+    ngx_stream_upstream_srv_conf_t   *uscf, **uscfp;
+    ngx_stream_upstream_main_conf_t  *umcf;
+
+    if (ngx_process != NGX_PROCESS_WORKER
+        && ngx_process != NGX_PROCESS_SINGLE)
+    {
+        return NGX_OK;
+    }
+
+    umcf = ngx_stream_cycle_get_module_main_conf(cycle,
+                                                 ngx_stream_upstream_module);
+
+    if (umcf == NULL) {
+        return NGX_OK;
+    }
+
+    uscfp = umcf->upstreams.elts;
+
+    for (i = 0; i < umcf->upstreams.nelts; i++) {
+
+        uscf = uscfp[i];
+
+        if (uscf->shm_zone == NULL) {
+            continue;
+        }
+
+        peers = uscf->peer.data;
+
+        do {
+            ngx_stream_upstream_rr_peers_wlock(peers);
+
+            for (peer = peers->resolve; peer; peer = peer->next) {
+
+                if (peer->host->worker != ngx_worker) {
+                    continue;
+                }
+
+                event = &peer->host->event;
+                ngx_memzero(event, sizeof(ngx_event_t));
+
+                event->data = uscf;
+                event->handler = ngx_stream_upstream_zone_resolve_timer;
+                event->log = cycle->log;
+                event->cancelable = 1;
+
+                ngx_stream_upstream_rr_peer_ref(peers, peer);
+                ngx_add_timer(event, 1);
+            }
+
+            ngx_stream_upstream_rr_peers_unlock(peers);
+
+            peers = peers->next;
+
+        } while (peers);
+    }
+
+    return NGX_OK;
+}
+
+
+static void
+ngx_stream_upstream_zone_resolve_timer(ngx_event_t *event)
+{
+    ngx_resolver_ctx_t              *ctx;
+    ngx_stream_upstream_host_t      *host;
+    ngx_stream_upstream_rr_peer_t   *template;
+    ngx_stream_upstream_rr_peers_t  *peers;
+    ngx_stream_upstream_srv_conf_t  *uscf;
+
+    host = (ngx_stream_upstream_host_t *) event;
+    uscf = event->data;
+    peers = host->peers;
+    template = host->peer;
+
+    if (template->zombie) {
+        (void) ngx_stream_upstream_rr_peer_unref(peers, template);
+
+        ngx_shmtx_lock(&peers->shpool->mutex);
+
+        if (host->service.len) {
+            ngx_slab_free_locked(peers->shpool, host->service.data);
+        }
+
+        ngx_slab_free_locked(peers->shpool, host->name.data);
+        ngx_slab_free_locked(peers->shpool, host);
+        ngx_shmtx_unlock(&peers->shpool->mutex);
+
+        return;
+    }
+
+    ctx = ngx_resolve_start(uscf->resolver, NULL);
+    if (ctx == NULL) {
+        goto retry;
+    }
+
+    if (ctx == NGX_NO_RESOLVER) {
+        ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                      "no resolver defined to resolve %V", &host->name);
+        return;
+    }
+
+    ctx->name = host->name;
+    ctx->handler = ngx_stream_upstream_zone_resolve_handler;
+    ctx->data = host;
+    ctx->timeout = uscf->resolver_timeout;
+    ctx->cancelable = 1;
+
+    if (ngx_resolve_name(ctx) == NGX_OK) {
+        return;
+    }
+
+retry:
+
+    ngx_add_timer(event, ngx_max(uscf->resolver_timeout, 1000));
+}
+
+
+static void
+ngx_stream_upstream_zone_resolve_handler(ngx_resolver_ctx_t *ctx)
+{
+    time_t                           now;
+    in_port_t                        port;
+    ngx_msec_t                       timer;
+    ngx_uint_t                       i, j;
+    ngx_event_t                     *event;
+    ngx_resolver_addr_t             *addr;
+    ngx_stream_upstream_host_t      *host;
+    ngx_stream_upstream_rr_peer_t   *peer, *template, **peerp;
+    ngx_stream_upstream_rr_peers_t  *peers;
+    ngx_stream_upstream_srv_conf_t  *uscf;
+
+    host = ctx->data;
+    event = &host->event;
+    uscf = event->data;
+    peers = host->peers;
+    template = host->peer;
+
+    ngx_stream_upstream_rr_peers_wlock(peers);
+
+    if (template->zombie) {
+        (void) ngx_stream_upstream_rr_peer_unref(peers, template);
+
+        ngx_stream_upstream_rr_peers_unlock(peers);
+
+        ngx_shmtx_lock(&peers->shpool->mutex);
+        ngx_slab_free_locked(peers->shpool, host->name.data);
+        ngx_slab_free_locked(peers->shpool, host);
+        ngx_shmtx_unlock(&peers->shpool->mutex);
+
+        ngx_resolve_name_done(ctx);
+
+        return;
+    }
+
+    now = ngx_time();
+
+    if (ctx->state) {
+        ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                      "%V could not be resolved (%i: %s)",
+                      &ctx->name, ctx->state,
+                      ngx_resolver_strerror(ctx->state));
+
+        if (ctx->state != NGX_RESOLVE_NXDOMAIN) {
+            ngx_stream_upstream_rr_peers_unlock(peers);
+
+            ngx_resolve_name_done(ctx);
+
+            ngx_add_timer(event, ngx_max(uscf->resolver_timeout, 1000));
+            return;
+        }
+
+        /* NGX_RESOLVE_NXDOMAIN */
+
+        ctx->naddrs = 0;
+    }
+
+#if (NGX_DEBUG)
+    {
+    u_char  text[NGX_SOCKADDR_STRLEN];
+    size_t  len;
+
+    for (i = 0; i < ctx->naddrs; i++) {
+        len = ngx_sock_ntop(ctx->addrs[i].sockaddr, ctx->addrs[i].socklen,
+                            text, NGX_SOCKADDR_STRLEN, 0);
+
+        ngx_log_debug3(NGX_LOG_DEBUG_STREAM, event->log, 0,
+                       "name %V was resolved to %*s", &host->name, len, text);
+    }
+    }
+#endif
+
+    for (peerp = &peers->peer; *peerp; /* void */ ) {
+        peer = *peerp;
+
+        if (peer->host != host) {
+            goto next;
+        }
+
+        for (j = 0; j < ctx->naddrs; j++) {
+
+            addr = &ctx->addrs[j];
+
+            if (addr->name.len == 0
+                && ngx_cmp_sockaddr(peer->sockaddr, peer->socklen,
+                                    addr->sockaddr, addr->socklen, 0)
+                   == NGX_OK)
+            {
+                addr->name.len = 1;
+                goto next;
+            }
+        }
+
+        *peerp = peer->next;
+        ngx_stream_upstream_zone_remove_peer_locked(peers, peer);
+
+        ngx_stream_upstream_zone_set_single(uscf);
+
+        continue;
+
+    next:
+
+        peerp = &peer->next;
+    }
+
+    for (i = 0; i < ctx->naddrs; i++) {
+
+        addr = &ctx->addrs[i];
+
+        if (addr->name.len == 1) {
+            addr->name.len = 0;
+            continue;
+        }
+
+        ngx_shmtx_lock(&peers->shpool->mutex);
+        peer = ngx_stream_upstream_zone_copy_peer(peers, NULL);
+        ngx_shmtx_unlock(&peers->shpool->mutex);
+        if (peer == NULL) {
+            ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                          "cannot add new server to upstream \"%V\", "
+                          "memory exhausted", peers->name);
+            break;
+        }
+
+        ngx_memcpy(peer->sockaddr, addr->sockaddr, addr->socklen);
+
+        port = ((struct sockaddr_in *) template->sockaddr)->sin_port;
+
+        switch (peer->sockaddr->sa_family) {
+#if (NGX_HAVE_INET6)
+        case AF_INET6:
+            ((struct sockaddr_in6 *) peer->sockaddr)->sin6_port = port;
+            break;
+#endif
+        default: /* AF_INET */
+            ((struct sockaddr_in *) peer->sockaddr)->sin_port = port;
+        }
+
+        peer->socklen = addr->socklen;
+
+        peer->name.len = ngx_sock_ntop(peer->sockaddr, peer->socklen,
+                                       peer->name.data, NGX_SOCKADDR_STRLEN, 1);
+
+        peer->host = template->host;
+        peer->server = template->server;
+
+        peer->weight = template->weight;
+        peer->effective_weight = peer->weight;
+        peer->max_conns = template->max_conns;
+        peer->max_fails = template->max_fails;
+        peer->fail_timeout = template->fail_timeout;
+        peer->down = template->down;
+
+        *peerp = peer;
+        peerp = &peer->next;
+
+        peers->number++;
+        peers->tries += (peer->down == 0);
+        peers->total_weight += peer->weight;
+        peers->weighted = (peers->total_weight != peers->number);
+        peer->id = (*peers->config)++;
+
+        ngx_stream_upstream_zone_set_single(uscf);
+    }
+
+    ngx_stream_upstream_rr_peers_unlock(peers);
+
+    timer = (ngx_msec_t) 1000 * (ctx->valid > now ? ctx->valid - now + 1 : 1);
+    timer = ngx_min(timer, uscf->resolver_timeout);
+
+    ngx_resolve_name_done(ctx);
+
+    ngx_add_timer(event, timer);
+}

From a.bavshin at nginx.com  Thu Jun 13 22:28:58 2024
From: a.bavshin at nginx.com (=?iso-8859-1?q?Aleksei_Bavshin?=)
Date: Thu, 13 Jun 2024 15:28:58 -0700
Subject: [PATCH 3 of 9] Upstream: construct upstream peers from DNS SRV records
In-Reply-To: <patchbomb.1718317735@fedora-wsl.local>
References: <patchbomb.1718317735@fedora-wsl.local>
Message-ID: <8b7fcded3983023229de.1718317738@fedora-wsl.local>

# HG changeset patch
# User Dmitry Volyntsev <xeioex at nginx.com>
# Date 1458229351 -10800
#      Thu Mar 17 18:42:31 2016 +0300
# Node ID 8b7fcded3983023229de1a6df5e2e0b857ee1bc9
# Parent  a2b1433abda29286189dad01fc3574df235e6aa8
Upstream: construct upstream peers from DNS SRV records.

diff --git a/src/http/modules/ngx_http_upstream_zone_module.c b/src/http/modules/ngx_http_upstream_zone_module.c
--- a/src/http/modules/ngx_http_upstream_zone_module.c
+++ b/src/http/modules/ngx_http_upstream_zone_module.c
@@ -361,6 +361,18 @@ ngx_http_upstream_zone_copy_peer(ngx_htt
             dst->host->name.len = src->host->name.len;
             ngx_memcpy(dst->host->name.data, src->host->name.data,
                        src->host->name.len);
+
+            if (src->host->service.len) {
+                dst->host->service.data = ngx_slab_alloc_locked(pool,
+                                                        src->host->service.len);
+                if (dst->host->service.data == NULL) {
+                    goto failed;
+                }
+
+                dst->host->service.len = src->host->service.len;
+                ngx_memcpy(dst->host->service.data, src->host->service.data,
+                           src->host->service.len);
+            }
         }
     }
 
@@ -369,6 +381,10 @@ ngx_http_upstream_zone_copy_peer(ngx_htt
 failed:
 
     if (dst->host) {
+        if (dst->host->name.data) {
+            ngx_slab_free_locked(pool, dst->host->name.data);
+        }
+
         ngx_slab_free_locked(pool, dst->host);
     }
 
@@ -533,6 +549,7 @@ ngx_http_upstream_zone_resolve_timer(ngx
     ctx->handler = ngx_http_upstream_zone_resolve_handler;
     ctx->data = host;
     ctx->timeout = uscf->resolver_timeout;
+    ctx->service = host->service;
     ctx->cancelable = 1;
 
     if (ngx_resolve_name(ctx) == NGX_OK) {
@@ -545,15 +562,28 @@ retry:
 }
 
 
+#define ngx_http_upstream_zone_addr_marked(addr)                              \
+    ((uintptr_t) (addr)->sockaddr & 1)
+
+#define ngx_http_upstream_zone_mark_addr(addr)                                \
+    (addr)->sockaddr = (struct sockaddr *) ((uintptr_t) (addr)->sockaddr | 1)
+
+#define ngx_http_upstream_zone_unmark_addr(addr)                              \
+    (addr)->sockaddr =                                                        \
+        (struct sockaddr *) ((uintptr_t) (addr)->sockaddr & ~((uintptr_t) 1))
+
 static void
 ngx_http_upstream_zone_resolve_handler(ngx_resolver_ctx_t *ctx)
 {
     time_t                         now;
+    u_short                        min_priority;
     in_port_t                      port;
+    ngx_str_t                     *server;
     ngx_msec_t                     timer;
-    ngx_uint_t                     i, j;
+    ngx_uint_t                     i, j, backup, addr_backup;
     ngx_event_t                   *event;
     ngx_resolver_addr_t           *addr;
+    ngx_resolver_srv_name_t       *srv;
     ngx_http_upstream_host_t      *host;
     ngx_http_upstream_rr_peer_t   *peer, *template, **peerp;
     ngx_http_upstream_rr_peers_t  *peers;
@@ -573,6 +603,11 @@ ngx_http_upstream_zone_resolve_handler(n
         ngx_http_upstream_rr_peers_unlock(peers);
 
         ngx_shmtx_lock(&peers->shpool->mutex);
+
+        if (host->service.len) {
+            ngx_slab_free_locked(peers->shpool, host->service.data);
+        }
+
         ngx_slab_free_locked(peers->shpool, host->name.data);
         ngx_slab_free_locked(peers->shpool, host);
         ngx_shmtx_unlock(&peers->shpool->mutex);
@@ -584,11 +619,32 @@ ngx_http_upstream_zone_resolve_handler(n
 
     now = ngx_time();
 
+    for (i = 0; i < ctx->nsrvs; i++) {
+        srv = &ctx->srvs[i];
+
+        if (srv->state) {
+            ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                          "%V could not be resolved (%i: %s) "
+                          "while resolving service %V of %V",
+                          &srv->name, srv->state,
+                          ngx_resolver_strerror(srv->state), &ctx->service,
+                          &ctx->name);
+        }
+    }
+
     if (ctx->state) {
-        ngx_log_error(NGX_LOG_ERR, event->log, 0,
-                      "%V could not be resolved (%i: %s)",
-                      &ctx->name, ctx->state,
-                      ngx_resolver_strerror(ctx->state));
+        if (ctx->service.len) {
+            ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                          "service %V of %V could not be resolved (%i: %s)",
+                          &ctx->service, &ctx->name, ctx->state,
+                          ngx_resolver_strerror(ctx->state));
+
+        } else {
+            ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                          "%V could not be resolved (%i: %s)",
+                          &ctx->name, ctx->state,
+                          ngx_resolver_strerror(ctx->state));
+        }
 
         if (ctx->state != NGX_RESOLVE_NXDOMAIN) {
             ngx_http_upstream_rr_peers_unlock(peers);
@@ -604,6 +660,13 @@ ngx_http_upstream_zone_resolve_handler(n
         ctx->naddrs = 0;
     }
 
+    backup = 0;
+    min_priority = 65535;
+
+    for (i = 0; i < ctx->naddrs; i++) {
+        min_priority = ngx_min(ctx->addrs[i].priority, min_priority);
+    }
+
 #if (NGX_DEBUG)
     {
     u_char  text[NGX_SOCKADDR_STRLEN];
@@ -611,14 +674,20 @@ ngx_http_upstream_zone_resolve_handler(n
 
     for (i = 0; i < ctx->naddrs; i++) {
         len = ngx_sock_ntop(ctx->addrs[i].sockaddr, ctx->addrs[i].socklen,
-                            text, NGX_SOCKADDR_STRLEN, 0);
+                            text, NGX_SOCKADDR_STRLEN, 1);
 
-        ngx_log_debug3(NGX_LOG_DEBUG_HTTP, event->log, 0,
-                       "name %V was resolved to %*s", &host->name, len, text);
+        ngx_log_debug7(NGX_LOG_DEBUG_HTTP, event->log, 0,
+                       "name %V was resolved to %*s "
+                       "s:\"%V\" n:\"%V\" w:%d %s",
+                       &host->name, len, text, &host->service,
+                       &ctx->addrs[i].name, ctx->addrs[i].weight,
+                       ctx->addrs[i].priority != min_priority ? "backup" : "");
     }
     }
 #endif
 
+again:
+
     for (peerp = &peers->peer; *peerp; /* void */ ) {
         peer = *peerp;
 
@@ -630,14 +699,39 @@ ngx_http_upstream_zone_resolve_handler(n
 
             addr = &ctx->addrs[j];
 
-            if (addr->name.len == 0
-                && ngx_cmp_sockaddr(peer->sockaddr, peer->socklen,
-                                    addr->sockaddr, addr->socklen, 0)
-                   == NGX_OK)
+            addr_backup = (addr->priority != min_priority);
+            if (addr_backup != backup) {
+                continue;
+            }
+
+            if (ngx_http_upstream_zone_addr_marked(addr)) {
+                continue;
+            }
+
+            if (ngx_cmp_sockaddr(peer->sockaddr, peer->socklen,
+                                 addr->sockaddr, addr->socklen,
+                                 host->service.len != 0)
+                != NGX_OK)
             {
-                addr->name.len = 1;
-                goto next;
+                continue;
             }
+
+            if (host->service.len) {
+                if (addr->name.len != peer->server.len
+                    || ngx_strncmp(addr->name.data, peer->server.data,
+                                   addr->name.len))
+                {
+                    continue;
+                }
+
+                if (template->weight == 1 && addr->weight != peer->weight) {
+                    continue;
+                }
+            }
+
+            ngx_http_upstream_zone_mark_addr(addr);
+
+            goto next;
         }
 
         *peerp = peer->next;
@@ -656,8 +750,13 @@ ngx_http_upstream_zone_resolve_handler(n
 
         addr = &ctx->addrs[i];
 
-        if (addr->name.len == 1) {
-            addr->name.len = 0;
+        addr_backup = (addr->priority != min_priority);
+        if (addr_backup != backup) {
+            continue;
+        }
+
+        if (ngx_http_upstream_zone_addr_marked(addr)) {
+            ngx_http_upstream_zone_unmark_addr(addr);
             continue;
         }
 
@@ -669,21 +768,14 @@ ngx_http_upstream_zone_resolve_handler(n
             ngx_log_error(NGX_LOG_ERR, event->log, 0,
                           "cannot add new server to upstream \"%V\", "
                           "memory exhausted", peers->name);
-            break;
+            goto done;
         }
 
         ngx_memcpy(peer->sockaddr, addr->sockaddr, addr->socklen);
 
-        port = ((struct sockaddr_in *) template->sockaddr)->sin_port;
-
-        switch (peer->sockaddr->sa_family) {
-#if (NGX_HAVE_INET6)
-        case AF_INET6:
-            ((struct sockaddr_in6 *) peer->sockaddr)->sin6_port = port;
-            break;
-#endif
-        default: /* AF_INET */
-            ((struct sockaddr_in *) peer->sockaddr)->sin_port = port;
+        if (host->service.len == 0) {
+            port = ngx_inet_get_port(template->sockaddr);
+            ngx_inet_set_port(peer->sockaddr, port);
         }
 
         peer->socklen = addr->socklen;
@@ -692,9 +784,30 @@ ngx_http_upstream_zone_resolve_handler(n
                                        peer->name.data, NGX_SOCKADDR_STRLEN, 1);
 
         peer->host = template->host;
-        peer->server = template->server;
+
+        server = host->service.len ? &addr->name : &template->server;
+
+        peer->server.data = ngx_slab_alloc(peers->shpool, server->len);
+        if (peer->server.data == NULL) {
+            ngx_http_upstream_rr_peer_free(peers, peer);
 
-        peer->weight = template->weight;
+            ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                          "cannot add new server to upstream \"%V\", "
+                          "memory exhausted", peers->name);
+            goto done;
+        }
+
+        peer->server.len = server->len;
+        ngx_memcpy(peer->server.data, server->data, server->len);
+
+        if (host->service.len == 0) {
+            peer->weight = template->weight;
+
+        } else {
+            peer->weight = (template->weight != 1 ? template->weight
+                                                  : addr->weight);
+        }
+
         peer->effective_weight = peer->weight;
         peer->max_conns = template->max_conns;
         peer->max_fails = template->max_fails;
@@ -713,8 +826,25 @@ ngx_http_upstream_zone_resolve_handler(n
         ngx_http_upstream_zone_set_single(uscf);
     }
 
+    if (host->service.len && peers->next) {
+        ngx_http_upstream_rr_peers_unlock(peers);
+
+        peers = peers->next;
+        backup = 1;
+
+        ngx_http_upstream_rr_peers_wlock(peers);
+
+        goto again;
+    }
+
+done:
+
     ngx_http_upstream_rr_peers_unlock(peers);
 
+    while (++i < ctx->naddrs) {
+        ngx_http_upstream_zone_unmark_addr(&ctx->addrs[i]);
+    }
+
     timer = (ngx_msec_t) 1000 * (ctx->valid > now ? ctx->valid - now + 1 : 1);
     timer = ngx_min(timer, uscf->resolver_timeout);
 
diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -6310,6 +6310,19 @@ ngx_http_upstream_server(ngx_conf_t *cf,
             resolve = 1;
             continue;
         }
+
+        if (ngx_strncmp(value[i].data, "service=", 8) == 0) {
+
+            us->service.len = value[i].len - 8;
+            us->service.data = &value[i].data[8];
+
+            if (us->service.len == 0) {
+                ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "service is empty");
+                return NGX_CONF_ERROR;
+            }
+
+            continue;
+        }
 #endif
 
         goto invalid;
@@ -6325,6 +6338,15 @@ ngx_http_upstream_server(ngx_conf_t *cf,
         /* resolve at run time */
         u.no_resolve = 1;
     }
+
+    if (us->service.len && !resolve) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "service upstream \"%V\" requires "
+                           "\"resolve\" parameter",
+                           &u.url);
+        return NGX_CONF_ERROR;
+    }
+
 #endif
 
     if (ngx_parse_url(cf->pool, &u) != NGX_OK) {
@@ -6340,6 +6362,22 @@ ngx_http_upstream_server(ngx_conf_t *cf,
 
 #if (NGX_HTTP_UPSTREAM_ZONE)
 
+    if (us->service.len && !u.no_port) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "service upstream \"%V\" may not have port",
+                           &us->name);
+
+        return NGX_CONF_ERROR;
+    }
+
+    if (us->service.len && u.naddrs) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "service upstream \"%V\" requires domain name",
+                           &us->name);
+
+        return NGX_CONF_ERROR;
+    }
+
     if (resolve && u.naddrs == 0) {
         ngx_addr_t  *addr;
 
diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h
+++ b/src/http/ngx_http_upstream.h
@@ -106,9 +106,10 @@ typedef struct {
 
 #if (NGX_HTTP_UPSTREAM_ZONE)
     ngx_str_t                        host;
+    ngx_str_t                        service;
 #endif
 
-    NGX_COMPAT_BEGIN(4)
+    NGX_COMPAT_BEGIN(2)
     NGX_COMPAT_END
 } ngx_http_upstream_server_t;
 
diff --git a/src/http/ngx_http_upstream_round_robin.c b/src/http/ngx_http_upstream_round_robin.c
--- a/src/http/ngx_http_upstream_round_robin.c
+++ b/src/http/ngx_http_upstream_round_robin.c
@@ -187,6 +187,7 @@ ngx_http_upstream_init_round_robin(ngx_c
                 }
 
                 peer[n].host->name = server[i].host;
+                peer[n].host->service = server[i].service;
 
                 peer[n].sockaddr = server[i].addrs[0].sockaddr;
                 peer[n].socklen = server[i].addrs[0].socklen;
@@ -319,6 +320,7 @@ ngx_http_upstream_init_round_robin(ngx_c
                 }
 
                 peer[n].host->name = server[i].host;
+                peer[n].host->service = server[i].service;
 
                 peer[n].sockaddr = server[i].addrs[0].sockaddr;
                 peer[n].socklen = server[i].addrs[0].socklen;
diff --git a/src/http/ngx_http_upstream_round_robin.h b/src/http/ngx_http_upstream_round_robin.h
--- a/src/http/ngx_http_upstream_round_robin.h
+++ b/src/http/ngx_http_upstream_round_robin.h
@@ -24,6 +24,7 @@ typedef struct {
     ngx_event_t                     event;         /* must be first */
     ngx_uint_t                      worker;
     ngx_str_t                       name;
+    ngx_str_t                       service;
     ngx_http_upstream_rr_peers_t   *peers;
     ngx_http_upstream_rr_peer_t    *peer;
 } ngx_http_upstream_host_t;
@@ -156,7 +157,7 @@ ngx_http_upstream_rr_peer_free_locked(ng
     ngx_slab_free_locked(peers->shpool, peer->sockaddr);
     ngx_slab_free_locked(peers->shpool, peer->name.data);
 
-    if (peer->server.data && (peer->host == NULL || peer->host->peer == peer)) {
+    if (peer->server.data) {
         ngx_slab_free_locked(peers->shpool, peer->server.data);
     }
 
diff --git a/src/stream/ngx_stream_upstream.c b/src/stream/ngx_stream_upstream.c
--- a/src/stream/ngx_stream_upstream.c
+++ b/src/stream/ngx_stream_upstream.c
@@ -527,6 +527,19 @@ ngx_stream_upstream_server(ngx_conf_t *c
             resolve = 1;
             continue;
         }
+
+        if (ngx_strncmp(value[i].data, "service=", 8) == 0) {
+
+            us->service.len = value[i].len - 8;
+            us->service.data = &value[i].data[8];
+
+            if (us->service.len == 0) {
+                ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "service is empty");
+                return NGX_CONF_ERROR;
+            }
+
+            continue;
+        }
 #endif
 
         goto invalid;
@@ -541,6 +554,15 @@ ngx_stream_upstream_server(ngx_conf_t *c
         /* resolve at run time */
         u.no_resolve = 1;
     }
+
+    if (us->service.len && !resolve) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "service upstream \"%V\" requires "
+                           "\"resolve\" parameter",
+                           &u.url);
+        return NGX_CONF_ERROR;
+    }
+
 #endif
 
     if (ngx_parse_url(cf->pool, &u) != NGX_OK) {
@@ -552,7 +574,12 @@ ngx_stream_upstream_server(ngx_conf_t *c
         return NGX_CONF_ERROR;
     }
 
-    if (u.no_port) {
+    if (u.no_port
+#if (NGX_STREAM_UPSTREAM_ZONE)
+        && us->service.len == 0
+#endif
+        )
+    {
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "no port in upstream \"%V\"", &u.url);
         return NGX_CONF_ERROR;
@@ -562,6 +589,22 @@ ngx_stream_upstream_server(ngx_conf_t *c
 
 #if (NGX_STREAM_UPSTREAM_ZONE)
 
+    if (us->service.len && !u.no_port) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "service upstream \"%V\" may not have port",
+                           &us->name);
+
+        return NGX_CONF_ERROR;
+    }
+
+    if (us->service.len && u.naddrs) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "service upstream \"%V\" requires domain name",
+                           &us->name);
+
+        return NGX_CONF_ERROR;
+    }
+
     if (resolve && u.naddrs == 0) {
         ngx_addr_t  *addr;
 
diff --git a/src/stream/ngx_stream_upstream.h b/src/stream/ngx_stream_upstream.h
--- a/src/stream/ngx_stream_upstream.h
+++ b/src/stream/ngx_stream_upstream.h
@@ -65,10 +65,8 @@ typedef struct {
 
 #if (NGX_STREAM_UPSTREAM_ZONE)
     ngx_str_t                          host;
+    ngx_str_t                          service;
 #endif
-
-    NGX_COMPAT_BEGIN(2)
-    NGX_COMPAT_END
 } ngx_stream_upstream_server_t;
 
 
diff --git a/src/stream/ngx_stream_upstream_round_robin.c b/src/stream/ngx_stream_upstream_round_robin.c
--- a/src/stream/ngx_stream_upstream_round_robin.c
+++ b/src/stream/ngx_stream_upstream_round_robin.c
@@ -194,6 +194,7 @@ ngx_stream_upstream_init_round_robin(ngx
                 }
 
                 peer[n].host->name = server[i].host;
+                peer[n].host->service = server[i].service;
 
                 peer[n].sockaddr = server[i].addrs[0].sockaddr;
                 peer[n].socklen = server[i].addrs[0].socklen;
@@ -325,6 +326,7 @@ ngx_stream_upstream_init_round_robin(ngx
                 }
 
                 peer[n].host->name = server[i].host;
+                peer[n].host->service = server[i].service;
 
                 peer[n].sockaddr = server[i].addrs[0].sockaddr;
                 peer[n].socklen = server[i].addrs[0].socklen;
diff --git a/src/stream/ngx_stream_upstream_round_robin.h b/src/stream/ngx_stream_upstream_round_robin.h
--- a/src/stream/ngx_stream_upstream_round_robin.h
+++ b/src/stream/ngx_stream_upstream_round_robin.h
@@ -24,6 +24,7 @@ typedef struct {
     ngx_event_t                      event;         /* must be first */
     ngx_uint_t                       worker;
     ngx_str_t                        name;
+    ngx_str_t                        service;
     ngx_stream_upstream_rr_peers_t  *peers;
     ngx_stream_upstream_rr_peer_t   *peer;
 } ngx_stream_upstream_host_t;
@@ -154,7 +155,7 @@ ngx_stream_upstream_rr_peer_free_locked(
     ngx_slab_free_locked(peers->shpool, peer->sockaddr);
     ngx_slab_free_locked(peers->shpool, peer->name.data);
 
-    if (peer->server.data && (peer->host == NULL || peer->host->peer == peer)) {
+    if (peer->server.data) {
         ngx_slab_free_locked(peers->shpool, peer->server.data);
     }
 
diff --git a/src/stream/ngx_stream_upstream_zone_module.c b/src/stream/ngx_stream_upstream_zone_module.c
--- a/src/stream/ngx_stream_upstream_zone_module.c
+++ b/src/stream/ngx_stream_upstream_zone_module.c
@@ -358,6 +358,18 @@ ngx_stream_upstream_zone_copy_peer(ngx_s
             dst->host->name.len = src->host->name.len;
             ngx_memcpy(dst->host->name.data, src->host->name.data,
                        src->host->name.len);
+
+            if (src->host->service.len) {
+                dst->host->service.data = ngx_slab_alloc_locked(pool,
+                                                        src->host->service.len);
+                if (dst->host->service.data == NULL) {
+                    goto failed;
+                }
+
+                dst->host->service.len = src->host->service.len;
+                ngx_memcpy(dst->host->service.data, src->host->service.data,
+                           src->host->service.len);
+            }
         }
     }
 
@@ -366,6 +378,10 @@ ngx_stream_upstream_zone_copy_peer(ngx_s
 failed:
 
     if (dst->host) {
+        if (dst->host->name.data) {
+            ngx_slab_free_locked(pool, dst->host->name.data);
+        }
+
         ngx_slab_free_locked(pool, dst->host);
     }
 
@@ -531,6 +547,7 @@ ngx_stream_upstream_zone_resolve_timer(n
     ctx->handler = ngx_stream_upstream_zone_resolve_handler;
     ctx->data = host;
     ctx->timeout = uscf->resolver_timeout;
+    ctx->service = host->service;
     ctx->cancelable = 1;
 
     if (ngx_resolve_name(ctx) == NGX_OK) {
@@ -543,15 +560,28 @@ retry:
 }
 
 
+#define ngx_stream_upstream_zone_addr_marked(addr)                            \
+    ((uintptr_t) (addr)->sockaddr & 1)
+
+#define ngx_stream_upstream_zone_mark_addr(addr)                              \
+    (addr)->sockaddr = (struct sockaddr *) ((uintptr_t) (addr)->sockaddr | 1)
+
+#define ngx_stream_upstream_zone_unmark_addr(addr)                            \
+    (addr)->sockaddr =                                                        \
+        (struct sockaddr *) ((uintptr_t) (addr)->sockaddr & ~((uintptr_t) 1))
+
 static void
 ngx_stream_upstream_zone_resolve_handler(ngx_resolver_ctx_t *ctx)
 {
     time_t                           now;
+    u_short                          min_priority;
     in_port_t                        port;
+    ngx_str_t                       *server;
     ngx_msec_t                       timer;
-    ngx_uint_t                       i, j;
+    ngx_uint_t                       i, j, backup, addr_backup;
     ngx_event_t                     *event;
     ngx_resolver_addr_t             *addr;
+    ngx_resolver_srv_name_t         *srv;
     ngx_stream_upstream_host_t      *host;
     ngx_stream_upstream_rr_peer_t   *peer, *template, **peerp;
     ngx_stream_upstream_rr_peers_t  *peers;
@@ -571,6 +601,11 @@ ngx_stream_upstream_zone_resolve_handler
         ngx_stream_upstream_rr_peers_unlock(peers);
 
         ngx_shmtx_lock(&peers->shpool->mutex);
+
+        if (host->service.len) {
+            ngx_slab_free_locked(peers->shpool, host->service.data);
+        }
+
         ngx_slab_free_locked(peers->shpool, host->name.data);
         ngx_slab_free_locked(peers->shpool, host);
         ngx_shmtx_unlock(&peers->shpool->mutex);
@@ -582,11 +617,32 @@ ngx_stream_upstream_zone_resolve_handler
 
     now = ngx_time();
 
+    for (i = 0; i < ctx->nsrvs; i++) {
+        srv = &ctx->srvs[i];
+
+        if (srv->state) {
+            ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                          "%V could not be resolved (%i: %s) "
+                          "while resolving service %V of %V",
+                          &srv->name, srv->state,
+                          ngx_resolver_strerror(srv->state), &ctx->service,
+                          &ctx->name);
+        }
+    }
+
     if (ctx->state) {
-        ngx_log_error(NGX_LOG_ERR, event->log, 0,
-                      "%V could not be resolved (%i: %s)",
-                      &ctx->name, ctx->state,
-                      ngx_resolver_strerror(ctx->state));
+        if (ctx->service.len) {
+            ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                          "service %V of %V could not be resolved (%i: %s)",
+                          &ctx->service, &ctx->name, ctx->state,
+                          ngx_resolver_strerror(ctx->state));
+
+        } else {
+            ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                          "%V could not be resolved (%i: %s)",
+                          &ctx->name, ctx->state,
+                          ngx_resolver_strerror(ctx->state));
+        }
 
         if (ctx->state != NGX_RESOLVE_NXDOMAIN) {
             ngx_stream_upstream_rr_peers_unlock(peers);
@@ -602,6 +658,13 @@ ngx_stream_upstream_zone_resolve_handler
         ctx->naddrs = 0;
     }
 
+    backup = 0;
+    min_priority = 65535;
+
+    for (i = 0; i < ctx->naddrs; i++) {
+        min_priority = ngx_min(ctx->addrs[i].priority, min_priority);
+    }
+
 #if (NGX_DEBUG)
     {
     u_char  text[NGX_SOCKADDR_STRLEN];
@@ -609,14 +672,20 @@ ngx_stream_upstream_zone_resolve_handler
 
     for (i = 0; i < ctx->naddrs; i++) {
         len = ngx_sock_ntop(ctx->addrs[i].sockaddr, ctx->addrs[i].socklen,
-                            text, NGX_SOCKADDR_STRLEN, 0);
+                            text, NGX_SOCKADDR_STRLEN, 1);
 
-        ngx_log_debug3(NGX_LOG_DEBUG_STREAM, event->log, 0,
-                       "name %V was resolved to %*s", &host->name, len, text);
+        ngx_log_debug7(NGX_LOG_DEBUG_STREAM, event->log, 0,
+                       "name %V was resolved to %*s "
+                       "s:\"%V\" n:\"%V\" w:%d %s",
+                       &host->name, len, text, &host->service,
+                       &ctx->addrs[i].name, ctx->addrs[i].weight,
+                       ctx->addrs[i].priority != min_priority ? "backup" : "");
     }
     }
 #endif
 
+again:
+
     for (peerp = &peers->peer; *peerp; /* void */ ) {
         peer = *peerp;
 
@@ -628,14 +697,39 @@ ngx_stream_upstream_zone_resolve_handler
 
             addr = &ctx->addrs[j];
 
-            if (addr->name.len == 0
-                && ngx_cmp_sockaddr(peer->sockaddr, peer->socklen,
-                                    addr->sockaddr, addr->socklen, 0)
-                   == NGX_OK)
+            addr_backup = (addr->priority != min_priority);
+            if (addr_backup != backup) {
+                continue;
+            }
+
+            if (ngx_stream_upstream_zone_addr_marked(addr)) {
+                continue;
+            }
+
+            if (ngx_cmp_sockaddr(peer->sockaddr, peer->socklen,
+                                 addr->sockaddr, addr->socklen,
+                                 host->service.len != 0)
+                != NGX_OK)
             {
-                addr->name.len = 1;
-                goto next;
+                continue;
             }
+
+            if (host->service.len) {
+                if (addr->name.len != peer->server.len
+                    || ngx_strncmp(addr->name.data, peer->server.data,
+                                   addr->name.len))
+                {
+                    continue;
+                }
+
+                if (template->weight == 1 && addr->weight != peer->weight) {
+                    continue;
+                }
+            }
+
+            ngx_stream_upstream_zone_mark_addr(addr);
+
+            goto next;
         }
 
         *peerp = peer->next;
@@ -654,33 +748,32 @@ ngx_stream_upstream_zone_resolve_handler
 
         addr = &ctx->addrs[i];
 
-        if (addr->name.len == 1) {
-            addr->name.len = 0;
+        addr_backup = (addr->priority != min_priority);
+        if (addr_backup != backup) {
+            continue;
+        }
+
+        if (ngx_stream_upstream_zone_addr_marked(addr)) {
+            ngx_stream_upstream_zone_unmark_addr(addr);
             continue;
         }
 
         ngx_shmtx_lock(&peers->shpool->mutex);
         peer = ngx_stream_upstream_zone_copy_peer(peers, NULL);
         ngx_shmtx_unlock(&peers->shpool->mutex);
+
         if (peer == NULL) {
             ngx_log_error(NGX_LOG_ERR, event->log, 0,
                           "cannot add new server to upstream \"%V\", "
                           "memory exhausted", peers->name);
-            break;
+            goto done;
         }
 
         ngx_memcpy(peer->sockaddr, addr->sockaddr, addr->socklen);
 
-        port = ((struct sockaddr_in *) template->sockaddr)->sin_port;
-
-        switch (peer->sockaddr->sa_family) {
-#if (NGX_HAVE_INET6)
-        case AF_INET6:
-            ((struct sockaddr_in6 *) peer->sockaddr)->sin6_port = port;
-            break;
-#endif
-        default: /* AF_INET */
-            ((struct sockaddr_in *) peer->sockaddr)->sin_port = port;
+        if (host->service.len == 0) {
+            port = ngx_inet_get_port(template->sockaddr);
+            ngx_inet_set_port(peer->sockaddr, port);
         }
 
         peer->socklen = addr->socklen;
@@ -689,9 +782,30 @@ ngx_stream_upstream_zone_resolve_handler
                                        peer->name.data, NGX_SOCKADDR_STRLEN, 1);
 
         peer->host = template->host;
-        peer->server = template->server;
+
+        server = host->service.len ? &addr->name : &template->server;
+
+        peer->server.data = ngx_slab_alloc(peers->shpool, server->len);
+        if (peer->server.data == NULL) {
+            ngx_stream_upstream_rr_peer_free(peers, peer);
 
-        peer->weight = template->weight;
+            ngx_log_error(NGX_LOG_ERR, event->log, 0,
+                          "cannot add new server to upstream \"%V\", "
+                          "memory exhausted", peers->name);
+            goto done;
+        }
+
+        peer->server.len = server->len;
+        ngx_memcpy(peer->server.data, server->data, server->len);
+
+        if (host->service.len == 0) {
+            peer->weight = template->weight;
+
+        } else {
+            peer->weight = (template->weight != 1 ? template->weight
+                                                  : addr->weight);
+        }
+
         peer->effective_weight = peer->weight;
         peer->max_conns = template->max_conns;
         peer->max_fails = template->max_fails;
@@ -710,8 +824,25 @@ ngx_stream_upstream_zone_resolve_handler
         ngx_stream_upstream_zone_set_single(uscf);
     }
 
+    if (host->service.len && peers->next) {
+        ngx_stream_upstream_rr_peers_unlock(peers);
+
+        peers = peers->next;
+        backup = 1;
+
+        ngx_stream_upstream_rr_peers_wlock(peers);
+
+        goto again;
+    }
+
+done:
+
     ngx_stream_upstream_rr_peers_unlock(peers);
 
+    while (++i < ctx->naddrs) {
+        ngx_stream_upstream_zone_unmark_addr(&ctx->addrs[i]);
+    }
+
     timer = (ngx_msec_t) 1000 * (ctx->valid > now ? ctx->valid - now + 1 : 1);
     timer = ngx_min(timer, uscf->resolver_timeout);
 

From a.bavshin at nginx.com  Thu Jun 13 22:28:59 2024
From: a.bavshin at nginx.com (=?iso-8859-1?q?Aleksei_Bavshin?=)
Date: Thu, 13 Jun 2024 15:28:59 -0700
Subject: [PATCH 4 of 9] Core: inheritance of non-reusable shared memory zones
In-Reply-To: <patchbomb.1718317735@fedora-wsl.local>
References: <patchbomb.1718317735@fedora-wsl.local>
Message-ID: <90ef6f74980d5e46c95a.1718317739@fedora-wsl.local>

# HG changeset patch
# User Ruslan Ermilov <ru at nginx.com>
# Date 1509736941 -10800
#      Fri Nov 03 22:22:21 2017 +0300
# Node ID 90ef6f74980d5e46c95aa32375a58bb8eb56122e
# Parent  8b7fcded3983023229de1a6df5e2e0b857ee1bc9
Core: inheritance of non-reusable shared memory zones.

When re-creating a non-reusable zone, make the pointer to the old zone
available during the new zone initialization.

diff --git a/src/core/ngx_cycle.c b/src/core/ngx_cycle.c
--- a/src/core/ngx_cycle.c
+++ b/src/core/ngx_cycle.c
@@ -38,7 +38,7 @@ static ngx_connection_t  dumb;
 ngx_cycle_t *
 ngx_init_cycle(ngx_cycle_t *old_cycle)
 {
-    void                *rv;
+    void                *rv, *data;
     char               **senv;
     ngx_uint_t           i, n;
     ngx_log_t           *log;
@@ -438,6 +438,8 @@ ngx_init_cycle(ngx_cycle_t *old_cycle)
         opart = &old_cycle->shared_memory.part;
         oshm_zone = opart->elts;
 
+        data = NULL;
+
         for (n = 0; /* void */ ; n++) {
 
             if (n >= opart->nelts) {
@@ -461,9 +463,13 @@ ngx_init_cycle(ngx_cycle_t *old_cycle)
                 continue;
             }
 
+            if (shm_zone[i].tag == oshm_zone[n].tag && shm_zone[i].noreuse) {
+                data = oshm_zone[n].data;
+                break;
+            }
+
             if (shm_zone[i].tag == oshm_zone[n].tag
-                && shm_zone[i].shm.size == oshm_zone[n].shm.size
-                && !shm_zone[i].noreuse)
+                && shm_zone[i].shm.size == oshm_zone[n].shm.size)
             {
                 shm_zone[i].shm.addr = oshm_zone[n].shm.addr;
 #if (NGX_WIN32)
@@ -490,7 +496,7 @@ ngx_init_cycle(ngx_cycle_t *old_cycle)
             goto failed;
         }
 
-        if (shm_zone[i].init(&shm_zone[i], NULL) != NGX_OK) {
+        if (shm_zone[i].init(&shm_zone[i], data) != NGX_OK) {
             goto failed;
         }
 

From a.bavshin at nginx.com  Thu Jun 13 22:29:00 2024
From: a.bavshin at nginx.com (=?iso-8859-1?q?Aleksei_Bavshin?=)
Date: Thu, 13 Jun 2024 15:29:00 -0700
Subject: [PATCH 5 of 9] Upstream: pre-resolve servers on reload
In-Reply-To: <patchbomb.1718317735@fedora-wsl.local>
References: <patchbomb.1718317735@fedora-wsl.local>
Message-ID: <238c1695d3b7450159ba.1718317740@fedora-wsl.local>

# HG changeset patch
# User Ruslan Ermilov <ru at nginx.com>
# Date 1509736943 -10800
#      Fri Nov 03 22:22:23 2017 +0300
# Node ID 238c1695d3b7450159ba0c03509584683bf25f9b
# Parent  90ef6f74980d5e46c95aa32375a58bb8eb56122e
Upstream: pre-resolve servers on reload.

After configuration is reloaded, it may take some time for the
re-resolvable upstream servers to resolve and become available
as peers.  During this time, client requests might get dropped.

Such servers are now pre-resolved using the "cache" of already
resolved peers from the old shared memory zone.

diff --git a/src/http/modules/ngx_http_upstream_zone_module.c b/src/http/modules/ngx_http_upstream_zone_module.c
--- a/src/http/modules/ngx_http_upstream_zone_module.c
+++ b/src/http/modules/ngx_http_upstream_zone_module.c
@@ -15,9 +15,15 @@ static char *ngx_http_upstream_zone(ngx_
 static ngx_int_t ngx_http_upstream_init_zone(ngx_shm_zone_t *shm_zone,
     void *data);
 static ngx_http_upstream_rr_peers_t *ngx_http_upstream_zone_copy_peers(
-    ngx_slab_pool_t *shpool, ngx_http_upstream_srv_conf_t *uscf);
+    ngx_slab_pool_t *shpool, ngx_http_upstream_srv_conf_t *uscf,
+    ngx_http_upstream_srv_conf_t *ouscf);
 static ngx_http_upstream_rr_peer_t *ngx_http_upstream_zone_copy_peer(
     ngx_http_upstream_rr_peers_t *peers, ngx_http_upstream_rr_peer_t *src);
+static ngx_int_t ngx_http_upstream_zone_preresolve(
+    ngx_http_upstream_rr_peer_t *resolve,
+    ngx_http_upstream_rr_peers_t *peers,
+    ngx_http_upstream_rr_peer_t *oresolve,
+    ngx_http_upstream_rr_peers_t *opeers);
 static void ngx_http_upstream_zone_set_single(
     ngx_http_upstream_srv_conf_t *uscf);
 static void ngx_http_upstream_zone_remove_peer_locked(
@@ -130,11 +136,11 @@ static ngx_int_t
 ngx_http_upstream_init_zone(ngx_shm_zone_t *shm_zone, void *data)
 {
     size_t                          len;
-    ngx_uint_t                      i;
+    ngx_uint_t                      i, j;
     ngx_slab_pool_t                *shpool;
     ngx_http_upstream_rr_peers_t   *peers, **peersp;
-    ngx_http_upstream_srv_conf_t   *uscf, **uscfp;
-    ngx_http_upstream_main_conf_t  *umcf;
+    ngx_http_upstream_srv_conf_t   *uscf, *ouscf, **uscfp, **ouscfp;
+    ngx_http_upstream_main_conf_t  *umcf, *oumcf;
 
     shpool = (ngx_slab_pool_t *) shm_zone->shm.addr;
     umcf = shm_zone->data;
@@ -171,6 +177,7 @@ ngx_http_upstream_init_zone(ngx_shm_zone
     /* copy peers to shared memory */
 
     peersp = (ngx_http_upstream_rr_peers_t **) (void *) &shpool->data;
+    oumcf = data;
 
     for (i = 0; i < umcf->upstreams.nelts; i++) {
         uscf = uscfp[i];
@@ -179,7 +186,38 @@ ngx_http_upstream_init_zone(ngx_shm_zone
             continue;
         }
 
-        peers = ngx_http_upstream_zone_copy_peers(shpool, uscf);
+        ouscf = NULL;
+
+        if (oumcf) {
+            ouscfp = oumcf->upstreams.elts;
+
+            for (j = 0; j < oumcf->upstreams.nelts; j++) {
+
+                 if (ouscfp[j]->shm_zone == NULL) {
+                     continue;
+                 }
+
+                 if (ouscfp[j]->shm_zone->shm.name.len != shm_zone->shm.name.len
+                     || ngx_memcmp(ouscfp[j]->shm_zone->shm.name.data,
+                                   shm_zone->shm.name.data,
+                                   shm_zone->shm.name.len)
+                        != 0)
+                 {
+                     continue;
+                 }
+
+                 if (ouscfp[j]->host.len == uscf->host.len
+                     && ngx_memcmp(ouscfp[j]->host.data, uscf->host.data,
+                                   uscf->host.len)
+                        == 0)
+                 {
+                     ouscf = ouscfp[j];
+                     break;
+                 }
+            }
+        }
+
+        peers = ngx_http_upstream_zone_copy_peers(shpool, uscf, ouscf);
         if (peers == NULL) {
             return NGX_ERROR;
         }
@@ -194,12 +232,14 @@ ngx_http_upstream_init_zone(ngx_shm_zone
 
 static ngx_http_upstream_rr_peers_t *
 ngx_http_upstream_zone_copy_peers(ngx_slab_pool_t *shpool,
-    ngx_http_upstream_srv_conf_t *uscf)
+    ngx_http_upstream_srv_conf_t *uscf, ngx_http_upstream_srv_conf_t *ouscf)
 {
     ngx_str_t                     *name;
     ngx_uint_t                    *config;
     ngx_http_upstream_rr_peer_t   *peer, **peerp;
-    ngx_http_upstream_rr_peers_t  *peers, *backup;
+    ngx_http_upstream_rr_peers_t  *peers, *opeers, *backup;
+
+    opeers = (ouscf ? ouscf->peer.data : NULL);
 
     config = ngx_slab_calloc(shpool, sizeof(ngx_uint_t));
     if (config == NULL) {
@@ -252,6 +292,16 @@ ngx_http_upstream_zone_copy_peers(ngx_sl
         peer->id = (*peers->config)++;
     }
 
+    if (opeers) {
+
+        if (ngx_http_upstream_zone_preresolve(peers->resolve, peers,
+                                              opeers->resolve, opeers)
+            != NGX_OK)
+        {
+            return NULL;
+        }
+    }
+
     if (peers->next == NULL) {
         goto done;
     }
@@ -291,10 +341,30 @@ ngx_http_upstream_zone_copy_peers(ngx_sl
 
     peers->next = backup;
 
+    if (opeers && opeers->next) {
+
+        if (ngx_http_upstream_zone_preresolve(peers->resolve, backup,
+                                              opeers->resolve, opeers->next)
+            != NGX_OK)
+        {
+            return NULL;
+        }
+
+        if (ngx_http_upstream_zone_preresolve(backup->resolve, backup,
+                                              opeers->next->resolve,
+                                              opeers->next)
+            != NGX_OK)
+        {
+            return NULL;
+        }
+    }
+
 done:
 
     uscf->peer.data = peers;
 
+    ngx_http_upstream_zone_set_single(uscf);
+
     return peers;
 }
 
@@ -406,6 +476,123 @@ failed:
 }
 
 
+static ngx_int_t
+ngx_http_upstream_zone_preresolve(ngx_http_upstream_rr_peer_t *resolve,
+    ngx_http_upstream_rr_peers_t *peers,
+    ngx_http_upstream_rr_peer_t *oresolve,
+    ngx_http_upstream_rr_peers_t *opeers)
+{
+    in_port_t                     port;
+    ngx_str_t                    *server;
+    ngx_http_upstream_host_t     *host;
+    ngx_http_upstream_rr_peer_t  *peer, *template, *opeer, **peerp;
+
+    if (resolve == NULL || oresolve == NULL) {
+        return NGX_OK;
+    }
+
+    for (peerp = &peers->peer; *peerp; peerp = &(*peerp)->next) {
+        /* void */
+    }
+
+    ngx_http_upstream_rr_peers_rlock(opeers);
+
+    for (template = resolve; template; template = template->next) {
+        for (opeer = oresolve; opeer; opeer = opeer->next) {
+
+            if (opeer->host->name.len != template->host->name.len
+                || ngx_memcmp(opeer->host->name.data,
+                              template->host->name.data,
+                              template->host->name.len)
+                   != 0)
+            {
+                continue;
+            }
+
+            if (opeer->host->service.len != template->host->service.len
+                || ngx_memcmp(opeer->host->service.data,
+                              template->host->service.data,
+                              template->host->service.len)
+                   != 0)
+            {
+                continue;
+            }
+
+            host = opeer->host;
+
+            for (opeer = opeers->peer; opeer; opeer = opeer->next) {
+
+                if (opeer->host != host) {
+                    continue;
+                }
+
+                peer = ngx_http_upstream_zone_copy_peer(peers, NULL);
+                if (peer == NULL) {
+                    ngx_http_upstream_rr_peers_unlock(opeers);
+                    return NGX_ERROR;
+                }
+
+                ngx_memcpy(peer->sockaddr, opeer->sockaddr, opeer->socklen);
+
+                if (template->host->service.len == 0) {
+                    port = ngx_inet_get_port(template->sockaddr);
+                    ngx_inet_set_port(peer->sockaddr, port);
+                }
+
+                peer->socklen = opeer->socklen;
+
+                peer->name.len = ngx_sock_ntop(peer->sockaddr, peer->socklen,
+                                               peer->name.data,
+                                               NGX_SOCKADDR_STRLEN, 1);
+
+                peer->host = template->host;
+
+                server = template->host->service.len ? &opeer->server
+                                                     : &template->server;
+
+                peer->server.data = ngx_slab_alloc(peers->shpool, server->len);
+                if (peer->server.data == NULL) {
+                    ngx_http_upstream_rr_peers_unlock(opeers);
+                    return NGX_ERROR;
+                }
+
+                ngx_memcpy(peer->server.data, server->data, server->len);
+                peer->server.len = server->len;
+
+                if (host->service.len == 0) {
+                    peer->weight = template->weight;
+
+                } else {
+                    peer->weight = (template->weight != 1 ? template->weight
+                                                          : opeer->weight);
+                }
+
+                peer->effective_weight = peer->weight;
+                peer->max_conns = template->max_conns;
+                peer->max_fails = template->max_fails;
+                peer->fail_timeout = template->fail_timeout;
+                peer->down = template->down;
+
+                peer->id = (*peers->config)++;
+
+                *peerp = peer;
+                peerp = &peer->next;
+
+                peers->number++;
+                peers->tries += (peer->down == 0);
+                peers->total_weight += peer->weight;
+                peers->weighted = (peers->total_weight != peers->number);
+            }
+
+            break;
+        }
+    }
+
+    ngx_http_upstream_rr_peers_unlock(opeers);
+    return NGX_OK;
+}
+
+
 static void
 ngx_http_upstream_zone_set_single(ngx_http_upstream_srv_conf_t *uscf)
 {
diff --git a/src/stream/ngx_stream_upstream_zone_module.c b/src/stream/ngx_stream_upstream_zone_module.c
--- a/src/stream/ngx_stream_upstream_zone_module.c
+++ b/src/stream/ngx_stream_upstream_zone_module.c
@@ -15,9 +15,15 @@ static char *ngx_stream_upstream_zone(ng
 static ngx_int_t ngx_stream_upstream_init_zone(ngx_shm_zone_t *shm_zone,
     void *data);
 static ngx_stream_upstream_rr_peers_t *ngx_stream_upstream_zone_copy_peers(
-    ngx_slab_pool_t *shpool, ngx_stream_upstream_srv_conf_t *uscf);
+    ngx_slab_pool_t *shpool, ngx_stream_upstream_srv_conf_t *uscf,
+    ngx_stream_upstream_srv_conf_t *ouscf);
 static ngx_stream_upstream_rr_peer_t *ngx_stream_upstream_zone_copy_peer(
     ngx_stream_upstream_rr_peers_t *peers, ngx_stream_upstream_rr_peer_t *src);
+static ngx_int_t ngx_stream_upstream_zone_preresolve(
+    ngx_stream_upstream_rr_peer_t *resolve,
+    ngx_stream_upstream_rr_peers_t *peers,
+    ngx_stream_upstream_rr_peer_t *oresolve,
+    ngx_stream_upstream_rr_peers_t *opeers);
 static void ngx_stream_upstream_zone_set_single(
     ngx_stream_upstream_srv_conf_t *uscf);
 static void ngx_stream_upstream_zone_remove_peer_locked(
@@ -127,11 +133,11 @@ static ngx_int_t
 ngx_stream_upstream_init_zone(ngx_shm_zone_t *shm_zone, void *data)
 {
     size_t                            len;
-    ngx_uint_t                        i;
+    ngx_uint_t                        i, j;
     ngx_slab_pool_t                  *shpool;
     ngx_stream_upstream_rr_peers_t   *peers, **peersp;
-    ngx_stream_upstream_srv_conf_t   *uscf, **uscfp;
-    ngx_stream_upstream_main_conf_t  *umcf;
+    ngx_stream_upstream_srv_conf_t   *uscf, *ouscf, **uscfp, **ouscfp;
+    ngx_stream_upstream_main_conf_t  *umcf, *oumcf;
 
     shpool = (ngx_slab_pool_t *) shm_zone->shm.addr;
     umcf = shm_zone->data;
@@ -168,6 +174,7 @@ ngx_stream_upstream_init_zone(ngx_shm_zo
     /* copy peers to shared memory */
 
     peersp = (ngx_stream_upstream_rr_peers_t **) (void *) &shpool->data;
+    oumcf = data;
 
     for (i = 0; i < umcf->upstreams.nelts; i++) {
         uscf = uscfp[i];
@@ -176,7 +183,38 @@ ngx_stream_upstream_init_zone(ngx_shm_zo
             continue;
         }
 
-        peers = ngx_stream_upstream_zone_copy_peers(shpool, uscf);
+        ouscf = NULL;
+
+        if (oumcf) {
+            ouscfp = oumcf->upstreams.elts;
+
+            for (j = 0; j < oumcf->upstreams.nelts; j++) {
+
+                 if (ouscfp[j]->shm_zone == NULL) {
+                     continue;
+                 }
+
+                 if (ouscfp[j]->shm_zone->shm.name.len != shm_zone->shm.name.len
+                     || ngx_memcmp(ouscfp[j]->shm_zone->shm.name.data,
+                                   shm_zone->shm.name.data,
+                                   shm_zone->shm.name.len)
+                        != 0)
+                 {
+                     continue;
+                 }
+
+                 if (ouscfp[j]->host.len == uscf->host.len
+                     && ngx_memcmp(ouscfp[j]->host.data, uscf->host.data,
+                                   uscf->host.len)
+                        == 0)
+                 {
+                     ouscf = ouscfp[j];
+                     break;
+                 }
+            }
+        }
+
+        peers = ngx_stream_upstream_zone_copy_peers(shpool, uscf, ouscf);
         if (peers == NULL) {
             return NGX_ERROR;
         }
@@ -191,12 +229,14 @@ ngx_stream_upstream_init_zone(ngx_shm_zo
 
 static ngx_stream_upstream_rr_peers_t *
 ngx_stream_upstream_zone_copy_peers(ngx_slab_pool_t *shpool,
-    ngx_stream_upstream_srv_conf_t *uscf)
+    ngx_stream_upstream_srv_conf_t *uscf, ngx_stream_upstream_srv_conf_t *ouscf)
 {
     ngx_str_t                       *name;
     ngx_uint_t                      *config;
     ngx_stream_upstream_rr_peer_t   *peer, **peerp;
-    ngx_stream_upstream_rr_peers_t  *peers, *backup;
+    ngx_stream_upstream_rr_peers_t  *peers, *opeers, *backup;
+
+    opeers = (ouscf ? ouscf->peer.data : NULL);
 
     config = ngx_slab_calloc(shpool, sizeof(ngx_uint_t));
     if (config == NULL) {
@@ -249,6 +289,16 @@ ngx_stream_upstream_zone_copy_peers(ngx_
         peer->id = (*peers->config)++;
     }
 
+    if (opeers) {
+
+        if (ngx_stream_upstream_zone_preresolve(peers->resolve, peers,
+                                                opeers->resolve, opeers)
+            != NGX_OK)
+        {
+            return NULL;
+        }
+    }
+
     if (peers->next == NULL) {
         goto done;
     }
@@ -288,10 +338,30 @@ ngx_stream_upstream_zone_copy_peers(ngx_
 
     peers->next = backup;
 
+    if (opeers && opeers->next) {
+
+        if (ngx_stream_upstream_zone_preresolve(peers->resolve, backup,
+                                                opeers->resolve, opeers->next)
+            != NGX_OK)
+        {
+            return NULL;
+        }
+
+        if (ngx_stream_upstream_zone_preresolve(backup->resolve, backup,
+                                                opeers->next->resolve,
+                                                opeers->next)
+            != NGX_OK)
+        {
+            return NULL;
+        }
+    }
+
 done:
 
     uscf->peer.data = peers;
 
+    ngx_stream_upstream_zone_set_single(uscf);
+
     return peers;
 }
 
@@ -403,6 +473,123 @@ failed:
 }
 
 
+static ngx_int_t
+ngx_stream_upstream_zone_preresolve(ngx_stream_upstream_rr_peer_t *resolve,
+    ngx_stream_upstream_rr_peers_t *peers,
+    ngx_stream_upstream_rr_peer_t *oresolve,
+    ngx_stream_upstream_rr_peers_t *opeers)
+{
+    in_port_t                       port;
+    ngx_str_t                      *server;
+    ngx_stream_upstream_host_t     *host;
+    ngx_stream_upstream_rr_peer_t  *peer, *template, *opeer, **peerp;
+
+    if (resolve == NULL || oresolve == NULL) {
+        return NGX_OK;
+    }
+
+    for (peerp = &peers->peer; *peerp; peerp = &(*peerp)->next) {
+        /* void */
+    }
+
+    ngx_stream_upstream_rr_peers_rlock(opeers);
+
+    for (template = resolve; template; template = template->next) {
+        for (opeer = oresolve; opeer; opeer = opeer->next) {
+
+            if (opeer->host->name.len != template->host->name.len
+                || ngx_memcmp(opeer->host->name.data,
+                              template->host->name.data,
+                              template->host->name.len)
+                   != 0)
+            {
+                continue;
+            }
+
+            if (opeer->host->service.len != template->host->service.len
+                || ngx_memcmp(opeer->host->service.data,
+                              template->host->service.data,
+                              template->host->service.len)
+                   != 0)
+            {
+                continue;
+            }
+
+            host = opeer->host;
+
+            for (opeer = opeers->peer; opeer; opeer = opeer->next) {
+
+                if (opeer->host != host) {
+                    continue;
+                }
+
+                peer = ngx_stream_upstream_zone_copy_peer(peers, NULL);
+                if (peer == NULL) {
+                    ngx_stream_upstream_rr_peers_unlock(opeers);
+                    return NGX_ERROR;
+                }
+
+                ngx_memcpy(peer->sockaddr, opeer->sockaddr, opeer->socklen);
+
+                if (template->host->service.len == 0) {
+                    port = ngx_inet_get_port(template->sockaddr);
+                    ngx_inet_set_port(peer->sockaddr, port);
+                }
+
+                peer->socklen = opeer->socklen;
+
+                peer->name.len = ngx_sock_ntop(peer->sockaddr, peer->socklen,
+                                               peer->name.data,
+                                               NGX_SOCKADDR_STRLEN, 1);
+
+                peer->host = template->host;
+
+                server = template->host->service.len ? &opeer->server
+                                                     : &template->server;
+
+                peer->server.data = ngx_slab_alloc(peers->shpool, server->len);
+                if (peer->server.data == NULL) {
+                    ngx_stream_upstream_rr_peers_unlock(opeers);
+                    return NGX_ERROR;
+                }
+
+                ngx_memcpy(peer->server.data, server->data, server->len);
+                peer->server.len = server->len;
+
+                if (host->service.len == 0) {
+                    peer->weight = template->weight;
+
+                } else {
+                    peer->weight = (template->weight != 1 ? template->weight
+                                                          : opeer->weight);
+                }
+
+                peer->effective_weight = peer->weight;
+                peer->max_conns = template->max_conns;
+                peer->max_fails = template->max_fails;
+                peer->fail_timeout = template->fail_timeout;
+                peer->down = template->down;
+
+                peer->id = (*peers->config)++;
+
+                *peerp = peer;
+                peerp = &peer->next;
+
+                peers->number++;
+                peers->tries += (peer->down == 0);
+                peers->total_weight += peer->weight;
+                peers->weighted = (peers->total_weight != peers->number);
+            }
+
+            break;
+        }
+    }
+
+    ngx_stream_upstream_rr_peers_unlock(opeers);
+    return NGX_OK;
+}
+
+
 static void
 ngx_stream_upstream_zone_set_single(ngx_stream_upstream_srv_conf_t *uscf)
 {

From a.bavshin at nginx.com  Thu Jun 13 22:29:01 2024
From: a.bavshin at nginx.com (=?iso-8859-1?q?Aleksei_Bavshin?=)
Date: Thu, 13 Jun 2024 15:29:01 -0700
Subject: [PATCH 6 of 9] Upstream: per-upstream resolver
In-Reply-To: <patchbomb.1718317735@fedora-wsl.local>
References: <patchbomb.1718317735@fedora-wsl.local>
Message-ID: <621ba257aeac3017ea83.1718317741@fedora-wsl.local>

# HG changeset patch
# User Vladimir Homutov <vl at nginx.com>
# Date 1571405595 -10800
#      Fri Oct 18 16:33:15 2019 +0300
# Node ID 621ba257aeac3017ea83b24fafa201e07c1c7756
# Parent  238c1695d3b7450159ba0c03509584683bf25f9b
Upstream: per-upstream resolver.

The "resolver" and "resolver_timeout" directives can now be specified
directly in the "upstream" block.

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -169,6 +169,10 @@ static ngx_int_t ngx_http_upstream_cooki
 static char *ngx_http_upstream(ngx_conf_t *cf, ngx_command_t *cmd, void *dummy);
 static char *ngx_http_upstream_server(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf);
+#if (NGX_HTTP_UPSTREAM_ZONE)
+static char *ngx_http_upstream_resolver(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf);
+#endif
 
 static ngx_int_t ngx_http_upstream_set_local(ngx_http_request_t *r,
   ngx_http_upstream_t *u, ngx_http_upstream_local_t *local);
@@ -339,6 +343,24 @@ static ngx_command_t  ngx_http_upstream_
       0,
       NULL },
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+
+    { ngx_string("resolver"),
+      NGX_HTTP_UPS_CONF|NGX_CONF_1MORE,
+      ngx_http_upstream_resolver,
+      NGX_HTTP_SRV_CONF_OFFSET,
+      0,
+      NULL },
+
+    { ngx_string("resolver_timeout"),
+      NGX_HTTP_UPS_CONF|NGX_CONF_TAKE1,
+      ngx_conf_set_msec_slot,
+      NGX_HTTP_SRV_CONF_OFFSET,
+      offsetof(ngx_http_upstream_srv_conf_t, resolver_timeout),
+      NULL },
+
+#endif
+
       ngx_null_command
 };
 
@@ -6438,6 +6460,32 @@ not_supported:
 }
 
 
+#if (NGX_HTTP_UPSTREAM_ZONE)
+
+static char *
+ngx_http_upstream_resolver(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
+{
+    ngx_http_upstream_srv_conf_t  *uscf = conf;
+
+    ngx_str_t  *value;
+
+    if (uscf->resolver) {
+        return "is duplicate";
+    }
+
+    value = cf->args->elts;
+
+    uscf->resolver = ngx_resolver_create(cf, &value[1], cf->args->nelts - 1);
+    if (uscf->resolver == NULL) {
+        return NGX_CONF_ERROR;
+    }
+
+    return NGX_CONF_OK;
+}
+
+#endif
+
+
 ngx_http_upstream_srv_conf_t *
 ngx_http_upstream_add(ngx_conf_t *cf, ngx_url_t *u, ngx_uint_t flags)
 {
@@ -6519,6 +6567,9 @@ ngx_http_upstream_add(ngx_conf_t *cf, ng
     uscf->line = cf->conf_file->line;
     uscf->port = u->port;
     uscf->no_port = u->no_port;
+#if (NGX_HTTP_UPSTREAM_ZONE)
+    uscf->resolver_timeout = NGX_CONF_UNSET_MSEC;
+#endif
 
     if (u->naddrs == 1 && (u->port || u->family == AF_UNIX)) {
         uscf->servers = ngx_array_create(cf->pool, 1,
diff --git a/src/http/ngx_http_upstream_round_robin.c b/src/http/ngx_http_upstream_round_robin.c
--- a/src/http/ngx_http_upstream_round_robin.c
+++ b/src/http/ngx_http_upstream_round_robin.c
@@ -104,15 +104,15 @@ ngx_http_upstream_init_round_robin(ngx_c
 
             clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
 
-            us->resolver = clcf->resolver;
-            us->resolver_timeout = clcf->resolver_timeout;
+            if (us->resolver == NULL) {
+                us->resolver = clcf->resolver;
+            }
 
             /*
-             * Without "resolver_timeout" in http{}, the value is unset.
-             * Even if we set it in ngx_http_core_merge_loc_conf(), it's
-             * still dependent on the module order and unreliable.
+             * Without "resolver_timeout" in http{} the merged value is unset.
              */
-            ngx_conf_init_msec_value(us->resolver_timeout, 30000);
+            ngx_conf_merge_msec_value(us->resolver_timeout,
+                                      clcf->resolver_timeout, 30000);
 
             if (resolve
                 && (us->resolver == NULL
diff --git a/src/stream/ngx_stream_upstream.c b/src/stream/ngx_stream_upstream.c
--- a/src/stream/ngx_stream_upstream.c
+++ b/src/stream/ngx_stream_upstream.c
@@ -22,6 +22,11 @@ static char *ngx_stream_upstream(ngx_con
     void *dummy);
 static char *ngx_stream_upstream_server(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf);
+#if (NGX_STREAM_UPSTREAM_ZONE)
+static char *ngx_stream_upstream_resolver(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf);
+#endif
+
 static void *ngx_stream_upstream_create_main_conf(ngx_conf_t *cf);
 static char *ngx_stream_upstream_init_main_conf(ngx_conf_t *cf, void *conf);
 
@@ -42,6 +47,24 @@ static ngx_command_t  ngx_stream_upstrea
       0,
       NULL },
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+
+    { ngx_string("resolver"),
+      NGX_STREAM_UPS_CONF|NGX_CONF_1MORE,
+      ngx_stream_upstream_resolver,
+      NGX_STREAM_SRV_CONF_OFFSET,
+      0,
+      NULL },
+
+    { ngx_string("resolver_timeout"),
+      NGX_STREAM_UPS_CONF|NGX_CONF_TAKE1,
+      ngx_conf_set_msec_slot,
+      NGX_STREAM_SRV_CONF_OFFSET,
+      offsetof(ngx_stream_upstream_srv_conf_t, resolver_timeout),
+      NULL },
+
+#endif
+
       ngx_null_command
 };
 
@@ -665,6 +688,32 @@ not_supported:
 }
 
 
+#if (NGX_STREAM_UPSTREAM_ZONE)
+
+static char *
+ngx_stream_upstream_resolver(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
+{
+    ngx_stream_upstream_srv_conf_t  *uscf = conf;
+
+    ngx_str_t  *value;
+
+    if (uscf->resolver) {
+        return "is duplicate";
+    }
+
+    value = cf->args->elts;
+
+    uscf->resolver = ngx_resolver_create(cf, &value[1], cf->args->nelts - 1);
+    if (uscf->resolver == NULL) {
+        return NGX_CONF_ERROR;
+    }
+
+    return NGX_CONF_OK;
+}
+
+#endif
+
+
 ngx_stream_upstream_srv_conf_t *
 ngx_stream_upstream_add(ngx_conf_t *cf, ngx_url_t *u, ngx_uint_t flags)
 {
@@ -743,6 +792,9 @@ ngx_stream_upstream_add(ngx_conf_t *cf, 
     uscf->line = cf->conf_file->line;
     uscf->port = u->port;
     uscf->no_port = u->no_port;
+#if (NGX_STREAM_UPSTREAM_ZONE)
+    uscf->resolver_timeout = NGX_CONF_UNSET_MSEC;
+#endif
 
     if (u->naddrs == 1 && (u->port || u->family == AF_UNIX)) {
         uscf->servers = ngx_array_create(cf->pool, 1,
diff --git a/src/stream/ngx_stream_upstream_round_robin.c b/src/stream/ngx_stream_upstream_round_robin.c
--- a/src/stream/ngx_stream_upstream_round_robin.c
+++ b/src/stream/ngx_stream_upstream_round_robin.c
@@ -111,15 +111,15 @@ ngx_stream_upstream_init_round_robin(ngx
             cscf = ngx_stream_conf_get_module_srv_conf(cf,
                                                        ngx_stream_core_module);
 
-            us->resolver = cscf->resolver;
-            us->resolver_timeout = cscf->resolver_timeout;
+            if (us->resolver == NULL) {
+                us->resolver = cscf->resolver;
+            }
 
             /*
-             * Without "resolver_timeout" in stream{}, the value is unset.
-             * Even if we set it in ngx_stream_core_merge_srv_conf(), it's
-             * still dependent on the module order and unreliable.
+             * Without "resolver_timeout" in stream{} the merged value is unset.
              */
-            ngx_conf_init_msec_value(us->resolver_timeout, 30000);
+            ngx_conf_merge_msec_value(us->resolver_timeout,
+                                      cscf->resolver_timeout, 30000);
 
             if (resolve
                 && (us->resolver == NULL

From a.bavshin at nginx.com  Thu Jun 13 22:29:02 2024
From: a.bavshin at nginx.com (=?iso-8859-1?q?Aleksei_Bavshin?=)
Date: Thu, 13 Jun 2024 15:29:02 -0700
Subject: [PATCH 7 of 9] Upstream: copy upstream zone DNS valid time during
 config reload
In-Reply-To: <patchbomb.1718317735@fedora-wsl.local>
References: <patchbomb.1718317735@fedora-wsl.local>
Message-ID: <8c8d8118c7ac0a0426f4.1718317742@fedora-wsl.local>

# HG changeset patch
# User Mini Hawthorne <mini at f5.com>
# Date 1689189645 25200
#      Wed Jul 12 12:20:45 2023 -0700
# Node ID 8c8d8118c7ac0a0426f48dbfed94e279dddff992
# Parent  621ba257aeac3017ea83b24fafa201e07c1c7756
Upstream: copy upstream zone DNS valid time during config reload.

Previously, all upstream DNS entries would be immediately re-resolved
on config reload.  With a large number of upstreams, this creates
a spike of DNS resolution requests.  These spikes can overwhelm the
DNS server or cause drops on the network.

This patch retains the TTL of previous resolutions across reloads
by copying each upstream's name's expiry time across configuration
cycles.  As a result, no additional resolutions are needed.

diff --git a/src/http/modules/ngx_http_upstream_zone_module.c b/src/http/modules/ngx_http_upstream_zone_module.c
--- a/src/http/modules/ngx_http_upstream_zone_module.c
+++ b/src/http/modules/ngx_http_upstream_zone_module.c
@@ -443,6 +443,8 @@ ngx_http_upstream_zone_copy_peer(ngx_htt
                 ngx_memcpy(dst->host->service.data, src->host->service.data,
                            src->host->service.len);
             }
+
+            dst->host->valid = src->host->valid;
         }
     }
 
@@ -547,6 +549,8 @@ ngx_http_upstream_zone_preresolve(ngx_ht
 
                 peer->host = template->host;
 
+                template->host->valid = host->valid;
+
                 server = template->host->service.len ? &opeer->server
                                                      : &template->server;
 
@@ -694,6 +698,8 @@ ngx_http_upstream_zone_init_worker(ngx_c
 static void
 ngx_http_upstream_zone_resolve_timer(ngx_event_t *event)
 {
+    time_t                         now, valid;
+    ngx_msec_t                     timer;
     ngx_resolver_ctx_t            *ctx;
     ngx_http_upstream_host_t      *host;
     ngx_http_upstream_rr_peer_t   *template;
@@ -705,6 +711,9 @@ ngx_http_upstream_zone_resolve_timer(ngx
     peers = host->peers;
     template = host->peer;
 
+    now = ngx_time();
+    valid = host->valid;
+
     if (template->zombie) {
         (void) ngx_http_upstream_rr_peer_unref(peers, template);
 
@@ -721,6 +730,10 @@ ngx_http_upstream_zone_resolve_timer(ngx
         return;
     }
 
+    if (valid > now) {
+        goto retry;
+    }
+
     ctx = ngx_resolve_start(uscf->resolver, NULL);
     if (ctx == NULL) {
         goto retry;
@@ -745,7 +758,11 @@ ngx_http_upstream_zone_resolve_timer(ngx
 
 retry:
 
-    ngx_add_timer(event, ngx_max(uscf->resolver_timeout, 1000));
+    /* don't delay zombie cleanup longer than resolver_timeout */
+    timer = (ngx_msec_t) 1000 * (valid > now ? valid - now + 1 : 1);
+    timer = ngx_min(timer, uscf->resolver_timeout);
+
+    ngx_add_timer(event, ngx_max(timer, 1000));
 }
 
 
@@ -1026,6 +1043,8 @@ again:
 
 done:
 
+    host->valid = ctx->valid;
+
     ngx_http_upstream_rr_peers_unlock(peers);
 
     while (++i < ctx->naddrs) {
diff --git a/src/http/ngx_http_upstream_round_robin.h b/src/http/ngx_http_upstream_round_robin.h
--- a/src/http/ngx_http_upstream_round_robin.h
+++ b/src/http/ngx_http_upstream_round_robin.h
@@ -25,6 +25,7 @@ typedef struct {
     ngx_uint_t                      worker;
     ngx_str_t                       name;
     ngx_str_t                       service;
+    time_t                          valid;
     ngx_http_upstream_rr_peers_t   *peers;
     ngx_http_upstream_rr_peer_t    *peer;
 } ngx_http_upstream_host_t;
diff --git a/src/stream/ngx_stream_upstream_round_robin.h b/src/stream/ngx_stream_upstream_round_robin.h
--- a/src/stream/ngx_stream_upstream_round_robin.h
+++ b/src/stream/ngx_stream_upstream_round_robin.h
@@ -25,6 +25,7 @@ typedef struct {
     ngx_uint_t                       worker;
     ngx_str_t                        name;
     ngx_str_t                        service;
+    time_t                           valid;
     ngx_stream_upstream_rr_peers_t  *peers;
     ngx_stream_upstream_rr_peer_t   *peer;
 } ngx_stream_upstream_host_t;
diff --git a/src/stream/ngx_stream_upstream_zone_module.c b/src/stream/ngx_stream_upstream_zone_module.c
--- a/src/stream/ngx_stream_upstream_zone_module.c
+++ b/src/stream/ngx_stream_upstream_zone_module.c
@@ -544,6 +544,8 @@ ngx_stream_upstream_zone_preresolve(ngx_
 
                 peer->host = template->host;
 
+                template->host->valid = host->valid;
+
                 server = template->host->service.len ? &opeer->server
                                                      : &template->server;
 
@@ -692,6 +694,8 @@ ngx_stream_upstream_zone_init_worker(ngx
 static void
 ngx_stream_upstream_zone_resolve_timer(ngx_event_t *event)
 {
+    time_t                           now, valid;
+    ngx_msec_t                       timer;
     ngx_resolver_ctx_t              *ctx;
     ngx_stream_upstream_host_t      *host;
     ngx_stream_upstream_rr_peer_t   *template;
@@ -703,6 +707,9 @@ ngx_stream_upstream_zone_resolve_timer(n
     peers = host->peers;
     template = host->peer;
 
+    now = ngx_time();
+    valid = host->valid;
+
     if (template->zombie) {
         (void) ngx_stream_upstream_rr_peer_unref(peers, template);
 
@@ -719,6 +726,10 @@ ngx_stream_upstream_zone_resolve_timer(n
         return;
     }
 
+    if (valid > now) {
+        goto retry;
+    }
+
     ctx = ngx_resolve_start(uscf->resolver, NULL);
     if (ctx == NULL) {
         goto retry;
@@ -743,7 +754,11 @@ ngx_stream_upstream_zone_resolve_timer(n
 
 retry:
 
-    ngx_add_timer(event, ngx_max(uscf->resolver_timeout, 1000));
+    /* don't delay zombie cleanup longer than resolver_timeout */
+    timer = (ngx_msec_t) 1000 * (valid > now ? valid - now + 1 : 1);
+    timer = ngx_min(timer, uscf->resolver_timeout);
+
+    ngx_add_timer(event, ngx_max(timer, 1000));
 }
 
 
@@ -1024,6 +1039,8 @@ again:
 
 done:
 
+    host->valid = ctx->valid;
+
     ngx_stream_upstream_rr_peers_unlock(peers);
 
     while (++i < ctx->naddrs) {

From a.bavshin at nginx.com  Thu Jun 13 22:29:03 2024
From: a.bavshin at nginx.com (=?iso-8859-1?q?Aleksei_Bavshin?=)
Date: Thu, 13 Jun 2024 15:29:03 -0700
Subject: [PATCH 8 of 9] Upstream: disable re-resolve functionality on Windows
In-Reply-To: <patchbomb.1718317735@fedora-wsl.local>
References: <patchbomb.1718317735@fedora-wsl.local>
Message-ID: <375fa42f1a6010692a87.1718317743@fedora-wsl.local>

# HG changeset patch
# User Aleksei Bavshin <a.bavshin at nginx.com>
# Date 1712181327 25200
#      Wed Apr 03 14:55:27 2024 -0700
# Node ID 375fa42f1a6010692a8782c4f03c6ad465d3f7f7
# Parent  8c8d8118c7ac0a0426f48dbfed94e279dddff992
Upstream: disable re-resolve functionality on Windows.

Following features are currently not implemented on Windows, making re-resolve
functionality unsafe to use:

 * 'noreuse' shared zones that are re-created on each configuration reload.
   The work scheduling logic is not prepared to handle simultaneous access to
   the shared zone from multiple generations of the worker processes.

 * 'ngx_worker' identification.
   It is possible to configure multiple worker processes on Windows, even if
   only one would actually handle the traffic.  All of the worker processes are
   currently identified as process 0, breaking scheduling and locking of the
   resolver tasks.

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -6327,7 +6327,7 @@ ngx_http_upstream_server(ngx_conf_t *cf,
             continue;
         }
 
-#if (NGX_HTTP_UPSTREAM_ZONE)
+#if (NGX_HTTP_UPSTREAM_ZONE && !(NGX_WIN32))
         if (ngx_strcmp(value[i].data, "resolve") == 0) {
             resolve = 1;
             continue;
diff --git a/src/stream/ngx_stream_upstream.c b/src/stream/ngx_stream_upstream.c
--- a/src/stream/ngx_stream_upstream.c
+++ b/src/stream/ngx_stream_upstream.c
@@ -545,7 +545,7 @@ ngx_stream_upstream_server(ngx_conf_t *c
             continue;
         }
 
-#if (NGX_STREAM_UPSTREAM_ZONE)
+#if (NGX_STREAM_UPSTREAM_ZONE && !(NGX_WIN32))
         if (ngx_strcmp(value[i].data, "resolve") == 0) {
             resolve = 1;
             continue;

From a.bavshin at nginx.com  Thu Jun 13 22:29:04 2024
From: a.bavshin at nginx.com (=?iso-8859-1?q?Aleksei_Bavshin?=)
Date: Thu, 13 Jun 2024 15:29:04 -0700
Subject: [PATCH 9 of 9] Tests: upstream configuration tests with re-resolvable
 servers
In-Reply-To: <patchbomb.1718317735@fedora-wsl.local>
References: <patchbomb.1718317735@fedora-wsl.local>
Message-ID: <ca287b2047ff417c3b23.1718317744@fedora-wsl.local>

# HG changeset patch
# User Aleksei Bavshin <a.bavshin at nginx.com>
# Date 1712098324 25200
#      Tue Apr 02 15:52:04 2024 -0700
# Node ID ca287b2047ff417c3b23a874d2e51c87553edc46
# Parent  375fa42f1a6010692a8782c4f03c6ad465d3f7f7
Tests: upstream configuration tests with re-resolvable servers.

Based on the NGINX Plus tests authored by Sergey Kandaurov.

diff --git a/stream_upstream_resolve.t b/stream_upstream_resolve.t
new file mode 100644
--- /dev/null
+++ b/stream_upstream_resolve.t
@@ -0,0 +1,378 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+
+# Stream tests for dynamic upstream configuration with re-resolvable servers.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+use IO::Select;
+use Socket qw/ CRLF /;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+use Test::Nginx::Stream qw/ stream /;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/stream http stream_upstream_zone/);
+
+$t->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+stream {
+    %%TEST_GLOBALS_STREAM%%
+
+    upstream u {
+        zone z 1m;
+        server example.net:%%PORT_8080%% max_fails=0 resolve;
+    }
+
+    # lower the retry timeout after empty reply
+    resolver 127.0.0.1:%%PORT_8983_UDP%% valid=1s;
+    # retry query shortly after DNS is started
+    resolver_timeout 1s;
+
+    log_format test $upstream_addr;
+
+    server {
+        listen      127.0.0.1:8082;
+        proxy_pass  u;
+        access_log  %%TESTDIR%%/cc.log test;
+        proxy_next_upstream   on;
+        proxy_connect_timeout 50ms;
+    }
+}
+EOF
+
+port(8084);
+
+$t->run_daemon(\&dns_daemon, port(8983), $t)
+	->waitforfile($t->testdir . '/' . port(8983));
+$t->try_run('no resolve in upstream server')->plan(11);
+
+###############################################################################
+
+my $p0 = port(8080);
+
+update_name({A => '127.0.0.201'});
+stream('127.0.0.1:' . port(8082))->read();
+
+# A changed
+
+update_name({A => '127.0.0.202'});
+stream('127.0.0.1:' . port(8082))->read();
+
+# 1 more A added
+
+update_name({A => '127.0.0.201 127.0.0.202'});
+stream('127.0.0.1:' . port(8082))->read();
+
+# 1 A removed, 2 AAAA added
+
+update_name({A => '127.0.0.201', AAAA => 'fe80::1 fe80::2'});
+stream('127.0.0.1:' . port(8082))->read();
+
+# all records removed
+
+update_name();
+stream('127.0.0.1:' . port(8082))->read();
+
+# A added after empty
+
+update_name({A => '127.0.0.201'});
+stream('127.0.0.1:' . port(8082))->read();
+
+# changed to CNAME
+
+update_name({CNAME => 'alias'}, 4);
+stream('127.0.0.1:' . port(8082))->read();
+
+# bad DNS reply should not affect existing upstream configuration
+
+update_name({ERROR => 'SERVFAIL'});
+stream('127.0.0.1:' . port(8082))->read();
+
+$t->stop();
+
+Test::Nginx::log_core('||', $t->read_file('cc.log'));
+
+open my $f, '<', "${\($t->testdir())}/cc.log" or die "Can't open cc.log: $!";
+my $line;
+
+like($f->getline(), qr/127.0.0.201:$p0/, 'log - A');
+
+# A changed
+
+like($f->getline(), qr/127.0.0.202:$p0/, 'log - A changed');
+
+# 1 more A added
+
+$line = $f->getline();
+like($line, qr/127.0.0.201:$p0/, 'log - A A 1');
+like($line, qr/127.0.0.202:$p0/, 'log - A A 2');
+
+# 1 A removed, 2 AAAA added
+
+$line = $f->getline();
+like($line, qr/127.0.0.201:$p0/, 'log - A AAAA AAAA 1');
+like($line, qr/\[fe80::1\]:$p0/, 'log - A AAAA AAAA 2');
+like($line, qr/\[fe80::2\]:$p0/, 'log - A AAAA AAAA 3');
+
+# all records removed
+
+like($f->getline(), qr/^u$/, 'log - empty response');
+
+# A added after empty
+
+like($f->getline(), qr/127.0.0.201:$p0/, 'log - A added 1');
+
+# changed to CNAME
+
+like($f->getline(), qr/127.0.0.203:$p0/, 'log - CNAME 1');
+
+# bad DNS reply should not affect existing upstream configuration
+
+like($f->getline(), qr/127.0.0.203:$p0/, 'log - ERROR 1');
+
+###############################################################################
+
+sub update_name {
+	my ($name, $plan) = @_;
+
+	$plan = 2 if !defined $plan;
+
+	sub sock {
+		IO::Socket::INET->new(
+			Proto => 'tcp',
+			PeerAddr => '127.0.0.1:' . port(8084)
+		)
+			or die "Can't connect to nginx: $!\n";
+	}
+
+	$name->{A} = '' unless $name->{A};
+	$name->{AAAA} = '' unless $name->{AAAA};
+	$name->{CNAME} = '' unless $name->{CNAME};
+	$name->{ERROR} = '' unless $name->{ERROR};
+
+	my $req =<<EOF;
+GET / HTTP/1.0
+Host: localhost
+X-A: $name->{A}
+X-AAAA: $name->{AAAA}
+X-CNAME: $name->{CNAME}
+X-ERROR: $name->{ERROR}
+
+EOF
+
+	my ($gen) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+	for (1 .. 10) {
+		my ($gen2) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+
+		# let resolver cache expire to finish upstream reconfiguration
+		select undef, undef, undef, 0.5;
+		last unless ($gen + $plan > $gen2);
+	}
+}
+
+###############################################################################
+
+sub reply_handler {
+	my ($recv_data, $h) = @_;
+
+	my (@name, @rdata);
+
+	use constant NOERROR	=> 0;
+	use constant SERVFAIL	=> 2;
+	use constant NXDOMAIN	=> 3;
+
+	use constant A		=> 1;
+	use constant CNAME	=> 5;
+	use constant AAAA	=> 28;
+	use constant DNAME	=> 39;
+	use constant IN		=> 1;
+
+	# default values
+
+	my ($hdr, $rcode, $ttl) = (0x8180, NOERROR, 1);
+	$h = {A => [ "127.0.0.201" ]} unless defined $h;
+
+	# decode name
+
+	my ($len, $offset) = (undef, 12);
+	while (1) {
+		$len = unpack("\@$offset C", $recv_data);
+		last if $len == 0;
+		$offset++;
+		push @name, unpack("\@$offset A$len", $recv_data);
+		$offset += $len;
+	}
+
+	$offset -= 1;
+	my ($id, $type, $class) = unpack("n x$offset n2", $recv_data);
+	my $name = join('.', @name);
+
+	if ($h->{ERROR}) {
+		$rcode = SERVFAIL;
+		goto bad;
+	}
+
+	if ($name eq 'example.net') {
+		if ($type == A && $h->{A}) {
+			map { push @rdata, rd_addr($ttl, $_) } @{$h->{A}};
+		}
+		if ($type == AAAA && $h->{AAAA}) {
+			map { push @rdata, rd_addr6($ttl, $_) } @{$h->{AAAA}};
+		}
+		my $cname = defined $h->{CNAME} ? $h->{CNAME} : 0;
+		if ($cname) {
+			push @rdata, pack("n3N nCa5n", 0xc00c, CNAME, IN, $ttl,
+				8, 5, $cname, 0xc00c);
+		}
+
+	} elsif ($name eq 'alias.example.net') {
+		if ($type == A) {
+			push @rdata, rd_addr($ttl, '127.0.0.203');
+		}
+	}
+
+bad:
+
+	Test::Nginx::log_core('||', "DNS: $name $type $rcode");
+
+	$len = @name;
+	pack("n6 (C/a*)$len x n2", $id, $hdr | $rcode, 1, scalar @rdata,
+		0, 0, @name, $type, $class) . join('', @rdata);
+}
+
+sub rd_addr {
+	my ($ttl, $addr) = @_;
+
+	my $code = 'split(/\./, $addr)';
+
+	pack 'n3N nC4', 0xc00c, A, IN, $ttl, eval "scalar $code", eval($code);
+}
+
+sub expand_ip6 {
+	my ($addr) = @_;
+
+	substr ($addr, index($addr, "::"), 2) =
+		join "0", map { ":" } (0 .. 8 - (split /:/, $addr) + 1);
+	map { hex "0" x (4 - length $_) . "$_" } split /:/, $addr;
+}
+
+sub rd_addr6 {
+	my ($ttl, $addr) = @_;
+
+	pack 'n3N nn8', 0xc00c, AAAA, IN, $ttl, 16, expand_ip6($addr);
+}
+
+sub dns_daemon {
+	my ($port, $t) = @_;
+	my ($data, $recv_data, $h);
+
+	my $socket = IO::Socket::INET->new(
+		LocalAddr => '127.0.0.1',
+		LocalPort => $port,
+		Proto => 'udp',
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $control = IO::Socket::INET->new(
+		Proto => 'tcp',
+		LocalHost => '127.0.0.1:' . port(8084),
+		Listen => 5,
+		Reuse => 1
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $sel = IO::Select->new($socket, $control);
+
+	local $SIG{PIPE} = 'IGNORE';
+
+	# signal we are ready
+
+	open my $fh, '>', $t->testdir() . '/' . $port;
+	close $fh;
+	my $cnt = 0;
+
+	while (my @ready = $sel->can_read) {
+		foreach my $fh (@ready) {
+			if ($control == $fh) {
+				my $new = $fh->accept;
+				$new->autoflush(1);
+				$sel->add($new);
+
+			} elsif ($socket == $fh) {
+				$fh->recv($recv_data, 65536);
+				$data = reply_handler($recv_data, $h);
+				$fh->send($data);
+				$cnt++;
+
+			} else {
+				$h = process_name($fh, $cnt);
+				$sel->remove($fh);
+				$fh->close;
+			}
+		}
+	}
+}
+
+# parse dns update
+
+sub process_name {
+	my ($client, $cnt) = @_;
+	my $port = $client->sockport();
+
+	my $headers = '';
+	my $uri = '';
+	my %h;
+
+	while (<$client>) {
+		$headers .= $_;
+		last if (/^\x0d?\x0a?$/);
+	}
+	return 1 if $headers eq '';
+
+	$uri = $1 if $headers =~ /^\S+\s+([^ ]+)\s+HTTP/i;
+	return 1 if $uri eq '';
+
+	$headers =~ /X-A: (.*)$/m;
+	map { push @{$h{A}}, $_ } split(/ /, $1);
+	$headers =~ /X-AAAA: (.*)$/m;
+	map { push @{$h{AAAA}}, $_ } split(/ /, $1);
+	$headers =~ /X-CNAME: (.*)$/m;
+	$h{CNAME} = $1;
+	$headers =~ /X-ERROR: (.*)$/m;
+	$h{ERROR} = $1;
+
+	Test::Nginx::log_core('||', "$port: response, 200");
+	print $client <<EOF;
+HTTP/1.1 200 OK
+Connection: close
+X-Gen: $cnt
+
+OK
+EOF
+
+	return \%h;
+}
+
+###############################################################################
diff --git a/stream_upstream_resolve_reload.t b/stream_upstream_resolve_reload.t
new file mode 100644
--- /dev/null
+++ b/stream_upstream_resolve_reload.t
@@ -0,0 +1,343 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+
+# Stream tests for dynamic upstream configuration with re-resolvable servers.
+# Ensure that upstream configuration is inherited on reload.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+use IO::Select;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+use Test::Nginx::Stream qw/ stream /;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/stream stream_upstream_zone http/);
+
+$t->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+stream {
+    %%TEST_GLOBALS_STREAM%%
+
+    upstream u {
+        zone z 1m;
+        server example.net:%%PORT_8081%% resolve;
+    }
+
+    upstream u2 {
+        zone z 1m;
+        server 127.0.0.203:%%PORT_8081%% max_fails=0;
+        server example.net:%%PORT_8081%% resolve max_fails=0;
+    }
+
+    # lower the retry timeout after empty reply
+    resolver 127.0.0.1:%%PORT_8980_UDP%% valid=1s;
+    # retry query shortly after DNS is started
+    resolver_timeout 1s;
+
+    log_format test $upstream_addr;
+
+    server {
+        listen 127.0.0.1:8082;
+        proxy_pass u;
+        proxy_connect_timeout 50ms;
+        access_log %%TESTDIR%%/cc.log test;
+    }
+
+    server {
+        listen 127.0.0.1:8083;
+        proxy_pass u2;
+        proxy_connect_timeout 50ms;
+        access_log %%TESTDIR%%/cc2.log test;
+    }
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    server {
+        listen       127.0.0.1:8080;
+        server_name  localhost;
+    }
+}
+
+EOF
+
+$t->run_daemon(\&dns_daemon, $t)->waitforfile($t->testdir . '/' . port(8980));
+$t->try_run('no resolve in upstream server')->plan(9);
+
+###############################################################################
+
+my $p = port(8081);
+
+update_name({A => '127.0.0.201'});
+stream('127.0.0.1:' . port(8082))->read();
+stream('127.0.0.1:' . port(8082))->read();
+stream('127.0.0.1:' . port(8083))->read();
+
+update_name({ERROR => 'SERVFAIL'}, 0);
+
+my $conf = $t->read_file('nginx.conf');
+$conf =~ s/$p/port(8082)/gmse;
+$t->write_file('nginx.conf', $conf);
+
+$t->reload();
+waitforworker($t);
+
+stream('127.0.0.1:' . port(8082))->read();
+stream('127.0.0.1:' . port(8082))->read();
+stream('127.0.0.1:' . port(8083))->read();
+
+update_name({A => '127.0.0.202'});
+stream('127.0.0.1:' . port(8082))->read();
+stream('127.0.0.1:' . port(8082))->read();
+stream('127.0.0.1:' . port(8083))->read();
+
+$t->stop();
+
+Test::Nginx::log_core('||', $t->read_file('cc.log'));
+
+open my $f, '<', "${\($t->testdir())}/cc.log" or die "Can't open cc.log: $!";
+
+like($f->getline(), qr/127.0.0.201:$p/, 'log - before');
+like($f->getline(), qr/127.0.0.201:$p/, 'log - before 2');
+
+$p = port(8082);
+
+like($f->getline(), qr/127.0.0.201:$p/, 'log - preresolve');
+like($f->getline(), qr/127.0.0.201:$p/, 'log - preresolve 2');
+
+like($f->getline(), qr/127.0.0.202:$p/, 'log - update');
+like($f->getline(), qr/127.0.0.202:$p/, 'log - update 2');
+
+Test::Nginx::log_core('||', $t->read_file('cc2.log'));
+
+$p = port(8081);
+
+open $f, '<', "${\($t->testdir())}/cc2.log" or die "Can't open cc2.log: $!";
+
+like($f->getline(), qr/127.0.0.(201:$p, 127.0.0.203|203:$p, 127.0.0.201):$p/,
+	'log many - before');
+
+$p = port(8082);
+
+like($f->getline(), qr/127.0.0.(201:$p, 127.0.0.203|203:$p, 127.0.0.201):$p/,
+	'log many - preresolve');
+
+like($f->getline(), qr/127.0.0.(202:$p, 127.0.0.203|203:$p, 127.0.0.202):$p/,
+	'log many - update');
+
+###############################################################################
+
+sub waitforworker {
+	my ($t) = @_;
+
+	for (1 .. 30) {
+		last if $t->read_file('error.log') =~ /exited with code/;
+		select undef, undef, undef, 0.2;
+	}
+}
+
+sub update_name {
+	my ($name, $plan) = @_;
+
+	$plan = 2 if !defined $plan;
+
+	sub sock {
+		IO::Socket::INET->new(
+			Proto => 'tcp',
+			PeerAddr => '127.0.0.1:' . port(8081)
+		)
+			or die "Can't connect to nginx: $!\n";
+	}
+
+	$name->{A} = '' unless $name->{A};
+	$name->{ERROR} = '' unless $name->{ERROR};
+
+	my $req =<<EOF;
+GET / HTTP/1.0
+Host: localhost
+X-A: $name->{A}
+X-ERROR: $name->{ERROR}
+
+EOF
+
+	my ($gen) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+	for (1 .. 10) {
+		my ($gen2) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+
+		# let resolver cache expire to finish upstream reconfiguration
+		select undef, undef, undef, 0.5;
+		last unless ($gen + $plan > $gen2);
+	}
+}
+
+###############################################################################
+
+sub reply_handler {
+	my ($recv_data, $h) = @_;
+
+	my (@name, @rdata);
+
+	use constant NOERROR	=> 0;
+	use constant SERVFAIL	=> 2;
+	use constant NXDOMAIN	=> 3;
+
+	use constant A		=> 1;
+	use constant IN		=> 1;
+
+	# default values
+
+	my ($hdr, $rcode, $ttl) = (0x8180, NOERROR, 1);
+	$h = {A => [ "127.0.0.201" ]} unless defined $h;
+
+	# decode name
+
+	my ($len, $offset) = (undef, 12);
+	while (1) {
+		$len = unpack("\@$offset C", $recv_data);
+		last if $len == 0;
+		$offset++;
+		push @name, unpack("\@$offset A$len", $recv_data);
+		$offset += $len;
+	}
+
+	$offset -= 1;
+	my ($id, $type, $class) = unpack("n x$offset n2", $recv_data);
+	my $name = join('.', @name);
+
+	if ($h->{ERROR}) {
+		$rcode = SERVFAIL;
+		goto bad;
+	}
+
+	if ($name eq 'example.net' && $type == A && $h->{A}) {
+		map { push @rdata, rd_addr($ttl, $_) } @{$h->{A}};
+	}
+
+bad:
+
+	Test::Nginx::log_core('||', "DNS: $name $type $rcode");
+
+	$len = @name;
+	pack("n6 (C/a*)$len x n2", $id, $hdr | $rcode, 1, scalar @rdata,
+		0, 0, @name, $type, $class) . join('', @rdata);
+}
+
+sub rd_addr {
+	my ($ttl, $addr) = @_;
+
+	my $code = 'split(/\./, $addr)';
+
+	pack 'n3N nC4', 0xc00c, A, IN, $ttl, eval "scalar $code", eval($code);
+}
+
+sub dns_daemon {
+	my ($t) = @_;
+	my ($data, $recv_data, $h);
+
+	my $socket = IO::Socket::INET->new(
+		LocalAddr => '127.0.0.1',
+		LocalPort => port(8980),
+		Proto => 'udp',
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $control = IO::Socket::INET->new(
+		Proto => 'tcp',
+		LocalHost => "127.0.0.1:" . port(8081),
+		Listen => 5,
+		Reuse => 1
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $sel = IO::Select->new($socket, $control);
+
+	local $SIG{PIPE} = 'IGNORE';
+
+	# signal we are ready
+
+	open my $fh, '>', $t->testdir() . '/' . port(8980);
+	close $fh;
+	my $cnt = 0;
+
+	while (my @ready = $sel->can_read) {
+		foreach my $fh (@ready) {
+			if ($control == $fh) {
+				my $new = $fh->accept;
+				$new->autoflush(1);
+				$sel->add($new);
+
+			} elsif ($socket == $fh) {
+				$fh->recv($recv_data, 65536);
+				$data = reply_handler($recv_data, $h);
+				$fh->send($data);
+				$cnt++;
+
+			} else {
+				$h = process_name($fh, $cnt);
+				$sel->remove($fh);
+				$fh->close;
+			}
+		}
+	}
+}
+
+# parse dns update
+
+sub process_name {
+	my ($client, $cnt) = @_;
+	my $port = $client->sockport();
+
+	my $headers = '';
+	my $uri = '';
+	my %h;
+
+	while (<$client>) {
+		$headers .= $_;
+		last if (/^\x0d?\x0a?$/);
+	}
+	return 1 if $headers eq '';
+
+	$uri = $1 if $headers =~ /^\S+\s+([^ ]+)\s+HTTP/i;
+	return 1 if $uri eq '';
+
+	$headers =~ /X-A: (.*)$/m;
+	map { push @{$h{A}}, $_ } split(/ /, $1);
+	$headers =~ /X-ERROR: (.*)$/m;
+	$h{ERROR} = $1;
+
+	Test::Nginx::log_core('||', "$port: response, 200");
+	print $client <<EOF;
+HTTP/1.1 200 OK
+Connection: close
+X-Gen: $cnt
+
+OK
+EOF
+
+	return \%h;
+}
+
+###############################################################################
diff --git a/stream_upstream_resolver.t b/stream_upstream_resolver.t
new file mode 100644
--- /dev/null
+++ b/stream_upstream_resolver.t
@@ -0,0 +1,290 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+
+# Tests for re-resolvable servers with resolver in stream upstream.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+use Test::Nginx::Stream qw/ stream /;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/stream stream_upstream_zone http/);
+
+$t->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+stream {
+    %%TEST_GLOBALS_STREAM%%
+
+    resolver 127.0.0.1:%%PORT_8980_UDP%%;
+
+    upstream u {
+        zone z 1m;
+        server example.net:%%PORT_8080%% resolve;
+    }
+
+    upstream u1 {
+        zone z 1m;
+        server example.net:%%PORT_8080%% resolve;
+        resolver 127.0.0.1:%%PORT_8981_UDP%%;
+    }
+
+    upstream u2 {
+        zone z 1m;
+        server example.net:%%PORT_8080%% resolve;
+        resolver 127.0.0.1:%%PORT_8982_UDP%%;
+        resolver_timeout 200s; # for coverage
+    }
+
+    log_format test $upstream_addr;
+
+    proxy_connect_timeout 50ms;
+
+    server {
+        listen       127.0.0.1:8081;
+        proxy_pass   u;
+
+        access_log %%TESTDIR%%/access.log test;
+    }
+
+    server {
+        listen       127.0.0.1:8082;
+        proxy_pass   u1;
+
+        access_log %%TESTDIR%%/access1.log test;
+    }
+
+    server {
+        listen       127.0.0.1:8083;
+        proxy_pass   u2;
+
+        access_log %%TESTDIR%%/access2.log test;
+    }
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    server {
+        listen       127.0.0.1:8080;
+        server_name  localhost;
+    }
+}
+
+EOF
+
+$t->run_daemon(\&dns_daemon, $t, port($_), port($_ + 10)) for (8980 .. 8982);
+$t->waitforfile($t->testdir . '/' . port($_)) for (8980 .. 8982);
+
+$t->try_run('no resolver in upstream')->plan(6);
+
+###############################################################################
+
+ok(waitfordns(8980), 'resolved');
+ok(waitfordns(8981), 'resolved in upstream 1');
+ok(waitfordns(8982), 'resolved in upstream 2');
+
+stream('127.0.0.1:' . port(8081))->read();
+stream('127.0.0.1:' . port(8082))->read();
+stream('127.0.0.1:' . port(8083))->read();
+
+$t->stop();
+
+like($t->read_file('access.log'), qr/127.0.0.200/, 'resolver');
+like($t->read_file('access1.log'), qr/127.0.0.201/, 'resolver upstream 1');
+like($t->read_file('access2.log'), qr/127.0.0.202/, 'resolver upstream 2');
+
+###############################################################################
+
+sub waitfordns {
+	my ($port, $plan) = @_;
+
+	$plan = 1 if !defined $plan;
+
+	sub sock {
+		my ($port) = @_;
+
+		IO::Socket::INET->new(
+			Proto => 'tcp',
+			PeerAddr => '127.0.0.1:' . port($port + 10)
+		)
+			or die "Can't connect to dns control socket: $!\n";
+	}
+
+	my $req =<<EOF;
+GET / HTTP/1.0
+Host: localhost
+
+EOF
+
+	for (1 .. 10) {
+		my ($gen) = http($req, socket => sock($port)) =~ /X-Gen: (\d+)/;
+		select undef, undef, undef, 0.5;
+		return 1 if $gen >= $plan;
+	}
+}
+
+###############################################################################
+
+sub reply_handler {
+	my ($recv_data, $port) = @_;
+
+	my (@name, @rdata);
+
+	use constant NOERROR	=> 0;
+	use constant A		=> 1;
+	use constant IN		=> 1;
+
+	# default values
+
+	my ($hdr, $rcode, $ttl) = (0x8180, NOERROR, 1);
+
+	# decode name
+
+	my ($len, $offset) = (undef, 12);
+	while (1) {
+		$len = unpack("\@$offset C", $recv_data);
+		last if $len == 0;
+		$offset++;
+		push @name, unpack("\@$offset A$len", $recv_data);
+		$offset += $len;
+	}
+
+	$offset -= 1;
+	my ($id, $type, $class) = unpack("n x$offset n2", $recv_data);
+
+	my $name = join('.', @name);
+	if ($name eq 'example.net' && $type == A) {
+		if ($port == port(8980)) {
+			push @rdata, rd_addr($ttl, "127.0.0.200");
+		}
+
+		if ($port == port(8981)) {
+			push @rdata, rd_addr($ttl, "127.0.0.201");
+		}
+
+		if ($port == port(8982)) {
+			push @rdata, rd_addr($ttl, "127.0.0.202");
+		}
+	}
+
+	Test::Nginx::log_core('||', "DNS: $name $type $rcode");
+
+	$len = @name;
+	pack("n6 (C/a*)$len x n2", $id, $hdr | $rcode, 1, scalar @rdata,
+		0, 0, @name, $type, $class) . join('', @rdata);
+}
+
+sub rd_addr {
+	my ($ttl, $addr) = @_;
+
+	my $code = 'split(/\./, $addr)';
+
+	pack 'n3N nC4', 0xc00c, A, IN, $ttl, eval "scalar $code", eval($code);
+}
+
+sub dns_daemon {
+	my ($t, $port, $control_port) = @_;
+
+	my ($data, $recv_data);
+	my $socket = IO::Socket::INET->new(
+		LocalAddr => '127.0.0.1',
+		LocalPort => $port,
+		Proto => 'udp',
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $control = IO::Socket::INET->new(
+		Proto => 'tcp',
+		LocalAddr => '127.0.0.1',
+		LocalPort => $control_port,
+		Listen => 5,
+		Reuse => 1
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $sel = IO::Select->new($socket, $control);
+
+	local $SIG{PIPE} = 'IGNORE';
+
+	# signal we are ready
+
+	open my $fh, '>', $t->testdir() . '/' . $port;
+	close $fh;
+	my $cnt = 0;
+
+	while (my @ready = $sel->can_read) {
+		foreach my $fh (@ready) {
+			if ($control == $fh) {
+				my $new = $fh->accept;
+				$new->autoflush(1);
+				$sel->add($new);
+
+			} elsif ($socket == $fh) {
+				$fh->recv($recv_data, 65536);
+				$data = reply_handler($recv_data, $port);
+				$fh->send($data);
+				$cnt++;
+
+			} else {
+				process_name($fh, $cnt);
+				$sel->remove($fh);
+				$fh->close;
+			}
+		}
+	}
+}
+
+# parse dns update
+
+sub process_name {
+	my ($client, $cnt) = @_;
+	my $port = $client->sockport();
+
+	my $headers = '';
+	my $uri = '';
+	my %h;
+
+	while (<$client>) {
+		$headers .= $_;
+		last if (/^\x0d?\x0a?$/);
+	}
+	return 1 if $headers eq '';
+
+	$uri = $1 if $headers =~ /^\S+\s+([^ ]+)\s+HTTP/i;
+	return 1 if $uri eq '';
+
+	Test::Nginx::log_core('||', "$port: response, 200");
+	print $client <<EOF;
+HTTP/1.1 200 OK
+Connection: close
+X-Gen: $cnt
+
+OK
+EOF
+
+	return \%h;
+}
+
+###############################################################################
diff --git a/stream_upstream_service.t b/stream_upstream_service.t
new file mode 100644
--- /dev/null
+++ b/stream_upstream_service.t
@@ -0,0 +1,544 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+
+# Stream tests for dynamic upstream configuration with service (SRV) feature.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+use IO::Select;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+use Test::Nginx::Stream qw/ stream /;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/stream stream_upstream_zone http/);
+
+$t->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+stream {
+    %%TEST_GLOBALS_STREAM%%
+
+    upstream u {
+        zone z 1m;
+        server example.net max_fails=0 resolve service=http;
+    }
+
+    upstream u2 {
+        zone z2 1m;
+        server example.net max_fails=0 resolve service=_http._tcp;
+    }
+
+    # lower the retry timeout after empty reply
+    resolver 127.0.0.1:%%PORT_8981_UDP%% valid=1s;
+    # retry query shortly after DNS is started
+    resolver_timeout 1s;
+
+    log_format test $upstream_addr;
+
+    server {
+        listen 127.0.0.1:8081;
+        proxy_pass u;
+        proxy_next_upstream   on;
+        proxy_connect_timeout 50ms;
+        access_log %%TESTDIR%%/cc.log test;
+    }
+
+    server {
+        listen 127.0.0.1:8082;
+        proxy_pass u2;
+        proxy_next_upstream   on;
+        proxy_connect_timeout 50ms;
+        access_log %%TESTDIR%%/cc.log test;
+    }
+}
+
+EOF
+
+port(8080);
+port(8084);
+
+$t->write_file('t', '');
+
+$t->run_daemon(\&dns_daemon, $t)->waitforfile($t->testdir . '/' . port(8981));
+port(8981, socket => 1)->close();
+$t->try_run('no resolve in upstream server')->plan(20);
+
+###############################################################################
+
+my ($p0, $p2, $p3) = (port(8080), port(8082), port(8083));
+
+update_name({A => '127.0.0.201', SRV => "1 5 $p0 example.net"});
+stream('127.0.0.1:' . port(8081))->read();
+
+# fully specified service
+
+stream('127.0.0.1:' . port(8082))->read();
+
+# A changed
+
+update_name({A => '127.0.0.202', SRV => "1 5 $p0 example.net"});
+stream('127.0.0.1:' . port(8081))->read();
+
+# 1 more A added
+
+update_name({A => '127.0.0.201 127.0.0.202', SRV => "1 5 $p0 example.net"});
+stream('127.0.0.1:' . port(8081))->read();
+
+# 1 A removed, 2 AAAA added
+
+update_name({A => '127.0.0.201', AAAA => 'fe80::1 fe80::2',
+	SRV => "1 5 $p0 example.net"});
+stream('127.0.0.1:' . port(8081))->read();
+
+# all records removed
+
+update_name({SRV => "1 5 $p0 example.net"});
+stream('127.0.0.1:' . port(8081))->read();
+
+# all SRV records removed
+
+update_name();
+stream('127.0.0.1:' . port(8081))->read();
+
+# A added after empty
+
+update_name({A => '127.0.0.201', SRV => "1 5 $p0 example.net"});
+stream('127.0.0.1:' . port(8081))->read();
+
+# SRV changed its weight
+
+update_name({A => '127.0.0.201', SRV => "1 6 $p0 example.net"});
+stream('127.0.0.1:' . port(8081))->read();
+
+# changed to CNAME
+
+update_name({CNAME => 'alias'}, 2, 2);
+stream('127.0.0.1:' . port(8081))->read();
+
+# bad SRV reply should not affect existing upstream configuration
+
+update_name({CNAME => 'alias', ERROR => 'SERVFAIL'}, 1, 0);
+stream('127.0.0.1:' . port(8081))->read();
+update_name({ERROR => ''}, 1, 0);
+
+# 2 equal SRV RR
+
+update_name({A => '127.0.0.201',
+	SRV => "1 5 $p0 example.net;1 5 $p0 example.net"});
+stream('127.0.0.1:' . port(8081))->read();
+
+# all equal records removed
+
+update_name();
+stream('127.0.0.1:' . port(8081))->read();
+
+# 2 different SRV RR
+
+update_name({A => '127.0.0.201',
+	SRV => "1 5 $p2 example.net;2 6 $p3 alias.example.net"}, 1, 2);
+stream('127.0.0.1:' . port(8081))->read();
+
+# all different records removed
+
+update_name();
+stream('127.0.0.1:' . port(8081))->read();
+
+# bad subordinate reply should not affect existing upstream configuration
+
+update_name({A => '127.0.0.201',
+	SRV => "1 5 $p0 example.net;1 5 $p0 example.net"});
+stream('127.0.0.1:' . port(8081))->read();
+
+update_name({A => '127.0.0.201', SERROR => 'SERVFAIL',
+	SRV => "1 5 $p0 example.net;1 5 $p0 example.net"});
+stream('127.0.0.1:' . port(8081))->read();
+
+$t->stop();
+
+Test::Nginx::log_core('||', $t->read_file('cc.log'));
+
+open my $f, '<', "${\($t->testdir())}/cc.log" or die "Can't open cc.log: $!";
+my $line;
+
+like($f->getline(), qr/127.0.0.201:$p0/, 'log - A');
+
+# fully specified service
+
+like($f->getline(), qr/127.0.0.201:$p0/, 'log - A full');
+
+# A changed
+
+like($f->getline(), qr/127.0.0.202:$p0/, 'log - A changed');
+
+# 1 more A added
+
+$line = $f->getline();
+like($line, qr/127.0.0.201:$p0/, 'log - A A 1');
+like($line, qr/127.0.0.202:$p0/, 'log - A A 2');
+
+# 1 A removed, 2 AAAA added
+
+$line = $f->getline();
+like($line, qr/127.0.0.201:$p0/, 'log - A AAAA AAAA 1');
+like($line, qr/\[fe80::1\]:$p0/, 'log - A AAAA AAAA 2');
+like($line, qr/\[fe80::2\]:$p0/, 'log - A AAAA AAAA 3');
+
+# all records removed
+
+like($f->getline(), qr/^u$/, 'log - empty response');
+
+# all SRV records removed
+
+like($f->getline(), qr/^u$/, 'log - empty response');
+
+# A added after empty
+
+like($f->getline(), qr/127.0.0.201:$p0/, 'log - A added 1');
+
+# SRV changed its weight
+
+like($f->getline(), qr/127.0.0.201:$p0/, 'log - SRV weight');
+
+# changed to CNAME
+
+like($f->getline(), qr/127.0.0.203:$p0/, 'log - CNAME');
+
+# bad SRV reply should not affect existing upstream configuration
+
+like($f->getline(), qr/127.0.0.203:$p0/, 'log - ERROR');
+
+# 2 equal SRV RR
+
+like($f->getline(), qr/127.0.0.201:$p0, 127.0.0.201:$p0/, 'log - SRV same');
+
+# all equal records removed
+
+like($f->getline(), qr/^u$/, 'log - SRV same removed');
+
+# 2 different SRV RR
+
+$line = $f->getline();
+like($line, qr/127.0.0.201:$p2, 127.0.0.203:$p3/, 'log - SRV diff');
+
+# all different records removed
+
+like($f->getline(), qr/^u$/, 'log - SRV diff removed');
+
+# bad subordinate reply should not affect existing upstream configuration
+
+like($f->getline(), qr/, /, 'log - subordinate good');
+like($f->getline(), qr/, /, 'log - subordinate error');
+
+###############################################################################
+
+sub update_name {
+	my ($name, $plan, $plan6) = @_;
+
+	$plan = 1, $plan6 = 0 if !defined $name;
+	$plan = $plan6 = 1 if !defined $plan;
+	$plan += $plan6 + $plan6;
+
+	sub sock {
+		IO::Socket::INET->new(
+			Proto => 'tcp',
+			PeerAddr => '127.0.0.1:' . port(8084)
+		)
+			or die "Can't connect to nginx: $!\n";
+	}
+
+	$name->{A} = '' unless $name->{A};
+	$name->{AAAA} = '' unless $name->{AAAA};
+	$name->{CNAME} = '' unless $name->{CNAME};
+	$name->{ERROR} = '' unless $name->{ERROR};
+	$name->{SERROR} = '' unless $name->{SERROR};
+	$name->{SRV} = '' unless $name->{SRV};
+
+	my $req =<<EOF;
+GET / HTTP/1.0
+Host: localhost
+X-A: $name->{A}
+X-AAAA: $name->{AAAA}
+X-CNAME: $name->{CNAME}
+X-ERROR: $name->{ERROR}
+X-SERROR: $name->{SERROR}
+X-SRV: $name->{SRV}
+
+EOF
+
+	my ($gen) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+	for (1 .. 10) {
+		my ($gen2) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+
+		# let resolver cache expire to finish upstream reconfiguration
+		select undef, undef, undef, 0.5;
+		last unless ($gen + $plan > $gen2);
+	}
+}
+
+###############################################################################
+
+sub reply_handler {
+	my ($recv_data, $h, $cnt, $tcp) = @_;
+
+	my (@name, @rdata);
+
+	use constant NOERROR	=> 0;
+	use constant FORMERR	=> 1;
+	use constant SERVFAIL	=> 2;
+	use constant NXDOMAIN	=> 3;
+
+	use constant A		=> 1;
+	use constant CNAME	=> 5;
+	use constant AAAA	=> 28;
+	use constant SRV	=> 33;
+
+	use constant IN		=> 1;
+
+	# default values
+
+	my ($hdr, $rcode, $ttl, $port) = (0x8180, NOERROR, 3600, port(8080));
+	$h = {A => [ "127.0.0.1" ], SRV => [ "1 5 $port example.net" ]}
+		unless defined $h;
+
+	# decode name
+
+	my ($len, $offset) = (undef, 12);
+	while (1) {
+		$len = unpack("\@$offset C", $recv_data);
+		last if $len == 0;
+		$offset++;
+		push @name, unpack("\@$offset A$len", $recv_data);
+		$offset += $len;
+	}
+
+	$offset -= 1;
+	my ($id, $type, $class) = unpack("n x$offset n2", $recv_data);
+	my $name = join('.', @name);
+
+	if ($h->{ERROR} && $type == SRV) {
+		$rcode = SERVFAIL;
+		goto bad;
+	}
+
+	# subordinate error
+
+	if ($h->{SERROR} && $type != SRV) {
+		$rcode = SERVFAIL;
+		goto bad;
+	}
+
+	if ($name eq '_http._tcp.example.net') {
+		if ($type == SRV && $h->{SRV}) {
+			map { push @rdata, rd_srv($ttl, (split ' ', $_)) }
+				@{$h->{SRV}};
+		}
+
+		my $cname = defined $h->{CNAME} ? $h->{CNAME} : 0;
+		if ($cname) {
+			push @rdata, pack("n3N nCa5n", 0xc00c, CNAME, IN, $ttl,
+				8, 5, "alias", 0xc00c + length("_http._tcp "));
+		}
+
+	} elsif ($name eq '_http._tcp.trunc.example.net' && $type == SRV) {
+		push @rdata, $tcp
+			? rd_srv($ttl, 1, 1, $port, 'tcp.example.net')
+			: rd_srv($ttl, 1, 1, $port, 'example.net');
+
+		$hdr |= 0x0300 if $name eq '_http._tcp.trunc.example.net'
+			and !$tcp;
+
+	} elsif ($name eq 'example.net' || $name eq 'tcp.example.net') {
+		if ($type == A && $h->{A}) {
+			map { push @rdata, rd_addr($ttl, $_) } @{$h->{A}};
+		}
+		if ($type == AAAA && $h->{AAAA}) {
+			map { push @rdata, rd_addr6($ttl, $_) } @{$h->{AAAA}};
+		}
+		my $cname = defined $h->{CNAME} ? $h->{CNAME} : 0;
+		if ($cname) {
+			push @rdata, pack("n3N nCa5n", 0xc00c, CNAME, IN, $ttl,
+				8, 5, $cname, 0xc00c);
+		}
+
+	} elsif ($name eq 'alias.example.net') {
+		if ($type == SRV) {
+			push @rdata, rd_srv($ttl, 1, 5, $port, 'example.net');
+		}
+		if ($type == A) {
+			push @rdata, rd_addr($ttl, '127.0.0.203');
+		}
+	}
+
+bad:
+
+	Test::Nginx::log_core('||', "DNS: $name $type $rcode");
+
+	$$cnt++ if $type == SRV || keys %$h;
+
+	$len = @name;
+	pack("n6 (C/a*)$len x n2", $id, $hdr | $rcode, 1, scalar @rdata,
+		0, 0, @name, $type, $class) . join('', @rdata);
+}
+
+sub rd_srv {
+	my ($ttl, $pri, $w, $port, $name) = @_;
+	my @rdname = split /\./, $name;
+	my $rdlen = length(join '', @rdname) + @rdname + 7;	# pri w port x
+
+	pack 'n3N n n3 (C/a*)* x',
+		0xc00c, SRV, IN, $ttl, $rdlen, $pri, $w, $port, @rdname;
+}
+
+sub rd_addr {
+	my ($ttl, $addr) = @_;
+
+	my $code = 'split(/\./, $addr)';
+
+	pack 'n3N nC4', 0xc00c, A, IN, $ttl, eval "scalar $code", eval($code);
+}
+
+sub expand_ip6 {
+	my ($addr) = @_;
+
+	substr ($addr, index($addr, "::"), 2) =
+		join "0", map { ":" } (0 .. 8 - (split /:/, $addr) + 1);
+	map { hex "0" x (4 - length $_) . "$_" } split /:/, $addr;
+}
+
+sub rd_addr6 {
+	my ($ttl, $addr) = @_;
+
+	pack 'n3N nn8', 0xc00c, AAAA, IN, $ttl, 16, expand_ip6($addr);
+}
+
+sub dns_daemon {
+	my ($t) = @_;
+	my ($data, $recv_data, $h);
+
+	my $socket = IO::Socket::INET->new(
+		LocalAddr => '127.0.0.1',
+		LocalPort => port(8981),
+		Proto => 'udp',
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $control = IO::Socket::INET->new(
+		Proto => 'tcp',
+		LocalHost => '127.0.0.1:' . port(8084),
+		Listen => 5,
+		Reuse => 1
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $tcp = port(8981, socket => 1);
+	my $sel = IO::Select->new($socket, $control, $tcp);
+
+	local $SIG{PIPE} = 'IGNORE';
+
+	# signal we are ready
+
+	open my $fh, '>', $t->testdir() . '/' . port(8981);
+	close $fh;
+	my $cnt = 0;
+
+	while (my @ready = $sel->can_read) {
+		foreach my $fh (@ready) {
+			if ($control == $fh || $tcp == $fh) {
+				my $new = $fh->accept;
+				$new->autoflush(1);
+				$sel->add($new);
+
+			} elsif ($socket == $fh) {
+				$fh->recv($recv_data, 65536);
+				$data = reply_handler($recv_data, $h, \$cnt);
+				$fh->send($data);
+
+			} elsif ($fh->sockport() == port(8084)) {
+				$h = process_name($fh, $cnt);
+				$sel->remove($fh);
+				$fh->close;
+
+			} elsif ($fh->sockport() == port(8981)) {
+				$fh->recv($recv_data, 65536);
+				unless (length $recv_data) {
+					$sel->remove($fh);
+					$fh->close;
+					next;
+				}
+
+again:
+				my $len = unpack("n", $recv_data);
+				my $data = substr $recv_data, 2, $len;
+				$data = reply_handler($data, $h, \$cnt, 1);
+				$data = pack("n", length $data) . $data;
+				$fh->send($data);
+				$recv_data = substr $recv_data, 2 + $len;
+				goto again if length $recv_data;
+			}
+		}
+	}
+}
+
+# parse dns update
+
+sub process_name {
+	my ($client, $cnt) = @_;
+	my $port = $client->sockport();
+
+	my $headers = '';
+	my $uri = '';
+	my %h;
+
+	while (<$client>) {
+		$headers .= $_;
+		last if (/^\x0d?\x0a?$/);
+	}
+	return 1 if $headers eq '';
+
+	$uri = $1 if $headers =~ /^\S+\s+([^ ]+)\s+HTTP/i;
+	return 1 if $uri eq '';
+
+	$headers =~ /X-A: (.*)$/m;
+	map { push @{$h{A}}, $_ } split(/ /, $1);
+	$headers =~ /X-AAAA: (.*)$/m;
+	map { push @{$h{AAAA}}, $_ } split(/ /, $1);
+	$headers =~ /X-SRV: (.*)$/m;
+	map { push @{$h{SRV}}, $_ } split(/;/, $1);
+	$headers =~ /X-CNAME: (.+)$/m and $h{CNAME} = $1;
+	$headers =~ /X-ERROR: (.+)$/m and $h{ERROR} = $1;
+	$headers =~ /X-SERROR: (.+)$/m and $h{SERROR} = $1;
+
+	Test::Nginx::log_core('||', "$port: response, 200");
+	print $client <<EOF;
+HTTP/1.1 200 OK
+Connection: close
+X-Gen: $cnt
+
+OK
+EOF
+
+	return \%h;
+}
+
+###############################################################################
diff --git a/stream_upstream_service_reload.t b/stream_upstream_service_reload.t
new file mode 100644
--- /dev/null
+++ b/stream_upstream_service_reload.t
@@ -0,0 +1,324 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+
+# Stream tests for dynamic upstream configuration with service (SRV) feature.
+# Ensure that upstream configuration is inherited on reload.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+use IO::Select;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+use Test::Nginx::Stream qw/ stream /;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/stream stream_upstream_zone http/);
+
+$t->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+stream {
+    %%TEST_GLOBALS_STREAM%%
+
+    upstream u {
+        zone z 1m;
+        server example.net resolve service=http;
+    }
+
+    # lower the retry timeout after empty reply
+    resolver 127.0.0.1:%%PORT_8980_UDP%% valid=1s;
+    # retry query shortly after DNS is started
+    resolver_timeout 1s;
+
+    log_format test $upstream_addr;
+
+    server {
+        listen 127.0.0.1:8082;
+        proxy_pass u;
+        proxy_connect_timeout 50ms;
+        access_log %%TESTDIR%%/cc.log test;
+    }
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    server {
+        listen       127.0.0.1:8080;
+        server_name  localhost;
+    }
+}
+
+EOF
+
+port(8081);
+
+$t->run_daemon(\&dns_daemon, $t)->waitforfile($t->testdir . '/' . port(8980));
+$t->try_run('no resolve in upstream server')->plan(6);
+
+###############################################################################
+
+update_name({A => '127.0.0.201', SRV => "1 5 8080 example.net"});
+stream('127.0.0.1:' . port(8082))->read();
+stream('127.0.0.1:' . port(8082))->read();
+
+update_name({ERROR => 'SERVFAIL'}, 0);
+
+$t->reload();
+waitforworker($t);
+
+stream('127.0.0.1:' . port(8082))->read();
+stream('127.0.0.1:' . port(8082))->read();
+
+update_name({A => '127.0.0.202', SRV => "1 5 8080 example.net"});
+stream('127.0.0.1:' . port(8082))->read();
+stream('127.0.0.1:' . port(8082))->read();
+
+$t->stop();
+
+Test::Nginx::log_core('||', $t->read_file('cc.log'));
+
+open my $f, '<', "${\($t->testdir())}/cc.log" or die "Can't open cc.log: $!";
+
+like($f->getline(), qr/127.0.0.201:8080/, 'log - before');
+like($f->getline(), qr/127.0.0.201:8080/, 'log - before 2');
+
+like($f->getline(), qr/127.0.0.201:8080/, 'log - preresolve');
+like($f->getline(), qr/127.0.0.201:8080/, 'log - preresolve 2');
+
+like($f->getline(), qr/127.0.0.202:8080/, 'log - update');
+like($f->getline(), qr/127.0.0.202:8080/, 'log - update 2');
+
+###############################################################################
+
+sub waitforworker {
+	my ($t) = @_;
+
+	for (1 .. 30) {
+		last if $t->read_file('error.log') =~ /exited with code/;
+		select undef, undef, undef, 0.2;
+	}
+}
+
+sub update_name {
+	my ($name, $plan) = @_;
+
+	$plan = 3 if !defined $plan;
+
+	sub sock {
+		IO::Socket::INET->new(
+			Proto => 'tcp',
+			PeerAddr => '127.0.0.1:' . port(8081)
+		)
+			or die "Can't connect to nginx: $!\n";
+	}
+
+	$name->{A} = '' unless $name->{A};
+	$name->{ERROR} = '' unless $name->{ERROR};
+	$name->{SRV} = '' unless $name->{SRV};
+
+	my $req =<<EOF;
+GET / HTTP/1.0
+Host: localhost
+X-A: $name->{A}
+X-ERROR: $name->{ERROR}
+X-SRV: $name->{SRV}
+
+EOF
+
+	my ($gen) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+	for (1 .. 10) {
+		my ($gen2) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+
+		# let resolver cache expire to finish upstream reconfiguration
+		select undef, undef, undef, 0.5;
+		last unless ($gen + $plan > $gen2);
+	}
+}
+
+###############################################################################
+
+sub reply_handler {
+	my ($recv_data, $h) = @_;
+
+	my (@name, @rdata);
+
+	use constant NOERROR	=> 0;
+	use constant SERVFAIL	=> 2;
+	use constant NXDOMAIN	=> 3;
+
+	use constant A		=> 1;
+	use constant SRV	=> 33;
+	use constant IN		=> 1;
+
+	# default values
+
+	my ($hdr, $rcode, $ttl, $port) = (0x8180, NOERROR, 3600, port(8080));
+	$h = {A => [ "127.0.0.201" ], SRV => [ "1 5 $port example.net" ]}
+		unless defined $h;
+
+	# decode name
+
+	my ($len, $offset) = (undef, 12);
+	while (1) {
+		$len = unpack("\@$offset C", $recv_data);
+		last if $len == 0;
+		$offset++;
+		push @name, unpack("\@$offset A$len", $recv_data);
+		$offset += $len;
+	}
+
+	$offset -= 1;
+	my ($id, $type, $class) = unpack("n x$offset n2", $recv_data);
+	my $name = join('.', @name);
+
+	if ($h->{ERROR}) {
+		$rcode = SERVFAIL;
+		goto bad;
+	}
+
+	if ($name eq 'example.net' && $type == A && $h->{A}) {
+		map { push @rdata, rd_addr($ttl, $_) } @{$h->{A}};
+
+	}
+	if ($name eq '_http._tcp.example.net' && $type == SRV && $h->{SRV}) {
+		map { push @rdata, rd_srv($ttl, (split ' ', $_)) }
+			@{$h->{SRV}};
+	}
+
+bad:
+
+	Test::Nginx::log_core('||', "DNS: $name $type $rcode");
+
+	$len = @name;
+	pack("n6 (C/a*)$len x n2", $id, $hdr | $rcode, 1, scalar @rdata,
+		0, 0, @name, $type, $class) . join('', @rdata);
+}
+
+sub rd_srv {
+	my ($ttl, $pri, $w, $port, $name) = @_;
+	my @rdname = split /\./, $name;
+	my $rdlen = length(join '', @rdname) + @rdname + 7;	# pri w port x
+
+	pack 'n3N n n3 (C/a*)* x',
+		0xc00c, SRV, IN, $ttl, $rdlen, $pri, $w, $port, @rdname;
+}
+
+sub rd_addr {
+	my ($ttl, $addr) = @_;
+
+	my $code = 'split(/\./, $addr)';
+
+	pack 'n3N nC4', 0xc00c, A, IN, $ttl, eval "scalar $code", eval($code);
+}
+
+sub dns_daemon {
+	my ($t) = @_;
+	my ($data, $recv_data, $h);
+
+	my $socket = IO::Socket::INET->new(
+		LocalAddr => '127.0.0.1',
+		LocalPort => port(8980),
+		Proto => 'udp',
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $control = IO::Socket::INET->new(
+		Proto => 'tcp',
+		LocalHost => "127.0.0.1:" . port(8081),
+		Listen => 5,
+		Reuse => 1
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $sel = IO::Select->new($socket, $control);
+
+	local $SIG{PIPE} = 'IGNORE';
+
+	# signal we are ready
+
+	open my $fh, '>', $t->testdir() . '/' . port(8980);
+	close $fh;
+	my $cnt = 0;
+
+	while (my @ready = $sel->can_read) {
+		foreach my $fh (@ready) {
+			if ($control == $fh) {
+				my $new = $fh->accept;
+				$new->autoflush(1);
+				$sel->add($new);
+
+			} elsif ($socket == $fh) {
+				$fh->recv($recv_data, 65536);
+				$data = reply_handler($recv_data, $h);
+				$fh->send($data);
+				$cnt++;
+
+			} else {
+				$h = process_name($fh, $cnt);
+				$sel->remove($fh);
+				$fh->close;
+			}
+		}
+	}
+}
+
+# parse dns update
+
+sub process_name {
+	my ($client, $cnt) = @_;
+	my $port = $client->sockport();
+
+	my $headers = '';
+	my $uri = '';
+	my %h;
+
+	while (<$client>) {
+		$headers .= $_;
+		last if (/^\x0d?\x0a?$/);
+	}
+	return 1 if $headers eq '';
+
+	$uri = $1 if $headers =~ /^\S+\s+([^ ]+)\s+HTTP/i;
+	return 1 if $uri eq '';
+
+	$headers =~ /X-A: (.*)$/m;
+	map { push @{$h{A}}, $_ } split(/ /, $1);
+	$headers =~ /X-SRV: (.*)$/m;
+	map { push @{$h{SRV}}, $_ } split(/;/, $1);
+	$headers =~ /X-ERROR: (.*)$/m;
+	$h{ERROR} = $1;
+
+	Test::Nginx::log_core('||', "$port: response, 200");
+	print $client <<EOF;
+HTTP/1.1 200 OK
+Connection: close
+X-Gen: $cnt
+
+OK
+EOF
+
+	return \%h;
+}
+
+###############################################################################
diff --git a/upstream_resolve.t b/upstream_resolve.t
new file mode 100644
--- /dev/null
+++ b/upstream_resolve.t
@@ -0,0 +1,368 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+
+# Tests for dynamic upstream configuration with re-resolvable servers.
+# Ensure that dns updates are properly applied.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+use IO::Select;
+use Socket qw/ CRLF /;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http proxy upstream_zone/);
+
+$t->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    upstream u {
+        zone z 1m;
+        server example.net:%%PORT_8080%% resolve max_fails=0;
+    }
+
+    # lower the retry timeout after empty reply
+    resolver 127.0.0.1:%%PORT_8982_UDP%% valid=1s;
+    # retry query shortly after DNS is started
+    resolver_timeout 1s;
+
+    server {
+        listen       127.0.0.1:8080;
+        listen       [::1]:%%PORT_8080%%;
+        server_name  localhost;
+
+        location / {
+            proxy_pass http://u/t;
+            proxy_connect_timeout 50ms;
+            add_header X-IP $upstream_addr;
+            error_page 502 504 redirect;
+        }
+
+        location /2 {
+            proxy_pass http://u/t;
+            add_header X-IP $upstream_addr;
+        }
+
+        location /t { }
+    }
+}
+
+EOF
+
+port(8083);
+
+$t->write_file('t', '');
+
+$t->run_daemon(\&dns_daemon, $t)->waitforfile($t->testdir . '/' . port(8982));
+$t->try_run('no resolve in upstream server')->plan(18);
+
+###############################################################################
+
+my ($r, @n);
+my $p0 = port(8080);
+
+update_name({A => '127.0.0.201'});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 1, 'A');
+like($r, qr/127.0.0.201:$p0/, 'A 1');
+
+# A changed
+
+update_name({A => '127.0.0.202'});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 1, 'A changed');
+like($r, qr/127.0.0.202:$p0/, 'A changed 1');
+
+# 1 more A added
+
+update_name({A => '127.0.0.201 127.0.0.202'});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 2, 'A A');
+like($r, qr/127.0.0.201:$p0/, 'A A 1');
+like($r, qr/127.0.0.202:$p0/, 'A A 2');
+
+# 1 A removed, 2 AAAA added
+
+update_name({A => '127.0.0.201', AAAA => 'fe80::1 fe80::2'});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 3, 'A AAAA AAAA responses');
+like($r, qr/127.0.0.201:$p0/, 'A AAAA AAAA 1');
+like($r, qr/\[fe80::1\]:$p0/, 'A AAAA AAAA 2');
+like($r, qr/\[fe80::1\]:$p0/, 'A AAAA AAAA 3');
+
+# all records removed
+
+update_name();
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 0, 'empty response');
+
+# A added after empty
+
+update_name({A => '127.0.0.201'});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 1, 'A added');
+like($r, qr/127.0.0.201:$p0/, 'A added 1');
+
+# changed to CNAME
+
+update_name({CNAME => 'alias'}, 4);
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 1, 'CNAME');
+like($r, qr/127.0.0.203:$p0/, 'CNAME 1');
+
+# bad DNS reply should not affect existing upstream configuration
+
+update_name({ERROR => 'SERVFAIL'});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 1, 'ERROR');
+like($r, qr/127.0.0.203:$p0/, 'ERROR 1');
+update_name({A => '127.0.0.1'});
+
+###############################################################################
+
+sub update_name {
+	my ($name, $plan) = @_;
+
+	$plan = 2 if !defined $plan;
+
+	sub sock {
+		IO::Socket::INET->new(
+			Proto => 'tcp',
+			PeerAddr => '127.0.0.1:' . port(8083)
+		)
+			or die "Can't connect to nginx: $!\n";
+	}
+
+	$name->{A} = '' unless $name->{A};
+	$name->{AAAA} = '' unless $name->{AAAA};
+	$name->{CNAME} = '' unless $name->{CNAME};
+	$name->{ERROR} = '' unless $name->{ERROR};
+
+	my $req =<<EOF;
+GET / HTTP/1.0
+Host: localhost
+X-A: $name->{A}
+X-AAAA: $name->{AAAA}
+X-CNAME: $name->{CNAME}
+X-ERROR: $name->{ERROR}
+
+EOF
+
+	my ($gen) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+	for (1 .. 10) {
+		my ($gen2) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+
+		# let resolver cache expire to finish upstream reconfiguration
+		select undef, undef, undef, 0.5;
+		last unless ($gen + $plan > $gen2);
+	}
+}
+
+###############################################################################
+
+sub reply_handler {
+	my ($recv_data, $h) = @_;
+
+	my (@name, @rdata);
+
+	use constant NOERROR	=> 0;
+	use constant SERVFAIL	=> 2;
+	use constant NXDOMAIN	=> 3;
+
+	use constant A		=> 1;
+	use constant CNAME	=> 5;
+	use constant AAAA	=> 28;
+	use constant DNAME	=> 39;
+	use constant IN		=> 1;
+
+	# default values
+
+	my ($hdr, $rcode, $ttl) = (0x8180, NOERROR, 1);
+	$h = {A => [ "127.0.0.201" ]} unless defined $h;
+
+	# decode name
+
+	my ($len, $offset) = (undef, 12);
+	while (1) {
+		$len = unpack("\@$offset C", $recv_data);
+		last if $len == 0;
+		$offset++;
+		push @name, unpack("\@$offset A$len", $recv_data);
+		$offset += $len;
+	}
+
+	$offset -= 1;
+	my ($id, $type, $class) = unpack("n x$offset n2", $recv_data);
+	my $name = join('.', @name);
+
+	if ($h->{ERROR}) {
+		$rcode = SERVFAIL;
+		goto bad;
+	}
+
+	if ($name eq 'example.net') {
+		if ($type == A && $h->{A}) {
+			map { push @rdata, rd_addr($ttl, $_) } @{$h->{A}};
+		}
+		if ($type == AAAA && $h->{AAAA}) {
+			map { push @rdata, rd_addr6($ttl, $_) } @{$h->{AAAA}};
+		}
+		my $cname = defined $h->{CNAME} ? $h->{CNAME} : 0;
+		if ($cname) {
+			push @rdata, pack("n3N nCa5n", 0xc00c, CNAME, IN, $ttl,
+				8, 5, $cname, 0xc00c);
+		}
+
+	} elsif ($name eq 'alias.example.net') {
+		if ($type == A) {
+			push @rdata, rd_addr($ttl, '127.0.0.203');
+		}
+	}
+
+bad:
+
+	Test::Nginx::log_core('||', "DNS: $name $type $rcode");
+
+	$len = @name;
+	pack("n6 (C/a*)$len x n2", $id, $hdr | $rcode, 1, scalar @rdata,
+		0, 0, @name, $type, $class) . join('', @rdata);
+}
+
+sub rd_addr {
+	my ($ttl, $addr) = @_;
+
+	my $code = 'split(/\./, $addr)';
+
+	pack 'n3N nC4', 0xc00c, A, IN, $ttl, eval "scalar $code", eval($code);
+}
+
+sub expand_ip6 {
+	my ($addr) = @_;
+
+	substr ($addr, index($addr, "::"), 2) =
+		join "0", map { ":" } (0 .. 8 - (split /:/, $addr) + 1);
+	map { hex "0" x (4 - length $_) . "$_" } split /:/, $addr;
+}
+
+sub rd_addr6 {
+	my ($ttl, $addr) = @_;
+
+	pack 'n3N nn8', 0xc00c, AAAA, IN, $ttl, 16, expand_ip6($addr);
+}
+
+sub dns_daemon {
+	my ($t) = @_;
+	my ($data, $recv_data, $h);
+
+	my $socket = IO::Socket::INET->new(
+		LocalAddr => '127.0.0.1',
+		LocalPort => port(8982),
+		Proto=> 'udp',
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $control = IO::Socket::INET->new(
+		Proto => 'tcp',
+		LocalHost => "127.0.0.1:" . port(8083),
+		Listen => 5,
+		Reuse => 1
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $sel = IO::Select->new($socket, $control);
+
+	local $SIG{PIPE} = 'IGNORE';
+
+	# signal we are ready
+
+	open my $fh, '>', $t->testdir() . '/' . port(8982);
+	close $fh;
+	my $cnt = 0;
+
+	while (my @ready = $sel->can_read) {
+		foreach my $fh (@ready) {
+			if ($control == $fh) {
+				my $new = $fh->accept;
+				$new->autoflush(1);
+				$sel->add($new);
+
+			} elsif ($socket == $fh) {
+				$fh->recv($recv_data, 65536);
+				$data = reply_handler($recv_data, $h);
+				$fh->send($data);
+				$cnt++;
+
+			} else {
+				$h = process_name($fh, $cnt);
+				$sel->remove($fh);
+				$fh->close;
+			}
+		}
+	}
+}
+
+# parse dns update
+
+sub process_name {
+	my ($client, $cnt) = @_;
+	my $port = $client->sockport();
+
+	my $headers = '';
+	my $uri = '';
+	my %h;
+
+	while (<$client>) {
+		$headers .= $_;
+		last if (/^\x0d?\x0a?$/);
+	}
+	return 1 if $headers eq '';
+
+	$uri = $1 if $headers =~ /^\S+\s+([^ ]+)\s+HTTP/i;
+	return 1 if $uri eq '';
+
+	$headers =~ /X-A: (.*)$/m;
+	map { push @{$h{A}}, $_ } split(/ /, $1);
+	$headers =~ /X-AAAA: (.*)$/m;
+	map { push @{$h{AAAA}}, $_ } split(/ /, $1);
+	$headers =~ /X-CNAME: (.*)$/m;
+	$h{CNAME} = $1;
+	$headers =~ /X-ERROR: (.*)$/m;
+	$h{ERROR} = $1;
+
+	Test::Nginx::log_core('||', "$port: response, 200");
+	print $client <<EOF;
+HTTP/1.1 200 OK
+Connection: close
+X-Gen: $cnt
+
+OK
+EOF
+
+	return \%h;
+}
+
+###############################################################################
diff --git a/upstream_resolve_reload.t b/upstream_resolve_reload.t
new file mode 100644
--- /dev/null
+++ b/upstream_resolve_reload.t
@@ -0,0 +1,304 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+
+# Tests for dynamic upstream configuration with re-resolvable servers.
+# Ensure that upstream configuration is inherited on reload.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+use IO::Select;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http proxy upstream_zone/);
+
+$t->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    upstream u {
+        zone z 1m;
+        server example.net:%%PORT_8081%% resolve;
+    }
+
+    upstream u2 {
+        zone z 1m;
+        server 127.0.0.203:%%PORT_8081%% max_fails=0;
+        server example.net:%%PORT_8081%% resolve max_fails=0;
+    }
+
+    # lower the retry timeout after empty reply
+    resolver 127.0.0.1:%%PORT_8980_UDP%% valid=1s;
+    # retry query shortly after DNS is started
+    resolver_timeout 1s;
+
+    server {
+        listen       127.0.0.1:8080;
+        server_name  localhost;
+
+        location / {
+            proxy_pass http://u;
+            proxy_connect_timeout 50ms;
+            add_header X-IP $upstream_addr always;
+        }
+
+        location /2 {
+            proxy_pass http://u2;
+            proxy_connect_timeout 50ms;
+            add_header X-IP $upstream_addr always;
+        }
+    }
+}
+
+EOF
+
+$t->run_daemon(\&dns_daemon, $t)->waitforfile($t->testdir . '/' . port(8980));
+$t->try_run('no resolve in upstream server')->plan(9);
+
+###############################################################################
+
+my $p = port(8081);
+
+update_name({A => '127.0.0.201'});
+like(http_get('/'), qr/X-IP: 127.0.0.201:$p/, 'reload - before - request');
+like(http_get('/'), qr/X-IP: 127.0.0.201:$p/, 'reload - before - request 2');
+like(http_get('/2'), qr/127.0.0.(201:$p, 127.0.0.203|203:$p, 127.0.0.201):$p/,
+	'reload - before - many');
+
+update_name({ERROR => 'SERVFAIL'}, 0);
+
+my $conf = $t->read_file('nginx.conf');
+$conf =~ s/$p/port(8082)/gmse;
+$p = port(8082);
+$t->write_file('nginx.conf', $conf);
+
+$t->reload();
+waitforworker($t);
+
+like(http_get('/'), qr/X-IP: 127.0.0.201:$p/, 'reload - preresolve - request');
+like(http_get('/'), qr/X-IP: 127.0.0.201:$p/, 'reload - preresolve - request 2');
+like(http_get('/2'), qr/127.0.0.(201:$p, 127.0.0.203|203:$p, 127.0.0.201):$p/,
+	'reload - preresolve - many');
+
+update_name({A => '127.0.0.202'});
+like(http_get('/'), qr/X-IP: 127.0.0.202:$p/, 'reload - update - request');
+like(http_get('/'), qr/X-IP: 127.0.0.202:$p/, 'reload - update - request 2');
+like(http_get('/2'), qr/127.0.0.(202:$p, 127.0.0.203|203:$p, 127.0.0.202):$p/,
+	'reload - update - many');
+
+###############################################################################
+
+sub waitforworker {
+	my ($t) = @_;
+
+	for (1 .. 30) {
+		last if $t->read_file('error.log') =~ /exited with code/;
+		select undef, undef, undef, 0.2;
+	}
+}
+
+sub update_name {
+	my ($name, $plan) = @_;
+
+	$plan = 2 if !defined $plan;
+
+	sub sock {
+		IO::Socket::INET->new(
+			Proto => 'tcp',
+			PeerAddr => '127.0.0.1:' . port(8081)
+		)
+			or die "Can't connect to nginx: $!\n";
+	}
+
+	$name->{A} = '' unless $name->{A};
+	$name->{ERROR} = '' unless $name->{ERROR};
+
+	my $req =<<EOF;
+GET / HTTP/1.0
+Host: localhost
+X-A: $name->{A}
+X-ERROR: $name->{ERROR}
+
+EOF
+
+	my ($gen) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+	for (1 .. 10) {
+		my ($gen2) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+
+		# let resolver cache expire to finish upstream reconfiguration
+		select undef, undef, undef, 0.5;
+		last unless ($gen + $plan > $gen2);
+	}
+}
+
+###############################################################################
+
+sub reply_handler {
+	my ($recv_data, $h) = @_;
+
+	my (@name, @rdata);
+
+	use constant NOERROR	=> 0;
+	use constant SERVFAIL	=> 2;
+	use constant NXDOMAIN	=> 3;
+
+	use constant A		=> 1;
+	use constant IN		=> 1;
+
+	# default values
+
+	my ($hdr, $rcode, $ttl) = (0x8180, NOERROR, 1);
+	$h = {A => [ "127.0.0.201" ]} unless defined $h;
+
+	# decode name
+
+	my ($len, $offset) = (undef, 12);
+	while (1) {
+		$len = unpack("\@$offset C", $recv_data);
+		last if $len == 0;
+		$offset++;
+		push @name, unpack("\@$offset A$len", $recv_data);
+		$offset += $len;
+	}
+
+	$offset -= 1;
+	my ($id, $type, $class) = unpack("n x$offset n2", $recv_data);
+	my $name = join('.', @name);
+
+	if ($h->{ERROR}) {
+		$rcode = SERVFAIL;
+		goto bad;
+	}
+
+	if ($name eq 'example.net' && $type == A && $h->{A}) {
+		map { push @rdata, rd_addr($ttl, $_) } @{$h->{A}};
+	}
+
+bad:
+
+	Test::Nginx::log_core('||', "DNS: $name $type $rcode");
+
+	$len = @name;
+	pack("n6 (C/a*)$len x n2", $id, $hdr | $rcode, 1, scalar @rdata,
+		0, 0, @name, $type, $class) . join('', @rdata);
+}
+
+sub rd_addr {
+	my ($ttl, $addr) = @_;
+
+	my $code = 'split(/\./, $addr)';
+
+	pack 'n3N nC4', 0xc00c, A, IN, $ttl, eval "scalar $code", eval($code);
+}
+
+sub dns_daemon {
+	my ($t) = @_;
+	my ($data, $recv_data, $h);
+
+	my $socket = IO::Socket::INET->new(
+		LocalAddr => '127.0.0.1',
+		LocalPort => port(8980),
+		Proto => 'udp',
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $control = IO::Socket::INET->new(
+		Proto => 'tcp',
+		LocalHost => "127.0.0.1:" . port(8081),
+		Listen => 5,
+		Reuse => 1
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $sel = IO::Select->new($socket, $control);
+
+	local $SIG{PIPE} = 'IGNORE';
+
+	# signal we are ready
+
+	open my $fh, '>', $t->testdir() . '/' . port(8980);
+	close $fh;
+	my $cnt = 0;
+
+	while (my @ready = $sel->can_read) {
+		foreach my $fh (@ready) {
+			if ($control == $fh) {
+				my $new = $fh->accept;
+				$new->autoflush(1);
+				$sel->add($new);
+
+			} elsif ($socket == $fh) {
+				$fh->recv($recv_data, 65536);
+				$data = reply_handler($recv_data, $h);
+				$fh->send($data);
+				$cnt++;
+
+			} else {
+				$h = process_name($fh, $cnt);
+				$sel->remove($fh);
+				$fh->close;
+			}
+		}
+	}
+}
+
+# parse dns update
+
+sub process_name {
+	my ($client, $cnt) = @_;
+	my $port = $client->sockport();
+
+	my $headers = '';
+	my $uri = '';
+	my %h;
+
+	while (<$client>) {
+		$headers .= $_;
+		last if (/^\x0d?\x0a?$/);
+	}
+	return 1 if $headers eq '';
+
+	$uri = $1 if $headers =~ /^\S+\s+([^ ]+)\s+HTTP/i;
+	return 1 if $uri eq '';
+
+	$headers =~ /X-A: (.*)$/m;
+	map { push @{$h{A}}, $_ } split(/ /, $1);
+	$headers =~ /X-ERROR: (.*)$/m;
+	$h{ERROR} = $1;
+
+	Test::Nginx::log_core('||', "$port: response, 200");
+	print $client <<EOF;
+HTTP/1.1 200 OK
+Connection: close
+X-Gen: $cnt
+
+OK
+EOF
+
+	return \%h;
+}
+
+###############################################################################
diff --git a/upstream_resolver.t b/upstream_resolver.t
new file mode 100644
--- /dev/null
+++ b/upstream_resolver.t
@@ -0,0 +1,265 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+
+# Tests for re-resolvable servers with resolver in http upstream.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+use IO::Select;
+use Socket qw/ CRLF /;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http proxy upstream_zone/);
+
+$t->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    resolver 127.0.0.1:%%PORT_8980_UDP%%;
+
+    upstream u {
+        zone z 1m;
+        server example.net:%%PORT_8080%% resolve;
+    }
+
+    upstream u1 {
+        zone z 1m;
+        server example.net:%%PORT_8080%% resolve;
+        resolver 127.0.0.1:%%PORT_8981_UDP%%;
+    }
+
+    upstream u2 {
+        zone z 1m;
+        server example.net:%%PORT_8080%% resolve;
+        resolver 127.0.0.1:%%PORT_8982_UDP%%;
+        resolver_timeout 200s; # for coverage
+    }
+
+    server {
+        listen       127.0.0.1:8080;
+        server_name  localhost;
+
+        location / {
+            proxy_pass http://$args/t;
+            proxy_connect_timeout 50ms;
+            add_header X-IP $upstream_addr;
+            error_page 502 504 redirect;
+        }
+
+    }
+}
+
+EOF
+
+$t->run_daemon(\&dns_daemon, $t, port($_), port($_ + 10)) for (8980 .. 8982);
+$t->waitforfile($t->testdir . '/' . port($_)) for (8980 .. 8982);
+
+$t->try_run('no resolver in upstream')->plan(6);
+
+###############################################################################
+
+ok(waitfordns(8980), 'resolved');
+ok(waitfordns(8981), 'resolved in upstream 1');
+ok(waitfordns(8982), 'resolved in upstream 2');
+
+like(http_get('/?u'), qr/127.0.0.200/, 'resolver');
+like(http_get('/?u1'), qr/127.0.0.201/, 'resolver upstream 1');
+like(http_get('/?u2'), qr/127.0.0.202/, 'resolver upstream 2');
+
+###############################################################################
+
+sub waitfordns {
+	my ($port, $plan) = @_;
+
+	$plan = 1 if !defined $plan;
+
+	sub sock {
+		my ($port) = @_;
+
+		IO::Socket::INET->new(
+			Proto => 'tcp',
+			PeerAddr => '127.0.0.1:' . port($port + 10)
+		)
+			or die "Can't connect to dns control socket: $!\n";
+	}
+
+	my $req =<<EOF;
+GET / HTTP/1.0
+Host: localhost
+
+EOF
+
+	for (1 .. 10) {
+		my ($gen) = http($req, socket => sock($port)) =~ /X-Gen: (\d+)/;
+		select undef, undef, undef, 0.5;
+		return 1 if $gen >= $plan;
+	}
+}
+
+###############################################################################
+
+sub reply_handler {
+	my ($recv_data, $port) = @_;
+
+	my (@name, @rdata);
+
+	use constant NOERROR	=> 0;
+	use constant A		=> 1;
+	use constant IN		=> 1;
+
+	# default values
+
+	my ($hdr, $rcode, $ttl) = (0x8180, NOERROR, 1);
+
+	# decode name
+
+	my ($len, $offset) = (undef, 12);
+	while (1) {
+		$len = unpack("\@$offset C", $recv_data);
+		last if $len == 0;
+		$offset++;
+		push @name, unpack("\@$offset A$len", $recv_data);
+		$offset += $len;
+	}
+
+	$offset -= 1;
+	my ($id, $type, $class) = unpack("n x$offset n2", $recv_data);
+
+	my $name = join('.', @name);
+	if ($name eq 'example.net' && $type == A) {
+		if ($port == port(8980)) {
+			push @rdata, rd_addr($ttl, "127.0.0.200");
+		}
+
+		if ($port == port(8981)) {
+			push @rdata, rd_addr($ttl, "127.0.0.201");
+		}
+
+		if ($port == port(8982)) {
+			push @rdata, rd_addr($ttl, "127.0.0.202");
+		}
+	}
+
+	Test::Nginx::log_core('||', "DNS: $name $type $rcode");
+
+	$len = @name;
+	pack("n6 (C/a*)$len x n2", $id, $hdr | $rcode, 1, scalar @rdata,
+		0, 0, @name, $type, $class) . join('', @rdata);
+}
+
+sub rd_addr {
+	my ($ttl, $addr) = @_;
+
+	my $code = 'split(/\./, $addr)';
+
+	pack 'n3N nC4', 0xc00c, A, IN, $ttl, eval "scalar $code", eval($code);
+}
+
+sub dns_daemon {
+	my ($t, $port, $control_port) = @_;
+
+	my ($data, $recv_data);
+	my $socket = IO::Socket::INET->new(
+		LocalAddr => '127.0.0.1',
+		LocalPort => $port,
+		Proto => 'udp',
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $control = IO::Socket::INET->new(
+		Proto => 'tcp',
+		LocalAddr => '127.0.0.1',
+		LocalPort => $control_port,
+		Listen => 5,
+		Reuse => 1
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $sel = IO::Select->new($socket, $control);
+
+	local $SIG{PIPE} = 'IGNORE';
+
+	# signal we are ready
+
+	open my $fh, '>', $t->testdir() . '/' . $port;
+	close $fh;
+	my $cnt = 0;
+
+	while (my @ready = $sel->can_read) {
+		foreach my $fh (@ready) {
+			if ($control == $fh) {
+				my $new = $fh->accept;
+				$new->autoflush(1);
+				$sel->add($new);
+
+			} elsif ($socket == $fh) {
+				$fh->recv($recv_data, 65536);
+				$data = reply_handler($recv_data, $port);
+				$fh->send($data);
+				$cnt++;
+
+			} else {
+				process_name($fh, $cnt);
+				$sel->remove($fh);
+				$fh->close;
+			}
+		}
+	}
+}
+
+# parse dns update
+
+sub process_name {
+	my ($client, $cnt) = @_;
+	my $port = $client->sockport();
+
+	my $headers = '';
+	my $uri = '';
+	my %h;
+
+	while (<$client>) {
+		$headers .= $_;
+		last if (/^\x0d?\x0a?$/);
+	}
+	return 1 if $headers eq '';
+
+	$uri = $1 if $headers =~ /^\S+\s+([^ ]+)\s+HTTP/i;
+	return 1 if $uri eq '';
+
+	Test::Nginx::log_core('||', "$port: response, 200");
+	print $client <<EOF;
+HTTP/1.1 200 OK
+Connection: close
+X-Gen: $cnt
+
+OK
+EOF
+
+	return \%h;
+}
+
+###############################################################################
diff --git a/upstream_service.t b/upstream_service.t
new file mode 100644
--- /dev/null
+++ b/upstream_service.t
@@ -0,0 +1,489 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+
+# Tests for dynamic upstream configuration with service (SRV) feature.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+use IO::Select;
+use Socket qw/ CRLF /;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http proxy upstream_zone/);
+
+$t->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    upstream u {
+        zone z 1m;
+        server example.net resolve service=http max_fails=0;
+    }
+
+    upstream u2 {
+        zone z2 1m;
+        server example.net resolve service=_http._tcp;
+    }
+
+    # lower the retry timeout after empty reply
+    resolver 127.0.0.1:%%PORT_8981_UDP%% valid=1s;
+    # retry query shortly after DNS is started
+    resolver_timeout 1s;
+
+    server {
+        listen       127.0.0.1:8080;
+        server_name  localhost;
+
+        add_header X-IP $upstream_addr;
+        error_page 502 504 redirect;
+        proxy_connect_timeout 50ms;
+
+        location / {
+            proxy_pass http://u/t;
+        }
+
+        location /full {
+            proxy_pass http://u2/t;
+        }
+
+        location /t { }
+    }
+}
+
+EOF
+
+port(8084);
+
+$t->write_file('t', '');
+
+$t->run_daemon(\&dns_daemon, $t)->waitforfile($t->testdir . '/' . port(8981));
+port(8981, socket => 1)->close();
+$t->try_run('no service in upstream server')->plan(30);
+
+###############################################################################
+
+my ($r, @n);
+my ($p0, $p2, $p3) = (port(8080), port(8082), port(8083));
+
+update_name({A => '127.0.0.201', SRV => "1 5 $p0 example.net"});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 1, 'A');
+like($r, qr/127.0.0.201:$p0/, 'A 1');
+
+# fully specified service
+
+$r = http_get('/full');
+is(@n = $r =~ /:$p0/g, 1, 'A full');
+like($r, qr/127.0.0.201:$p0/, 'A full 1');
+
+# A changed
+
+update_name({A => '127.0.0.202', SRV => "1 5 $p0 example.net"});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 1, 'A changed');
+like($r, qr/127.0.0.202:$p0/, 'A changed 1');
+
+# 1 more A added
+
+update_name({A => '127.0.0.201 127.0.0.202', SRV => "1 5 $p0 example.net"});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 2, 'A A');
+like($r, qr/127.0.0.201:$p0/, 'A A 1');
+like($r, qr/127.0.0.202:$p0/, 'A A 2');
+
+# 1 A removed, 2 AAAA added
+
+update_name({A => '127.0.0.201', AAAA => 'fe80::1 fe80::2',
+	SRV => "1 5 $p0 example.net"});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 3, 'A AAAA AAAA responses');
+like($r, qr/127.0.0.201:$p0/, 'A AAAA AAAA 1');
+like($r, qr/\[fe80::1\]:$p0/, 'A AAAA AAAA 2');
+like($r, qr/\[fe80::1\]:$p0/, 'A AAAA AAAA 3');
+
+# all records removed
+
+update_name({SRV => "1 5 $p0 example.net"});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 0, 'empty SRV response');
+
+# all SRV records removed
+
+update_name();
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 0, 'empty response');
+
+# A added after empty
+
+update_name({A => '127.0.0.201', SRV => "1 5 $p0 example.net"});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 1, 'A added');
+like($r, qr/127.0.0.201:$p0/, 'A added 1');
+
+# SRV changed its weight
+
+update_name({A => '127.0.0.201', SRV => "1 6 $p0 example.net"});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 1, 'SRV weight');
+like($r, qr/127.0.0.201:$p0/, 'SRV weight 1');
+
+# changed to CNAME
+
+update_name({CNAME => 'alias'}, 2, 2);
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 1, 'CNAME');
+like($r, qr/127.0.0.203:$p0/, 'CNAME 1');
+
+# bad SRV reply should not affect existing upstream configuration
+
+update_name({CNAME => 'alias', ERROR => 'SERVFAIL'}, 1, 0);
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 1, 'ERROR');
+like($r, qr/127.0.0.203:$p0/, 'ERROR 1');
+update_name({ERROR => ''}, 1, 0);
+
+# 2 equal SRV RR
+
+update_name({A => '127.0.0.201',
+	SRV => "1 5 $p0 example.net;1 5 $p0 example.net"});
+$r = http_get('/');
+is(@n = $r =~ /:$p0/g, 2, 'SRV same');
+like($r, qr/127.0.0.201:$p0, 127.0.0.201:$p0/, 'SRV same peers');
+
+# all equal records removed
+
+update_name();
+$r = http_get('/');
+is(@n = $r =~ /:($p0|$p2|$p3)/g, 0, 'SRV same removed');
+
+# 2 different SRV RR
+
+update_name({A => '127.0.0.201',
+	SRV => "1 5 $p2 example.net;2 6 $p3 alias.example.net"}, 1, 2);
+$r = http_get('/');
+is(@n = $r =~ /:($p2|$p3)/g, 2, 'SRV diff');
+like($r, qr/127.0.0.201:$p2/, 'SRV diff 1');
+like($r, qr/127.0.0.203:$p3/, 'SRV diff 2');
+
+# all different records removed
+
+update_name();
+$r = http_get('/');
+is(@n = $r =~ /:($p0|$p2|$p3)/g, 0, 'SRV diff removed');
+
+###############################################################################
+
+sub update_name {
+	my ($name, $plan, $plan6) = @_;
+
+	$plan = 1, $plan6 = 0 if !defined $name;
+	$plan = $plan6 = 1 if !defined $plan;
+	$plan += $plan6 + $plan6;
+
+	sub sock {
+		IO::Socket::INET->new(
+			Proto => 'tcp',
+			PeerAddr => '127.0.0.1:' . port(8084)
+		)
+			or die "Can't connect to nginx: $!\n";
+	}
+
+	$name->{A} = '' unless $name->{A};
+	$name->{AAAA} = '' unless $name->{AAAA};
+	$name->{CNAME} = '' unless $name->{CNAME};
+	$name->{ERROR} = '' unless $name->{ERROR};
+	$name->{SERROR} = '' unless $name->{SERROR};
+	$name->{SRV} = '' unless $name->{SRV};
+
+	my $req =<<EOF;
+GET / HTTP/1.0
+Host: localhost
+X-A: $name->{A}
+X-AAAA: $name->{AAAA}
+X-CNAME: $name->{CNAME}
+X-ERROR: $name->{ERROR}
+X-SERROR: $name->{SERROR}
+X-SRV: $name->{SRV}
+
+EOF
+
+	my ($gen) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+	for (1 .. 10) {
+		my ($gen2) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+
+		# let resolver cache expire to finish upstream reconfiguration
+		select undef, undef, undef, 0.5;
+		last unless ($gen + $plan > $gen2);
+	}
+}
+
+###############################################################################
+
+sub reply_handler {
+	my ($recv_data, $h, $cnt, $tcp) = @_;
+
+	my (@name, @rdata);
+
+	use constant NOERROR	=> 0;
+	use constant FORMERR	=> 1;
+	use constant SERVFAIL	=> 2;
+	use constant NXDOMAIN	=> 3;
+
+	use constant A		=> 1;
+	use constant CNAME	=> 5;
+	use constant AAAA	=> 28;
+	use constant SRV	=> 33;
+
+	use constant IN		=> 1;
+
+	# default values
+
+	my ($hdr, $rcode, $ttl, $port) = (0x8180, NOERROR, 3600, port(8080));
+	$h = {A => [ "127.0.0.1" ], SRV => [ "1 5 $port example.net" ]}
+		unless defined $h;
+
+	# decode name
+
+	my ($len, $offset) = (undef, 12);
+	while (1) {
+		$len = unpack("\@$offset C", $recv_data);
+		last if $len == 0;
+		$offset++;
+		push @name, unpack("\@$offset A$len", $recv_data);
+		$offset += $len;
+	}
+
+	$offset -= 1;
+	my ($id, $type, $class) = unpack("n x$offset n2", $recv_data);
+	my $name = join('.', @name);
+
+	if ($h->{ERROR} && $type == SRV) {
+		$rcode = SERVFAIL;
+		goto bad;
+	}
+
+	# subordinate error
+
+	if ($h->{SERROR} && $type != SRV) {
+		$rcode = SERVFAIL;
+		goto bad;
+	}
+
+	if ($name eq '_http._tcp.example.net') {
+		if ($type == SRV && $h->{SRV}) {
+			map { push @rdata, rd_srv($ttl, (split ' ', $_)) }
+				@{$h->{SRV}};
+		}
+
+		my $cname = defined $h->{CNAME} ? $h->{CNAME} : 0;
+		if ($cname) {
+			push @rdata, pack("n3N nCa5n", 0xc00c, CNAME, IN, $ttl,
+				8, 5, "alias", 0xc00c + length("_http._tcp "));
+		}
+
+	} elsif ($name eq '_http._tcp.trunc.example.net' && $type == SRV) {
+		push @rdata, $tcp
+			? rd_srv($ttl, 1, 1, $port, 'tcp.example.net')
+			: rd_srv($ttl, 1, 1, $port, 'example.net');
+
+		$hdr |= 0x0300 if $name eq '_http._tcp.trunc.example.net'
+			and !$tcp;
+
+	} elsif ($name eq 'example.net' || $name eq 'tcp.example.net') {
+		if ($type == A && $h->{A}) {
+			map { push @rdata, rd_addr($ttl, $_) } @{$h->{A}};
+		}
+		if ($type == AAAA && $h->{AAAA}) {
+			map { push @rdata, rd_addr6($ttl, $_) } @{$h->{AAAA}};
+		}
+		my $cname = defined $h->{CNAME} ? $h->{CNAME} : 0;
+		if ($cname) {
+			push @rdata, pack("n3N nCa5n", 0xc00c, CNAME, IN, $ttl,
+				8, 5, $cname, 0xc00c);
+		}
+
+	} elsif ($name eq 'alias.example.net') {
+		if ($type == SRV) {
+			push @rdata, rd_srv($ttl, 1, 5, $port, 'example.net');
+		}
+		if ($type == A) {
+			push @rdata, rd_addr($ttl, '127.0.0.203');
+		}
+	}
+
+bad:
+
+	Test::Nginx::log_core('||', "DNS: $name $type $rcode");
+
+	$$cnt++ if $type == SRV || keys %$h;
+
+	$len = @name;
+	pack("n6 (C/a*)$len x n2", $id, $hdr | $rcode, 1, scalar @rdata,
+		0, 0, @name, $type, $class) . join('', @rdata);
+}
+
+sub rd_srv {
+	my ($ttl, $pri, $w, $port, $name) = @_;
+	my @rdname = split /\./, $name;
+	my $rdlen = length(join '', @rdname) + @rdname + 7;	# pri w port x
+
+	pack 'n3N n n3 (C/a*)* x',
+		0xc00c, SRV, IN, $ttl, $rdlen, $pri, $w, $port, @rdname;
+}
+
+sub rd_addr {
+	my ($ttl, $addr) = @_;
+
+	my $code = 'split(/\./, $addr)';
+
+	pack 'n3N nC4', 0xc00c, A, IN, $ttl, eval "scalar $code", eval($code);
+}
+
+sub expand_ip6 {
+	my ($addr) = @_;
+
+	substr ($addr, index($addr, "::"), 2) =
+		join "0", map { ":" } (0 .. 8 - (split /:/, $addr) + 1);
+	map { hex "0" x (4 - length $_) . "$_" } split /:/, $addr;
+}
+
+sub rd_addr6 {
+	my ($ttl, $addr) = @_;
+
+	pack 'n3N nn8', 0xc00c, AAAA, IN, $ttl, 16, expand_ip6($addr);
+}
+
+sub dns_daemon {
+	my ($t) = @_;
+	my ($data, $recv_data, $h);
+
+	my $socket = IO::Socket::INET->new(
+		LocalAddr => '127.0.0.1',
+		LocalPort => port(8981),
+		Proto => 'udp',
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $control = IO::Socket::INET->new(
+		Proto => 'tcp',
+		LocalHost => '127.0.0.1:' . port(8084),
+		Listen => 5,
+		Reuse => 1
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $tcp = port(8981, socket => 1);
+	my $sel = IO::Select->new($socket, $control, $tcp);
+
+	local $SIG{PIPE} = 'IGNORE';
+
+	# signal we are ready
+
+	open my $fh, '>', $t->testdir() . '/' . port(8981);
+	close $fh;
+	my $cnt = 0;
+
+	while (my @ready = $sel->can_read) {
+		foreach my $fh (@ready) {
+			if ($control == $fh || $tcp == $fh) {
+				my $new = $fh->accept;
+				$new->autoflush(1);
+				$sel->add($new);
+
+			} elsif ($socket == $fh) {
+				$fh->recv($recv_data, 65536);
+				$data = reply_handler($recv_data, $h, \$cnt);
+				$fh->send($data);
+
+			} elsif ($fh->sockport() == port(8084)) {
+				$h = process_name($fh, $cnt);
+				$sel->remove($fh);
+				$fh->close;
+
+			} elsif ($fh->sockport() == port(8981)) {
+				$fh->recv($recv_data, 65536);
+				unless (length $recv_data) {
+					$sel->remove($fh);
+					$fh->close;
+					next;
+				}
+
+again:
+				my $len = unpack("n", $recv_data);
+				my $data = substr $recv_data, 2, $len;
+				$data = reply_handler($data, $h, \$cnt, 1);
+				$data = pack("n", length $data) . $data;
+				$fh->send($data);
+				$recv_data = substr $recv_data, 2 + $len;
+				goto again if length $recv_data;
+			}
+		}
+	}
+}
+
+# parse dns update
+
+sub process_name {
+	my ($client, $cnt) = @_;
+	my $port = $client->sockport();
+
+	my $headers = '';
+	my $uri = '';
+	my %h;
+
+	while (<$client>) {
+		$headers .= $_;
+		last if (/^\x0d?\x0a?$/);
+	}
+	return 1 if $headers eq '';
+
+	$uri = $1 if $headers =~ /^\S+\s+([^ ]+)\s+HTTP/i;
+	return 1 if $uri eq '';
+
+	$headers =~ /X-A: (.*)$/m;
+	map { push @{$h{A}}, $_ } split(/ /, $1);
+	$headers =~ /X-AAAA: (.*)$/m;
+	map { push @{$h{AAAA}}, $_ } split(/ /, $1);
+	$headers =~ /X-SRV: (.*)$/m;
+	map { push @{$h{SRV}}, $_ } split(/;/, $1);
+	$headers =~ /X-CNAME: (.+)$/m and $h{CNAME} = $1;
+	$headers =~ /X-ERROR: (.+)$/m and $h{ERROR} = $1;
+	$headers =~ /X-SERROR: (.+)$/m and $h{SERROR} = $1;
+
+	Test::Nginx::log_core('||', "$port: response, 200");
+	print $client <<EOF;
+HTTP/1.1 200 OK
+Connection: close
+X-Gen: $cnt
+
+OK
+EOF
+
+	return \%h;
+}
+
+###############################################################################
diff --git a/upstream_service_reload.t b/upstream_service_reload.t
new file mode 100644
--- /dev/null
+++ b/upstream_service_reload.t
@@ -0,0 +1,301 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+
+# Tests for dynamic upstream configuration with service (SRV) feature.
+# Ensure that upstream configuration is inherited on reload.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+use IO::Select;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http upstream_zone/);
+
+$t->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    upstream u {
+        zone z 1m;
+        server example.net resolve service=http;
+    }
+
+    # lower the retry timeout after empty reply
+    resolver 127.0.0.1:%%PORT_8980_UDP%% valid=1s;
+    # retry query shortly after DNS is started
+    resolver_timeout 1s;
+
+    server {
+        listen       127.0.0.1:8080;
+        server_name  localhost;
+
+        location / {
+            proxy_pass http://u;
+            proxy_connect_timeout 50ms;
+            add_header X-IP $upstream_addr always;
+        }
+    }
+}
+
+EOF
+
+port(8081);
+
+$t->run_daemon(\&dns_daemon, $t)->waitforfile($t->testdir . '/' . port(8980));
+$t->try_run('no resolve in upstream server')->plan(6);
+
+###############################################################################
+
+update_name({A => '127.0.0.201', SRV => "1 5 42 example.net"});
+like(http_get('/'), qr/X-IP: 127.0.0.201:42/, 'reload - before - request');
+like(http_get('/'), qr/X-IP: 127.0.0.201:42/, 'reload - before - request 2');
+
+update_name({ERROR => 'SERVFAIL'}, 0);
+
+$t->reload();
+waitforworker($t);
+
+like(http_get('/'), qr/X-IP: 127.0.0.201:42/, 'reload - preresolve - request');
+like(http_get('/'), qr/X-IP: 127.0.0.201:42/, 'reload - preresolve - request 2');
+
+update_name({A => '127.0.0.202', SRV => "1 5 42 example.net"});
+like(http_get('/'), qr/X-IP: 127.0.0.202:42/, 'reload - update - request');
+like(http_get('/'), qr/X-IP: 127.0.0.202:42/, 'reload - update - request 2');
+
+###############################################################################
+
+sub waitforworker {
+	my ($t) = @_;
+
+	for (1 .. 30) {
+		last if $t->read_file('error.log') =~ /exited with code/;
+		select undef, undef, undef, 0.2;
+	}
+}
+
+sub update_name {
+	my ($name, $plan) = @_;
+
+	$plan = 3 if !defined $plan;
+
+	sub sock {
+		IO::Socket::INET->new(
+			Proto => 'tcp',
+			PeerAddr => '127.0.0.1:' . port(8081)
+		)
+			or die "Can't connect to nginx: $!\n";
+	}
+
+	$name->{A} = '' unless $name->{A};
+	$name->{ERROR} = '' unless $name->{ERROR};
+	$name->{SRV} = '' unless $name->{SRV};
+
+	my $req =<<EOF;
+GET / HTTP/1.0
+Host: localhost
+X-A: $name->{A}
+X-ERROR: $name->{ERROR}
+X-SRV: $name->{SRV}
+
+EOF
+
+	my ($gen) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+	for (1 .. 10) {
+		my ($gen2) = http($req, socket => sock()) =~ /X-Gen: (\d+)/;
+
+		# let resolver cache expire to finish upstream reconfiguration
+		select undef, undef, undef, 0.5;
+		last unless ($gen + $plan > $gen2);
+	}
+}
+
+###############################################################################
+
+sub reply_handler {
+	my ($recv_data, $h) = @_;
+
+	my (@name, @rdata);
+
+	use constant NOERROR	=> 0;
+	use constant SERVFAIL	=> 2;
+	use constant NXDOMAIN	=> 3;
+
+	use constant A		=> 1;
+	use constant SRV	=> 33;
+	use constant IN		=> 1;
+
+	# default values
+
+	my ($hdr, $rcode, $ttl, $port) = (0x8180, NOERROR, 3600, port(8080));
+	$h = {A => [ "127.0.0.201" ], SRV => [ "1 5 $port example.net" ]}
+		unless defined $h;
+
+	# decode name
+
+	my ($len, $offset) = (undef, 12);
+	while (1) {
+		$len = unpack("\@$offset C", $recv_data);
+		last if $len == 0;
+		$offset++;
+		push @name, unpack("\@$offset A$len", $recv_data);
+		$offset += $len;
+	}
+
+	$offset -= 1;
+	my ($id, $type, $class) = unpack("n x$offset n2", $recv_data);
+	my $name = join('.', @name);
+
+	if ($h->{ERROR}) {
+		$rcode = SERVFAIL;
+		goto bad;
+	}
+
+	if ($name eq 'example.net' && $type == A && $h->{A}) {
+		map { push @rdata, rd_addr($ttl, $_) } @{$h->{A}};
+
+	}
+	if ($name eq '_http._tcp.example.net' && $type == SRV && $h->{SRV}) {
+		map { push @rdata, rd_srv($ttl, (split ' ', $_)) }
+			@{$h->{SRV}};
+	}
+
+bad:
+
+	Test::Nginx::log_core('||', "DNS: $name $type $rcode");
+
+	$len = @name;
+	pack("n6 (C/a*)$len x n2", $id, $hdr | $rcode, 1, scalar @rdata,
+		0, 0, @name, $type, $class) . join('', @rdata);
+}
+
+sub rd_srv {
+	my ($ttl, $pri, $w, $port, $name) = @_;
+	my @rdname = split /\./, $name;
+	my $rdlen = length(join '', @rdname) + @rdname + 7;	# pri w port x
+
+	pack 'n3N n n3 (C/a*)* x',
+		0xc00c, SRV, IN, $ttl, $rdlen, $pri, $w, $port, @rdname;
+}
+
+sub rd_addr {
+	my ($ttl, $addr) = @_;
+
+	my $code = 'split(/\./, $addr)';
+
+	pack 'n3N nC4', 0xc00c, A, IN, $ttl, eval "scalar $code", eval($code);
+}
+
+sub dns_daemon {
+	my ($t) = @_;
+	my ($data, $recv_data, $h);
+
+	my $socket = IO::Socket::INET->new(
+		LocalAddr => '127.0.0.1',
+		LocalPort => port(8980),
+		Proto => 'udp',
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $control = IO::Socket::INET->new(
+		Proto => 'tcp',
+		LocalHost => "127.0.0.1:" . port(8081),
+		Listen => 5,
+		Reuse => 1
+	)
+		or die "Can't create listening socket: $!\n";
+
+	my $sel = IO::Select->new($socket, $control);
+
+	local $SIG{PIPE} = 'IGNORE';
+
+	# signal we are ready
+
+	open my $fh, '>', $t->testdir() . '/' . port(8980);
+	close $fh;
+	my $cnt = 0;
+
+	while (my @ready = $sel->can_read) {
+		foreach my $fh (@ready) {
+			if ($control == $fh) {
+				my $new = $fh->accept;
+				$new->autoflush(1);
+				$sel->add($new);
+
+			} elsif ($socket == $fh) {
+				$fh->recv($recv_data, 65536);
+				$data = reply_handler($recv_data, $h);
+				$fh->send($data);
+				$cnt++;
+
+			} else {
+				$h = process_name($fh, $cnt);
+				$sel->remove($fh);
+				$fh->close;
+			}
+		}
+	}
+}
+
+# parse dns update
+
+sub process_name {
+	my ($client, $cnt) = @_;
+	my $port = $client->sockport();
+
+	my $headers = '';
+	my $uri = '';
+	my %h;
+
+	while (<$client>) {
+		$headers .= $_;
+		last if (/^\x0d?\x0a?$/);
+	}
+	return 1 if $headers eq '';
+
+	$uri = $1 if $headers =~ /^\S+\s+([^ ]+)\s+HTTP/i;
+	return 1 if $uri eq '';
+
+	$headers =~ /X-A: (.*)$/m;
+	map { push @{$h{A}}, $_ } split(/ /, $1);
+	$headers =~ /X-SRV: (.*)$/m;
+	map { push @{$h{SRV}}, $_ } split(/;/, $1);
+	$headers =~ /X-ERROR: (.*)$/m;
+	$h{ERROR} = $1;
+
+	Test::Nginx::log_core('||', "$port: response, 200");
+	print $client <<EOF;
+HTTP/1.1 200 OK
+Connection: close
+X-Gen: $cnt
+
+OK
+EOF
+
+	return \%h;
+}
+
+###############################################################################

From hnakamur at gmail.com  Fri Jun 14 07:29:08 2024
From: hnakamur at gmail.com (Hiroaki Nakamura)
Date: Fri, 14 Jun 2024 16:29:08 +0900
Subject: [PATCH 0 of 3] Update Age response header correctly
Message-ID: <CAN-DUMSa6MiyQJ-thvRVGbPtyLzvdPkTgmMu27ttgfOCmHiSfA@mail.gmail.com>

Hello!

Currently nginx does not update Age response header when it
receives responses from upstreams nor when it sends a cached
response.

This causes the problem that nginx might send an expired
cache.

With this patchset, nginx updates Age response header
correctly when it receives responses from upstream and when
it sends a cached response as specified in RFC 9111 [1].

- patch 1: Update Age response header when nginx receives
           a response from upstreams and when it sends a
           cached response.
- patch 2: Save response time and corrected initial age
           to cache file header.
- patch 3: Tests: Update and add test files to patchset
           for convenience.
           Contents of the "*.tt" files are to be put into
           nginx-tests repository.

Link: https://www.rfc-editor.org/rfc/rfc9111 [1]

Thanks!
Hiroaki Nakamura

From hnakamur at gmail.com  Fri Jun 14 07:29:14 2024
From: hnakamur at gmail.com (Hiroaki Nakamura)
Date: Fri, 14 Jun 2024 16:29:14 +0900
Subject: [PATCH 1 of 3] Correctly calculate and set Age header
Message-ID: <CAN-DUMSrWHmEYDrPweXcoonLHvad6HUg0Q3kib7Gfk9pCgYdpA@mail.gmail.com>

# HG changeset patch
# User Hiroaki Nakamura <hnakamur at gmail.com>
# Date 1718345844 -32400
#      Fri Jun 14 15:17:24 2024 +0900
# Branch correct_age
# Node ID 493817cf38580a71430f9c1293e29bdfbf243075
# Parent  2f636ec9c71aaa7129f3f79fa1e584e02f8748f0
Correctly calculate and set Age header.
Implement the calculation of the Age header as specified in
"RFC 9111: HTTP Caching"
https://www.rfc-editor.org/rfc/rfc9111.html

diff -r 2f636ec9c71a -r 493817cf3858 src/http/ngx_http_cache.h
--- a/src/http/ngx_http_cache.h Fri Jun 14 15:17:14 2024 +0900
+++ b/src/http/ngx_http_cache.h Fri Jun 14 15:17:24 2024 +0900
@@ -59,6 +59,8 @@
     size_t                           body_start;
     off_t                            fs_size;
     ngx_msec_t                       lock_time;
+    time_t                           response_time;
+    time_t                           corrected_initial_age;
 } ngx_http_file_cache_node_t;


@@ -75,6 +77,8 @@
     time_t                           error_sec;
     time_t                           last_modified;
     time_t                           date;
+    time_t                           response_time;
+    time_t                           corrected_initial_age;

     ngx_str_t                        etag;
     ngx_str_t                        vary;
diff -r 2f636ec9c71a -r 493817cf3858 src/http/ngx_http_file_cache.c
--- a/src/http/ngx_http_file_cache.c    Fri Jun 14 15:17:14 2024 +0900
+++ b/src/http/ngx_http_file_cache.c    Fri Jun 14 15:17:24 2024 +0900
@@ -971,6 +971,8 @@
     fcn->uniq = 0;
     fcn->body_start = 0;
     fcn->fs_size = 0;
+    fcn->response_time = 0;
+    fcn->corrected_initial_age = 0;

 done:

@@ -980,6 +982,8 @@

     c->uniq = fcn->uniq;
     c->error = fcn->error;
+    c->response_time = fcn->response_time;
+    c->corrected_initial_age = fcn->corrected_initial_age;
     c->node = fcn;

 failed:
@@ -1624,6 +1628,7 @@
 ngx_int_t
 ngx_http_cache_send(ngx_http_request_t *r)
 {
+    time_t             resident_time, current_age;
     ngx_int_t          rc;
     ngx_buf_t         *b;
     ngx_chain_t        out;
@@ -1646,6 +1651,17 @@
         return NGX_HTTP_INTERNAL_SERVER_ERROR;
     }

+    /*
+     * Update age response header.
+     * https://www.rfc-editor.org/rfc/rfc9111.html#name-calculating-age
+     */
+    resident_time = ngx_time() - c->response_time;
+    current_age = c->corrected_initial_age + resident_time;
+    r->headers_out.age_n = current_age;
+    ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
+                   "http file cache send, resp:%O, resident:%d, age:%d",
+                   c->response_time, resident_time, current_age);
+
     rc = ngx_http_send_header(r);

     if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) {
diff -r 2f636ec9c71a -r 493817cf3858 src/http/ngx_http_header_filter_module.c
--- a/src/http/ngx_http_header_filter_module.c  Fri Jun 14 15:17:14 2024 +0900
+++ b/src/http/ngx_http_header_filter_module.c  Fri Jun 14 15:17:24 2024 +0900
@@ -322,6 +322,10 @@
         len += sizeof("Last-Modified: Mon, 28 Sep 1970 06:00:00 GMT" CRLF) - 1;
     }

+    if (r->headers_out.age_n != -1) {
+        len += sizeof("Age: ") - 1 + NGX_OFF_T_LEN + 2;
+    }
+
     c = r->connection;

     if (r->headers_out.location
@@ -518,6 +522,10 @@
         *b->last++ = CR; *b->last++ = LF;
     }

+    if (r->headers_out.age_n != -1) {
+        b->last = ngx_sprintf(b->last, "Age: %O" CRLF, r->headers_out.age_n);
+    }
+
     if (host.data) {

         p = b->last + sizeof("Location: ") - 1;
diff -r 2f636ec9c71a -r 493817cf3858 src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c       Fri Jun 14 15:17:14 2024 +0900
+++ b/src/http/ngx_http_request.c       Fri Jun 14 15:17:24 2024 +0900
@@ -646,6 +646,7 @@
     r->headers_in.keep_alive_n = -1;
     r->headers_out.content_length_n = -1;
     r->headers_out.last_modified_time = -1;
+    r->headers_out.age_n = -1;

     r->uri_changes = NGX_HTTP_MAX_URI_CHANGES + 1;
     r->subrequests = NGX_HTTP_MAX_SUBREQUESTS + 1;
diff -r 2f636ec9c71a -r 493817cf3858 src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h       Fri Jun 14 15:17:14 2024 +0900
+++ b/src/http/ngx_http_request.h       Fri Jun 14 15:17:24 2024 +0900
@@ -290,6 +290,7 @@
     off_t                             content_offset;
     time_t                            date_time;
     time_t                            last_modified_time;
+    off_t                             age_n;
 } ngx_http_headers_out_t;


diff -r 2f636ec9c71a -r 493817cf3858 src/http/ngx_http_special_response.c
--- a/src/http/ngx_http_special_response.c      Fri Jun 14 15:17:14 2024 +0900
+++ b/src/http/ngx_http_special_response.c      Fri Jun 14 15:17:24 2024 +0900
@@ -581,6 +581,7 @@

     r->headers_out.content_length_n = -1;
     r->headers_out.last_modified_time = -1;
+    r->headers_out.age_n = -1;
 }


diff -r 2f636ec9c71a -r 493817cf3858 src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c      Fri Jun 14 15:17:14 2024 +0900
+++ b/src/http/ngx_http_upstream.c      Fri Jun 14 15:17:24 2024 +0900
@@ -50,6 +50,8 @@
     ngx_http_upstream_t *u);
 static ngx_int_t ngx_http_upstream_test_next(ngx_http_request_t *r,
     ngx_http_upstream_t *u);
+static void ngx_http_upstream_update_age(ngx_http_request_t *r,
+    ngx_http_upstream_t *u, time_t now);
 static ngx_int_t ngx_http_upstream_intercept_errors(ngx_http_request_t *r,
     ngx_http_upstream_t *u);
 static ngx_int_t ngx_http_upstream_test_connect(ngx_connection_t *c);
@@ -132,6 +134,8 @@
     ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t ngx_http_upstream_process_vary(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset);
+static ngx_int_t ngx_http_upstream_process_age(ngx_http_request_t *r,
+    ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t ngx_http_upstream_copy_header_line(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t
@@ -319,6 +323,10 @@
                  ngx_http_upstream_copy_header_line,
                  offsetof(ngx_http_headers_out_t, content_encoding), 0 },

+    { ngx_string("Age"),
+                 ngx_http_upstream_process_age, 0,
+                 ngx_http_upstream_ignore_header_line, 0, 0 },
+
     { ngx_null_string, NULL, 0, NULL, 0, 0 }
 };

@@ -499,6 +507,7 @@

     u->headers_in.content_length_n = -1;
     u->headers_in.last_modified_time = -1;
+    u->headers_in.age_n = -1;

     return NGX_OK;
 }
@@ -1068,6 +1077,7 @@
     ngx_memzero(&u->headers_in, sizeof(ngx_http_upstream_headers_in_t));
     u->headers_in.content_length_n = -1;
     u->headers_in.last_modified_time = -1;
+    u->headers_in.age_n = -1;

     if (ngx_list_init(&u->headers_in.headers, r->pool, 8,
                       sizeof(ngx_table_elt_t))
@@ -1547,6 +1557,7 @@
     ngx_memzero(u->state, sizeof(ngx_http_upstream_state_t));

     u->start_time = ngx_current_msec;
+    u->request_time = ngx_time();

     u->state->response_time = (ngx_msec_t) -1;
     u->state->connect_time = (ngx_msec_t) -1;
@@ -2006,6 +2017,7 @@
     ngx_memzero(&u->headers_in, sizeof(ngx_http_upstream_headers_in_t));
     u->headers_in.content_length_n = -1;
     u->headers_in.last_modified_time = -1;
+    u->headers_in.age_n = -1;

     if (ngx_list_init(&u->headers_in.headers, r->pool, 8,
                       sizeof(ngx_table_elt_t))
@@ -2520,6 +2532,8 @@
         return;
     }

+    ngx_http_upstream_update_age(r, u, ngx_time());
+
     ngx_http_upstream_send_response(r, u);
 }

@@ -2606,6 +2620,7 @@
                        "http upstream not modified");

         now = ngx_time();
+        ngx_http_upstream_update_age(r, u, now);

         valid = r->cache->valid_sec;
         updating = r->cache->updating_sec;
@@ -2639,7 +2654,12 @@
             valid = ngx_http_file_cache_valid(u->conf->cache_valid,
                                               u->headers_in.status_n);
             if (valid) {
-                valid = now + valid;
+                ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
+                               "adjust cache valid_sec:%O, "
+                               "valid:%O, init_age:%d for 304",
+                               now + valid - r->cache->corrected_initial_age,
+                               valid, r->cache->corrected_initial_age);
+                valid = now + valid - r->cache->corrected_initial_age;
             }
         }

@@ -2663,6 +2683,59 @@
 }


+static void
+ngx_http_upstream_update_age(ngx_http_request_t *r, ngx_http_upstream_t *u,
+    time_t now)
+{
+    time_t  response_time, date, apparent_age, response_delay, age_value,
+            corrected_age_value, corrected_initial_age;
+
+    /*
+     * Update age response header.
+     * https://www.rfc-editor.org/rfc/rfc9111.html#name-calculating-age
+     */
+    response_time = now;
+    if (u->headers_in.date != NULL) {
+        date = ngx_parse_http_time(u->headers_in.date->value.data,
+                                   u->headers_in.date->value.len);
+        if (date == NGX_ERROR) {
+            date = now;
+        }
+    } else {
+        date = now;
+    }
+    apparent_age = ngx_max(0, response_time - date);
+
+    response_delay = response_time - u->request_time;
+    age_value = u->headers_in.age_n != -1 ? u->headers_in.age_n : 0;
+    corrected_age_value = age_value + response_delay;
+
+    corrected_initial_age = ngx_max(apparent_age, corrected_age_value);
+    r->headers_out.age_n = corrected_initial_age;
+
+    ngx_log_debug8(NGX_LOG_DEBUG_HTTP, u->peer.connection->log, 0,
+                   "http upstream set age:%O, req:%O, resp:%O, date:%O, "
+                   "a_age:%O, resp_delay:%O, u_age:%O, c_age:%O",
+                   corrected_initial_age, u->request_time, response_time, date,
+                   apparent_age, response_delay, u->headers_in.age_n,
+                   corrected_age_value);
+
+#if (NGX_HTTP_CACHE)
+    if (r->cache) {
+        r->cache->response_time = response_time;
+        r->cache->corrected_initial_age = corrected_initial_age;
+        if (u->headers_in.adjusting_valid_sec) {
+            r->cache->valid_sec -= corrected_initial_age;
+            ngx_log_debug2(NGX_LOG_DEBUG_HTTP, u->peer.connection->log, 0,
+                           "http upstream adjusted cache "
+                           "valid_sec:%O, init_age:%O",
+                           r->cache->valid_sec, corrected_initial_age);
+        }
+    }
+#endif
+}
+
+
 static ngx_int_t
 ngx_http_upstream_intercept_errors(ngx_http_request_t *r,
     ngx_http_upstream_t *u)
@@ -2738,6 +2811,7 @@
                                                           status);
                         if (valid) {
                             r->cache->valid_sec = ngx_time() + valid;
+                            u->headers_in.adjusting_valid_sec = 1;
                         }
                     }

@@ -2945,6 +3019,7 @@
     r->headers_out.status_line = u->headers_in.status_line;

     r->headers_out.content_length_n = u->headers_in.content_length_n;
+    r->headers_out.age_n = u->headers_in.age_n;

     r->disable_not_modified = !u->cacheable;

@@ -4609,6 +4684,7 @@

                 if (valid) {
                     r->cache->valid_sec = ngx_time() + valid;
+                    u->headers_in.adjusting_valid_sec = 1;
                     r->cache->error = rc;
                 }
             }
@@ -4884,6 +4960,7 @@
         }

         r->cache->valid_sec = ngx_time() + n;
+        u->headers_in.adjusting_valid_sec = 1;
         u->headers_in.expired = 0;
     }

@@ -5046,6 +5123,7 @@

         default:
             r->cache->valid_sec = ngx_time() + n;
+            u->headers_in.adjusting_valid_sec = 1;
             u->headers_in.no_cache = 0;
             u->headers_in.expired = 0;
             return NGX_OK;
@@ -5312,6 +5390,39 @@


 static ngx_int_t
+ngx_http_upstream_process_age(ngx_http_request_t *r,
+    ngx_table_elt_t *h, ngx_uint_t offset)
+{
+    ngx_http_upstream_t  *u;
+
+    u = r->upstream;
+
+    if (u->headers_in.age) {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "upstream sent duplicate header line: "%V: %V", "
+                      "previous value: "%V: %V"",
+                      &h->key, &h->value,
+                      &u->headers_in.age->key,
+                      &u->headers_in.age->value);
+        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
+    }
+
+    h->next = NULL;
+    u->headers_in.age = h;
+    u->headers_in.age_n = ngx_atoof(h->value.data, h->value.len);
+
+    if (u->headers_in.age_n == NGX_ERROR) {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "upstream sent invalid "Age" header: "
+                      ""%V: %V"", &h->key, &h->value);
+        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
+    }
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
 ngx_http_upstream_copy_header_line(ngx_http_request_t *r, ngx_table_elt_t *h,
     ngx_uint_t offset)
 {
diff -r 2f636ec9c71a -r 493817cf3858 src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h      Fri Jun 14 15:17:14 2024 +0900
+++ b/src/http/ngx_http_upstream.h      Fri Jun 14 15:17:24 2024 +0900
@@ -287,14 +287,17 @@

     ngx_table_elt_t                 *cache_control;
     ngx_table_elt_t                 *set_cookie;
+    ngx_table_elt_t                 *age;

     off_t                            content_length_n;
     time_t                           last_modified_time;
+    off_t                            age_n;

     unsigned                         connection_close:1;
     unsigned                         chunked:1;
     unsigned                         no_cache:1;
     unsigned                         expired:1;
+    unsigned                         adjusting_valid_sec:1;
 } ngx_http_upstream_headers_in_t;


@@ -369,6 +372,7 @@
                                          ngx_table_elt_t *h);

     ngx_msec_t                       start_time;
+    time_t                           request_time;

     ngx_http_upstream_state_t       *state;

diff -r 2f636ec9c71a -r 493817cf3858 src/http/v2/ngx_http_v2.h
--- a/src/http/v2/ngx_http_v2.h Fri Jun 14 15:17:14 2024 +0900
+++ b/src/http/v2/ngx_http_v2.h Fri Jun 14 15:17:24 2024 +0900
@@ -398,6 +398,7 @@
 #define NGX_HTTP_V2_STATUS_404_INDEX      13
 #define NGX_HTTP_V2_STATUS_500_INDEX      14

+#define NGX_HTTP_V2_AGE_INDEX             21
 #define NGX_HTTP_V2_CONTENT_LENGTH_INDEX  28
 #define NGX_HTTP_V2_CONTENT_TYPE_INDEX    31
 #define NGX_HTTP_V2_DATE_INDEX            33
diff -r 2f636ec9c71a -r 493817cf3858 src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c   Fri Jun 14 15:17:14 2024 +0900
+++ b/src/http/v2/ngx_http_v2_filter_module.c   Fri Jun 14 15:17:24 2024 +0900
@@ -256,6 +256,10 @@
         len += 1 + ngx_http_v2_literal_size("Wed, 31 Dec 1986 18:00:00 GMT");
     }

+    if (r->headers_out.age_n != -1) {
+        len += 1 + ngx_http_v2_integer_octets(NGX_OFF_T_LEN) + NGX_OFF_T_LEN;
+    }
+
     if (r->headers_out.location && r->headers_out.location->value.len) {

         if (r->headers_out.location->value.data[0] == '/'
@@ -543,6 +547,18 @@
         pos = ngx_http_v2_write_value(pos, pos, len, tmp);
     }

+    if (r->headers_out.age_n != -1) {
+        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 output header: "age: %O"",
+                       r->headers_out.age_n);
+
+        *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_AGE_INDEX);
+
+        p = pos;
+        pos = ngx_sprintf(pos + 1, "%O", r->headers_out.age_n);
+        *p = NGX_HTTP_V2_ENCODE_RAW | (u_char) (pos - p - 1);
+    }
+
     if (r->headers_out.location && r->headers_out.location->value.len) {
         ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
                        "http2 output header: "location: %V"",
diff -r 2f636ec9c71a -r 493817cf3858 src/http/v3/ngx_http_v3_filter_module.c
--- a/src/http/v3/ngx_http_v3_filter_module.c   Fri Jun 14 15:17:14 2024 +0900
+++ b/src/http/v3/ngx_http_v3_filter_module.c   Fri Jun 14 15:17:24 2024 +0900
@@ -13,6 +13,7 @@
 /* static table indices */
 #define NGX_HTTP_V3_HEADER_AUTHORITY                 0
 #define NGX_HTTP_V3_HEADER_PATH_ROOT                 1
+#define NGX_HTTP_V3_HEADER_AGE_ZERO                  2
 #define NGX_HTTP_V3_HEADER_CONTENT_LENGTH_ZERO       4
 #define NGX_HTTP_V3_HEADER_DATE                      6
 #define NGX_HTTP_V3_HEADER_LAST_MODIFIED             10
@@ -213,6 +214,15 @@
                                   sizeof("Mon, 28 Sep 1970 06:00:00 GMT") - 1);
     }

+    if (r->headers_out.age_n > 0) {
+        len += ngx_http_v3_encode_field_lri(NULL, 0,
+                                            NGX_HTTP_V3_HEADER_AGE_ZERO,
+                                            NULL, NGX_OFF_T_LEN);
+    } else if (r->headers_out.age_n == 0) {
+        len += ngx_http_v3_encode_field_ri(NULL, 0,
+                                           NGX_HTTP_V3_HEADER_AGE_ZERO);
+    }
+
     if (r->headers_out.location && r->headers_out.location->value.len) {

         if (r->headers_out.location->value.data[0] == '/'
@@ -452,6 +462,27 @@
                                               p, n);
     }

+    if (r->headers_out.age_n != -1) {
+        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0,
+                       "http3 output header: "age: %O"",
+                       r->headers_out.age_n);
+
+        if (r->headers_out.age_n > 0) {
+            p = ngx_sprintf(b->last, "%O", r->headers_out.age_n);
+            n = p - b->last;
+
+            b->last = (u_char *) ngx_http_v3_encode_field_lri(b->last, 0,
+                                                   NGX_HTTP_V3_HEADER_AGE_ZERO,
+                                                   NULL, n);
+
+            b->last = ngx_sprintf(b->last, "%O", r->headers_out.age_n);
+
+        } else {
+            b->last = (u_char *) ngx_http_v3_encode_field_ri(b->last, 0,
+                                                  NGX_HTTP_V3_HEADER_AGE_ZERO);
+        }
+    }
+
     if (r->headers_out.location && r->headers_out.location->value.len) {
         ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0,
                        "http3 output header: "location: %V"",

From hnakamur at gmail.com  Fri Jun 14 07:29:24 2024
From: hnakamur at gmail.com (Hiroaki Nakamura)
Date: Fri, 14 Jun 2024 16:29:24 +0900
Subject: [PATCH 3 of 3] Tests: Update and add tests for Age header
Message-ID: <CAN-DUMRW4-vyFG=AuqM4=1zt9N3nvXLW2ZGPhRecbDBcjX9PYg@mail.gmail.com>

# HG changeset patch
# User Hiroaki Nakamura <hnakamur at gmail.com>
# Date 1718345855 -32400
#      Fri Jun 14 15:17:35 2024 +0900
# Branch correct_age
# Node ID 8473b99b2c169b50bc71c9c07a1f10750b0cfc73
# Parent  51355c41fc134954e58b1534d232291d70252be7
Tests: Update and add tests for Age header.

diff -r 51355c41fc13 -r 8473b99b2c16 h2_proxy_cache_age.t
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/h2_proxy_cache_age.t      Fri Jun 14 15:17:35 2024 +0900
@@ -0,0 +1,198 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+# (C) Hiroaki Nakamura
+
+# Tests for age in HTTP/2 proxy cache.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+use Test::Nginx::HTTP2;
+
+use POSIX qw/ ceil /;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http http_v2 proxy cache/)->plan(8)
+       ->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    proxy_cache_path %%TESTDIR%%/cache    keys_zone=NAME:1m;
+    proxy_cache_path %%TESTDIR%%/cache2   keys_zone=NAME2:1m;
+
+    map $arg_slow $rate {
+        default 8k;
+        1       90;
+    }
+
+    server {
+        listen       127.0.0.1:8080 http2;
+        server_name  localhost;
+
+        location / {
+            proxy_pass             http://127.0.0.1:8081;
+            proxy_cache            NAME;
+            proxy_http_version     1.1;
+            proxy_cache_revalidate on;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8081;
+        server_name  localhost;
+
+        location / {
+            proxy_pass             http://127.0.0.1:8082;
+            proxy_cache            NAME2;
+            proxy_http_version     1.1;
+            proxy_cache_revalidate on;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8082;
+        server_name  localhost;
+
+        location / {
+            add_header Cache-Control s-maxage=$arg_ttl;
+            limit_rate $rate;
+        }
+    }
+}
+
+EOF
+
+$t->write_file('t.html', 'SEE-THIS');
+
+# suppress deprecation warning
+
+open OLDERR, ">&", *STDERR; close STDERR;
+$t->run();
+open STDERR, ">&", *OLDERR;
+
+###############################################################################
+
+my $s = Test::Nginx::HTTP2->new();
+
+my ($path, $sid, $frames, $frame, $t1, $resident_time);
+
+# normal origin
+
+wait_until_next_second();
+
+$path = '/t.html?ttl=2';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 0, 'age first');
+
+select undef, undef, undef, 2.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 2, 'age hit');
+
+select undef, undef, undef, 1.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 0, 'age updated');
+
+SKIP: {
+skip 'no exec on win32', 3 if $^O eq 'MSWin32';
+
+# slow origin
+
+wait_until_next_second();
+
+$path = '/t.html?ttl=6&slow=1';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 4, 'slow origin first');
+
+select undef, undef, undef, 2.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 6, 'slow origin hit');
+
+select undef, undef, undef, 1.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 5, 'slow origin updated');
+
+}
+
+# update age after restart
+
+wait_until_next_second();
+
+$path = '/t.html?ttl=20';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 0, 'age before restart');
+$t1 = time();
+
+$t->stop();
+
+open OLDERR, ">&", *STDERR; close STDERR;
+$t->run();
+open STDERR, ">&", *OLDERR;
+
+$resident_time = time() - $t1;
+
+$s = Test::Nginx::HTTP2->new();
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, $resident_time, 'age after restart');
+
+$t->stop();
+
+###############################################################################
+
+# Wait until the next second boundary.
+# Calling this before sending a request increases the likelihood that the
+# timestamp value does not cross into the next second while sending a request
+# and receiving a response.
+sub wait_until_next_second {
+    my $now = time();
+    my $next_second = ceil($now);
+    my $sleep = $next_second - $now;
+    select undef, undef, undef, $sleep;
+}
+
+###############################################################################
diff -r 51355c41fc13 -r 8473b99b2c16 h2_ssl_proxy_cache_age.t
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/h2_ssl_proxy_cache_age.t  Fri Jun 14 15:17:35 2024 +0900
@@ -0,0 +1,243 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+# (C) Hiroaki Nakamura
+
+# Tests for age in HTTP/2 ssl proxy cache.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+use Test::Nginx::HTTP2;
+
+use POSIX qw/ ceil /;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()
+       ->has(qw/http http_ssl http_v2 proxy cache socket_ssl/)->plan(10)
+       ->has_daemon('openssl');
+
+$t->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    proxy_cache_path %%TESTDIR%%/cache   keys_zone=NAME:1m;
+    proxy_cache_path %%TESTDIR%%/cache2  keys_zone=NAME2:1m;
+
+    map $arg_slow $rate {
+        default 8k;
+        1       90;
+    }
+
+    server {
+        listen       127.0.0.1:8080 http2 ssl;
+        server_name  localhost;
+
+        ssl_certificate_key localhost.key;
+        ssl_certificate localhost.crt;
+
+        location / {
+            proxy_pass   http://127.0.0.1:8081;
+            proxy_cache  NAME;
+            proxy_http_version 1.1;
+            proxy_cache_revalidate on;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8081;
+        server_name  localhost;
+
+        location / {
+            proxy_pass             http://127.0.0.1:8082;
+            proxy_cache            NAME2;
+            proxy_http_version     1.1;
+            proxy_cache_revalidate on;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8082;
+        server_name  localhost;
+
+        location / {
+            add_header Cache-Control s-maxage=$arg_ttl;
+            limit_rate $rate;
+        }
+    }
+}
+
+EOF
+
+$t->write_file('openssl.conf', <<EOF);
+[ req ]
+default_bits = 2048
+encrypt_key = no
+distinguished_name = req_distinguished_name
+[ req_distinguished_name ]
+EOF
+
+my $d = $t->testdir();
+
+foreach my $name ('localhost') {
+       system('openssl req -x509 -new '
+               . "-config $d/openssl.conf -subj /CN=$name/ "
+               . "-out $d/$name.crt -keyout $d/$name.key "
+               . ">>$d/openssl.out 2>&1") == 0
+               or die "Can't create certificate for $name: $!
";
+}
+
+$t->write_file('t.html', 'SEE-THIS');
+
+open OLDERR, ">&", *STDERR; close STDERR;
+$t->run();
+open STDERR, ">&", *OLDERR;
+
+###############################################################################
+
+my $s = getconn(port(8080));
+ok($s, 'ssl connection');
+
+my ($path, $sid, $frames, $frame, $t1, $resident_time);
+
+# normal origin
+
+wait_until_next_second();
+
+$path = '/t.html?ttl=2';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 0, 'age first');
+
+select undef, undef, undef, 2.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 2, 'age hit');
+
+select undef, undef, undef, 1.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 0, 'age updated');
+
+# slow origin
+
+SKIP: {
+skip 'no exec on win32', 3 if $^O eq 'MSWin32';
+
+wait_until_next_second();
+
+$path = '/t.html?ttl=6&slow=1';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 4, 'slow origin first');
+
+select undef, undef, undef, 2.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 6, 'slow origin hit');
+
+select undef, undef, undef, 1.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 5, 'slow origin updated');
+
+}
+
+# update age after restart
+
+$path = '/t.html?ttl=20';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 0, 'age before restart');
+$t1 = time();
+
+$t->stop();
+
+open OLDERR, ">&", *STDERR; close STDERR;
+$t->run();
+open STDERR, ">&", *OLDERR;
+
+$resident_time = time() - $t1;
+
+$s = getconn(port(8080));
+ok($s, 'ssl connection');
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, $resident_time, 'age after restart');
+
+$t->stop();
+
+###############################################################################
+
+sub getconn {
+       my ($port) = @_;
+       my $s;
+
+       eval {
+               my $sock = Test::Nginx::HTTP2::new_socket($port, SSL => 1,
+                       alpn => 'h2');
+               $s = Test::Nginx::HTTP2->new($port, socket => $sock)
+                       if $sock->alpn_selected();
+       };
+
+       return $s if defined $s;
+
+       eval {
+               my $sock = Test::Nginx::HTTP2::new_socket($port, SSL => 1,
+                       npn => 'h2');
+               $s = Test::Nginx::HTTP2->new($port, socket => $sock)
+                       if $sock->next_proto_negotiated();
+       };
+
+       return $s;
+}
+
+# Wait until the next second boundary.
+# Calling this before sending a request increases the likelihood that the
+# timestamp value does not cross into the next second while sending a request
+# and receiving a response.
+sub wait_until_next_second {
+    my $now = time();
+    my $next_second = ceil($now);
+    my $sleep = $next_second - $now;
+    select undef, undef, undef, $sleep;
+}
+
+###############################################################################
diff -r 51355c41fc13 -r 8473b99b2c16 h3_proxy_cache_age.t
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/h3_proxy_cache_age.t      Fri Jun 14 15:17:35 2024 +0900
@@ -0,0 +1,210 @@
+#!/usr/bin/perl
+
+# (C) Sergey Kandaurov
+# (C) Nginx, Inc.
+# (C) Hiroaki Nakamura
+
+# Tests for age in HTTP/3 proxy cache.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+use Test::Nginx::HTTP3;
+
+use POSIX qw/ ceil /;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http http_v3 proxy cryptx/)
+       ->has_daemon('openssl')->plan(8)
+       ->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    ssl_certificate_key localhost.key;
+    ssl_certificate localhost.crt;
+
+    log_format test $uri:$status:$request_completion;
+
+    proxy_cache_path %%TESTDIR%%/cache    keys_zone=NAME:1m;
+
+    map $arg_slow $rate {
+        default 8k;
+        1       90;
+    }
+
+    server {
+        listen       127.0.0.1:%%PORT_8980_UDP%% quic;
+        server_name  localhost;
+
+        location / {
+            proxy_pass http://127.0.0.1:8081/;
+            proxy_cache NAME;
+            proxy_http_version 1.1;
+            proxy_cache_revalidate on;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8081;
+        server_name  localhost;
+
+        location / {
+            proxy_pass             http://127.0.0.1:8082;
+            proxy_cache            NAME;
+            proxy_http_version     1.1;
+            proxy_cache_revalidate on;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8082;
+        server_name  localhost;
+
+        location / {
+            add_header Cache-Control s-maxage=$arg_ttl;
+            limit_rate $rate;
+        }
+    }
+}
+
+EOF
+
+$t->write_file('openssl.conf', <<EOF);
+[ req ]
+default_bits = 2048
+encrypt_key = no
+distinguished_name = req_distinguished_name
+[ req_distinguished_name ]
+EOF
+
+my $d = $t->testdir();
+
+foreach my $name ('localhost') {
+       system('openssl req -x509 -new '
+               . "-config $d/openssl.conf -subj /CN=$name/ "
+               . "-out $d/$name.crt -keyout $d/$name.key "
+               . ">>$d/openssl.out 2>&1") == 0
+               or die "Can't create certificate for $name: $!
";
+}
+
+my $content = 'SEE-THIS';
+$t->write_file('t.html', $content);
+$t->run();
+
+###############################################################################
+
+my $s = Test::Nginx::HTTP3->new();
+
+my ($path, $sid, $frames, $frame, $t1, $resident_time);
+
+# normal origin
+
+wait_until_next_second();
+
+$path = '/t.html?ttl=2';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 0, 'age first');
+
+select undef, undef, undef, 2.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 2, 'age hit');
+
+select undef, undef, undef, 1.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 0, 'age updated');
+
+# slow origin
+
+wait_until_next_second();
+
+$path = '/t.html?ttl=6&slow=1';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 4, 'slow origin first');
+
+select undef, undef, undef, 2.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 6, 'slow origin hit');
+
+select undef, undef, undef, 1.0;
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 5, 'slow origin updated');
+
+# update age after restart
+
+$path = '/t.html?ttl=20';
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, 0, 'age before restart');
+$t1 = time();
+
+$t->stop();
+
+open OLDERR, ">&", *STDERR; close STDERR;
+$t->run();
+open STDERR, ">&", *OLDERR;
+
+$resident_time = time() - $t1;
+
+$s = Test::Nginx::HTTP3->new();
+
+$sid = $s->new_stream({ path => $path });
+$frames = $s->read(all => [{ sid => $sid, fin => 1 }]);
+($frame) = grep { $_->{type} eq "HEADERS" } @$frames;
+is($frame->{headers}->{'age'}, $resident_time, 'age after restart');
+
+$t->stop();
+
+###############################################################################
+
+# Wait until the next second boundary.
+# Calling this before sending a request increases the likelihood that the
+# timestamp value does not cross into the next second while sending a request
+# and receiving a response.
+sub wait_until_next_second {
+    my $now = time();
+    my $next_second = ceil($now);
+    my $sleep = $next_second - $now;
+    select undef, undef, undef, $sleep;
+}
+
+###############################################################################
diff -r 51355c41fc13 -r 8473b99b2c16 proxy_cache_age.t
--- /dev/null   Thu Jan 01 00:00:00 1970 +0000
+++ b/proxy_cache_age.t Fri Jun 14 15:17:35 2024 +0900
@@ -0,0 +1,176 @@
+#!/usr/bin/perl
+
+# (C) Maxim Dounin
+# (C) Hiroaki Nakamura
+
+# Tests for age in http proxy cache.
+
+###############################################################################
+
+use warnings;
+use strict;
+
+use Test::More;
+use Socket qw/ CRLF /;
+
+BEGIN { use FindBin; chdir($FindBin::Bin); }
+
+use lib 'lib';
+use Test::Nginx;
+
+use POSIX qw/ ceil /;
+
+###############################################################################
+
+select STDERR; $| = 1;
+select STDOUT; $| = 1;
+
+my $t = Test::Nginx->new()->has(qw/http proxy cache/)->plan(8)
+       ->write_file_expand('nginx.conf', <<'EOF');
+
+%%TEST_GLOBALS%%
+
+daemon off;
+
+events {
+}
+
+http {
+    %%TEST_GLOBALS_HTTP%%
+
+    proxy_cache_path   %%TESTDIR%%/cache  levels=1:2
+                       keys_zone=NAME:1m;
+    proxy_cache_path   %%TESTDIR%%/cache2  levels=1:2
+                       keys_zone=NAME2:1m;
+
+    map $arg_slow $rate {
+        default 8k;
+        1       100;
+    }
+
+    server {
+        listen       127.0.0.1:8080;
+        server_name  localhost;
+
+        location / {
+            proxy_pass             http://127.0.0.1:8081;
+            proxy_cache            NAME;
+            proxy_http_version     1.1;
+            proxy_cache_revalidate on;
+            add_header parent_date $upstream_http_date;
+            add_header child_msec $msec;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8081;
+        server_name  localhost;
+
+        location / {
+            proxy_pass             http://127.0.0.1:8082;
+            proxy_cache            NAME2;
+            proxy_http_version     1.1;
+            proxy_cache_revalidate on;
+            add_header origin_date $upstream_http_date;
+            add_header parent_msec $msec;
+        }
+    }
+
+    server {
+        listen       127.0.0.1:8082;
+        server_name  localhost;
+
+        location / {
+            add_header Cache-Control $http_x_cache_control;
+            limit_rate $rate;
+            add_header origin_msec $msec;
+        }
+    }
+}
+
+EOF
+
+$t->write_file('t.html', 'SEE-THIS');
+$t->write_file('t2.html', 'SEE-THIS');
+$t->write_file('t3.html', 'SEE-THIS');
+
+$t->run();
+
+###############################################################################
+
+# normal origin
+
+wait_until_next_second();
+
+like(get('/t.html', 's-maxage=2'), qr/
Age: 0
/, 'age first');
+
+sleep 2;
+
+like(get('/t.html', 's-maxage=2'), qr/
Age: 2
/, 'age hit');
+
+sleep 1;
+
+like(http_get('/t.html'), qr/
Age: 0
/, 'age updated');
+
+# slow origin
+
+SKIP: {
+skip 'no exec on win32', 3 if $^O eq 'MSWin32';
+
+wait_until_next_second();
+
+like(get('/t2.html?slow=1', 's-maxage=6'), qr/
Age: 4
/,
+    'slow origin first');
+
+sleep 2;
+
+like(http_get('/t2.html?slow=1'), qr/
Age: 6
/, 'slow origin hit');
+
+sleep 1;
+
+like(http_get('/t2.html?slow=1'), qr/
Age: 5
/, 'slow origin updated');
+
+}
+
+# update age after restart
+
+wait_until_next_second();
+
+like(get('/t3.html', 's-maxage=20'), qr/
Age: 0
/, 'age before restart');
+my $t1 = time();
+
+$t->stop();
+
+$t->run();
+
+my $resident_time = time() - $t1;
+like(http_get('/t3.html'), qr/
Age: $resident_time
/,
+    'age after restart');
+
+$t->stop();
+
+###############################################################################
+
+sub get {
+       my ($url, $extra, %extra) = @_;
+       return http(<<EOF, %extra);
+GET $url HTTP/1.1
+Host: localhost
+Connection: close
+X-Cache-Control: $extra
+
+EOF
+}
+
+# Wait until the next second boundary.
+# Calling this before sending a request increases the likelihood that the
+# timestamp value does not cross into the next second while sending a request
+# and receiving a response.
+sub wait_until_next_second {
+    my $now = time();
+    my $next_second = ceil($now);
+    my $sleep = $next_second - $now;
+    select undef, undef, undef, $sleep;
+}
+
+###############################################################################
diff -r 51355c41fc13 -r 8473b99b2c16 proxy_cache_use_stale.t
--- a/proxy_cache_use_stale.t   Fri Jun 14 15:17:30 2024 +0900
+++ b/proxy_cache_use_stale.t   Fri Jun 14 15:17:35 2024 +0900
@@ -206,7 +206,8 @@

 $t->write_file('t6.html', 'SEE-THAT');

-my $s = get('/t6.html', 'max-age=1, stale-while-revalidate=2', start => 1);
+# max-age must be 5 here since response delay is 4 seconds.
+my $s = get('/t6.html', 'max-age=5, stale-while-revalidate=2', start => 1);
 select undef, undef, undef, 0.2;
 like(http_get('/t6.html'), qr/UPDATING.*SEE-THIS/s, 's-w-r - updating');
 like(http_end($s), qr/STALE.*SEE-THIS/s, 's-w-r - updating stale');

From hnakamur at gmail.com  Fri Jun 14 07:29:19 2024
From: hnakamur at gmail.com (Hiroaki Nakamura)
Date: Fri, 14 Jun 2024 16:29:19 +0900
Subject: [PATCH 2 of 3] Save response time and corrected initial age to file
 cache header
Message-ID: <CAN-DUMRk50c=byfaYu8ZRCVf2je63T=kzNq_pCaO6NHMmFc2QQ@mail.gmail.com>

# HG changeset patch
# User Hiroaki Nakamura <hnakamur at gmail.com>
# Date 1718345850 -32400
#      Fri Jun 14 15:17:30 2024 +0900
# Branch correct_age
# Node ID 51355c41fc134954e58b1534d232291d70252be7
# Parent  493817cf38580a71430f9c1293e29bdfbf243075
Save response time and corrected initial age to file cache header.

diff -r 493817cf3858 -r 51355c41fc13 src/http/ngx_http_cache.h
--- a/src/http/ngx_http_cache.h Fri Jun 14 15:17:24 2024 +0900
+++ b/src/http/ngx_http_cache.h Fri Jun 14 15:17:30 2024 +0900
@@ -27,7 +27,7 @@
 #define NGX_HTTP_CACHE_ETAG_LEN      128
 #define NGX_HTTP_CACHE_VARY_LEN      128

-#define NGX_HTTP_CACHE_VERSION       5
+#define NGX_HTTP_CACHE_VERSION       6


 typedef struct {
@@ -145,6 +145,8 @@
     u_char                           vary_len;
     u_char                           vary[NGX_HTTP_CACHE_VARY_LEN];
     u_char                           variant[NGX_HTTP_CACHE_KEY_LEN];
+    time_t                           response_time;
+    time_t                           corrected_initial_age;
 } ngx_http_file_cache_header_t;


diff -r 493817cf3858 -r 51355c41fc13 src/http/ngx_http_file_cache.c
--- a/src/http/ngx_http_file_cache.c    Fri Jun 14 15:17:24 2024 +0900
+++ b/src/http/ngx_http_file_cache.c    Fri Jun 14 15:17:30 2024 +0900
@@ -627,6 +627,8 @@
     c->body_start = h->body_start;
     c->etag.len = h->etag_len;
     c->etag.data = h->etag;
+    c->response_time = h->response_time;
+    c->corrected_initial_age = h->corrected_initial_age;

     r->cached = 1;

@@ -1330,6 +1332,8 @@
     h->valid_msec = (u_short) c->valid_msec;
     h->header_start = (u_short) c->header_start;
     h->body_start = (u_short) c->body_start;
+    h->response_time = c->response_time;
+    h->corrected_initial_age = c->corrected_initial_age;

     if (c->etag.len <= NGX_HTTP_CACHE_ETAG_LEN) {
         h->etag_len = (u_char) c->etag.len;
@@ -1594,6 +1598,8 @@
     h.valid_msec = (u_short) c->valid_msec;
     h.header_start = (u_short) c->header_start;
     h.body_start = (u_short) c->body_start;
+    h.response_time = c->response_time;
+    h.corrected_initial_age = c->corrected_initial_age;

     if (c->etag.len <= NGX_HTTP_CACHE_ETAG_LEN) {
         h.etag_len = (u_char) c->etag.len;

From vbart at wbsrv.ru  Sat Jun 15 18:31:52 2024
From: vbart at wbsrv.ru (Valentin V. Bartenev)
Date: Sat, 15 Jun 2024 21:31:52 +0300
Subject: [PATCH] Fixed checking for no port in the upstream block references
Message-ID: <f09d4844-2540-4265-a6dd-c252f10ac60f@wbsrv.ru>

# HG changeset patch
# User Valentin Bartenev <vbart at wbsrv.ru>
# Date 1718475780 -10800
#      Sat Jun 15 21:23:00 2024 +0300
# Node ID 6cda0d648e42c173e9eb2de5b5baab6a2d34595b
# Parent  02e9411009b987f408214ab4a8b6b6093f843bcd
Fixed checking for no port in the upstream block references.

A configurations like this one:

   server {
       proxy_pass backend:80;
   }

   server {
       proxy_pass backend;
   }

   upstream backend {
       server 127.0.0.1:443;
   }

is rejected, since a "proxy_pass" refence of the upstream block cannot have
a port specified.  But if the "proxy_pass" directive with no port appeared
first then the check for no port didn't work properly and such configurations
were allowed:

   server {
       proxy_pass backend;
   }

   server {
       proxy_pass backend:80;
   }

   upstream backend {
       server 127.0.0.1:443;
   }

So, the fix is to continue lookup for all upstreams in the list checking
for duplicate references with port specified.

The issue was present in both "stream" and "http" modules.  And in case of
there assumes port 80 (or 443 for https) by default.  For example, in the
config below:

   server {
       location / {
           proxy_pass http://backend;
       }
   }

   server {
       location / {
           proxy_pass http://backend:80;
       }
   }

   upstream backend {
       server 127.0.0.1:8000;
   }

the second "proxy_pass" is aggregated with the first one, but the "no_port"
flag was left intact.  In order, to properly detect such cases now the flag
is reset at the first case of reference with an explicit port set and also
the position of such reference is stored for a proper error handling.

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -6341,6 +6341,7 @@ ngx_http_upstream_add(ngx_conf_t *cf, ng
      umcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_upstream_module);
  
      uscfp = umcf->upstreams.elts;
+    uscf = NULL;
  
      for (i = 0; i < umcf->upstreams.nelts; i++) {
  
@@ -6383,11 +6384,23 @@ ngx_http_upstream_add(ngx_conf_t *cf, ng
          if (flags & NGX_HTTP_UPSTREAM_CREATE) {
              uscfp[i]->flags = flags;
              uscfp[i]->port = 0;
+            uscf = uscfp[i];
+            continue;
+        }
+
+        if (uscfp[i]->no_port && !u->no_port) {
+            uscfp[i]->no_port = 0;
+            uscfp[i]->file_name = cf->conf_file->file.name.data;
+            uscfp[i]->line = cf->conf_file->line;
          }
  
          return uscfp[i];
      }
  
+    if (uscf) {
+        return uscf;
+    }
+
      uscf = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_srv_conf_t));
      if (uscf == NULL) {
          return NULL;
diff --git a/src/stream/ngx_stream_upstream.c b/src/stream/ngx_stream_upstream.c
--- a/src/stream/ngx_stream_upstream.c
+++ b/src/stream/ngx_stream_upstream.c
@@ -583,6 +583,7 @@ ngx_stream_upstream_add(ngx_conf_t *cf,
      umcf = ngx_stream_conf_get_module_main_conf(cf, ngx_stream_upstream_module);
  
      uscfp = umcf->upstreams.elts;
+    uscf = NULL;
  
      for (i = 0; i < umcf->upstreams.nelts; i++) {
  
@@ -622,11 +623,17 @@ ngx_stream_upstream_add(ngx_conf_t *cf,
  
          if (flags & NGX_STREAM_UPSTREAM_CREATE) {
              uscfp[i]->flags = flags;
+            uscf = uscfp[i];
+            continue;
          }
  
          return uscfp[i];
      }
  
+    if (uscf) {
+        return uscf;
+    }
+
      uscf = ngx_pcalloc(cf->pool, sizeof(ngx_stream_upstream_srv_conf_t));
      if (uscf == NULL) {
          return NULL;


From jordanc.carter at outlook.com  Sun Jun 16 00:43:01 2024
From: jordanc.carter at outlook.com (J Carter)
Date: Sun, 16 Jun 2024 01:43:01 +0100
Subject: [PATCH 1 of 2] Rewritten host header validation to follow
 generic parsing rules
In-Reply-To: <tfnlaj4oafo7xpqf2gu45gxkbpo5zusdbc55u6b3m32wn6lnbi@w6bkdippzokq>
References: <e82a7318ed48fdbc1273.1716805303@enoparse.local>
 <LO0P123MB6319FF781672708FBF0C61CD9EF12@LO0P123MB6319.GBRP123.PROD.OUTLOOK.COM>
 <tfnlaj4oafo7xpqf2gu45gxkbpo5zusdbc55u6b3m32wn6lnbi@w6bkdippzokq>
Message-ID: <LO0P123MB63199377B350D991F601389A9ECC2@LO0P123MB6319.GBRP123.PROD.OUTLOOK.COM>

Hello Sergey,

I had some trouble importing your patches in one go due to missing
whitespace after patches 2 and 3. In case anyone else has the same
issue just add 1 new line after each of those patches.

On Fri, 7 Jun 2024 20:48:23 +0400
Sergey Kandaurov <pluknet at nginx.com> wrote:

> On Tue, May 28, 2024 at 12:53:46PM +0100, J Carter wrote:
> > Hello Sergey,
> > 
> > On Mon, 27 May 2024 14:21:43 +0400
> > Sergey Kandaurov <pluknet at nginx.com> wrote:
> > 
> > > # HG changeset patch
> > > # User Sergey Kandaurov <pluknet at nginx.com>
> > > # Date 1716805272 -14400
> > > #      Mon May 27 14:21:12 2024 +0400
> > > # Node ID e82a7318ed48fdbc1273771bc96357e9dc232975
> > > # Parent  f58b6f6362387eeace46043a6fc0bceb56a6786a
> > > Rewritten host header validation to follow generic parsing rules.
> > > 
> > > It now uses a generic model of state-based machine, with more strict
> > > parsing rules borrowed from ngx_http_validate_host(),
> > 
> > I think you mean "borrowed from ngx_http_parse_request_line()".
> > 
> 
> Sure, tnx.
> 
> The problem is that both functions make a subset of parsing
> and validation, and these sets just partially intersect.
> In particular, ngx_http_parse_request_line() currently detects
> invalid characters in a port subcomponent of authority, and
> ngx_http_validate_host() handles a trailing dot.
> So I think it makes sense to make them unifined, this will also
> remove the need to validate host in absolute URIs.
> Below is an updated version with both parsers further polished
> (stream part is excluded for now).
> 
> Also, it may have sense to rename ngx_http_validate_host()
> to something like ngx_http_parse_host(), similar to
> ngx_http_parse_uri(), out of this series.

Makes sense.

> 
> # HG changeset patch
> # User Sergey Kandaurov <pluknet at nginx.com>
> # Date 1717777582 -14400
> #      Fri Jun 07 20:26:22 2024 +0400
> # Node ID 0cba4301e4980871de7aceb46acddf8f2b5a7318
> # Parent  02e9411009b987f408214ab4a8b6b6093f843bcd
> Improved parsing of host in absolute URIs.
> 
> When the request line is in the absolute-URI form, a host identified
> by a registered name (reg-name) is now restricted to start with an
> alphanumeric character (see RFC 1123, RFC 3986).  Previously, empty
> domain labels or host starting with a hyphen were accepted.
> 
> Additionally, host with a trailing dot is taken into account.
> 
> diff --git a/src/http/ngx_http_parse.c b/src/http/ngx_http_parse.c
> --- a/src/http/ngx_http_parse.c
> +++ b/src/http/ngx_http_parse.c
> @@ -113,8 +113,10 @@ ngx_http_parse_request_line(ngx_http_req
>          sw_schema_slash_slash,
>          sw_host_start,
>          sw_host,
> +        sw_host_dot,
>          sw_host_end,
>          sw_host_ip_literal,
> +        sw_host_ip_literal_dot,
>          sw_port,
>          sw_after_slash_in_uri,
>          sw_check_uri,
> @@ -354,27 +356,50 @@ ngx_http_parse_request_line(ngx_http_req
>                  break;
>              }
>  
> -            state = sw_host;
> +            if (ch == '.' || ch == '-') {
> +                return NGX_HTTP_PARSE_INVALID_REQUEST;

One inconsistency I noticed while testing these patches is the difference
in the usefulness of error logs between invalid host header vs invalid
host value in absolute uri. 

Absolute uri parsing simply includes a vague "error while parsing
client request line" for everything, including failed host checks,
whereas host header failure error is naturally more specific to what
the error is. Perhaps improving that is beyond scope of this series
however.

> +            }
>  
>              /* fall through */
>  
>          case sw_host:
> +        case sw_host_dot:
> +
> +            if (ch == '.') {
> +                if (state == sw_host_dot) {
> +                    return NGX_HTTP_PARSE_INVALID_REQUEST;
> +                }
> +
> +                state = sw_host_dot;
> +                break;
> +            }
>  
>              c = (u_char) (ch | 0x20);
>              if (c >= 'a' && c <= 'z') {
> +                state = sw_host;
> +                break;
> +            }
> +
> +            if ((ch >= '0' && ch <= '9') || ch == '-') {
> +                state = sw_host;
>                  break;
>              }
>  
> -            if ((ch >= '0' && ch <= '9') || ch == '.' || ch == '-') {
> -                break;
> +            if (state == sw_host_start) {
> +                return NGX_HTTP_PARSE_INVALID_REQUEST;
> +            }
> +
> +            if (state == sw_host_dot) {
> +                r->host_end = p - 1;
> +
> +            } else {
> +                r->host_end = p;
>              }
>  
>              /* fall through */
>  
>          case sw_host_end:
>  
> -            r->host_end = p;
> -
>              switch (ch) {
>              case ':':
>                  state = sw_port;
> @@ -404,6 +429,18 @@ ngx_http_parse_request_line(ngx_http_req
>              break;
>  
>          case sw_host_ip_literal:
> +        case sw_host_ip_literal_dot:
> +
> +            if (ch == '.') {
> +                if (state == sw_host_ip_literal_dot) {
> +                    return NGX_HTTP_PARSE_INVALID_REQUEST;
> +                }
> +
> +                state = sw_host_ip_literal_dot;
> +                break;
> +            }
> +
> +            state = sw_host_ip_literal;
>  
>              if (ch >= '0' && ch <= '9') {
>                  break;
> @@ -418,10 +455,10 @@ ngx_http_parse_request_line(ngx_http_req
>              case ':':
>                  break;
>              case ']':
> +                r->host_end = p + 1;
>                  state = sw_host_end;
>                  break;
>              case '-':
> -            case '.':
>              case '_':
>              case '~':
>                  /* unreserved */
> # HG changeset patch
> # User Sergey Kandaurov <pluknet at nginx.com>
> # Date 1717777646 -14400
> #      Fri Jun 07 20:27:26 2024 +0400
> # Node ID 3f7ac1d90a6d4eceabaa5ce45dabf53efd99ed67
> # Parent  0cba4301e4980871de7aceb46acddf8f2b5a7318
> Rewritten host validation to match host parsing in absolute URIs.
> 
> It is reimplemented based on ngx_http_parse_request_line() state machine.
> This introduces several changes, in particular:
> - host name with underscores is rejected;
> - a port subcomponent is restricted to digits;
> - for IP literals, a missing closing bracket and trailing dot are detected.
> 

I see the changes in this patch cause a fail on 'double port hack' 
test from http_host.t. I'm not sure on the context behind that test, 
as it's quite an odd thing to test for (perhaps a workaround for something?)

The author of that test was Valentin Bartenev, perhaps he will kindly
let us know the context if he remembers, and we do not.

> diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
> --- a/src/http/ngx_http_request.c
> +++ b/src/http/ngx_http_request.c
> @@ -2145,72 +2145,157 @@ ngx_int_t
>  ngx_http_validate_host(ngx_str_t *host, ngx_pool_t *pool, ngx_uint_t alloc)
>  {
>      u_char  *h, ch;
> -    size_t   i, dot_pos, host_len;
> +    size_t   i, host_len;
>  
>      enum {
> -        sw_usual = 0,
> -        sw_literal,
> -        sw_rest
> +        sw_host_start = 0,
> +        sw_host,
> +        sw_host_dot,
> +        sw_host_end,
> +        sw_host_ip_literal,
> +        sw_host_ip_literal_dot,
> +        sw_port,
>      } state;
>  
> -    dot_pos = host->len;
>      host_len = host->len;
>  
>      h = host->data;
>  
> -    state = sw_usual;
> +    state = sw_host_start;
>  
>      for (i = 0; i < host->len; i++) {
>          ch = h[i];
>  
> -        switch (ch) {
> -
> -        case '.':
> -            if (dot_pos == i - 1) {
> +        switch (state) {
> +
> +        case sw_host_start:
> +
> +            if (ch == '[') {
> +                state = sw_host_ip_literal;
> +                break;
> +            }
> +
> +            if (ch == '.' || ch == '-') {
>                  return NGX_DECLINED;
>              }
> -            dot_pos = i;
> -            break;
> -
> -        case ':':
> -            if (state == sw_usual) {
> -                host_len = i;
> -                state = sw_rest;
> -            }
> -            break;
> -
> -        case '[':
> -            if (i == 0) {
> -                state = sw_literal;
> -            }
> -            break;
> -
> -        case ']':
> -            if (state == sw_literal) {
> -                host_len = i + 1;
> -                state = sw_rest;
> -            }
> -            break;
> -
> -        default:
> -
> -            if (ngx_path_separator(ch)) {
> -                return NGX_DECLINED;
> -            }
> -
> -            if (ch <= 0x20 || ch == 0x7f) {
> -                return NGX_DECLINED;
> +
> +            /* fall through */
> +
> +        case sw_host:
> +        case sw_host_dot:
> +
> +            if (ch == '.') {
> +                if (state == sw_host_dot) {
> +                    return NGX_DECLINED;
> +                }
> +
> +                state = sw_host_dot;
> +                break;
>              }
>  
>              if (ch >= 'A' && ch <= 'Z') {
>                  alloc = 1;
> +                state = sw_host;
> +                break;
>              }
>  
> +            if (ch >= 'a' && ch <= 'z') {
> +                state = sw_host;
> +                break;
> +            }
> +
> +            if ((ch >= '0' && ch <= '9') || ch == '-') {
> +                state = sw_host;
> +                break;
> +            }
> +
> +            if (state == sw_host_dot) {
> +                host_len = i - 1;
> +
> +            } else {
> +                host_len = i;
> +            }
> +
> +            /* fall through */
> +
> +        case sw_host_end:
> +
> +            if (ch == ':') {
> +                state = sw_port;
> +                break;
> +            }
> +            return NGX_DECLINED;
> +
> +        case sw_host_ip_literal:
> +        case sw_host_ip_literal_dot:
> +
> +            if (ch == '.') {
> +                if (state == sw_host_ip_literal_dot) {
> +                    return NGX_DECLINED;
> +                }
> +
> +                state = sw_host_ip_literal_dot;
> +                break;
> +            }
> +
> +            state = sw_host_ip_literal;
> +
> +            if (ch >= 'A' && ch <= 'Z') {
> +                alloc = 1;
> +                break;
> +            }
> +
> +            if (ch >= 'a' && ch <= 'z') {
> +                break;
> +            }
> +
> +            if (ch >= '0' && ch <= '9') {
> +                break;
> +            }
> +
> +            switch (ch) {
> +            case ':':
> +                break;
> +            case ']':
> +                host_len = i + 1;
> +                state = sw_host_end;
> +                break;
> +            case '-':
> +            case '_':
> +            case '~':
> +                /* unreserved */
> +                break;
> +            case '!':
> +            case '$':
> +            case '&':
> +            case '\'':
> +            case '(':
> +            case ')':
> +            case '*':
> +            case '+':
> +            case ',':
> +            case ';':
> +            case '=':
> +                /* sub-delims */
> +                break;
> +            default:
> +                return NGX_DECLINED;
> +            }
>              break;
> +
> +        case sw_port:
> +            if (ch >= '0' && ch <= '9') {
> +                break;
> +            }

Although likely beyond scope of this series, it does seem a bit
inconsistent to reject request for non-numeric characters in port, while
not also doing so for obviously incorrect port values.

For example: Host: 127.0.0.1:77777 is accepted.

> +            return NGX_DECLINED;
>          }
>      }
>  
> -    if (dot_pos == host_len - 1) {
> +    if (state == sw_host_ip_literal) {
> +        return NGX_DECLINED;
> +    }
> +
> +    if (h[host_len - 1] == '.') {
>          host_len--;
>      }
>  
> # HG changeset patch
> # User Sergey Kandaurov <pluknet at nginx.com>
> # Date 1717777737 -14400
> #      Fri Jun 07 20:28:57 2024 +0400
> # Node ID 722bcffe3d3c9ff4314a2813227c47dc5eff660e
> # Parent  3f7ac1d90a6d4eceabaa5ce45dabf53efd99ed67
> Skip host validation in absolute URIs.
> 
> Now that parsing of host in the absolute-URI form and of the host header
> were made the same in previous changes, it makes no sense to validate
> host once again.  Only host case normalization to lowercase is applied
> (RFC 3986, 6.2.2.1) after parsing absolute URI as this is out of scope.
> 
> No functional changes intended.
> 
> diff --git a/src/http/ngx_http_parse.c b/src/http/ngx_http_parse.c
> --- a/src/http/ngx_http_parse.c
> +++ b/src/http/ngx_http_parse.c
> @@ -374,8 +374,13 @@ ngx_http_parse_request_line(ngx_http_req
>                  break;
>              }
>  
> -            c = (u_char) (ch | 0x20);
> -            if (c >= 'a' && c <= 'z') {
> +            if (ch >= 'A' && ch <= 'Z') {
> +                r->host_normalize = 1;
> +                state = sw_host;
> +                break;
> +            }
> +
> +            if (ch >= 'a' && ch <= 'z') {
>                  state = sw_host;
>                  break;
>              }
> @@ -446,8 +451,12 @@ ngx_http_parse_request_line(ngx_http_req
>                  break;
>              }
>  
> -            c = (u_char) (ch | 0x20);
> -            if (c >= 'a' && c <= 'z') {
> +            if (ch >= 'A' && ch <= 'Z') {
> +                r->host_normalize = 1;
> +                break;
> +            }
> +
> +            if (ch >= 'a' && ch <= 'z') {
>                  break;
>              }
>  
> diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
> --- a/src/http/ngx_http_request.c
> +++ b/src/http/ngx_http_request.c
> @@ -1147,18 +1147,15 @@ ngx_http_process_request_line(ngx_event_
>                  host.len = r->host_end - r->host_start;
>                  host.data = r->host_start;
>  
> -                rc = ngx_http_validate_host(&host, r->pool, 0);
> -
> -                if (rc == NGX_DECLINED) {
> -                    ngx_log_error(NGX_LOG_INFO, c->log, 0,
> -                                  "client sent invalid host in request line");
> -                    ngx_http_finalize_request(r, NGX_HTTP_BAD_REQUEST);
> -                    break;
> -                }
> -
> -                if (rc == NGX_ERROR) {
> -                    ngx_http_close_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
> -                    break;
> +                if (r->host_normalize) {
> +                    host.data = ngx_pnalloc(r->pool, host.len);
> +                    if (host.data == NULL) {
> +                        ngx_http_close_request(r,
> +                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
> +                        break;
> +                    }
> +
> +                    ngx_strlow(host.data, r->host_start, host.len);
>                  }
>  
>                  if (ngx_http_set_virtual_server(r, &host) == NGX_ERROR) {
> diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
> --- a/src/http/ngx_http_request.h
> +++ b/src/http/ngx_http_request.h
> @@ -466,6 +466,9 @@ struct ngx_http_request_s {
>  
>      unsigned                          http_state:4;
>  
> +    /* host with upper case */
> +    unsigned                          host_normalize:1;
> +
>      /* URI with "/." and on Win32 with "//" */
>      unsigned                          complex_uri:1;
>  

Functionally everything appears to be correct in these patches. 

I do have a question about real world impact of these changes
particularly around underscore in Host (following on from concerns
raised by others out-of-band). Do we have any statistics on the use of
underscores in Host, today, on the open internet? 

If not, is it worth attempting to source that information before
making these changes to better assess the impact / if 'no underscore in
host header' enforcement should be optional, behind a flag?

From vbart at wbsrv.ru  Sun Jun 16 11:28:09 2024
From: vbart at wbsrv.ru (Valentin V. Bartenev)
Date: Sun, 16 Jun 2024 14:28:09 +0300
Subject: [PATCH] Fixed checking for no ports in the upstream block references
In-Reply-To: <f09d4844-2540-4265-a6dd-c252f10ac60f@wbsrv.ru>
References: <f09d4844-2540-4265-a6dd-c252f10ac60f@wbsrv.ru>
Message-ID: <994dfd57-3315-43b0-8ae1-c987f299bb3b@wbsrv.ru>

Improved commit message.
The previous one was a bit messy with missing fragments.


# HG changeset patch
# User Valentin Bartenev <vbart at wbsrv.ru>
# Date 1718536452 -10800
#      Sun Jun 16 14:14:12 2024 +0300
# Node ID 4fbc38ad3c8a8ed7986aaaa76b8aceda5ed70ef3
# Parent  02e9411009b987f408214ab4a8b6b6093f843bcd
Fixed checking for no ports in the upstream block references.

Configurations like this one:

   server {
       proxy_pass backend:80;
   }

   server {
       proxy_pass backend;
   }

   upstream backend {
       server 127.0.0.1:443;
   }

are rejected, since a "proxy_pass" reference of the upstream block cannot
have a port specified.  But if the "proxy_pass" directive with no port
appeared first then the check for no port didn't work properly and such
configurations were allowed:

   server {
       proxy_pass backend;
   }

   server {
       proxy_pass backend:80;
   }

   upstream backend {
       server 127.0.0.1:443;
   }

So, the fix is to continue lookup for all upstreams in the list checking
for duplicate references with port specified.

The issue was present in both "stream" and "http" modules.  And in case of
the "http" module, the fix is a bit more tricky there, since its "proxy_pass"
directive assumes port 80 (or 443 for https) by default.  For example, in the
configuration below:

   server {
       location / {
           proxy_pass http://backend;
       }
   }

   server {
       location / {
           proxy_pass http://backend:80;
       }
   }

   upstream backend {
       server 127.0.0.1:8000;
   }

the second "proxy_pass" is aggregated with the first one, but the "no_port"
flag was left intact.  In order to properly detect such cases, now the flag
is reset in the first case of reference with an explicit port set and also
the position of such reference is stored for a proper error handling.

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -6341,6 +6341,7 @@ ngx_http_upstream_add(ngx_conf_t *cf, ng
      umcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_upstream_module);
  
      uscfp = umcf->upstreams.elts;
+    uscf = NULL;
  
      for (i = 0; i < umcf->upstreams.nelts; i++) {
  
@@ -6383,11 +6384,23 @@ ngx_http_upstream_add(ngx_conf_t *cf, ng
          if (flags & NGX_HTTP_UPSTREAM_CREATE) {
              uscfp[i]->flags = flags;
              uscfp[i]->port = 0;
+            uscf = uscfp[i];
+            continue;
+        }
+
+        if (uscfp[i]->no_port && !u->no_port) {
+            uscfp[i]->no_port = 0;
+            uscfp[i]->file_name = cf->conf_file->file.name.data;
+            uscfp[i]->line = cf->conf_file->line;
          }
  
          return uscfp[i];
      }
  
+    if (uscf) {
+        return uscf;
+    }
+
      uscf = ngx_pcalloc(cf->pool, sizeof(ngx_http_upstream_srv_conf_t));
      if (uscf == NULL) {
          return NULL;
diff --git a/src/stream/ngx_stream_upstream.c b/src/stream/ngx_stream_upstream.c
--- a/src/stream/ngx_stream_upstream.c
+++ b/src/stream/ngx_stream_upstream.c
@@ -583,6 +583,7 @@ ngx_stream_upstream_add(ngx_conf_t *cf,
      umcf = ngx_stream_conf_get_module_main_conf(cf, ngx_stream_upstream_module);
  
      uscfp = umcf->upstreams.elts;
+    uscf = NULL;
  
      for (i = 0; i < umcf->upstreams.nelts; i++) {
  
@@ -622,11 +623,17 @@ ngx_stream_upstream_add(ngx_conf_t *cf,
  
          if (flags & NGX_STREAM_UPSTREAM_CREATE) {
              uscfp[i]->flags = flags;
+            uscf = uscfp[i];
+            continue;
          }
  
          return uscfp[i];
      }
  
+    if (uscf) {
+        return uscf;
+    }
+
      uscf = ngx_pcalloc(cf->pool, sizeof(ngx_stream_upstream_srv_conf_t));
      if (uscf == NULL) {
          return NULL;


From noreply at nginx.com  Tue Jun 25 17:41:04 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Tue, 25 Jun 2024 17:41:04 +0000
Subject: [njs] Version 0.8.5.
Message-ID: <hg.a419f9189f55.1719337264.-3104155734389529393@dev.nginx>

details:   https://hg.nginx.org/njs/rev/a419f9189f55
branches:  
changeset: 2359:a419f9189f55
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Jun 24 17:09:07 2024 -0700
description:
Version 0.8.5.

diffstat:

 CHANGES |  50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 50 insertions(+), 0 deletions(-)

diffs (57 lines):

diff -r 3ba1433803ee -r a419f9189f55 CHANGES
--- a/CHANGES	Thu May 30 22:22:48 2024 -0700
+++ b/CHANGES	Mon Jun 24 17:09:07 2024 -0700
@@ -1,3 +1,53 @@
+Changes with njs 0.8.5                                       25 Jun 2024
+
+    nginx modules:
+
+    *) Change: r.variables.var, r.requestText, r.responseText,
+       s.variables.var, and the "data" argument of the s.on() callback
+       with "upload" or "download" event types will now convert bytes
+       invalid in UTF-8 encoding into the replacement character.  When
+       working with binary data, use r.rawVariables.var, r.requestBuffer,
+       r.responseBuffer, s.rawVariables.var, and the "upstream" or
+       "downstream" event type for s.on() instead.
+
+    *) Feature: added timeout argument for shared dictionary methods
+       add(), set() and incr().
+
+    *) Bugfix: fixed checking for duplicate js_set variables.
+
+    *) Bugfix: fixed request Host header when the port is non-standard.
+
+    *) Bugfix: fixed handling of a zero-length request body in ngx.fetch()
+       and r.subrequest().
+
+    *) Bugfix: fixed heap-buffer-overflow in Headers.get().
+
+    *) Bugfix: fixed r.subrequest() error handling.
+
+    Core:
+
+    *) Feature: added zlib module for QuickJS engine.
+
+    *) Bugfix: fixed zlib.inflate().
+
+    *) Bugfix: fixed String.prototype.replaceAll() with zero-length
+       argument.
+
+    *) Bugfix: fixed retval handling after an exception in
+       Array.prototype.toSpliced(), Array.prototype.toReversed(),
+       Array.prototype.toSorted().
+
+    *) Bugfix: fixed RegExp.prototype[@@replace]() with replacements
+       containing "$'", "$\`" and strings with Unicode characters.
+
+    *) Bugfix: fixed a one-byte overread in decodeURI() and
+       decodeURIComponent().
+
+    *) Bugfix: fixed tracking of argument scope.
+
+    *) Bugfix: fixed integer overflow in Date.parse().
+
+
 Changes with njs 0.8.4                                       16 Apr 2024
 
     nginx modules:

From arut at nginx.com  Thu Jun 27 13:33:58 2024
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Thu, 27 Jun 2024 17:33:58 +0400
Subject: [PATCH] Stream: allow servers with no handler
Message-ID: <252582428b60f3422aa3.1719495238@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1719494996 -14400
#      Thu Jun 27 17:29:56 2024 +0400
# Node ID 252582428b60f3422aa3b25dac8cca94edd43c34
# Parent  4fbc38ad3c8a8ed7986aaaa76b8aceda5ed70ef3
Stream: allow servers with no handler.

Previously handlers were mandatory.  However they are not always needed.
For example, a server configured with ssl_reject_handshake does not need a
handler.  Such servers required a fake handler to pass the check.  Now handler
absence check is moved to runtime.  If handler is missing, the connection is
closed with 500 code.

diff --git a/src/stream/ngx_stream_core_module.c b/src/stream/ngx_stream_core_module.c
--- a/src/stream/ngx_stream_core_module.c
+++ b/src/stream/ngx_stream_core_module.c
@@ -458,6 +458,13 @@ ngx_stream_core_content_phase(ngx_stream
         return NGX_OK;
     }
 
+    if (cscf->handler == NULL) {
+        ngx_log_debug0(NGX_LOG_DEBUG_STREAM, c->log, 0,
+                       "no handler for server");
+        ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+        return NGX_OK;
+    }
+
     cscf->handler(s);
 
     return NGX_OK;
@@ -734,13 +741,6 @@ ngx_stream_core_merge_srv_conf(ngx_conf_
         conf->resolver = prev->resolver;
     }
 
-    if (conf->handler == NULL) {
-        ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
-                      "no handler for server in %s:%ui",
-                      conf->file_name, conf->line);
-        return NGX_CONF_ERROR;
-    }
-
     if (conf->error_log == NULL) {
         if (prev->error_log) {
             conf->error_log = prev->error_log;

From noreply at nginx.com  Sat Jun 29 02:52:04 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Sat, 29 Jun 2024 02:52:04 +0000
Subject: [njs] Modules: removed not needed previous location initialization.
Message-ID: <hg.5b5f45340f9f.1719629524.1952099648323381796@dev.nginx>

details:   https://hg.nginx.org/njs/rev/5b5f45340f9f
branches:  
changeset: 2360:5b5f45340f9f
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Jun 17 21:35:02 2024 -0700
description:
Modules: removed not needed previous location initialization.

diffstat:

 nginx/ngx_js.c |  6 ------
 1 files changed, 0 insertions(+), 6 deletions(-)

diffs (16 lines):

diff -r a419f9189f55 -r 5b5f45340f9f nginx/ngx_js.c
--- a/nginx/ngx_js.c	Mon Jun 24 17:09:07 2024 -0700
+++ b/nginx/ngx_js.c	Mon Jun 17 21:35:02 2024 -0700
@@ -1542,12 +1542,6 @@ ngx_js_merge_vm(ngx_conf_t *cf, ngx_js_l
     ngx_array_t          *imports, *preload_objects, *paths;
     ngx_js_named_path_t  *import, *pi, *pij, *preload;
 
-    if (prev->imports != NGX_CONF_UNSET_PTR && prev->vm == NULL) {
-        if (init_vm(cf, (ngx_js_loc_conf_t *) prev) != NGX_OK) {
-            return NGX_ERROR;
-        }
-    }
-
     if (conf->imports == NGX_CONF_UNSET_PTR
         && conf->paths == NGX_CONF_UNSET_PTR
         && conf->preload_objects == NGX_CONF_UNSET_PTR)

From noreply at nginx.com  Sat Jun 29 02:52:06 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Sat, 29 Jun 2024 02:52:06 +0000
Subject: [njs] HTTP: simplified check for subrequest from a subrequest.
Message-ID: <hg.8b182df14819.1719629526.1952099648323381796@dev.nginx>

details:   https://hg.nginx.org/njs/rev/8b182df14819
branches:  
changeset: 2361:8b182df14819
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Jun 12 23:31:08 2024 -0700
description:
HTTP: simplified check for subrequest from a subrequest.

diffstat:

 nginx/ngx_http_js_module.c |  4 ++--
 nginx/t/js_subrequests.t   |  5 +++--
 2 files changed, 5 insertions(+), 4 deletions(-)

diffs (43 lines):

diff -r 5b5f45340f9f -r 8b182df14819 nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c	Mon Jun 17 21:35:02 2024 -0700
+++ b/nginx/ngx_http_js_module.c	Wed Jun 12 23:31:08 2024 -0700
@@ -3153,7 +3153,7 @@ ngx_http_js_ext_subrequest(njs_vm_t *vm,
 
     ctx = ngx_http_get_module_ctx(r, ngx_http_js_module);
 
-    if (ctx->vm != vm) {
+    if (r->main != r) {
         njs_vm_error(vm, "subrequest can only be created for "
                          "the primary request");
         return NJS_ERROR;
@@ -3511,7 +3511,7 @@ ngx_http_js_ext_get_parent(njs_vm_t *vm,
     ctx = r->parent ? ngx_http_get_module_ctx(r->parent, ngx_http_js_module)
                     : NULL;
 
-    if (ctx == NULL || ctx->vm != vm) {
+    if (ctx == NULL) {
         njs_value_undefined_set(retval);
         return NJS_DECLINED;
     }
diff -r 5b5f45340f9f -r 8b182df14819 nginx/t/js_subrequests.t
--- a/nginx/t/js_subrequests.t	Mon Jun 17 21:35:02 2024 -0700
+++ b/nginx/t/js_subrequests.t	Wed Jun 12 23:31:08 2024 -0700
@@ -590,6 +590,9 @@ local $TODO = 'not yet' unless has_versi
 
 http_get('/sr_error_in_callback');
 
+ok(index($t->read_file('error.log'), 'subrequest can only be created for') > 0,
+   'subrequest creation failed');
+
 }
 
 $t->stop();
@@ -602,8 +605,6 @@ ok(index($t->read_file('error.log'), 'fa
 	'subrequest invalid args exception');
 ok(index($t->read_file('error.log'), 'too big subrequest response') > 0,
 	'subrequest too large body');
-ok(index($t->read_file('error.log'), 'subrequest creation failed') > 0,
-	'subrequest creation failed');
 ok(index($t->read_file('error.log'),
 		'js subrequest: failed to get the parent context') > 0,
 	'zero parent ctx');

From noreply at nginx.com  Sat Jun 29 02:53:03 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Sat, 29 Jun 2024 02:53:03 +0000
Subject: =?utf-8?q?=5Bnjs=5D_Fixed_=E2=80=98length=E2=80=99_may_be_used_uninitialized?=
 =?utf-8?q?_in_Array=2Eprototype=2Epop=28=29=2E?=
Message-ID: <hg.146d4699a4c4.1719629583.-2089952106619681529@dev.nginx>

details:   https://hg.nginx.org/njs/rev/146d4699a4c4
branches:  
changeset: 2362:146d4699a4c4
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Thu Jun 20 17:26:14 2024 -0700
description:
Fixed ‘length’ may be used uninitialized in Array.prototype.pop().

When building by GCC with -O3 and -flto flags the following
warning was reported:
src/njs_array.c: In function ‘njs_array_prototype_pop’:
src/njs_array.c:1009:8: error: ‘length’ may be used uninitialized in
this function [-Werror=maybe-uninitialized]
1009 | if (length == 0) {
     |     ^

Returning a specific code in njs_value_to_number() helps GCC
to infer that there are only 2 return values are possible and
both of them are handled.

diffstat:

 src/njs_value_conversion.h |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff -r 8b182df14819 -r 146d4699a4c4 src/njs_value_conversion.h
--- a/src/njs_value_conversion.h	Wed Jun 12 23:31:08 2024 -0700
+++ b/src/njs_value_conversion.h	Thu Jun 20 17:26:14 2024 -0700
@@ -17,7 +17,7 @@ njs_value_to_number(njs_vm_t *vm, njs_va
     if (njs_slow_path(!njs_is_primitive(value))) {
         ret = njs_value_to_primitive(vm, &primitive, value, 0);
         if (njs_slow_path(ret != NJS_OK)) {
-            return ret;
+            return NJS_ERROR;
         }
 
         value = &primitive;

From noreply at nginx.com  Sat Jun 29 02:53:05 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Sat, 29 Jun 2024 02:53:05 +0000
Subject: =?utf-8?q?=5Bnjs=5D_Fixed_=E2=80=98ctx=2Ecodepoint=E2=80=99_may_be_used_unin?=
 =?utf-8?q?itialized=2E?=
Message-ID: <hg.f358df45922c.1719629585.-2089952106619681529@dev.nginx>

details:   https://hg.nginx.org/njs/rev/f358df45922c
branches:  
changeset: 2363:f358df45922c
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Jun 21 17:58:32 2024 -0700
description:
Fixed ‘ctx.codepoint’ may be used uninitialized.

When building by GCC 13 with -O3 and -flto flags the following
warning was reported:
In function ‘njs_utf8_decode’,
inlined from ‘njs_text_encoder_encode_into’ at
src/njs_encoding.c:214:14:
src/njs_utf8.c:191:42: error: ‘ctx.codepoint’ may be used
uninitialized [-Werror=maybe-uninitialized]
 191 |         ctx->codepoint = (ctx->codepoint << 6) | (c & 0x3F);

diffstat:

 src/njs_utf8.h |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (11 lines):

diff -r 146d4699a4c4 -r f358df45922c src/njs_utf8.h
--- a/src/njs_utf8.h	Thu Jun 20 17:26:14 2024 -0700
+++ b/src/njs_utf8.h	Fri Jun 21 17:58:32 2024 -0700
@@ -128,6 +128,7 @@ njs_utf8_decode_init(njs_unicode_decode_
 {
     ctx->need = 0x00;
     ctx->lower = 0x00;
+    ctx->codepoint = 0;
 }
 
 

From noreply at nginx.com  Sat Jun 29 02:53:07 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Sat, 29 Jun 2024 02:53:07 +0000
Subject: [njs] Modules: adding NUL byte at the end of the module body.
Message-ID: <hg.818ab94037d7.1719629587.-2089952106619681529@dev.nginx>

details:   https://hg.nginx.org/njs/rev/818ab94037d7
branches:  
changeset: 2364:818ab94037d7
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Jun 18 23:47:31 2024 -0700
description:
Modules: adding NUL byte at the end of the module body.

Even though QuickJS takes length as an argument, when parsing the code,
it expects NUL byte at the end.

diffstat:

 nginx/ngx_js.c |  4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diffs (21 lines):

diff -r f358df45922c -r 818ab94037d7 nginx/ngx_js.c
--- a/nginx/ngx_js.c	Fri Jun 21 17:58:32 2024 -0700
+++ b/nginx/ngx_js.c	Tue Jun 18 23:47:31 2024 -0700
@@ -1841,7 +1841,7 @@ ngx_js_module_read(njs_mp_t *mp, int fd,
 
     text->length = sb.st_size;
 
-    text->start = njs_mp_alloc(mp, text->length);
+    text->start = njs_mp_alloc(mp, text->length + 1);
     if (text->start == NULL) {
         goto fail;
     }
@@ -1852,6 +1852,8 @@ ngx_js_module_read(njs_mp_t *mp, int fd,
         goto fail;
     }
 
+    text->start[text->length] = '\0';
+
     return NJS_OK;
 
 fail:

From noreply at nginx.com  Sat Jun 29 04:57:03 2024
From: noreply at nginx.com (noreply at nginx.com)
Date: Sat, 29 Jun 2024 04:57:03 +0000
Subject: [njs] Fixed maybe-uninitialized warning in error creation.
Message-ID: <hg.9f59e7e13899.1719637023.2364989759688239044@dev.nginx>

details:   https://hg.nginx.org/njs/rev/9f59e7e13899
branches:  
changeset: 2365:9f59e7e13899
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Thu Jun 20 17:11:24 2024 -0700
description:
Fixed maybe-uninitialized warning in error creation.

Ensuring that buf is always initialized in njs_throw_error_va()
and njs_error_fmt_new(), by requiring fmt to always be non NULL.

This fixes GCC warnings like:
169 | njs_unicode_decode_t ctx;
| ^
In function ‘njs_utf8_length’,
   inlined from ‘njs_error_new’ at src/njs_error.c:39:14,
   inlined from ‘njs_throw_error_va’ at src/njs_error.c:69:5:
   src/njs_utf8.h:141:12: error: ‘buf’ may be used uninitialized
   [-Werror=maybe-uninitialized]
   141 | return njs_utf8_stream_length(&ctx, p, len, 1, 1, NULL);

diffstat:

 src/njs_error.c          |  16 ++++------------
 src/njs_number.c         |   2 +-
 src/njs_string.c         |   8 ++++----
 src/test/njs_unit_test.c |  40 ++++++++++++++++++++--------------------
 4 files changed, 29 insertions(+), 37 deletions(-)

diffs (215 lines):

diff -r 818ab94037d7 -r 9f59e7e13899 src/njs_error.c
--- a/src/njs_error.c	Tue Jun 18 23:47:31 2024 -0700
+++ b/src/njs_error.c	Thu Jun 20 17:11:24 2024 -0700
@@ -60,11 +60,7 @@ njs_throw_error_va(njs_vm_t *vm, njs_obj
 {
     u_char   buf[NJS_MAX_ERROR_STR], *p;
 
-    p = buf;
-
-    if (fmt != NULL) {
-        p = njs_vsprintf(buf, buf + sizeof(buf), fmt, args);
-    }
+    p = njs_vsprintf(buf, buf + sizeof(buf), fmt, args);
 
     njs_error_new(vm, &vm->exception, proto, buf, p - buf);
 }
@@ -88,13 +84,9 @@ njs_error_fmt_new(njs_vm_t *vm, njs_valu
     va_list  args;
     u_char   buf[NJS_MAX_ERROR_STR], *p;
 
-    p = buf;
-
-    if (fmt != NULL) {
-        va_start(args, fmt);
-        p = njs_vsprintf(buf, buf + sizeof(buf), fmt, args);
-        va_end(args);
-    }
+    va_start(args, fmt);
+    p = njs_vsprintf(buf, buf + sizeof(buf), fmt, args);
+    va_end(args);
 
     njs_error_new(vm, dst, njs_vm_proto(vm, type), buf, p - buf);
 }
diff -r 818ab94037d7 -r 9f59e7e13899 src/njs_number.c
--- a/src/njs_number.c	Tue Jun 18 23:47:31 2024 -0700
+++ b/src/njs_number.c	Thu Jun 20 17:11:24 2024 -0700
@@ -543,7 +543,7 @@ njs_number_prototype_to_string(njs_vm_t 
         }
 
         if (radix < 2 || radix > 36 || radix != (int) radix) {
-            njs_range_error(vm, NULL);
+            njs_range_error(vm, "radix argument must be between 2 and 36");
             return NJS_ERROR;
         }
 
diff -r 818ab94037d7 -r 9f59e7e13899 src/njs_string.c
--- a/src/njs_string.c	Tue Jun 18 23:47:31 2024 -0700
+++ b/src/njs_string.c	Thu Jun 20 17:11:24 2024 -0700
@@ -1695,7 +1695,7 @@ njs_string_from_char_code(njs_vm_t *vm, 
 
 range_error:
 
-    njs_range_error(vm, NULL);
+    njs_range_error(vm, "invalid code point");
 
     return NJS_ERROR;
 }
@@ -2503,7 +2503,7 @@ njs_string_prototype_repeat(njs_vm_t *vm
     }
 
     if (njs_slow_path(n < 0 || n == INT64_MAX)) {
-        njs_range_error(vm, NULL);
+        njs_range_error(vm, "invalid count value");
         return NJS_ERROR;
     }
 
@@ -2517,7 +2517,7 @@ njs_string_prototype_repeat(njs_vm_t *vm
     max = NJS_STRING_MAX_LENGTH / string.size;
 
     if (njs_slow_path(n >= max)) {
-        njs_range_error(vm, NULL);
+        njs_range_error(vm, "invalid string length");
         return NJS_ERROR;
     }
 
@@ -2583,7 +2583,7 @@ njs_string_prototype_pad(njs_vm_t *vm, n
     }
 
     if (njs_slow_path(new_length >= NJS_STRING_MAX_LENGTH)) {
-        njs_range_error(vm, NULL);
+        njs_range_error(vm, "invalid string length");
         return NJS_ERROR;
     }
 
diff -r 818ab94037d7 -r 9f59e7e13899 src/test/njs_unit_test.c
--- a/src/test/njs_unit_test.c	Tue Jun 18 23:47:31 2024 -0700
+++ b/src/test/njs_unit_test.c	Thu Jun 20 17:11:24 2024 -0700
@@ -630,13 +630,13 @@ static njs_unit_test_t  njs_test[] =
       njs_str("Infinity") },
 
     { njs_str("Infinity.toString(NaN)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: radix argument must be between 2 and 36") },
 
     { njs_str("Infinity.toString({})"),
-      njs_str("RangeError") },
+      njs_str("RangeError: radix argument must be between 2 and 36") },
 
     { njs_str("Infinity.toString(Infinity)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: radix argument must be between 2 and 36") },
 
     { njs_str("NaN.toString()"),
       njs_str("NaN") },
@@ -648,13 +648,13 @@ static njs_unit_test_t  njs_test[] =
       njs_str("NaN") },
 
     { njs_str("NaN.toString(Infinity)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: radix argument must be between 2 and 36") },
 
     { njs_str("NaN.toString({})"),
-      njs_str("RangeError") },
+      njs_str("RangeError: radix argument must be between 2 and 36") },
 
     { njs_str("NaN.toString(NaN)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: radix argument must be between 2 and 36") },
 
     { njs_str("1.2312313132.toString(14)"),
       njs_str("1.3346da6d5d455c") },
@@ -8383,13 +8383,13 @@ static njs_unit_test_t  njs_test[] =
       njs_str("0") },
 
     { njs_str("String.fromCodePoint('_')"),
-      njs_str("RangeError") },
+      njs_str("RangeError: invalid code point") },
 
     { njs_str("String.fromCharCode(65.14)"),
       njs_str("A") },
 
     { njs_str("String.fromCodePoint(3.14)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: invalid code point") },
 
     { njs_str("String.fromCharCode(65.14 + 65536)"),
       njs_str("A") },
@@ -8428,7 +8428,7 @@ static njs_unit_test_t  njs_test[] =
       njs_str("\n") },
 
     { njs_str("String.fromCodePoint(1114111 + 1)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: invalid code point") },
 
     { njs_str("String.fromCharCode(65, 90) + String.fromCodePoint(65, 90)"),
       njs_str("AZAZ") },
@@ -9809,22 +9809,22 @@ static njs_unit_test_t  njs_test[] =
       njs_str("") },
 
     { njs_str("'abc'.repeat(Infinity)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: invalid count value") },
 
     { njs_str("'abc'.repeat(-1)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: invalid count value") },
 
     { njs_str("''.repeat(-1)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: invalid count value") },
 
     { njs_str("'a'.repeat(2147483647)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: invalid string length") },
 
     { njs_str("'a'.repeat(2147483648)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: invalid string length") },
 
     { njs_str("'a'.repeat(Infinity)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: invalid count value") },
 
     { njs_str("'a'.repeat(NaN)"),
       njs_str("") },
@@ -9839,10 +9839,10 @@ static njs_unit_test_t  njs_test[] =
       njs_str("") },
 
     { njs_str("'aaaaaaaa'.repeat(2**64+1)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: invalid count value") },
 
     { njs_str("''.repeat(Infinity)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: invalid count value") },
 
     { njs_str("''.repeat(NaN)"),
       njs_str("") },
@@ -9866,7 +9866,7 @@ static njs_unit_test_t  njs_test[] =
       njs_str("abc") },
 
     { njs_str("'abc'.padStart(2147483647)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: invalid string length") },
 
     { njs_str("'abc'.padStart(2147483646, '')"),
       njs_str("abc") },
@@ -9920,7 +9920,7 @@ static njs_unit_test_t  njs_test[] =
       njs_str("я     ") },
 
     { njs_str("'я'.padEnd(2147483647)"),
-      njs_str("RangeError") },
+      njs_str("RangeError: invalid string length") },
 
     { njs_str("'я'.padEnd(2147483646, '')"),
       njs_str("я") },
@@ -23050,7 +23050,7 @@ static njs_unit_test_t  njs_backtraces_t
               "    at main (:1)\n") },
 
     { njs_str("''.repeat(-1)"),
-      njs_str("RangeError\n"
+      njs_str("RangeError: invalid count value\n"
               "    at String.prototype.repeat (native)\n"
               "    at main (:1)\n") },