From rekgrpth at gmail.com  Sun May  1 02:46:05 2022
From: rekgrpth at gmail.com (RekGRpth)
Date: Sun, 1 May 2022 07:46:05 +0500
Subject: [PATCH] upstream keepalive: save and restore connection data
In-Reply-To: <Ym211QrjRxiYEvHK@mdounin.ru>
References: <7b38618adf6317fc2683.1651295022@gws.t72.ru> <Ym211QrjRxiYEvHK@mdounin.ru>
Message-ID: <CAPgh2mLN7ucbnrY+T_C5i886ncxnPVcQhOQEU7mmf9i1bJWTRA@mail.gmail.com>

now, for example, I do very dirty hack for this:
https://github.com/RekGRpth/ngx_pg_module/blob/06eab53160fcff2b19e73ed5bc787f534c2a4c2e/ngx_pg_module.c#L1382-L1385
for save and https://github.com/RekGRpth/ngx_pg_module/blob/06eab53160fcff2b19e73ed5bc787f534c2a4c2e/ngx_pg_module.c#L1362
for restore

вс, 1 мая 2022 г. в 03:19, Maxim Dounin <mdounin at mdounin.ru>:
>
> Hello!
>
> On Sat, Apr 30, 2022 at 10:03:42AM +0500, RekGRpth wrote:
>
> > # HG changeset patch
> > # User RekGRpth <rekgrpth at gmail.com>
> > # Date 1651294527 -18000
> > #      Sat Apr 30 09:55:27 2022 +0500
> > # Node ID 7b38618adf6317fc268315b8134a5e2b1bc96269
> > # Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
> > upstream keepalive: save and restore connection data
> >
> > save connection data before keep it
> > and
> > restore connection data after yuild it
> >
> > diff -r a736a7a613ea -r 7b38618adf63 src/http/modules/ngx_http_upstream_keepalive_module.c
> > --- a/src/http/modules/ngx_http_upstream_keepalive_module.c   Tue Feb 08 17:35:27 2022 +0300
> > +++ b/src/http/modules/ngx_http_upstream_keepalive_module.c   Sat Apr 30 09:55:27 2022 +0500
> > @@ -34,6 +34,8 @@
> >      socklen_t                          socklen;
> >      ngx_sockaddr_t                     sockaddr;
> >
> > +    void                              *data;
> > +
> >  } ngx_http_upstream_keepalive_cache_t;
> >
> >
> > @@ -284,7 +286,7 @@
> >
> >      c->idle = 0;
> >      c->sent = 0;
> > -    c->data = NULL;
> > +    c->data = item->data;
> >      c->log = pc->log;
> >      c->read->log = pc->log;
> >      c->write->log = pc->log;
> > @@ -390,6 +392,7 @@
> >      c->write->handler = ngx_http_upstream_keepalive_dummy_handler;
> >      c->read->handler = ngx_http_upstream_keepalive_close_handler;
> >
> > +    item->data = c->data;
> >      c->data = item;
> >      c->idle = 1;
> >      c->log = ngx_cycle->log;
>
> Thanks for the patch.
>
> You may want to be more specific on what problem you are trying to
> solve by this patch.
>
> --
> Maxim Dounin
> http://mdounin.ru/


From A.Bavshin at F5.com  Sun May  1 06:45:32 2022
From: A.Bavshin at F5.com (Aleksei Bavshin)
Date: Sun, 1 May 2022 06:45:32 +0000
Subject: [PATCH] Resolver: relax validation of response flags to allow AD and
 CD
Message-ID:  <CO6PR01MB7482056549AC2618FDDE729DC0FE9@CO6PR01MB7482.prod.exchangelabs.com>

A correct nameserver implementation should not be sending any of these flags to a non-security aware client. CD must be copied from a query to the corresponding response, and AD must only be set if all the RRsets are authentic and DO or AD were present in the query. The behavior was allowed in the early DNSSEC standards, though, and the prohibition in the RFC is not strict. The change is, in fact, motivated by a report about some DNS server passing CD in a response to the nginx resolver.

There's a valid concern that CD may mean that the upstream nameserver did not check the response signatures and passed the answer as is. There's nothing we can do with such nameserver from the nginx side, though - we can't prevent it from sending unexpected or unchecked responses. And there's already an error message if an RRSIG record makes it to the answer we receive:
    2022/04/28 17:34:10 [error] 27847#27847: unexpected RR type 46 in DNS response

# HG changeset patch
# User Aleksei Bavshin <a.bavshin at f5.com>
# Date 1651176054 25200
#      Thu Apr 28 13:00:54 2022 -0700
# Node ID 5a570e610b375d1f3442a5b0fc1844be3909d103
# Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
Resolver: relax validation of response flags to allow AD and CD

The check introduced in aebdca7e8f8f was using reserved must-be-zero bits
definition from RFC1035. Later RFCs for DNS Security Extensions (4035 & 6840)
allocated AD and CD from these reserved bits.

It's said that AD and CD SHOULD only appear in the reply for a query from
a security aware resolver, but that is not a strict prohibition, and it was
not included in the early set of DNSSEC RFCs. We may encounter these bits in
a reply when the upstream nameserver implements pre-RFC4035 (with RFC6840
clarifications) standard or uses overly aggressive caching.

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -1746,7 +1746,7 @@ ngx_resolver_process_response(ngx_resolv
                    (response->nar_hi << 8) + response->nar_lo);
 
     /* response to a standard query */
-    if ((flags & 0xf870) != 0x8000 || (trunc && tcp)) {
+    if ((flags & 0xf840) != 0x8000 || (trunc && tcp)) {
         ngx_log_error(r->log_level, r->log, 0,
                       "invalid %s DNS response %ui fl:%04Xi",
                       tcp ? "TCP" : "UDP", ident, flags);


From mdounin at mdounin.ru  Sun May  1 15:37:40 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sun, 1 May 2022 18:37:40 +0300
Subject: [PATCH] Resolver: relax validation of response flags to allow AD
 and CD
In-Reply-To: <CO6PR01MB7482056549AC2618FDDE729DC0FE9@CO6PR01MB7482.prod.exchangelabs.com>
References: <CO6PR01MB7482056549AC2618FDDE729DC0FE9@CO6PR01MB7482.prod.exchangelabs.com>
Message-ID: <Ym6pRADlyh0tVsZD@mdounin.ru>

Hello!

On Sun, May 01, 2022 at 06:45:32AM +0000, Aleksei Bavshin via nginx-devel wrote:

> A correct nameserver implementation should not be sending any of these flags to a non-security aware client. CD must be copied from a query to the corresponding response, and AD must only be set if all the RRsets are authentic and DO or AD were present in the query. The behavior was allowed in the early DNSSEC standards, though, and the prohibition in the RFC is not strict. The change is, in fact, motivated by a report about some DNS server passing CD in a response to the nginx resolver.
> 
> There's a valid concern that CD may mean that the upstream nameserver did not check the response signatures and passed the answer as is. There's nothing we can do with such nameserver from the nginx side, though - we can't prevent it from sending unexpected or unchecked responses. And there's already an error message if an RRSIG record makes it to the answer we receive:
>     2022/04/28 17:34:10 [error] 27847#27847: unexpected RR type 46 in DNS response
> 
> # HG changeset patch
> # User Aleksei Bavshin <a.bavshin at f5.com>
> # Date 1651176054 25200
> #      Thu Apr 28 13:00:54 2022 -0700
> # Node ID 5a570e610b375d1f3442a5b0fc1844be3909d103
> # Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
> Resolver: relax validation of response flags to allow AD and CD
> 
> The check introduced in aebdca7e8f8f was using reserved must-be-zero bits
> definition from RFC1035. Later RFCs for DNS Security Extensions (4035 & 6840)
> allocated AD and CD from these reserved bits.
> 
> It's said that AD and CD SHOULD only appear in the reply for a query from
> a security aware resolver, but that is not a strict prohibition, and it was
> not included in the early set of DNSSEC RFCs. We may encounter these bits in
> a reply when the upstream nameserver implements pre-RFC4035 (with RFC6840
> clarifications) standard or uses overly aggressive caching.

Thank you for the patch.

Quoting RFC 4035, section 3:

   ... The CD bit
   is controlled by resolvers; a security-aware name server MUST copy
   the CD bit from a query into the corresponding response.

That is, response with CD bit set strictly prohibited by RFC 4035 
unless it was set by the resolver, i.e., nginx.

Similarly, presence of the AD bit, while not strictly prohibited 
by RFC 6840 (which only says "SHOULD only set the AD bit" when 
talking to a security-aware resolver), is a clear violation of RFC 
1035.  If it appears in the response, this indicates that the DNS 
server is not expected to be used with non-security-aware 
resolvers, such as the one in nginx.

Summing the above, unless there are practical reasons for the 
change, a better approach might be to fix affected DNS servers, if 
any, or switch to using more interoperable DNS servers with nginx.

Hope this helps.

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Sun May  1 15:53:27 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sun, 1 May 2022 18:53:27 +0300
Subject: [PATCH] upstream keepalive: save and restore connection data
In-Reply-To: <CAPgh2mLN7ucbnrY+T_C5i886ncxnPVcQhOQEU7mmf9i1bJWTRA@mail.gmail.com>
References: <7b38618adf6317fc2683.1651295022@gws.t72.ru>
 <Ym211QrjRxiYEvHK@mdounin.ru>
 <CAPgh2mLN7ucbnrY+T_C5i886ncxnPVcQhOQEU7mmf9i1bJWTRA@mail.gmail.com>
Message-ID: <Ym6s9+o4uIUaloAx@mdounin.ru>

Hello!

On Sun, May 01, 2022 at 07:46:05AM +0500, RekGRpth wrote:

> now, for example, I do very dirty hack for this:
> https://github.com/RekGRpth/ngx_pg_module/blob/06eab53160fcff2b19e73ed5bc787f534c2a4c2e/ngx_pg_module.c#L1382-L1385
> for save and https://github.com/RekGRpth/ngx_pg_module/blob/06eab53160fcff2b19e73ed5bc787f534c2a4c2e/ngx_pg_module.c#L1362
> for restore

As far as I understand, the module in question misuses parts of 
the upstream infrastructure, notably upstream blocks and 
balancers, for its own needs.  It looks somewhat strange to me to 
request changes in the infrastructure which corresponds to the 
misuse in question and not the use of the infrastructure in the 
upstream module itself.

Either way, a better and well supported approach to save any 
connection-specific data within a cached connection would be to do 
so via a connection's pool cleanup, see gRPC proxy module for an 
example:

http://hg.nginx.org/nginx/file/a736a7a613ea/src/http/modules/ngx_http_grpc_module.c#l4196

Hope this helps.

-- 
Maxim Dounin
http://mdounin.ru/


From rekgrpth at gmail.com  Mon May  2 03:41:40 2022
From: rekgrpth at gmail.com (RekGRpth)
Date: Mon, 2 May 2022 08:41:40 +0500
Subject: [PATCH] upstream keepalive: save and restore connection data
In-Reply-To: <CAPgh2mLN7ucbnrY+T_C5i886ncxnPVcQhOQEU7mmf9i1bJWTRA@mail.gmail.com>
References: <7b38618adf6317fc2683.1651295022@gws.t72.ru> <Ym211QrjRxiYEvHK@mdounin.ru>
 <CAPgh2mLN7ucbnrY+T_C5i886ncxnPVcQhOQEU7mmf9i1bJWTRA@mail.gmail.com>
Message-ID: <CAPgh2mK-u6QRQGihJ+9wr=9XYFn0H2b+cK1UoEr5AXi75awbEQ@mail.gmail.com>

I think, my suggested in patch solution is more clear and easy and
beauty, than using cleanup handler to save any connection-specific
data within a cached connection.

вс, 1 мая 2022 г. в 07:46, RekGRpth <rekgrpth at gmail.com>:
>
> now, for example, I do very dirty hack for this:
> https://github.com/RekGRpth/ngx_pg_module/blob/06eab53160fcff2b19e73ed5bc787f534c2a4c2e/ngx_pg_module.c#L1382-L1385
> for save and https://github.com/RekGRpth/ngx_pg_module/blob/06eab53160fcff2b19e73ed5bc787f534c2a4c2e/ngx_pg_module.c#L1362
> for restore
>
> вс, 1 мая 2022 г. в 03:19, Maxim Dounin <mdounin at mdounin.ru>:
> >
> > Hello!
> >
> > On Sat, Apr 30, 2022 at 10:03:42AM +0500, RekGRpth wrote:
> >
> > > # HG changeset patch
> > > # User RekGRpth <rekgrpth at gmail.com>
> > > # Date 1651294527 -18000
> > > #      Sat Apr 30 09:55:27 2022 +0500
> > > # Node ID 7b38618adf6317fc268315b8134a5e2b1bc96269
> > > # Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
> > > upstream keepalive: save and restore connection data
> > >
> > > save connection data before keep it
> > > and
> > > restore connection data after yuild it
> > >
> > > diff -r a736a7a613ea -r 7b38618adf63 src/http/modules/ngx_http_upstream_keepalive_module.c
> > > --- a/src/http/modules/ngx_http_upstream_keepalive_module.c   Tue Feb 08 17:35:27 2022 +0300
> > > +++ b/src/http/modules/ngx_http_upstream_keepalive_module.c   Sat Apr 30 09:55:27 2022 +0500
> > > @@ -34,6 +34,8 @@
> > >      socklen_t                          socklen;
> > >      ngx_sockaddr_t                     sockaddr;
> > >
> > > +    void                              *data;
> > > +
> > >  } ngx_http_upstream_keepalive_cache_t;
> > >
> > >
> > > @@ -284,7 +286,7 @@
> > >
> > >      c->idle = 0;
> > >      c->sent = 0;
> > > -    c->data = NULL;
> > > +    c->data = item->data;
> > >      c->log = pc->log;
> > >      c->read->log = pc->log;
> > >      c->write->log = pc->log;
> > > @@ -390,6 +392,7 @@
> > >      c->write->handler = ngx_http_upstream_keepalive_dummy_handler;
> > >      c->read->handler = ngx_http_upstream_keepalive_close_handler;
> > >
> > > +    item->data = c->data;
> > >      c->data = item;
> > >      c->idle = 1;
> > >      c->log = ngx_cycle->log;
> >
> > Thanks for the patch.
> >
> > You may want to be more specific on what problem you are trying to
> > solve by this patch.
> >
> > --
> > Maxim Dounin
> > http://mdounin.ru/


From pluknet at nginx.com  Mon May  2 12:42:46 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 02 May 2022 12:42:46 +0000
Subject: [nginx] Configure: recognize arm64 machine name as a synonym for
 aarch64.
Message-ID: <hg.35afae4b3dff.1651495366.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/35afae4b3dff
branches:  
changeset: 8010:35afae4b3dff
user:      Sergey Kandaurov <pluknet at nginx.com>
date:      Fri Apr 29 17:38:01 2022 +0400
description:
Configure: recognize arm64 machine name as a synonym for aarch64.

In particular, this sets a reasonable cacheline size on FreeBSD and macOS,
which prefer to use this name and both lack _SC_LEVEL1_DCACHE_LINESIZE.

diffstat:

 auto/os/conf |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff -r a736a7a613ea -r 35afae4b3dff auto/os/conf
--- a/auto/os/conf	Tue Feb 08 17:35:27 2022 +0300
+++ b/auto/os/conf	Fri Apr 29 17:38:01 2022 +0400
@@ -110,7 +110,7 @@ case "$NGX_MACHINE" in
         NGX_MACH_CACHE_LINE=64
     ;;
 
-    aarch64 )
+    aarch64 | arm64)
         have=NGX_ALIGNMENT value=16 . auto/define
         NGX_MACH_CACHE_LINE=64
     ;;


From kiprasm at gmail.com  Mon May  2 13:36:09 2022
From: kiprasm at gmail.com (=?iso-8859-1?q?Kipras_Mancevicius?=)
Date: Mon, 02 May 2022 16:36:09 +0300
Subject: [PATCH] Do not break downstream connection on proxy_cache write error
Message-ID: <e6e708fd7958987027f6.1651498569@kipras-Lenovo>

# HG changeset patch
# User Kipras Mancevicius <kiprasm at gmail.com>
# Date 1651495240 -10800
#      Mon May 02 15:40:40 2022 +0300
# Branch proxy_cache_write_disk_full_fix
# Node ID e6e708fd7958987027f6de7748d948e5015ed32b
# Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
Do not break downstream connection on proxy_cache write error

Continue sending response to downstream connection and delete temp cache file on
upstream connection finalization.

diff -r a736a7a613ea -r e6e708fd7958 src/core/ngx_file.c
--- a/src/core/ngx_file.c	Tue Feb 08 17:35:27 2022 +0300
+++ b/src/core/ngx_file.c	Mon May 02 15:40:40 2022 +0300
@@ -765,6 +765,13 @@
                               ngx_delete_file_n " \"%s\" failed", name);
 
             }
+        } else {
+            /*
+             * if the file failed to fully copy (e.g. target disk is full) - it
+             * may remain (partially written) in the target disk. Attempt to
+             * delete it.
+             */
+            ngx_delete_file(name);
         }
 
         ngx_free(name);
diff -r a736a7a613ea -r e6e708fd7958 src/event/ngx_event_pipe.c
--- a/src/event/ngx_event_pipe.c	Tue Feb 08 17:35:27 2022 +0300
+++ b/src/event/ngx_event_pipe.c	Mon May 02 15:40:40 2022 +0300
@@ -753,10 +753,21 @@
         out = p->writing;
         p->writing = NULL;
 
-        n = ngx_write_chain_to_temp_file(p->temp_file, NULL);
+        /*
+         * stop writing to cache on cache write fail, but keep sending response
+         * downstream. Incomplete temp cache file will be deleted later.
+         */
+        if (p->cache_write_fail) {
+            return NGX_BUSY;
+        } else {
+            n = ngx_write_chain_to_temp_file(p->temp_file, NULL);
 
-        if (n == NGX_ERROR) {
-            return NGX_ABORT;
+            if (n == NGX_ERROR) {
+                p->cache_write_fail = 1;
+                p->cacheable = 0;
+
+                return NGX_BUSY;
+            }
         }
 
         goto done;
@@ -777,6 +788,23 @@
         out = p->in;
     }
 
+    /*
+     * stop writing to cache on cache write fail, but keep sending response
+     * downstream. Incomplete temp cache file will be deleted later.
+     */
+    if (p->cache_write_fail) {
+        return NGX_BUSY;
+    } else {
+        n = ngx_write_chain_to_temp_file(p->temp_file, out);
+
+        if (n == NGX_ERROR) {
+            p->cache_write_fail = 1;
+            p->cacheable = 0;
+
+            return NGX_BUSY;
+        }
+    }
+
     if (!p->cacheable) {
 
         size = 0;
diff -r a736a7a613ea -r e6e708fd7958 src/event/ngx_event_pipe.h
--- a/src/event/ngx_event_pipe.h	Tue Feb 08 17:35:27 2022 +0300
+++ b/src/event/ngx_event_pipe.h	Mon May 02 15:40:40 2022 +0300
@@ -66,6 +66,7 @@
     unsigned           downstream_error:1;
     unsigned           cyclic_temp_file:1;
     unsigned           aio:1;
+    unsigned           cache_write_fail:1;
 
     ngx_int_t          allocated;
     ngx_bufs_t         bufs;
diff -r a736a7a613ea -r e6e708fd7958 src/http/ngx_http_file_cache.c
--- a/src/http/ngx_http_file_cache.c	Tue Feb 08 17:35:27 2022 +0300
+++ b/src/http/ngx_http_file_cache.c	Mon May 02 15:40:40 2022 +0300
@@ -1377,6 +1377,16 @@
 
     cache = c->file_cache;
 
+    if (r->upstream && r->upstream->pipe && r->upstream->pipe->cache_write_fail) {
+        /*
+         * if cache file failed to be written (e.g. cache disk is full) - we
+         * kept sending response downstream and then delete the incomplete temp
+         * cache file here (after response is served downstream).
+         */
+        ngx_http_file_cache_free(cache, tf);
+        return;
+    }
+
     c->updated = 1;
     c->updating = 0;
 


From kiprasm at gmail.com  Mon May  2 13:51:20 2022
From: kiprasm at gmail.com (Kipras)
Date: Mon, 2 May 2022 16:51:20 +0300
Subject: [PATCH] Do not break downstream connection on proxy_cache write error
Message-ID: <CADNd3rsOqfJzmASM178d0_Pd+trkpmLPDBiacQvzPX3tRGP_fA@mail.gmail.com>

Hi all, this is the first time i'm submitting a patch for Nginx, i hope i'm
doing this correctly, apologies if not :)
(i want to provide some context behind the patch, so i'm following up the
patch with a message sent directly to the group, using the same topic, hope
it will end up in the right thread)

This is a fix for: https://trac.nginx.org/nginx/ticket/1945

In short: when HTTP proxy_pass is used with proxy_cache and while writing
the response to the cache the write fails (for example target disk is full)
- Nginx closes the downstream connection (the response breaks).

With this change, in this situation (when a cache write fails):
* the downstream connection is not closed (response continues to be sent
from upstream to the client)
* Nginx stops attempting to write this response to the cache
* Nginx deletes the temporary cache file after the response is fully served
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220502/9e7a2c47/attachment.htm>

From scott_mitchell at apple.com  Mon May  2 15:19:44 2022
From: scott_mitchell at apple.com (Scott Mitchell)
Date: Mon, 02 May 2022 08:19:44 -0700
Subject: Proposal: Change H2 RST_STREAM error code on timeout
 PROTOCOL_ERROR->CANCEL
References: <2EA529D6-56A4-43D3-B642-8AD0F88748AB@apple.com>
Message-ID: <96D9E871-3F66-44CB-A4E8-23C69A84A8D5@apple.com>

Rational is captured in the patch set below (tests also updated). Please let me know if this is acceptable.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: nginx_h2_cancel.patch
Type: application/octet-stream
Size: 1222 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220502/a267b43f/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nginx_h2_cancel_tests.patch
Type: application/octet-stream
Size: 1381 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220502/a267b43f/attachment-0001.obj>
-------------- next part --------------


Thanks,
Scott


From mdounin at mdounin.ru  Mon May  2 20:08:24 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 2 May 2022 23:08:24 +0300
Subject: [PATCH] upstream keepalive: save and restore connection data
In-Reply-To: <CAPgh2mK-u6QRQGihJ+9wr=9XYFn0H2b+cK1UoEr5AXi75awbEQ@mail.gmail.com>
References: <7b38618adf6317fc2683.1651295022@gws.t72.ru>
 <Ym211QrjRxiYEvHK@mdounin.ru>
 <CAPgh2mLN7ucbnrY+T_C5i886ncxnPVcQhOQEU7mmf9i1bJWTRA@mail.gmail.com>
 <CAPgh2mK-u6QRQGihJ+9wr=9XYFn0H2b+cK1UoEr5AXi75awbEQ@mail.gmail.com>
Message-ID: <YnA6OHQGDioMriOk@mdounin.ru>

Hello!

On Mon, May 02, 2022 at 08:41:40AM +0500, RekGRpth wrote:

> I think, my suggested in patch solution is more clear and easy and
> beauty, than using cleanup handler to save any connection-specific
> data within a cached connection.

To re-iterate:

- Your suggested patch does nothing for nginx itself where the 
  upstream module infrastructure is used by the upstream module, 
  and c->data points to the request, so saving it will just waste 
  resources.  Your patch only makes sense for your module (which 
  is basically a hack by design).

- There is a proper way to do what you are trying to do, not 
  limited to your module.  You are free to use it, much like other 
  modules do.

Hope this helps.

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Mon May  2 21:01:52 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 3 May 2022 00:01:52 +0300
Subject: [PATCH] Do not break downstream connection on proxy_cache write
 error
In-Reply-To: <e6e708fd7958987027f6.1651498569@kipras-Lenovo>
References: <e6e708fd7958987027f6.1651498569@kipras-Lenovo>
Message-ID: <YnBGwLuyGn0If3ST@mdounin.ru>

Hello!

On Mon, May 02, 2022 at 04:36:09PM +0300, Kipras Mancevicius wrote:

> # HG changeset patch
> # User Kipras Mancevicius <kiprasm at gmail.com>
> # Date 1651495240 -10800
> #      Mon May 02 15:40:40 2022 +0300
> # Branch proxy_cache_write_disk_full_fix
> # Node ID e6e708fd7958987027f6de7748d948e5015ed32b
> # Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
> Do not break downstream connection on proxy_cache write error
> 
> Continue sending response to downstream connection and delete temp cache file on
> upstream connection finalization.
> 
> diff -r a736a7a613ea -r e6e708fd7958 src/core/ngx_file.c
> --- a/src/core/ngx_file.c	Tue Feb 08 17:35:27 2022 +0300
> +++ b/src/core/ngx_file.c	Mon May 02 15:40:40 2022 +0300
> @@ -765,6 +765,13 @@
>                                ngx_delete_file_n " \"%s\" failed", name);
>  
>              }
> +        } else {
> +            /*
> +             * if the file failed to fully copy (e.g. target disk is full) - it
> +             * may remain (partially written) in the target disk. Attempt to
> +             * delete it.
> +             */
> +            ngx_delete_file(name);

Trying to delete a file "just in case" looks clearly wrong.  If at 
all, this is to be done in ngx_copy_file(), the function which 
created the file and knows what in fact happened.  Further, it is 
not clear if it's to be done at all: in many practical cases an 
incomplete file is better than no file at all, and this is a 
generic function.

Also, this looks irrelevant to the issue being addressed.

>          }
>  
>          ngx_free(name);
> diff -r a736a7a613ea -r e6e708fd7958 src/event/ngx_event_pipe.c
> --- a/src/event/ngx_event_pipe.c	Tue Feb 08 17:35:27 2022 +0300
> +++ b/src/event/ngx_event_pipe.c	Mon May 02 15:40:40 2022 +0300
> @@ -753,10 +753,21 @@
>          out = p->writing;
>          p->writing = NULL;
>  
> -        n = ngx_write_chain_to_temp_file(p->temp_file, NULL);
> +        /*
> +         * stop writing to cache on cache write fail, but keep sending response
> +         * downstream. Incomplete temp cache file will be deleted later.
> +         */
> +        if (p->cache_write_fail) {
> +            return NGX_BUSY;
> +        } else {
> +            n = ngx_write_chain_to_temp_file(p->temp_file, NULL);

This looks terribly wrong: if this code will happen actually 
return NGX_BUSY, a thread-based operation won't be completed, 
potentially resulting in a connection hang and/or a socket leak.

>  
> -        if (n == NGX_ERROR) {
> -            return NGX_ABORT;
> +            if (n == NGX_ERROR) {
> +                p->cache_write_fail = 1;
> +                p->cacheable = 0;
> +
> +                return NGX_BUSY;

This looks wrong: if an error happens as a result of a 
thread-based operation, it is incorrect to just ignore this.  Data 
being written needs to be recovered somehow.

> +            }
>          }
>  
>          goto done;
> @@ -777,6 +788,23 @@
>          out = p->in;
>      }
>  
> +    /*
> +     * stop writing to cache on cache write fail, but keep sending response
> +     * downstream. Incomplete temp cache file will be deleted later.
> +     */
> +    if (p->cache_write_fail) {
> +        return NGX_BUSY;
> +    } else {
> +        n = ngx_write_chain_to_temp_file(p->temp_file, out);
> +
> +        if (n == NGX_ERROR) {
> +            p->cache_write_fail = 1;
> +            p->cacheable = 0;
> +
> +            return NGX_BUSY;
> +        }
> +    }
> +

This also looks wrong:

- The code tries to write everything in out, ignoring things like 
  temp_file_write_size and max_temp_file_size.

- In case of success, the following code will write the data 
  again, resulting in file corruption and likely various other 
  issues.

Further, the following writes (which are the only writes 
meaningful for the current request, as you don't try to update 
p->out) are not really protected by any error checking, so the 
initial objective of the patch does not seem to be achieved.

Please also note that the ticket in question says "without 
introducing unreasonable complexity in the source code" for a 
reason.

Hope this helps.

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Mon May  2 23:14:49 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 3 May 2022 02:14:49 +0300
Subject: Proposal: Change H2 RST_STREAM error code on timeout
 PROTOCOL_ERROR->CANCEL
In-Reply-To: <96D9E871-3F66-44CB-A4E8-23C69A84A8D5@apple.com>
References: <2EA529D6-56A4-43D3-B642-8AD0F88748AB@apple.com>
 <96D9E871-3F66-44CB-A4E8-23C69A84A8D5@apple.com>
Message-ID: <YnBl6ZFqQ7Vz3ILy@mdounin.ru>

Hello!

On Mon, May 02, 2022 at 08:19:44AM -0700, Scott Mitchell via nginx-devel wrote:

> Rational is captured in the patch set below (tests also 
> updated). Please let me know if this is acceptable.
> 

> # HG changeset patch
> # User Scott Mitchell <scott_mitchell at apple.com>
> # Date 1651280176 25200
> #      Fri Apr 29 17:56:16 2022 -0700
> # Node ID ac2a48aa4efb86cd34189dd1b62fe24c5afbab70
> # Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
> H2: RST_STREAM(CANCEL) on timeout instead of PROTOCOL_ERROR
> 
> When a timeout occurs this condition maybe temporary and retryable. For example if the network
> bandwidth is temporarily throttled. If a PROTOCOL_ERROR is returned the client is unlikely to
> retry due to unknown error conditions. Returning CANCEL provides more context as an application
> level error that the client is more likely to retry.
> 
> diff -r a736a7a613ea -r ac2a48aa4efb src/http/v2/ngx_http_v2.c
> --- a/src/http/v2/ngx_http_v2.c	Tue Feb 08 17:35:27 2022 +0300
> +++ b/src/http/v2/ngx_http_v2.c	Fri Apr 29 17:56:16 2022 -0700
> @@ -4636,7 +4636,7 @@
>  
>          if (!stream->out_closed) {
>              if (ngx_http_v2_send_rst_stream(h2c, node->id,
> -                                      fc->timedout ? NGX_HTTP_V2_PROTOCOL_ERROR
> +                                      fc->timedout ? NGX_HTTP_V2_CANCEL
>                                                     : NGX_HTTP_V2_INTERNAL_ERROR)
>                  != NGX_OK)
>              {

Thanks for the patch.

The CANCEL error code is defined by RFC 7540 as follows:

   CANCEL (0x8):  Used by the endpoint to indicate that the stream is no
      longer needed.

This does not seem to apply to the timeout case. In contrast, 
PROTOCOL_ERROR mostly matches the particular error condition (that 
is, client failed to provide further frames as expected per 
protocol in a reasonable time), and it is also defined to be a 
"catch-all" error:

   PROTOCOL_ERROR (0x1):  The endpoint detected an unspecific protocol
      error.  This error is for use when a more specific error code is
      not available.

As such, PROTOCOL_ERROR looks more appropriate in the particular 
case from the protocol point of view.

If the particular error condition is temporary and retry is 
desired, it is probably something to do per user action, and the 
particular error code shouldn't matter as long as retry is 
requested by the user.

Hope this helps.

-- 
Maxim Dounin
http://mdounin.ru/


From scott_mitchell at apple.com  Tue May  3 01:27:48 2022
From: scott_mitchell at apple.com (Scott Mitchell)
Date: Mon, 02 May 2022 18:27:48 -0700
Subject: Proposal: Change H2 RST_STREAM error code on timeout
 PROTOCOL_ERROR->CANCEL
In-Reply-To: <YnBl6ZFqQ7Vz3ILy@mdounin.ru>
References: <2EA529D6-56A4-43D3-B642-8AD0F88748AB@apple.com>
 <96D9E871-3F66-44CB-A4E8-23C69A84A8D5@apple.com> <YnBl6ZFqQ7Vz3ILy@mdounin.ru>
Message-ID: <8C3552C8-7668-44AD-BC6D-E68A26524181@apple.com>

Thanks for the additional details :)

Agreed there is ambiguity in the RFC for error codes, and retryability will require additional context either way (idempotent, safe, ..). Some points to consider:
- PROTOCOL_ERROR is allowed as a “catch all”, but the client can’t differentiate from other “real protocol errors” (e.g. incorrect frame format, sequencing, ..) which generally aren’t retryable.
- CANCEL also doesn’t tell the client “why”, but it is known to be application level (as opposed to a protocol issue)

wdyt? Since there is no “right answer” understood if this is deemed undesirable or too risky.

-Scott

> On May 2, 2022, at 4:14 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> Hello!
> 
> On Mon, May 02, 2022 at 08:19:44AM -0700, Scott Mitchell via nginx-devel wrote:
> 
>> Rational is captured in the patch set below (tests also 
>> updated). Please let me know if this is acceptable.
>> 
> 
>> # HG changeset patch
>> # User Scott Mitchell <scott_mitchell at apple.com>
>> # Date 1651280176 25200
>> #      Fri Apr 29 17:56:16 2022 -0700
>> # Node ID ac2a48aa4efb86cd34189dd1b62fe24c5afbab70
>> # Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
>> H2: RST_STREAM(CANCEL) on timeout instead of PROTOCOL_ERROR
>> 
>> When a timeout occurs this condition maybe temporary and retryable. For example if the network
>> bandwidth is temporarily throttled. If a PROTOCOL_ERROR is returned the client is unlikely to
>> retry due to unknown error conditions. Returning CANCEL provides more context as an application
>> level error that the client is more likely to retry.
>> 
>> diff -r a736a7a613ea -r ac2a48aa4efb src/http/v2/ngx_http_v2.c
>> --- a/src/http/v2/ngx_http_v2.c	Tue Feb 08 17:35:27 2022 +0300
>> +++ b/src/http/v2/ngx_http_v2.c	Fri Apr 29 17:56:16 2022 -0700
>> @@ -4636,7 +4636,7 @@
>> 
>>         if (!stream->out_closed) {
>>             if (ngx_http_v2_send_rst_stream(h2c, node->id,
>> -                                      fc->timedout ? NGX_HTTP_V2_PROTOCOL_ERROR
>> +                                      fc->timedout ? NGX_HTTP_V2_CANCEL
>>                                                    : NGX_HTTP_V2_INTERNAL_ERROR)
>>                 != NGX_OK)
>>             {
> 
> Thanks for the patch.
> 
> The CANCEL error code is defined by RFC 7540 as follows:
> 
>   CANCEL (0x8):  Used by the endpoint to indicate that the stream is no
>      longer needed.
> 
> This does not seem to apply to the timeout case. In contrast, 
> PROTOCOL_ERROR mostly matches the particular error condition (that 
> is, client failed to provide further frames as expected per 
> protocol in a reasonable time), and it is also defined to be a 
> "catch-all" error:
> 
>   PROTOCOL_ERROR (0x1):  The endpoint detected an unspecific protocol
>      error.  This error is for use when a more specific error code is
>      not available.
> 
> As such, PROTOCOL_ERROR looks more appropriate in the particular 
> case from the protocol point of view.
> 
> If the particular error condition is temporary and retry is 
> desired, it is probably something to do per user action, and the 
> particular error code shouldn't matter as long as retry is 
> requested by the user.
> 
> Hope this helps.
> 
> -- 
> Maxim Dounin
> http://mdounin.ru/


From xeioex at nginx.com  Wed May  4 23:58:40 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 04 May 2022 23:58:40 +0000
Subject: [njs] Implement OrdinaryDelete() which does not throw exception.
Message-ID: <hg.723873b4e315.1651708720.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/723873b4e315
branches:  
changeset: 1848:723873b4e315
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed May 04 16:23:46 2022 -0700
description:
Implement OrdinaryDelete() which does not throw exception.

diffstat:

 src/njs_array.c  |   4 ++--
 src/njs_value.c  |  14 +++++++++-----
 src/njs_value.h  |   4 ++--
 src/njs_vmcode.c |   2 +-
 4 files changed, 14 insertions(+), 10 deletions(-)

diffs (85 lines):

diff -r 900d3ba62bcb -r 723873b4e315 src/njs_array.c
--- a/src/njs_array.c	Thu Apr 28 17:59:03 2022 -0700
+++ b/src/njs_array.c	Wed May 04 16:23:46 2022 -0700
@@ -243,7 +243,7 @@ njs_array_length_set(njs_vm_t *vm, njs_v
                 idx = njs_string_to_index(&keys->start[i]);
                 if (idx >= length) {
                     ret = njs_value_property_delete(vm, value, &keys->start[i],
-                                                    NULL);
+                                                    NULL, 1);
                     if (njs_slow_path(ret == NJS_ERROR)) {
                         goto done;
                     }
@@ -1064,7 +1064,7 @@ njs_array_prototype_unshift(njs_vm_t *vm
 
         while (from > 0) {
             ret = njs_value_property_delete(vm, this, &keys->start[--from],
-                                            &entry);
+                                            &entry, 1);
             if (njs_slow_path(ret == NJS_ERROR)) {
                 njs_array_destroy(vm, keys);
                 return ret;
diff -r 900d3ba62bcb -r 723873b4e315 src/njs_value.c
--- a/src/njs_value.c	Thu Apr 28 17:59:03 2022 -0700
+++ b/src/njs_value.c	Wed May 04 16:23:46 2022 -0700
@@ -1327,7 +1327,7 @@ fail:
 
 njs_int_t
 njs_value_property_delete(njs_vm_t *vm, njs_value_t *value, njs_value_t *key,
-    njs_value_t *removed)
+    njs_value_t *removed, njs_bool_t thrw)
 {
     njs_int_t             ret;
     njs_object_prop_t     *prop;
@@ -1343,10 +1343,14 @@ njs_value_property_delete(njs_vm_t *vm, 
     prop = pq.lhq.value;
 
     if (njs_slow_path(!prop->configurable)) {
-        njs_key_string_get(vm, &pq.key,  &pq.lhq.key);
-        njs_type_error(vm, "Cannot delete property \"%V\" of %s",
-                       &pq.lhq.key, njs_type_string(value->type));
-        return NJS_ERROR;
+        if (thrw) {
+            njs_key_string_get(vm, &pq.key,  &pq.lhq.key);
+            njs_type_error(vm, "Cannot delete property \"%V\" of %s",
+                           &pq.lhq.key, njs_type_string(value->type));
+            return NJS_ERROR;
+        }
+
+        return NJS_OK;
     }
 
     switch (prop->type) {
diff -r 900d3ba62bcb -r 723873b4e315 src/njs_value.h
--- a/src/njs_value.h	Thu Apr 28 17:59:03 2022 -0700
+++ b/src/njs_value.h	Wed May 04 16:23:46 2022 -0700
@@ -1062,7 +1062,7 @@ njs_int_t njs_value_property(njs_vm_t *v
 njs_int_t njs_value_property_set(njs_vm_t *vm, njs_value_t *value,
     njs_value_t *key, njs_value_t *setval);
 njs_int_t njs_value_property_delete(njs_vm_t *vm, njs_value_t *value,
-    njs_value_t *key, njs_value_t *removed);
+    njs_value_t *key, njs_value_t *removed, njs_bool_t thrw);
 njs_int_t njs_value_to_object(njs_vm_t *vm, njs_value_t *value);
 
 void njs_symbol_conversion_failed(njs_vm_t *vm, njs_bool_t to_string);
@@ -1110,7 +1110,7 @@ njs_value_property_i64_delete(njs_vm_t *
         return ret;
     }
 
-    return njs_value_property_delete(vm, value, &key, removed);
+    return njs_value_property_delete(vm, value, &key, removed, 1);
 }
 
 
diff -r 900d3ba62bcb -r 723873b4e315 src/njs_vmcode.c
--- a/src/njs_vmcode.c	Thu Apr 28 17:59:03 2022 -0700
+++ b/src/njs_vmcode.c	Wed May 04 16:23:46 2022 -0700
@@ -500,7 +500,7 @@ next:
                 break;
 
             case NJS_VMCODE_PROPERTY_DELETE:
-                ret = njs_value_property_delete(vm, value1, value2, NULL);
+                ret = njs_value_property_delete(vm, value1, value2, NULL, 1);
                 if (njs_fast_path(ret != NJS_ERROR)) {
                     vm->retval = njs_value_true;
 


From xeioex at nginx.com  Wed May  4 23:58:42 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 04 May 2022 23:58:42 +0000
Subject: [njs] Fixed JSON.parse() when reviver function is provided.
Message-ID: <hg.80ed74a0e205.1651708722.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/80ed74a0e205
branches:  
changeset: 1849:80ed74a0e205
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed May 04 16:44:48 2022 -0700
description:
Fixed JSON.parse() when reviver function is provided.

This closes #480 issue on Github.

diffstat:

 src/njs_json.c           |  289 +++++++++++++++-------------------------------
 src/test/njs_benchmark.c |   10 +
 src/test/njs_unit_test.c |   25 ++++
 3 files changed, 131 insertions(+), 193 deletions(-)

diffs (417 lines):

diff -r 723873b4e315 -r 80ed74a0e205 src/njs_json.c
--- a/src/njs_json.c	Wed May 04 16:23:46 2022 -0700
+++ b/src/njs_json.c	Wed May 04 16:44:48 2022 -0700
@@ -28,27 +28,17 @@ typedef struct {
     int64_t                    length;
     njs_array_t                *keys;
     njs_value_t                *key;
-    njs_object_prop_t          *prop;
+    njs_value_t                prop;
 } njs_json_state_t;
 
 
 typedef struct {
     njs_value_t                retval;
 
-    njs_uint_t                 depth;
-#define NJS_JSON_MAX_DEPTH     32
-    njs_json_state_t           states[NJS_JSON_MAX_DEPTH];
-
-    njs_function_t             *function;
-} njs_json_parse_t;
-
-
-typedef struct {
-    njs_value_t                retval;
-
     njs_vm_t                   *vm;
 
     njs_uint_t                 depth;
+#define NJS_JSON_MAX_DEPTH     32
     njs_json_state_t           states[NJS_JSON_MAX_DEPTH];
 
     njs_value_t                replacer;
@@ -72,10 +62,9 @@ njs_inline uint32_t njs_json_unicode(con
 static const u_char *njs_json_skip_space(const u_char *start,
     const u_char *end);
 
-static njs_int_t njs_json_parse_iterator(njs_vm_t *vm, njs_json_parse_t *parse,
-    njs_value_t *value);
-static njs_int_t njs_json_parse_iterator_call(njs_vm_t *vm,
-    njs_json_parse_t *parse, njs_json_state_t *state);
+static njs_int_t njs_json_internalize_property(njs_vm_t *vm,
+    njs_function_t *reviver, njs_value_t *holder, njs_value_t *name,
+    njs_int_t depth, njs_value_t *retval);
 static void njs_json_parse_exception(njs_json_parse_ctx_t *ctx,
     const char *msg, const u_char *pos);
 
@@ -108,15 +97,13 @@ njs_json_parse(njs_vm_t *vm, njs_value_t
     njs_index_t unused)
 {
     njs_int_t             ret;
-    njs_value_t           *text, value, lvalue;
+    njs_value_t           *text, value, lvalue, wrapper;
+    njs_object_t          *obj;
     const u_char          *p, *end;
-    njs_json_parse_t      *parse, json_parse;
     const njs_value_t     *reviver;
     njs_string_prop_t     string;
     njs_json_parse_ctx_t  ctx;
 
-    parse = &json_parse;
-
     text = njs_lvalue_arg(&lvalue, args, nargs, 1);
 
     if (njs_slow_path(!njs_is_string(text))) {
@@ -156,11 +143,16 @@ njs_json_parse(njs_vm_t *vm, njs_value_t
 
     reviver = njs_arg(args, nargs, 2);
 
-    if (njs_slow_path(njs_is_function(reviver) && njs_is_object(&value))) {
-        parse->function = njs_function(reviver);
-        parse->depth = 0;
-
-        return njs_json_parse_iterator(vm, parse, &value);
+    if (njs_slow_path(njs_is_function(reviver))) {
+        obj = njs_json_wrap_value(vm, &wrapper, &value);
+        if (njs_slow_path(obj == NULL)) {
+            return NJS_ERROR;
+        }
+
+        return njs_json_internalize_property(vm, njs_function(reviver),
+                                             &wrapper,
+                                             njs_value_arg(&njs_string_empty),
+                                             0, &vm->retval);
     }
 
     vm->retval = value;
@@ -851,195 +843,106 @@ njs_json_skip_space(const u_char *start,
 }
 
 
-static njs_json_state_t *
-njs_json_push_parse_state(njs_vm_t *vm, njs_json_parse_t *parse,
-    njs_value_t *value)
-{
-    njs_json_state_t  *state;
-
-    if (njs_slow_path(parse->depth >= NJS_JSON_MAX_DEPTH)) {
-        njs_type_error(vm, "Nested too deep or a cyclic structure");
-        return NULL;
-    }
-
-    state = &parse->states[parse->depth++];
-    state->value = *value;
-    state->index = 0;
-    state->prop = NULL;
-    state->keys = njs_value_own_enumerate(vm, value, NJS_ENUM_KEYS,
-                                          NJS_ENUM_STRING, 0);
-    if (state->keys == NULL) {
-        return NULL;
-    }
-
-    return state;
-}
-
-
-njs_inline njs_json_state_t *
-njs_json_pop_parse_state(njs_vm_t *vm, njs_json_parse_t *parse)
+static njs_int_t
+njs_json_internalize_property(njs_vm_t *vm, njs_function_t *reviver,
+    njs_value_t *holder, njs_value_t *name, njs_int_t depth,
+    njs_value_t *retval)
 {
-    njs_json_state_t  *state;
-
-    state = &parse->states[parse->depth - 1];
-    njs_array_destroy(vm, state->keys);
-    state->keys = NULL;
-
-    if (parse->depth > 1) {
-        parse->depth--;
-        return &parse->states[parse->depth - 1];
+    int64_t       k, length;
+    njs_int_t     ret;
+    njs_value_t   val, new_elem, index;
+    njs_value_t   arguments[3];
+    njs_array_t   *keys;
+
+    if (njs_slow_path(depth++ >= NJS_JSON_MAX_DEPTH)) {
+        njs_type_error(vm, "Nested too deep or a cyclic structure");
+        return NJS_ERROR;
     }
 
-    return NULL;
-}
-
-
-static njs_int_t
-njs_json_parse_iterator(njs_vm_t *vm, njs_json_parse_t *parse,
-    njs_value_t *object)
-{
-    njs_int_t             ret;
-    njs_value_t           *key, wrapper;
-    njs_object_t          *obj;
-    njs_json_state_t      *state;
-    njs_object_prop_t     *prop;
-    njs_property_query_t  pq;
-
-    obj = njs_json_wrap_value(vm, &wrapper, object);
-    if (njs_slow_path(obj == NULL)) {
+    ret = njs_value_property(vm, holder, name, &val);
+    if (njs_slow_path(ret == NJS_ERROR)) {
         return NJS_ERROR;
     }
 
-    state = njs_json_push_parse_state(vm, parse, &wrapper);
-    if (njs_slow_path(state == NULL)) {
-        return NJS_ERROR;
-    }
-
-    for ( ;; ) {
-        if (state->index < state->keys->length) {
-            njs_property_query_init(&pq, NJS_PROPERTY_QUERY_SET, 0);
-
-            key = &state->keys->start[state->index];
-
-            ret = njs_property_query(vm, &pq, &state->value, key);
-            if (njs_slow_path(ret != NJS_OK)) {
-                if (ret == NJS_DECLINED) {
-                    state->index++;
-                    continue;
-                }
-
+    keys = NULL;
+
+    if (njs_is_object(&val)) {
+        if (!njs_is_array(&val)) {
+            keys = njs_array_keys(vm, &val, 0);
+            if (njs_slow_path(keys == NULL)) {
                 return NJS_ERROR;
             }
 
-            prop = pq.lhq.value;
-
-            if (prop->type == NJS_WHITEOUT) {
-                state->index++;
-                continue;
-            }
-
-            state->prop = prop;
-
-            if (prop->type == NJS_PROPERTY && njs_is_object(&prop->value)) {
-                state = njs_json_push_parse_state(vm, parse, &prop->value);
-                if (state == NULL) {
-                    return NJS_ERROR;
+            for (k = 0; k < keys->length; k++) {
+                ret = njs_json_internalize_property(vm, reviver, &val,
+                                                    &keys->start[k], depth,
+                                                    &new_elem);
+
+                if (njs_slow_path(ret != NJS_OK)) {
+                    goto done;
                 }
 
-                continue;
-            }
-
-            if (prop->type == NJS_PROPERTY_REF
-                && njs_is_object(prop->value.data.u.value))
-            {
-                state = njs_json_push_parse_state(vm, parse,
-                                                  prop->value.data.u.value);
-                if (state == NULL) {
-                    return NJS_ERROR;
+                if (njs_is_undefined(&new_elem)) {
+                    ret = njs_value_property_delete(vm, &val, &keys->start[k],
+                                                    NULL, 0);
+
+                } else {
+                    ret = njs_value_property_set(vm, &val, &keys->start[k],
+                                                 &new_elem);
                 }
 
-                continue;
+                if (njs_slow_path(ret == NJS_ERROR)) {
+                    goto done;
+                }
             }
 
         } else {
-            state = njs_json_pop_parse_state(vm, parse);
-            if (state == NULL) {
-                vm->retval = parse->retval;
-                return NJS_OK;
+
+            ret = njs_object_length(vm, &val, &length);
+            if (njs_slow_path(ret == NJS_ERROR)) {
+                return NJS_ERROR;
             }
-        }
-
-        ret = njs_json_parse_iterator_call(vm, parse, state);
-        if (njs_slow_path(ret != NJS_OK)) {
-            return ret;
+
+            for (k = 0; k < length; k++) {
+                ret = njs_int64_to_string(vm, &index, k);
+                if (njs_slow_path(ret != NJS_OK)) {
+                    return NJS_ERROR;
+                }
+
+                ret = njs_json_internalize_property(vm, reviver, &val, &index,
+                                                    depth, &new_elem);
+
+                if (njs_slow_path(ret != NJS_OK)) {
+                    return NJS_ERROR;
+                }
+
+                if (njs_is_undefined(&new_elem)) {
+                    ret = njs_value_property_delete(vm, &val, &index, NULL, 0);
+
+                } else {
+                    ret = njs_value_property_set(vm, &val, &index, &new_elem);
+                }
+
+                if (njs_slow_path(ret == NJS_ERROR)) {
+                    return NJS_ERROR;
+                }
+            }
         }
     }
-}
-
-
-static njs_int_t
-njs_json_parse_iterator_call(njs_vm_t *vm, njs_json_parse_t *parse,
-    njs_json_state_t *state)
-{
-    njs_int_t          ret;
-    njs_value_t        arguments[3], *value;
-    njs_object_prop_t  *prop;
-
-    prop = state->prop;
-
-    arguments[0] = state->value;
-    arguments[1] = state->keys->start[state->index++];
-
-    switch (prop->type) {
-    case NJS_PROPERTY:
-        arguments[2] = prop->value;
-
-        ret = njs_function_apply(vm, parse->function, arguments, 3,
-                                 &parse->retval);
-        if (njs_slow_path(ret != NJS_OK)) {
-            return ret;
-        }
-
-        if (njs_is_undefined(&parse->retval)) {
-            prop->type = NJS_WHITEOUT;
-
-        } else {
-            prop->value = parse->retval;
-        }
-
-        break;
-
-    case NJS_PROPERTY_REF:
-        value = prop->value.data.u.value;
-        arguments[2] = *value;
-
-        ret = njs_function_apply(vm, parse->function, arguments, 3,
-                                 &parse->retval);
-        if (njs_slow_path(ret != NJS_OK)) {
-            return ret;
-        }
-
-        if (njs_is_undefined(&parse->retval)) {
-            ret = njs_value_property_i64_delete(vm, &state->value,
-                                                state->index - 1, NULL);
-
-        } else {
-            ret = njs_value_property_i64_set(vm, &state->value,
-                                             state->index - 1, &parse->retval);
-        }
-
-        if (njs_slow_path(ret == NJS_ERROR)) {
-            return NJS_ERROR;
-        }
-
-        break;
-
-    default:
-        njs_internal_error(vm, "njs_json_parse_iterator_call() unexpected "
-                         "property type:%s", njs_prop_type_string(prop->type));
+
+    njs_value_assign(&arguments[0], holder);
+    njs_value_assign(&arguments[1], name);
+    njs_value_assign(&arguments[2], &val);
+
+    ret = njs_function_apply(vm, reviver, arguments, 3, retval);
+
+done:
+
+    if (keys != NULL) {
+        njs_array_destroy(vm, keys);
     }
 
-    return NJS_OK;
+    return ret;
 }
 
 
diff -r 723873b4e315 -r 80ed74a0e205 src/test/njs_benchmark.c
--- a/src/test/njs_benchmark.c	Wed May 04 16:23:46 2022 -0700
+++ b/src/test/njs_benchmark.c	Wed May 04 16:44:48 2022 -0700
@@ -208,6 +208,16 @@ static njs_benchmark_test_t  njs_test[] 
       njs_str("123"),
       1000000 },
 
+    { "JSON.parse large",
+      njs_str("JSON.parse(JSON.stringify([Array(2**16)]))[0].length"),
+      njs_str("65536"),
+      10 },
+
+    { "JSON.parse reviver large",
+      njs_str("JSON.parse(JSON.stringify([Array(2**16)]), v=>v)"),
+      njs_str(""),
+      10 },
+
     { "for loop 100M",
       njs_str("var i; for (i = 0; i < 100000000; i++); i"),
       njs_str("100000000"),
diff -r 723873b4e315 -r 80ed74a0e205 src/test/njs_unit_test.c
--- a/src/test/njs_unit_test.c	Wed May 04 16:23:46 2022 -0700
+++ b/src/test/njs_unit_test.c	Wed May 04 16:44:48 2022 -0700
@@ -17203,6 +17203,31 @@ static njs_unit_test_t  njs_test[] =
               "JSON.parse('[1]', func);"),
       njs_str("") },
 
+    { njs_str("JSON.parse(JSON.stringify([Array(2**16)]), v => v)"),
+      njs_str("") },
+
+    { njs_str("var order = []; function reviver(k, v) { order.push(k); };"
+              "JSON.parse('{\"p1\":0,\"p2\":0,\"p1\":0,\"2\":0,\"1\":0}', reviver);"
+              "order"),
+      njs_str("1,2,p1,p2,") },
+
+    { njs_str("function reviver(k, v) {"
+              "    if (k == '0') Object.defineProperty(this, '1', {configurable: false});"
+              "    if (k == '1') return;"
+              "    return v;"
+              " };"
+              "JSON.parse('[1, 2]', reviver)"),
+      njs_str("1,2") },
+
+    { njs_str("JSON.parse('0', (k, v) => {throw 'Oops'})"),
+      njs_str("Oops") },
+
+    { njs_str("JSON.parse('{\"a\":1}', (k, v) => {if (k == 'a') {throw 'Oops'}; return v;})"),
+      njs_str("Oops") },
+
+    { njs_str("JSON.parse('[2,3,43]', (k, v) => {if (v == 43) {throw 'Oops'}; return v;})"),
+      njs_str("Oops") },
+
     /* JSON.stringify() */
 
     { njs_str("JSON.stringify()"),


From rekgrpth at gmail.com  Thu May  5 04:22:02 2022
From: rekgrpth at gmail.com (RekGRpth)
Date: Thu, 5 May 2022 09:22:02 +0500
Subject: [PATCH] upstream keepalive: save and restore connection data
In-Reply-To: <CAPgh2mK-u6QRQGihJ+9wr=9XYFn0H2b+cK1UoEr5AXi75awbEQ@mail.gmail.com>
References: <7b38618adf6317fc2683.1651295022@gws.t72.ru> <Ym211QrjRxiYEvHK@mdounin.ru>
 <CAPgh2mLN7ucbnrY+T_C5i886ncxnPVcQhOQEU7mmf9i1bJWTRA@mail.gmail.com> <CAPgh2mK-u6QRQGihJ+9wr=9XYFn0H2b+cK1UoEr5AXi75awbEQ@mail.gmail.com>
Message-ID: <CAPgh2m+os28C=rWdhm3Fwi4Vv0ic60SMK+jYQk17hXXA+ipU_w@mail.gmail.com>

1) The proper way is use cleanup handler? Is this right?
https://github.com/RekGRpth/ngx_pg_module/blob/25db8706e6268f8197002012e9044be64f698ef3/ngx_pg_module.c#L1422-L1424
2) Why my module is a hack by design?

пн, 2 мая 2022 г. в 08:41, RekGRpth <rekgrpth at gmail.com>:
>
> I think, my suggested in patch solution is more clear and easy and
> beauty, than using cleanup handler to save any connection-specific
> data within a cached connection.
>
> вс, 1 мая 2022 г. в 07:46, RekGRpth <rekgrpth at gmail.com>:
> >
> > now, for example, I do very dirty hack for this:
> > https://github.com/RekGRpth/ngx_pg_module/blob/06eab53160fcff2b19e73ed5bc787f534c2a4c2e/ngx_pg_module.c#L1382-L1385
> > for save and https://github.com/RekGRpth/ngx_pg_module/blob/06eab53160fcff2b19e73ed5bc787f534c2a4c2e/ngx_pg_module.c#L1362
> > for restore
> >
> > вс, 1 мая 2022 г. в 03:19, Maxim Dounin <mdounin at mdounin.ru>:
> > >
> > > Hello!
> > >
> > > On Sat, Apr 30, 2022 at 10:03:42AM +0500, RekGRpth wrote:
> > >
> > > > # HG changeset patch
> > > > # User RekGRpth <rekgrpth at gmail.com>
> > > > # Date 1651294527 -18000
> > > > #      Sat Apr 30 09:55:27 2022 +0500
> > > > # Node ID 7b38618adf6317fc268315b8134a5e2b1bc96269
> > > > # Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
> > > > upstream keepalive: save and restore connection data
> > > >
> > > > save connection data before keep it
> > > > and
> > > > restore connection data after yuild it
> > > >
> > > > diff -r a736a7a613ea -r 7b38618adf63 src/http/modules/ngx_http_upstream_keepalive_module.c
> > > > --- a/src/http/modules/ngx_http_upstream_keepalive_module.c   Tue Feb 08 17:35:27 2022 +0300
> > > > +++ b/src/http/modules/ngx_http_upstream_keepalive_module.c   Sat Apr 30 09:55:27 2022 +0500
> > > > @@ -34,6 +34,8 @@
> > > >      socklen_t                          socklen;
> > > >      ngx_sockaddr_t                     sockaddr;
> > > >
> > > > +    void                              *data;
> > > > +
> > > >  } ngx_http_upstream_keepalive_cache_t;
> > > >
> > > >
> > > > @@ -284,7 +286,7 @@
> > > >
> > > >      c->idle = 0;
> > > >      c->sent = 0;
> > > > -    c->data = NULL;
> > > > +    c->data = item->data;
> > > >      c->log = pc->log;
> > > >      c->read->log = pc->log;
> > > >      c->write->log = pc->log;
> > > > @@ -390,6 +392,7 @@
> > > >      c->write->handler = ngx_http_upstream_keepalive_dummy_handler;
> > > >      c->read->handler = ngx_http_upstream_keepalive_close_handler;
> > > >
> > > > +    item->data = c->data;
> > > >      c->data = item;
> > > >      c->idle = 1;
> > > >      c->log = ngx_cycle->log;
> > >
> > > Thanks for the patch.
> > >
> > > You may want to be more specific on what problem you are trying to
> > > solve by this patch.
> > >
> > > --
> > > Maxim Dounin
> > > http://mdounin.ru/


From mdounin at mdounin.ru  Thu May  5 21:34:36 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 6 May 2022 00:34:36 +0300
Subject: [PATCH] upstream keepalive: save and restore connection data
In-Reply-To: <CAPgh2m+os28C=rWdhm3Fwi4Vv0ic60SMK+jYQk17hXXA+ipU_w@mail.gmail.com>
References: <7b38618adf6317fc2683.1651295022@gws.t72.ru>
 <Ym211QrjRxiYEvHK@mdounin.ru>
 <CAPgh2mLN7ucbnrY+T_C5i886ncxnPVcQhOQEU7mmf9i1bJWTRA@mail.gmail.com>
 <CAPgh2mK-u6QRQGihJ+9wr=9XYFn0H2b+cK1UoEr5AXi75awbEQ@mail.gmail.com>
 <CAPgh2m+os28C=rWdhm3Fwi4Vv0ic60SMK+jYQk17hXXA+ipU_w@mail.gmail.com>
Message-ID: <YnRC7PckfLQEW7Rw@mdounin.ru>

Hello!

On Thu, May 05, 2022 at 09:22:02AM +0500, RekGRpth wrote:

> 1) The proper way is use cleanup handler? Is this right?
> https://github.com/RekGRpth/ngx_pg_module/blob/25db8706e6268f8197002012e9044be64f698ef3/ngx_pg_module.c#L1422-L1424

Quick look suggests it should work.

> 2) Why my module is a hack by design?

As already outlined in previous responses, your module misuses 
parts of the upstream module infrastructure, notably upstream 
blocks and balancers, for its own needs.  This is not something 
supported: upstream module infrastructure is expected to be used 
by the upstream module itself, and not arbitrary external modules.  
What you are doing might (and likely will) stop working on any 
non-trivial changes in the upstream module, even if there will be 
no internal API changes.

-- 
Maxim Dounin
http://mdounin.ru/


From xeioex at nginx.com  Fri May  6 03:27:15 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 06 May 2022 03:27:15 +0000
Subject: [njs] Improved surrogate pairs support for PCRE2 backend.
Message-ID: <hg.ded5304adaf0.1651807635.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/ded5304adaf0
branches:  
changeset: 1850:ded5304adaf0
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Thu May 05 20:25:05 2022 -0700
description:
Improved surrogate pairs support for PCRE2 backend.

In collaboration with Javier Evans.

diffstat:

 external/njs_regex.c     |  20 +++++++++++++++++++-
 src/test/njs_unit_test.c |   5 +++++
 2 files changed, 24 insertions(+), 1 deletions(-)

diffs (46 lines):

diff -r 80ed74a0e205 -r ded5304adaf0 external/njs_regex.c
--- a/external/njs_regex.c	Wed May 04 16:44:48 2022 -0700
+++ b/external/njs_regex.c	Thu May 05 20:25:05 2022 -0700
@@ -60,8 +60,26 @@ njs_regex_compile_ctx_t *
 njs_regex_compile_ctx_create(njs_regex_generic_ctx_t *ctx)
 {
 #ifdef NJS_HAVE_PCRE2
+    pcre2_compile_context  *cc;
 
-    return pcre2_compile_context_create(ctx);
+    cc = pcre2_compile_context_create(ctx);
+
+#ifdef PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES
+    if (njs_fast_path(cc != NULL)) {
+        /* Workaround for surrogate pairs in regular expressions
+         *
+         * This option is needed because njs, unlike the standard ECMAScript,
+         * stores and processes strings in UTF-8 encoding.
+         * PCRE2 does not support surrogate pairs by default when it
+         * is compiled for UTF-8 only strings. But many polyfills
+         * and transpilers use such surrogate pairs expressions.
+         */
+        pcre2_set_compile_extra_options(cc,
+                                        PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES);
+    }
+#endif
+
+    return cc;
 
 #else
 
diff -r 80ed74a0e205 -r ded5304adaf0 src/test/njs_unit_test.c
--- a/src/test/njs_unit_test.c	Wed May 04 16:44:48 2022 -0700
+++ b/src/test/njs_unit_test.c	Thu May 05 20:25:05 2022 -0700
@@ -10841,6 +10841,11 @@ static njs_unit_test_t  njs_test[] =
       njs_str("true") },
 #endif
 
+#ifdef PCRE2_EXTRA_ALLOW_SURROGATE_ESCAPES
+    { njs_str("/\\u200d\\ud800-/"),
+      njs_str("/\\u200d\\ud800-/") },
+#endif
+
     { njs_str("/(\\.(?!com|org)|\\/)/.test('ah.info')"),
       njs_str("true") },
 


From rekgrpth at gmail.com  Fri May  6 03:36:57 2022
From: rekgrpth at gmail.com (RekGRpth)
Date: Fri, 6 May 2022 08:36:57 +0500
Subject: [PATCH] upstream keepalive: save and restore connection data
In-Reply-To: <CAPgh2m+os28C=rWdhm3Fwi4Vv0ic60SMK+jYQk17hXXA+ipU_w@mail.gmail.com>
References: <7b38618adf6317fc2683.1651295022@gws.t72.ru> <Ym211QrjRxiYEvHK@mdounin.ru>
 <CAPgh2mLN7ucbnrY+T_C5i886ncxnPVcQhOQEU7mmf9i1bJWTRA@mail.gmail.com>
 <CAPgh2mK-u6QRQGihJ+9wr=9XYFn0H2b+cK1UoEr5AXi75awbEQ@mail.gmail.com> <CAPgh2m+os28C=rWdhm3Fwi4Vv0ic60SMK+jYQk17hXXA+ipU_w@mail.gmail.com>
Message-ID: <CAPgh2mKpJigdeEtxK+M5L0P+hB4OVLiCaLQGjim3exjUPuq0mw@mail.gmail.com>

In which places my module misuses parts of the upstream module
infrastructure, notably upstream blocks and balancers, for its own
needs?

чт, 5 мая 2022 г. в 09:22, RekGRpth <rekgrpth at gmail.com>:
>
> 1) The proper way is use cleanup handler? Is this right?
> https://github.com/RekGRpth/ngx_pg_module/blob/25db8706e6268f8197002012e9044be64f698ef3/ngx_pg_module.c#L1422-L1424
> 2) Why my module is a hack by design?
>
> пн, 2 мая 2022 г. в 08:41, RekGRpth <rekgrpth at gmail.com>:
> >
> > I think, my suggested in patch solution is more clear and easy and
> > beauty, than using cleanup handler to save any connection-specific
> > data within a cached connection.
> >
> > вс, 1 мая 2022 г. в 07:46, RekGRpth <rekgrpth at gmail.com>:
> > >
> > > now, for example, I do very dirty hack for this:
> > > https://github.com/RekGRpth/ngx_pg_module/blob/06eab53160fcff2b19e73ed5bc787f534c2a4c2e/ngx_pg_module.c#L1382-L1385
> > > for save and https://github.com/RekGRpth/ngx_pg_module/blob/06eab53160fcff2b19e73ed5bc787f534c2a4c2e/ngx_pg_module.c#L1362
> > > for restore
> > >
> > > вс, 1 мая 2022 г. в 03:19, Maxim Dounin <mdounin at mdounin.ru>:
> > > >
> > > > Hello!
> > > >
> > > > On Sat, Apr 30, 2022 at 10:03:42AM +0500, RekGRpth wrote:
> > > >
> > > > > # HG changeset patch
> > > > > # User RekGRpth <rekgrpth at gmail.com>
> > > > > # Date 1651294527 -18000
> > > > > #      Sat Apr 30 09:55:27 2022 +0500
> > > > > # Node ID 7b38618adf6317fc268315b8134a5e2b1bc96269
> > > > > # Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
> > > > > upstream keepalive: save and restore connection data
> > > > >
> > > > > save connection data before keep it
> > > > > and
> > > > > restore connection data after yuild it
> > > > >
> > > > > diff -r a736a7a613ea -r 7b38618adf63 src/http/modules/ngx_http_upstream_keepalive_module.c
> > > > > --- a/src/http/modules/ngx_http_upstream_keepalive_module.c   Tue Feb 08 17:35:27 2022 +0300
> > > > > +++ b/src/http/modules/ngx_http_upstream_keepalive_module.c   Sat Apr 30 09:55:27 2022 +0500
> > > > > @@ -34,6 +34,8 @@
> > > > >      socklen_t                          socklen;
> > > > >      ngx_sockaddr_t                     sockaddr;
> > > > >
> > > > > +    void                              *data;
> > > > > +
> > > > >  } ngx_http_upstream_keepalive_cache_t;
> > > > >
> > > > >
> > > > > @@ -284,7 +286,7 @@
> > > > >
> > > > >      c->idle = 0;
> > > > >      c->sent = 0;
> > > > > -    c->data = NULL;
> > > > > +    c->data = item->data;
> > > > >      c->log = pc->log;
> > > > >      c->read->log = pc->log;
> > > > >      c->write->log = pc->log;
> > > > > @@ -390,6 +392,7 @@
> > > > >      c->write->handler = ngx_http_upstream_keepalive_dummy_handler;
> > > > >      c->read->handler = ngx_http_upstream_keepalive_close_handler;
> > > > >
> > > > > +    item->data = c->data;
> > > > >      c->data = item;
> > > > >      c->idle = 1;
> > > > >      c->log = ngx_cycle->log;
> > > >
> > > > Thanks for the patch.
> > > >
> > > > You may want to be more specific on what problem you are trying to
> > > > solve by this patch.
> > > >
> > > > --
> > > > Maxim Dounin
> > > > http://mdounin.ru/


From mdounin at mdounin.ru  Fri May  6 13:28:17 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 6 May 2022 16:28:17 +0300
Subject: [PATCH] upstream keepalive: save and restore connection data
In-Reply-To: <CAPgh2mKpJigdeEtxK+M5L0P+hB4OVLiCaLQGjim3exjUPuq0mw@mail.gmail.com>
References: <7b38618adf6317fc2683.1651295022@gws.t72.ru>
 <Ym211QrjRxiYEvHK@mdounin.ru>
 <CAPgh2mLN7ucbnrY+T_C5i886ncxnPVcQhOQEU7mmf9i1bJWTRA@mail.gmail.com>
 <CAPgh2mK-u6QRQGihJ+9wr=9XYFn0H2b+cK1UoEr5AXi75awbEQ@mail.gmail.com>
 <CAPgh2m+os28C=rWdhm3Fwi4Vv0ic60SMK+jYQk17hXXA+ipU_w@mail.gmail.com>
 <CAPgh2mKpJigdeEtxK+M5L0P+hB4OVLiCaLQGjim3exjUPuq0mw@mail.gmail.com>
Message-ID: <YnUicee5irGtrlDE@mdounin.ru>

Hello!

On Fri, May 06, 2022 at 08:36:57AM +0500, RekGRpth wrote:

> In which places my module misuses parts of the upstream module
> infrastructure, notably upstream blocks and balancers, for its own
> needs?

If you really don't understand, I'm afraid I can't help.  Sorry 
about that.

-- 
Maxim Dounin
http://mdounin.ru/


From rekgrpth at gmail.com  Fri May  6 13:54:13 2022
From: rekgrpth at gmail.com (RekGRpth)
Date: Fri, 6 May 2022 18:54:13 +0500
Subject: [PATCH] upstream keepalive: save and restore connection data
In-Reply-To: <CAPgh2mKpJigdeEtxK+M5L0P+hB4OVLiCaLQGjim3exjUPuq0mw@mail.gmail.com>
References: <7b38618adf6317fc2683.1651295022@gws.t72.ru> <Ym211QrjRxiYEvHK@mdounin.ru>
 <CAPgh2mLN7ucbnrY+T_C5i886ncxnPVcQhOQEU7mmf9i1bJWTRA@mail.gmail.com>
 <CAPgh2mK-u6QRQGihJ+9wr=9XYFn0H2b+cK1UoEr5AXi75awbEQ@mail.gmail.com>
 <CAPgh2m+os28C=rWdhm3Fwi4Vv0ic60SMK+jYQk17hXXA+ipU_w@mail.gmail.com> <CAPgh2mKpJigdeEtxK+M5L0P+hB4OVLiCaLQGjim3exjUPuq0mw@mail.gmail.com>
Message-ID: <CAPgh2m+YQ3ekWJ3D8i78_A6P12n8P4Yi82yD5N1Beiro2kzTGQ@mail.gmail.com>

I really very want to understand, point me please.


From xeioex at nginx.com  Sat May  7 01:56:26 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Sat, 07 May 2022 01:56:26 +0000
Subject: [njs] Fixed Object.defineProperty() when a recursive descriptor is
 provided.
Message-ID: <hg.3f6f3ba16741.1651888586.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/3f6f3ba16741
branches:  
changeset: 1851:3f6f3ba16741
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri May 06 18:55:07 2022 -0700
description:
Fixed Object.defineProperty() when a recursive descriptor is provided.

This closes #481 issue on Github.

diffstat:

 src/njs_array.c          |   4 ++++
 src/test/njs_unit_test.c |  10 ++++++++++
 2 files changed, 14 insertions(+), 0 deletions(-)

diffs (34 lines):

diff -r ded5304adaf0 -r 3f6f3ba16741 src/njs_array.c
--- a/src/njs_array.c	Thu May 05 20:25:05 2022 -0700
+++ b/src/njs_array.c	Fri May 06 18:55:07 2022 -0700
@@ -142,6 +142,10 @@ njs_array_convert_to_slow_array(njs_vm_t
     njs_value_t        index, value;
     njs_object_prop_t  *prop;
 
+    if (njs_slow_path(!array->object.fast_array)) {
+        return NJS_OK;
+    }
+
     njs_set_array(&value, array);
     array->object.fast_array = 0;
 
diff -r ded5304adaf0 -r 3f6f3ba16741 src/test/njs_unit_test.c
--- a/src/test/njs_unit_test.c	Thu May 05 20:25:05 2022 -0700
+++ b/src/test/njs_unit_test.c	Fri May 06 18:55:07 2022 -0700
@@ -13837,6 +13837,16 @@ static njs_unit_test_t  njs_test[] =
               "d.enumerable && d.writable && d.configurable"),
       njs_str("true") },
 
+    { njs_str("const arr = [1,2];"
+              "function f(arg) {"
+              "        const desc = {get: arg};"
+              "        Object.defineProperty(desc, 'set', desc);"
+              "        Object.defineProperty(arr, 1, desc);"
+              "}"
+              "f(f);"
+              "njs.dump(arr)"),
+      njs_str("[1,'[Getter]']") },
+
     { njs_str("Object.defineProperties()"),
       njs_str("TypeError: Object.defineProperties is called on non-object") },
 


From xeioex at nginx.com  Sat May  7 01:56:28 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Sat, 07 May 2022 01:56:28 +0000
Subject: [njs] Fixed Array.prototype.fill() for typed-arrays.
Message-ID: <hg.5c7e02885c26.1651888588.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/5c7e02885c26
branches:  
changeset: 1852:5c7e02885c26
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri May 06 18:55:35 2022 -0700
description:
Fixed Array.prototype.fill() for typed-arrays.

This closes #478 issue on Github.

diffstat:

 src/njs_typed_array.c    |  2 --
 src/test/njs_unit_test.c |  3 +++
 2 files changed, 3 insertions(+), 2 deletions(-)

diffs (25 lines):

diff -r 3f6f3ba16741 -r 5c7e02885c26 src/njs_typed_array.c
--- a/src/njs_typed_array.c	Fri May 06 18:55:07 2022 -0700
+++ b/src/njs_typed_array.c	Fri May 06 18:55:35 2022 -0700
@@ -702,8 +702,6 @@ njs_typed_array_set_value(njs_vm_t *vm, 
 
     njs_typed_array_prop_set(vm, array, index, num);
 
-    njs_set_number(setval, num);
-
     return NJS_OK;
 }
 
diff -r 3f6f3ba16741 -r 5c7e02885c26 src/test/njs_unit_test.c
--- a/src/test/njs_unit_test.c	Fri May 06 18:55:07 2022 -0700
+++ b/src/test/njs_unit_test.c	Fri May 06 18:55:35 2022 -0700
@@ -5409,6 +5409,9 @@ static njs_unit_test_t  njs_test[] =
                  "Array.prototype.fill.call(o, 2).a"),
       njs_str("4") },
 
+    { njs_str("Array.prototype.fill.call(new Int32Array(1))"),
+      njs_str("0") },
+
     { njs_str("ArrayBuffer()"),
       njs_str("TypeError: Constructor ArrayBuffer requires 'new'") },
 


From mths at google.com  Mon May  9 09:46:54 2022
From: mths at google.com (Mathias Bynens)
Date: Mon, 9 May 2022 11:46:54 +0200
Subject: [PATCH] Correct JavaScript MIME types + extensions per RFC 9239
Message-ID: <CADizRgZC4NXLhjuBRyDq5hRCNNf_ctMye+ZqKN0-BWmXdbFF3w@mail.gmail.com>

# HG changeset patch
# User Mathias Bynens <mathias at qiwi.be>
# Date 1652088302 -7200
#      Mon May 09 11:25:02 2022 +0200
# Branch fix-js-mime-and-extensions
# Node ID 7944657540c64d92a5bdc3932710c977d984b54d
# Parent  35afae4b3dffff6718c0cab3ceb16b9de207c20a
Correct JavaScript MIME types + extensions per RFC 9239

This patch updates the MIME type configuration per RFC 9239.
https://www.rfc-editor.org/rfc/rfc9239

First, the recommended MIME type is now `text/javascript`:

> The most widely supported media type in use is `text/javascript`; all
> others are considered historical and obsolete aliases of
`text/javascript`.

Second, the `.mjs` extension is now explicitly registered:

> The `.mjs` file extension signals that the file represents a JavaScript
> module. Execution environments that rely on file extensions to
> determine how to process inputs parse `.mjs` files using the Module
> grammar of [ECMA-262].

IANA template: https://www.iana.org/assignments/media-types/text/javascript

diff -r 35afae4b3dff -r 7944657540c6 conf/mime.types
--- a/conf/mime.types Fri Apr 29 17:38:01 2022 +0400
+++ b/conf/mime.types Mon May 09 11:25:02 2022 +0200
@@ -5,7 +5,7 @@
     text/xml                                         xml;
     image/gif                                        gif;
     image/jpeg                                       jpeg jpg;
-    application/javascript                           js;
+    text/javascript                                  js mjs;
     application/atom+xml                             atom;
     application/rss+xml                              rss;

diff -r 35afae4b3dff -r 7944657540c6
src/http/modules/ngx_http_charset_filter_module.c
--- a/src/http/modules/ngx_http_charset_filter_module.c Fri Apr 29 17:38:01
2022 +0400
+++ b/src/http/modules/ngx_http_charset_filter_module.c Mon May 09 11:25:02
2022 +0200
@@ -128,7 +128,7 @@
     ngx_string("text/xml"),
     ngx_string("text/plain"),
     ngx_string("text/vnd.wap.wml"),
-    ngx_string("application/javascript"),
+    ngx_string("text/javascript"),
     ngx_string("application/rss+xml"),
     ngx_null_string
 };
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220509/9555bdce/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nginx-rfc9239-patch
Type: application/octet-stream
Size: 2122 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220509/9555bdce/attachment.obj>

From kiprasm at gmail.com  Mon May  9 13:12:40 2022
From: kiprasm at gmail.com (Kipras)
Date: Mon, 9 May 2022 16:12:40 +0300
Subject: [PATCH] Do not break downstream connection on proxy_cache write error
In-Reply-To: <YnBGwLuyGn0If3ST@mdounin.ru>
References: <e6e708fd7958987027f6.1651498569@kipras-Lenovo> <YnBGwLuyGn0If3ST@mdounin.ru>
Message-ID: <CADNd3rvs_=nS0U+3wLpLjHT5kr3Vo__zHXKvFrbzrUW2sgxn1w@mail.gmail.com>

Hello,

Thank you for the very detailed feedback.

A couple of mistakes from my side, when submitting the patch:

Also, this looks irrelevant to the issue being addressed.


Yep, this is not relevant to the issue being addressed, i think i added this
change here by mistake, my bad.


Also, i missed to add another part of the patch, which removes this block
(the
original call to ngx_write_chain_to_temp_file()):

    n = ngx_write_chain_to_temp_file(p->temp_file, out);


>     if (n == NGX_ERROR) {
>         p->cache_write_fail = 1;
>         return NGX_ABORT;
>     }



I'm guessing you were referring to that block in this comment: (?)

- In case of success, the following code will write the data
>   again, resulting in file corruption and likely various other
>   issues.



Honestly, all the ins and outs of the function
ngx_event_pipe_write_chain_to_temp_file() are not quite clear to me. But
with
this change the issue of closing the client connection abruptly without
finishing to serve the response (when proxy cache failed to write to disk)
goes
away (at least in my tests).
However, i surely cannot guarantee that nothing else will break :)
(but fwiw i did run the nginx-tests suite with this change, and all tests
pass)

I will do more testing and try to analyze what exactly is happening in this
function.

Thank you.

P.S. if i could ask - what does the p->cacheable flag mean? The
ngx_event_pipe_t
struct is not documented in
http://nginx.org/en/docs/dev/development_guide.html
and it is kinda hard to understand all the pipes/buffers and the buffer
shadowing mechanism/purpose just from reading the code :)

On Tue, May 3, 2022 at 12:02 AM Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Mon, May 02, 2022 at 04:36:09PM +0300, Kipras Mancevicius wrote:
>
> > # HG changeset patch
> > # User Kipras Mancevicius <kiprasm at gmail.com>
> > # Date 1651495240 -10800
> > #      Mon May 02 15:40:40 2022 +0300
> > # Branch proxy_cache_write_disk_full_fix
> > # Node ID e6e708fd7958987027f6de7748d948e5015ed32b
> > # Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
> > Do not break downstream connection on proxy_cache write error
> >
> > Continue sending response to downstream connection and delete temp cache
> file on
> > upstream connection finalization.
> >
> > diff -r a736a7a613ea -r e6e708fd7958 src/core/ngx_file.c
> > --- a/src/core/ngx_file.c     Tue Feb 08 17:35:27 2022 +0300
> > +++ b/src/core/ngx_file.c     Mon May 02 15:40:40 2022 +0300
> > @@ -765,6 +765,13 @@
> >                                ngx_delete_file_n " \"%s\" failed", name);
> >
> >              }
> > +        } else {
> > +            /*
> > +             * if the file failed to fully copy (e.g. target disk is
> full) - it
> > +             * may remain (partially written) in the target disk.
> Attempt to
> > +             * delete it.
> > +             */
> > +            ngx_delete_file(name);
>
> Trying to delete a file "just in case" looks clearly wrong.  If at
> all, this is to be done in ngx_copy_file(), the function which
> created the file and knows what in fact happened.  Further, it is
> not clear if it's to be done at all: in many practical cases an
> incomplete file is better than no file at all, and this is a
> generic function.
>
> Also, this looks irrelevant to the issue being addressed.
>
> >          }
> >
> >          ngx_free(name);
> > diff -r a736a7a613ea -r e6e708fd7958 src/event/ngx_event_pipe.c
> > --- a/src/event/ngx_event_pipe.c      Tue Feb 08 17:35:27 2022 +0300
> > +++ b/src/event/ngx_event_pipe.c      Mon May 02 15:40:40 2022 +0300
> > @@ -753,10 +753,21 @@
> >          out = p->writing;
> >          p->writing = NULL;
> >
> > -        n = ngx_write_chain_to_temp_file(p->temp_file, NULL);
> > +        /*
> > +         * stop writing to cache on cache write fail, but keep sending
> response
> > +         * downstream. Incomplete temp cache file will be deleted later.
> > +         */
> > +        if (p->cache_write_fail) {
> > +            return NGX_BUSY;
> > +        } else {
> > +            n = ngx_write_chain_to_temp_file(p->temp_file, NULL);
>
> This looks terribly wrong: if this code will happen actually
> return NGX_BUSY, a thread-based operation won't be completed,
> potentially resulting in a connection hang and/or a socket leak.
>
> >
> > -        if (n == NGX_ERROR) {
> > -            return NGX_ABORT;
> > +            if (n == NGX_ERROR) {
> > +                p->cache_write_fail = 1;
> > +                p->cacheable = 0;
> > +
> > +                return NGX_BUSY;
>
> This looks wrong: if an error happens as a result of a
> thread-based operation, it is incorrect to just ignore this.  Data
> being written needs to be recovered somehow.
>
> > +            }
> >          }
> >
> >          goto done;
> > @@ -777,6 +788,23 @@
> >          out = p->in;
> >      }
> >
> > +    /*
> > +     * stop writing to cache on cache write fail, but keep sending
> response
> > +     * downstream. Incomplete temp cache file will be deleted later.
> > +     */
> > +    if (p->cache_write_fail) {
> > +        return NGX_BUSY;
> > +    } else {
> > +        n = ngx_write_chain_to_temp_file(p->temp_file, out);
> > +
> > +        if (n == NGX_ERROR) {
> > +            p->cache_write_fail = 1;
> > +            p->cacheable = 0;
> > +
> > +            return NGX_BUSY;
> > +        }
> > +    }
> > +
>
> This also looks wrong:
>
> - The code tries to write everything in out, ignoring things like
>   temp_file_write_size and max_temp_file_size.
>
> - In case of success, the following code will write the data
>   again, resulting in file corruption and likely various other
>   issues.
>
> Further, the following writes (which are the only writes
> meaningful for the current request, as you don't try to update
> p->out) are not really protected by any error checking, so the
> initial objective of the patch does not seem to be achieved.
>
> Please also note that the ticket in question says "without
> introducing unreasonable complexity in the source code" for a
> reason.
>
> Hope this helps.
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx-devel mailing list -- nginx-devel at nginx.org
> To unsubscribe send an email to nginx-devel-leave at nginx.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220509/8473d235/attachment.htm>

From mdounin at mdounin.ru  Mon May  9 15:16:11 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 9 May 2022 18:16:11 +0300
Subject: [PATCH] Do not break downstream connection on proxy_cache write
 error
In-Reply-To: <CADNd3rvs_=nS0U+3wLpLjHT5kr3Vo__zHXKvFrbzrUW2sgxn1w@mail.gmail.com>
References: <e6e708fd7958987027f6.1651498569@kipras-Lenovo>
 <YnBGwLuyGn0If3ST@mdounin.ru>
 <CADNd3rvs_=nS0U+3wLpLjHT5kr3Vo__zHXKvFrbzrUW2sgxn1w@mail.gmail.com>
Message-ID: <YnkwO5J561RME0aD@mdounin.ru>

Hello!

On Mon, May 09, 2022 at 04:12:40PM +0300, Kipras wrote:

[...]

> P.S. if i could ask - what does the p->cacheable flag mean?

It basically means "save the whole to response to the temporary 
file, it will be used for caching/store".  Note well that as of 
now, it is only set at the very start of the event pipe operation, 
when initializing u->pipe in ngx_http_upstream_send_response(), 
and it might not be safe to simply switch it off at some point.

> The
> ngx_event_pipe_t
> struct is not documented in
> http://nginx.org/en/docs/dev/development_guide.html
> and it is kinda hard to understand all the pipes/buffers and the buffer
> shadowing mechanism/purpose just from reading the code :)

The code is basically the only documentation available about the 
code (history and commit logs might be also helpful).  The 
development guide is mostly about explaining some basic things, 
and it is focused on how to write modules, not on explaining how 
existing modules work.  If you want to change the existing code, 
you'll have to read the code carefully to understand what the code 
does.

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Mon May  9 18:26:21 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 9 May 2022 21:26:21 +0300
Subject: [PATCH] Correct JavaScript MIME types + extensions per RFC 9239
In-Reply-To: <CADizRgZC4NXLhjuBRyDq5hRCNNf_ctMye+ZqKN0-BWmXdbFF3w@mail.gmail.com>
References: <CADizRgZC4NXLhjuBRyDq5hRCNNf_ctMye+ZqKN0-BWmXdbFF3w@mail.gmail.com>
Message-ID: <YnlczZbl0TtbIx8Y@mdounin.ru>

Hello!

On Mon, May 09, 2022 at 11:46:54AM +0200, Mathias Bynens via nginx-devel wrote:

> # HG changeset patch
> # User Mathias Bynens <mathias at qiwi.be>
> # Date 1652088302 -7200
> #      Mon May 09 11:25:02 2022 +0200
> # Branch fix-js-mime-and-extensions
> # Node ID 7944657540c64d92a5bdc3932710c977d984b54d
> # Parent  35afae4b3dffff6718c0cab3ceb16b9de207c20a
> Correct JavaScript MIME types + extensions per RFC 9239
> 
> This patch updates the MIME type configuration per RFC 9239.
> https://www.rfc-editor.org/rfc/rfc9239
> 
> First, the recommended MIME type is now `text/javascript`:
> 
> > The most widely supported media type in use is `text/javascript`; all
> > others are considered historical and obsolete aliases of
> `text/javascript`.
> 
> Second, the `.mjs` extension is now explicitly registered:
> 
> > The `.mjs` file extension signals that the file represents a JavaScript
> > module. Execution environments that rely on file extensions to
> > determine how to process inputs parse `.mjs` files using the Module
> > grammar of [ECMA-262].
> 
> IANA template: https://www.iana.org/assignments/media-types/text/javascript
> 
> diff -r 35afae4b3dff -r 7944657540c6 conf/mime.types
> --- a/conf/mime.types Fri Apr 29 17:38:01 2022 +0400
> +++ b/conf/mime.types Mon May 09 11:25:02 2022 +0200
> @@ -5,7 +5,7 @@
>      text/xml                                         xml;
>      image/gif                                        gif;
>      image/jpeg                                       jpeg jpg;
> -    application/javascript                           js;
> +    text/javascript                                  js mjs;
>      application/atom+xml                             atom;
>      application/rss+xml                              rss;
> 
> diff -r 35afae4b3dff -r 7944657540c6
> src/http/modules/ngx_http_charset_filter_module.c
> --- a/src/http/modules/ngx_http_charset_filter_module.c Fri Apr 29 17:38:01
> 2022 +0400
> +++ b/src/http/modules/ngx_http_charset_filter_module.c Mon May 09 11:25:02
> 2022 +0200
> @@ -128,7 +128,7 @@
>      ngx_string("text/xml"),
>      ngx_string("text/plain"),
>      ngx_string("text/vnd.wap.wml"),
> -    ngx_string("application/javascript"),
> +    ngx_string("text/javascript"),
>      ngx_string("application/rss+xml"),
>      ngx_null_string
>  };

Thanks for the patch.

I've added a link to the patch to the most relevant ticket 
(https://trac.nginx.org/nginx/ticket/2216).

-- 
Maxim Dounin
http://mdounin.ru/


From pluknet at nginx.com  Wed May 11 15:32:32 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Wed, 11 May 2022 19:32:32 +0400
Subject: [PATCH 02 of 20] FastCGI: combining headers with identical names
 (ticket #1724)
In-Reply-To: <61b29233a55216c6fa72.1650493122@vm-bsd.mdounin.ru>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <61b29233a55216c6fa72.1650493122@vm-bsd.mdounin.ru>
Message-ID: <20220511153232.c3fd4i573xt64hnc@Y9MQ9X2QVV>

On Thu, Apr 21, 2022 at 01:18:42AM +0300, Maxim Dounin wrote:
> # HG changeset patch
> # User Maxim Dounin <mdounin at mdounin.ru>
> # Date 1650492316 -10800
> #      Thu Apr 21 01:05:16 2022 +0300
> # Node ID 61b29233a55216c6fa72e23b93a4a28d76a9fb94
> # Parent  e70fb0fdfbc0fb7b7e9f493cc2eb65de617b115a
> FastCGI: combining headers with identical names (ticket #1724).
> 
> FastCGI responder is expected to receive CGI/1.1 environment variables
> in the parameters (see section "6.2 Responder" of the FastCGI specification).
> Obviously enough, there cannot be multiple environment variables with
> the same name.
> 
> Further, CGI specification (RFC 3875, section "4.1.18. Protocol-Specific
> Meta-Variables") explicitly requires to combine headers: "If multiple
> header fields with the same field-name are received then the server MUST
> rewrite them as a single value having the same semantics".
> 
> diff --git a/src/core/ngx_hash.h b/src/core/ngx_hash.h
> --- a/src/core/ngx_hash.h
> +++ b/src/core/ngx_hash.h
> @@ -89,12 +89,15 @@ typedef struct {
>  } ngx_hash_keys_arrays_t;
>  
>  
> -typedef struct {
> +typedef struct ngx_table_elt_s  ngx_table_elt_t;
> +
> +struct ngx_table_elt_s {
>      ngx_uint_t        hash;
>      ngx_str_t         key;
>      ngx_str_t         value;
>      u_char           *lowcase_key;
> -} ngx_table_elt_t;
> +    ngx_table_elt_t  *next;
> +};
>  
>  
>  void *ngx_hash_find(ngx_hash_t *hash, ngx_uint_t key, u_char *name, size_t len);
> diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c
> --- a/src/http/modules/ngx_http_fastcgi_module.c
> +++ b/src/http/modules/ngx_http_fastcgi_module.c
> @@ -835,14 +835,14 @@ static ngx_int_t
>  ngx_http_fastcgi_create_request(ngx_http_request_t *r)
>  {
>      off_t                         file_pos;
> -    u_char                        ch, *pos, *lowcase_key;
> +    u_char                        ch, sep, *pos, *lowcase_key;
>      size_t                        size, len, key_len, val_len, padding,
>                                    allocated;
>      ngx_uint_t                    i, n, next, hash, skip_empty, header_params;
>      ngx_buf_t                    *b;
>      ngx_chain_t                  *cl, *body;
>      ngx_list_part_t              *part;
> -    ngx_table_elt_t              *header, **ignored;
> +    ngx_table_elt_t              *header, *hn, **ignored;
>      ngx_http_upstream_t          *u;
>      ngx_http_script_code_pt       code;
>      ngx_http_script_engine_t      e, le;
> @@ -900,7 +900,11 @@ ngx_http_fastcgi_create_request(ngx_http
>          allocated = 0;
>          lowcase_key = NULL;
>  
> -        if (params->number) {
> +        if (ngx_http_link_multi_headers(r) != NGX_OK) {
> +            return NGX_ERROR;
> +        }
> +
> +        if (params->number || r->headers_in.multi) {
>              n = 0;
>              part = &r->headers_in.headers.part;
>  
> @@ -930,6 +934,12 @@ ngx_http_fastcgi_create_request(ngx_http
>                  i = 0;
>              }
>  
> +            for (n = 0; n < header_params; n++) {
> +                if (&header[i] == ignored[n]) {
> +                    goto next_length;
> +                }
> +            }
> +
>              if (params->number) {
>                  if (allocated < header[i].key.len) {
>                      allocated = header[i].key.len + 16;
> @@ -959,15 +969,23 @@ ngx_http_fastcgi_create_request(ngx_http
>                      ignored[header_params++] = &header[i];
>                      continue;
>                  }
> -
> -                n += sizeof("HTTP_") - 1;
> -
> -            } else {
> -                n = sizeof("HTTP_") - 1 + header[i].key.len;
>              }
>  
> -            len += ((n > 127) ? 4 : 1) + ((header[i].value.len > 127) ? 4 : 1)
> -                + n + header[i].value.len;
> +            key_len = sizeof("HTTP_") - 1 + header[i].key.len;
> +
> +            val_len = header[i].value.len;
> +
> +            for (hn = header[i].next; hn; hn = hn->next) {
> +                val_len += hn->value.len + 2;
> +                ignored[header_params++] = hn;
> +            }
> +
> +            len += ((key_len > 127) ? 4 : 1) + key_len
> +                   + ((val_len > 127) ? 4 : 1) + val_len;
> +
> +        next_length:
> +
> +            continue;
>          }
>      }
>  
> @@ -1109,7 +1127,7 @@ ngx_http_fastcgi_create_request(ngx_http
>  
>              for (n = 0; n < header_params; n++) {
>                  if (&header[i] == ignored[n]) {
> -                    goto next;
> +                    goto next_value;
>                  }
>              }
>  
> @@ -1125,6 +1143,11 @@ ngx_http_fastcgi_create_request(ngx_http
>              }
>  
>              val_len = header[i].value.len;
> +
> +            for (hn = header[i].next; hn; hn = hn->next) {
> +                val_len += hn->value.len + 2;
> +            }
> +
>              if (val_len > 127) {
>                  *b->last++ = (u_char) (((val_len >> 24) & 0x7f) | 0x80);
>                  *b->last++ = (u_char) ((val_len >> 16) & 0xff);
> @@ -1150,13 +1173,34 @@ ngx_http_fastcgi_create_request(ngx_http
>                  *b->last++ = ch;
>              }
>  
> -            b->last = ngx_copy(b->last, header[i].value.data, val_len);
> +            b->last = ngx_copy(b->last, header[i].value.data,
> +                               header[i].value.len);
> +
> +            if (header[i].next) {
> +
> +                if (header[i].key.len == sizeof("Cookie") - 1
> +                    && ngx_strncasecmp(header[i].key.data, (u_char *) "Cookie",
> +                                       sizeof("Cookie") - 1)
> +                       == 0)
> +                {
> +                    sep = ';';
> +
> +                } else {
> +                    sep = ',';
> +                }
> +
> +                for (hn = header[i].next; hn; hn = hn->next) {
> +                    *b->last++ = sep;
> +                    *b->last++ = ' ';
> +                    b->last = ngx_copy(b->last, hn->value.data, hn->value.len);
> +                }
> +            }
>  
>              ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
>                             "fastcgi param: \"%*s: %*s\"",
>                             key_len, b->last - (key_len + val_len),
>                             val_len, b->last - val_len);
> -        next:
> +        next_value:
>  
>              continue;
>          }

Overall, fastcgi/uwsgi/scgi parts look good, some observations below:

- fastcgi_param (still) overrides client headers with the same name
e.g., "fastcgi_param header value;" overrides client header "header: value"

- multiple directives itself aren't linked emitting separate params:
fastcgi_param FOO BAR;
fastcgi_param FOO BAZ;

I don't think it's a major point though, it can be handled
in configuration in principle.

BTW, what about proxy modifications? they aren't required but could be useful

> diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
> --- a/src/http/ngx_http_core_module.c
> +++ b/src/http/ngx_http_core_module.c
> @@ -2802,6 +2802,78 @@ ngx_http_get_forwarded_addr_internal(ngx
>  }
>  
>  
> +ngx_int_t
> +ngx_http_link_multi_headers(ngx_http_request_t *r)
> +{
> +    ngx_uint_t        i, j;
> +    ngx_list_part_t  *part, *ppart;
> +    ngx_table_elt_t  *header, *pheader, **ph;
> +
> +    if (r->headers_in.multi_linked) {
> +        return NGX_OK;
> +    }

multi_linked is never set, was the intension to avoid doing the work
twice on repetitive invocation? it doesn't seem to be possible.
I'd just axe it.

> +
> +    part = &r->headers_in.headers.part;
> +    header = part->elts;
> +
> +    for (i = 0; /* void */; i++) {
> +
> +        if (i >= part->nelts) {
> +            if (part->next == NULL) {
> +                break;
> +            }
> +
> +            part = part->next;
> +            header = part->elts;
> +            i = 0;
> +        }
> +
> +        header[i].next = NULL;
> +
> +        /*
> +         * search for previous headers with the same name;
> +         * if there are any, link to them
> +         */
> +
> +        ppart = &r->headers_in.headers.part;
> +        pheader = part->elts;

pheader = ppart->elts;

> +
> +        for (j = 0; /* void */; j++) {
> +
> +            if (j >= ppart->nelts) {
> +                if (ppart->next == NULL) {
> +                    break;
> +                }
> +
> +                ppart = ppart->next;
> +                pheader = ppart->elts;
> +                i = 0;

j = 0;

> +            }
> +
> +            if (part == ppart && i == j) {
> +                break;
> +            }
> +
> +            if (header[i].key.len == pheader[j].key.len
> +                && ngx_strncasecmp(header[i].key.data, pheader[j].key.data,
> +                                   header[i].key.len)
> +                   == 0)
> +            {
> +                ph = &pheader[j].next;
> +                while (*ph) { ph = &(*ph)->next; }
> +                *ph = &header[i];
> +
> +                r->headers_in.multi = 1;
> +
> +                break;
> +            }
> +        }
> +    }
> +
> +    return NGX_OK;
> +}
> +
> +
>  static char *
>  ngx_http_core_server(ngx_conf_t *cf, ngx_command_t *cmd, void *dummy)
>  {
> diff --git a/src/http/ngx_http_core_module.h b/src/http/ngx_http_core_module.h
> --- a/src/http/ngx_http_core_module.h
> +++ b/src/http/ngx_http_core_module.h
> @@ -532,6 +532,8 @@ ngx_int_t ngx_http_get_forwarded_addr(ng
>      ngx_array_t *headers, ngx_str_t *value, ngx_array_t *proxies,
>      int recursive);
>  
> +ngx_int_t ngx_http_link_multi_headers(ngx_http_request_t *r);
> +
>  
>  extern ngx_module_t  ngx_http_core_module;
>  
> diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
> --- a/src/http/ngx_http_request.h
> +++ b/src/http/ngx_http_request.h
> @@ -242,6 +242,8 @@ typedef struct {
>  
>      unsigned                          connection_type:2;
>      unsigned                          chunked:1;
> +    unsigned                          multi:1;
> +    unsigned                          multi_linked:1;
>      unsigned                          msie:1;
>      unsigned                          msie6:1;
>      unsigned                          opera:1;
> 
> _______________________________________________
> nginx-devel mailing list -- nginx-devel at nginx.org
> To unsubscribe send an email to nginx-devel-leave at nginx.org


From pluknet at nginx.com  Wed May 11 16:09:10 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Wed, 11 May 2022 20:09:10 +0400
Subject: [PATCH 05 of 20] Combining unknown headers during variables
 lookup (ticket #1316)
In-Reply-To: <2ea48b5e4643a818cd81.1650493125@vm-bsd.mdounin.ru>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <2ea48b5e4643a818cd81.1650493125@vm-bsd.mdounin.ru>
Message-ID: <20220511160910.pycqf36eqvpyf66c@Y9MQ9X2QVV>

On Thu, Apr 21, 2022 at 01:18:45AM +0300, Maxim Dounin wrote:
> # HG changeset patch
> # User Maxim Dounin <mdounin at mdounin.ru>
> # Date 1650492321 -10800
> #      Thu Apr 21 01:05:21 2022 +0300
> # Node ID 2ea48b5e4643a818cd81179f040f0b36be9050d6
> # Parent  3618f35ac3c8833cfd4a4c80295ab94eb924a9a7
> Combining unknown headers during variables lookup (ticket #1316).
> 
> Previously, $http_*, $sent_http_*, $upstream_http_*, and $upstream_trailer_*
> variables returned only the first header (with a few specially handled
> exceptions: $http_cookie, $http_x_forwarded_for, $sent_http_cache_control,
> $sent_http_link).

forgot to mention "sent_trailer_"

jftr, this is an API change, used at least in mod_zip for for "upstream_http_"

> 
> With this change, all headers are returned, combined together.  For
> example, $http_foo variable will be "a, b" if there are "Foo: a" and
> "Foo: b" headers in the request.
> 
> Note that $upstream_http_set_cookie will also return all "Set-Cookie"
> headers (ticket #1843), though this might not be what one want, since
> the "Set-Cookie" header does not follow the list syntax (see RFC 7230,
> section 3.2.2).
> 
> diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
> --- a/src/http/ngx_http_upstream.c
> +++ b/src/http/ngx_http_upstream.c
> @@ -5703,7 +5703,7 @@ ngx_http_upstream_header_variable(ngx_ht
>          return NGX_OK;
>      }
>  
> -    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
> +    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
>                                           &r->upstream->headers_in.headers.part,
>                                           sizeof("upstream_http_") - 1);
>  }
> @@ -5718,7 +5718,7 @@ ngx_http_upstream_trailer_variable(ngx_h
>          return NGX_OK;
>      }
>  
> -    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
> +    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
>                                          &r->upstream->headers_in.trailers.part,
>                                          sizeof("upstream_trailer_") - 1);
>  }
> diff --git a/src/http/ngx_http_variables.c b/src/http/ngx_http_variables.c
> --- a/src/http/ngx_http_variables.c
> +++ b/src/http/ngx_http_variables.c
> @@ -919,7 +919,7 @@ static ngx_int_t
>  ngx_http_variable_unknown_header_in(ngx_http_request_t *r,
>      ngx_http_variable_value_t *v, uintptr_t data)
>  {
> -    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
> +    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
>                                              &r->headers_in.headers.part,
>                                              sizeof("http_") - 1);
>  }
> @@ -929,7 +929,7 @@ static ngx_int_t
>  ngx_http_variable_unknown_header_out(ngx_http_request_t *r,
>      ngx_http_variable_value_t *v, uintptr_t data)
>  {
> -    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
> +    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
>                                              &r->headers_out.headers.part,
>                                              sizeof("sent_http_") - 1);
>  }
> @@ -939,19 +939,26 @@ static ngx_int_t
>  ngx_http_variable_unknown_trailer_out(ngx_http_request_t *r,
>      ngx_http_variable_value_t *v, uintptr_t data)
>  {
> -    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
> +    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
>                                              &r->headers_out.trailers.part,
>                                              sizeof("sent_trailer_") - 1);
>  }
>  
>  
>  ngx_int_t
> -ngx_http_variable_unknown_header(ngx_http_variable_value_t *v, ngx_str_t *var,
> +ngx_http_variable_unknown_header(ngx_http_request_t *r,
> +    ngx_http_variable_value_t *v, ngx_str_t *var,
>      ngx_list_part_t *part, size_t prefix)
>  {
> -    u_char            ch;
> +    u_char           *p, ch;
> +    size_t            len;
>      ngx_uint_t        i, n;
> -    ngx_table_elt_t  *header;
> +    ngx_table_elt_t  *header, *h, **ph;
> +
> +    ph = &h;
> +#if (NGX_SUPPRESS_WARN)
> +    len = 0;
> +#endif
>  
>      header = part->elts;
>  
> @@ -971,7 +978,11 @@ ngx_http_variable_unknown_header(ngx_htt
>              continue;
>          }
>  
> -        for (n = 0; n + prefix < var->len && n < header[i].key.len; n++) {
> +        if (header[i].key.len != var->len - prefix) {
> +            continue;
> +        }
> +
> +        for (n = 0; n < var->len - prefix; n++) {
>              ch = header[i].key.data[n];
>  
>              if (ch >= 'A' && ch <= 'Z') {
> @@ -986,18 +997,59 @@ ngx_http_variable_unknown_header(ngx_htt
>              }
>          }
>  
> -        if (n + prefix == var->len && n == header[i].key.len) {
> -            v->len = header[i].value.len;
> -            v->valid = 1;
> -            v->no_cacheable = 0;
> -            v->not_found = 0;
> -            v->data = header[i].value.data;
> -
> -            return NGX_OK;
> +        if (n != var->len - prefix) {
> +            continue;
>          }
> +
> +        len += header[i].value.len + 2;
> +
> +        *ph = &header[i];
> +        ph = &header[i].next;
>      }
>  
> -    v->not_found = 1;
> +    *ph = NULL;
> +
> +    if (h == NULL) {
> +        v->not_found = 1;
> +        return NGX_OK;
> +    }
> +
> +    len -= 2;
> +
> +    if (h->next == NULL) {
> +
> +        v->len = h->value.len;

could be just len for consistency with common case

> +        v->valid = 1;
> +        v->no_cacheable = 0;
> +        v->not_found = 0;
> +        v->data = h->value.data;
> +
> +        return NGX_OK;
> +    }
> +
> +    p = ngx_pnalloc(r->pool, len);
> +    if (p == NULL) {
> +        return NGX_ERROR;
> +    }
> +
> +    v->len = len;
> +    v->valid = 1;
> +    v->no_cacheable = 0;
> +    v->not_found = 0;
> +    v->data = p;
> +
> +    for ( ;; ) {
> +
> +        p = ngx_copy(p, h->value.data, h->value.len);
> +
> +        if (h->next == NULL) {
> +            break;
> +        }
> +
> +        *p++ = ','; *p++ = ' ';

what about Set-Cookie separator ";" support?
looks like can be easly integrated (on top off):

diff --git a/src/http/ngx_http_variables.c b/src/http/ngx_http_variables.c
--- a/src/http/ngx_http_variables.c
+++ b/src/http/ngx_http_variables.c
@@ -950,7 +950,7 @@ ngx_http_variable_unknown_header(ngx_htt
     ngx_http_variable_value_t *v, ngx_str_t *var,
     ngx_list_part_t *part, size_t prefix)
 {
-    u_char           *p, ch;
+    u_char           *p, ch, sep;
     size_t            len;
     ngx_uint_t        i, n;
     ngx_table_elt_t  *header, *h, **ph;
@@ -1018,7 +1018,7 @@ ngx_http_variable_unknown_header(ngx_htt
 
     if (h->next == NULL) {
 
-        v->len = h->value.len;
+        v->len = len;
         v->valid = 1;
         v->no_cacheable = 0;
         v->not_found = 0;
@@ -1038,6 +1038,17 @@ ngx_http_variable_unknown_header(ngx_htt
     v->not_found = 0;
     v->data = p;
 
+    if (h->key.len == sizeof("Set-Cookie") - 1
+        && ngx_strncasecmp(h->key.data, (u_char *) "Set-Cookie",
+                           sizeof("Cookie") - 1)
+           == 0)
+    {
+        sep = ';';
+
+    } else {
+        sep = ',';
+    }
+
     for ( ;; ) {
 
         p = ngx_copy(p, h->value.data, h->value.len);
@@ -1046,7 +1057,7 @@ ngx_http_variable_unknown_header(ngx_htt
             break;
         }
 
-        *p++ = ','; *p++ = ' ';
+        *p++ = sep; *p++ = ' ';
 
         h = h->next;
     }

> +
> +        h = h->next;
> +    }
>  
>      return NGX_OK;
>  }
> @@ -1879,7 +1931,7 @@ ngx_http_variable_sent_location(ngx_http
>  
>      ngx_str_set(&name, "sent_http_location");
>  
> -    return ngx_http_variable_unknown_header(v, &name,
> +    return ngx_http_variable_unknown_header(r, v, &name,
>                                              &r->headers_out.headers.part,
>                                              sizeof("sent_http_") - 1);
>  }
> diff --git a/src/http/ngx_http_variables.h b/src/http/ngx_http_variables.h
> --- a/src/http/ngx_http_variables.h
> +++ b/src/http/ngx_http_variables.h
> @@ -57,8 +57,9 @@ ngx_http_variable_value_t *ngx_http_get_
>  ngx_http_variable_value_t *ngx_http_get_variable(ngx_http_request_t *r,
>      ngx_str_t *name, ngx_uint_t key);
>  
> -ngx_int_t ngx_http_variable_unknown_header(ngx_http_variable_value_t *v,
> -    ngx_str_t *var, ngx_list_part_t *part, size_t prefix);
> +ngx_int_t ngx_http_variable_unknown_header(ngx_http_request_t *r,
> +    ngx_http_variable_value_t *v, ngx_str_t *var, ngx_list_part_t *part,
> +    size_t prefix);
>  
>  
>  #if (NGX_PCRE)
> 


From pluknet at nginx.com  Wed May 11 19:21:15 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Wed, 11 May 2022 23:21:15 +0400
Subject: [PATCH 06 of 20] Reworked multi headers to use linked lists
In-Reply-To: <50fe52f516ff9c148aa9.1650493126@vm-bsd.mdounin.ru>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <50fe52f516ff9c148aa9.1650493126@vm-bsd.mdounin.ru>
Message-ID: <20220511192115.h3openel4fzpzikd@Y9MQ9X2QVV>

On Thu, Apr 21, 2022 at 01:18:46AM +0300, Maxim Dounin wrote:
> # HG changeset patch
> # User Maxim Dounin <mdounin at mdounin.ru>
> # Date 1650492323 -10800
> #      Thu Apr 21 01:05:23 2022 +0300
> # Node ID 50fe52f516ff9c148aa9e7dfcc1c31cc6a4929ae
> # Parent  2ea48b5e4643a818cd81179f040f0b36be9050d6
> Reworked multi headers to use linked lists.
> 
> Multi headers are now using linked lists instead of arrays.  Notably,
> the following fields were changed: r->headers_in.cookies (renamed
> to r->headers_in.cookie), r->headers_in.x_forwarded_for,
> r->headers_out.cache_control, r->headers_out.link, u->headers_in.cache_control
> u->headers_in.cookies (renamed to u->headers_in.set_cookie).

jftr, this implies updating a bunch of 3rd party modules, including njs, lua,
naxsi, nginx-clojure, headers-more, geoip2_module, ngx_http_sticky_module

> 
> The r->headers_in.cookies and u->headers_in.cookies fields were renamed
> to r->headers_in.cookie and u->headers_in.set_cookie to match header names.
> 
> The ngx_http_parse_multi_header_lines() and ngx_http_parse_set_cookie_lines()
> functions were changed accordingly.
> 
> With this change, multi headers are now essentially equivalent to normal
> headers, and following changes will further make them equivalent.
> 
> diff --git a/src/http/modules/ngx_http_geo_module.c b/src/http/modules/ngx_http_geo_module.c
> --- a/src/http/modules/ngx_http_geo_module.c
> +++ b/src/http/modules/ngx_http_geo_module.c
> @@ -327,15 +327,15 @@ static ngx_int_t
>  ngx_http_geo_addr(ngx_http_request_t *r, ngx_http_geo_ctx_t *ctx,
>      ngx_addr_t *addr)
>  {
> -    ngx_array_t  *xfwd;
> +    ngx_table_elt_t  *xfwd;
>  
>      if (ngx_http_geo_real_addr(r, ctx, addr) != NGX_OK) {
>          return NGX_ERROR;
>      }
>  
> -    xfwd = &r->headers_in.x_forwarded_for;
> +    xfwd = r->headers_in.x_forwarded_for;
>  
> -    if (xfwd->nelts > 0 && ctx->proxies != NULL) {
> +    if (xfwd != NULL && ctx->proxies != NULL) {
>          (void) ngx_http_get_forwarded_addr(r, addr, xfwd, NULL,
>                                             ctx->proxies, ctx->proxy_recursive);
>      }
> diff --git a/src/http/modules/ngx_http_geoip_module.c b/src/http/modules/ngx_http_geoip_module.c
> --- a/src/http/modules/ngx_http_geoip_module.c
> +++ b/src/http/modules/ngx_http_geoip_module.c
> @@ -240,16 +240,16 @@ static u_long
>  ngx_http_geoip_addr(ngx_http_request_t *r, ngx_http_geoip_conf_t *gcf)
>  {
>      ngx_addr_t           addr;
> -    ngx_array_t         *xfwd;
> +    ngx_table_elt_t     *xfwd;
>      struct sockaddr_in  *sin;
>  
>      addr.sockaddr = r->connection->sockaddr;
>      addr.socklen = r->connection->socklen;
>      /* addr.name = r->connection->addr_text; */
>  
> -    xfwd = &r->headers_in.x_forwarded_for;
> +    xfwd = r->headers_in.x_forwarded_for;
>  
> -    if (xfwd->nelts > 0 && gcf->proxies != NULL) {
> +    if (xfwd != NULL && gcf->proxies != NULL) {
>          (void) ngx_http_get_forwarded_addr(r, &addr, xfwd, NULL,
>                                             gcf->proxies, gcf->proxy_recursive);
>      }
> @@ -292,7 +292,7 @@ static geoipv6_t
>  ngx_http_geoip_addr_v6(ngx_http_request_t *r, ngx_http_geoip_conf_t *gcf)
>  {
>      ngx_addr_t            addr;
> -    ngx_array_t          *xfwd;
> +    ngx_table_elt_t      *xfwd;
>      in_addr_t             addr4;
>      struct in6_addr       addr6;
>      struct sockaddr_in   *sin;
> @@ -302,9 +302,9 @@ ngx_http_geoip_addr_v6(ngx_http_request_
>      addr.socklen = r->connection->socklen;
>      /* addr.name = r->connection->addr_text; */
>  
> -    xfwd = &r->headers_in.x_forwarded_for;
> +    xfwd = r->headers_in.x_forwarded_for;
>  
> -    if (xfwd->nelts > 0 && gcf->proxies != NULL) {
> +    if (xfwd != NULL && gcf->proxies != NULL) {
>          (void) ngx_http_get_forwarded_addr(r, &addr, xfwd, NULL,
>                                             gcf->proxies, gcf->proxy_recursive);
>      }
> diff --git a/src/http/modules/ngx_http_headers_filter_module.c b/src/http/modules/ngx_http_headers_filter_module.c
> --- a/src/http/modules/ngx_http_headers_filter_module.c
> +++ b/src/http/modules/ngx_http_headers_filter_module.c
> @@ -329,8 +329,7 @@ ngx_http_set_expires(ngx_http_request_t 
>      time_t               now, expires_time, max_age;
>      ngx_str_t            value;
>      ngx_int_t            rc;
> -    ngx_uint_t           i;
> -    ngx_table_elt_t     *e, *cc, **ccp;
> +    ngx_table_elt_t     *e, *cc;
>      ngx_http_expires_t   expires;
>  
>      expires = conf->expires;
> @@ -371,38 +370,28 @@ ngx_http_set_expires(ngx_http_request_t 
>      len = sizeof("Mon, 28 Sep 1970 06:00:00 GMT");
>      e->value.len = len - 1;
>  
> -    ccp = r->headers_out.cache_control.elts;
> -
> -    if (ccp == NULL) {
> +    cc = r->headers_out.cache_control;
>  
> -        if (ngx_array_init(&r->headers_out.cache_control, r->pool,
> -                           1, sizeof(ngx_table_elt_t *))
> -            != NGX_OK)
> -        {
> -            return NGX_ERROR;
> -        }
> +    if (cc == NULL) {
>  
>          cc = ngx_list_push(&r->headers_out.headers);
>          if (cc == NULL) {
>              return NGX_ERROR;
>          }
>  
> +        r->headers_out.cache_control = cc;
> +        cc->next = NULL;
> +
>          cc->hash = 1;
>          ngx_str_set(&cc->key, "Cache-Control");
>  
> -        ccp = ngx_array_push(&r->headers_out.cache_control);
> -        if (ccp == NULL) {
> -            return NGX_ERROR;
> +    } else {
> +        for (cc = cc->next; cc; cc = cc->next) {
> +            cc->hash = 0;
>          }
>  
> -        *ccp = cc;
> -
> -    } else {
> -        for (i = 1; i < r->headers_out.cache_control.nelts; i++) {
> -            ccp[i]->hash = 0;
> -        }
> -
> -        cc = ccp[0];
> +        cc = r->headers_out.cache_control;
> +        cc->next = NULL;
>      }
>  
>      if (expires == NGX_HTTP_EXPIRES_EPOCH) {
> @@ -564,22 +553,12 @@ static ngx_int_t
>  ngx_http_add_multi_header_lines(ngx_http_request_t *r,
>      ngx_http_header_val_t *hv, ngx_str_t *value)
>  {
> -    ngx_array_t      *pa;
>      ngx_table_elt_t  *h, **ph;
>  
>      if (value->len == 0) {
>          return NGX_OK;
>      }
>  
> -    pa = (ngx_array_t *) ((char *) &r->headers_out + hv->offset);
> -
> -    if (pa->elts == NULL) {
> -        if (ngx_array_init(pa, r->pool, 1, sizeof(ngx_table_elt_t *)) != NGX_OK)
> -        {
> -            return NGX_ERROR;
> -        }
> -    }
> -
>      h = ngx_list_push(&r->headers_out.headers);
>      if (h == NULL) {
>          return NGX_ERROR;
> @@ -589,12 +568,12 @@ ngx_http_add_multi_header_lines(ngx_http
>      h->key = hv->key;
>      h->value = *value;
>  
> -    ph = ngx_array_push(pa);
> -    if (ph == NULL) {
> -        return NGX_ERROR;
> -    }
> +    ph = (ngx_table_elt_t **) ((char *) &r->headers_out + hv->offset);
> +
> +    while (*ph) { ph = &(*ph)->next; }
>  
>      *ph = h;
> +    h->next = NULL;
>  
>      return NGX_OK;
>  }
> diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c
> --- a/src/http/modules/ngx_http_proxy_module.c
> +++ b/src/http/modules/ngx_http_proxy_module.c
> @@ -2559,22 +2559,20 @@ static ngx_int_t
>  ngx_http_proxy_add_x_forwarded_for_variable(ngx_http_request_t *r,
>      ngx_http_variable_value_t *v, uintptr_t data)
>  {
> -    size_t             len;
> -    u_char            *p;
> -    ngx_uint_t         i, n;
> -    ngx_table_elt_t  **h;
> +    size_t            len;
> +    u_char           *p;
> +    ngx_table_elt_t  *h, *xfwd;
>  
>      v->valid = 1;
>      v->no_cacheable = 0;
>      v->not_found = 0;
>  
> -    n = r->headers_in.x_forwarded_for.nelts;
> -    h = r->headers_in.x_forwarded_for.elts;
> +    xfwd = r->headers_in.x_forwarded_for;
>  
>      len = 0;
>  
> -    for (i = 0; i < n; i++) {
> -        len += h[i]->value.len + sizeof(", ") - 1;
> +    for (h = xfwd; h; h = h->next) {
> +        len += h->value.len + sizeof(", ") - 1;
>      }
>  
>      if (len == 0) {

The only minor point in a whole patch (which I won't insist):
a special case for xfwd absense can be slightly reshuffled now
to avoid a do-nothing loop.

diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c
--- a/src/http/modules/ngx_http_proxy_module.c
+++ b/src/http/modules/ngx_http_proxy_module.c
@@ -2569,18 +2569,18 @@ ngx_http_proxy_add_x_forwarded_for_varia
 
     xfwd = r->headers_in.x_forwarded_for;
 
+    if (xfwd == NULL) {
+        v->len = r->connection->addr_text.len;
+        v->data = r->connection->addr_text.data;
+        return NGX_OK;
+    }
+
     len = 0;
 
     for (h = xfwd; h; h = h->next) {
         len += h->value.len + sizeof(", ") - 1;
     }
 
-    if (len == 0) {
-        v->len = r->connection->addr_text.len;
-        v->data = r->connection->addr_text.data;
-        return NGX_OK;
-    }
-
     len += r->connection->addr_text.len;
 
     p = ngx_pnalloc(r->pool, len);

Otherwise, looks good.

[..]


From pluknet at nginx.com  Wed May 11 19:40:48 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Wed, 11 May 2022 23:40:48 +0400
Subject: [PATCH 07 of 20] All non-unique input headers are now linked
 lists
In-Reply-To: <238d1f5f438735432822.1650493127@vm-bsd.mdounin.ru>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <238d1f5f438735432822.1650493127@vm-bsd.mdounin.ru>
Message-ID: <20220511194048.df6wpfybtpn4dxjg@Y9MQ9X2QVV>

On Thu, Apr 21, 2022 at 01:18:47AM +0300, Maxim Dounin wrote:
> # HG changeset patch
> # User Maxim Dounin <mdounin at mdounin.ru>
> # Date 1650492324 -10800
> #      Thu Apr 21 01:05:24 2022 +0300
> # Node ID 238d1f5f438735432822689605fa37b1b0e01517
> # Parent  50fe52f516ff9c148aa9e7dfcc1c31cc6a4929ae
> All non-unique input headers are now linked lists.
> 
> The ngx_http_process_multi_header_lines() function is removed, as it is
> exactly equivalent to ngx_http_process_header_line().  Similarly,
> ngx_http_variable_header() is used instead of ngx_http_variable_headers().
> 
> diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
> --- a/src/http/ngx_http_request.c
> +++ b/src/http/ngx_http_request.c
> @@ -22,8 +22,6 @@ static ngx_int_t ngx_http_process_header
>      ngx_table_elt_t *h, ngx_uint_t offset);
>  static ngx_int_t ngx_http_process_unique_header_line(ngx_http_request_t *r,
>      ngx_table_elt_t *h, ngx_uint_t offset);
> -static ngx_int_t ngx_http_process_multi_header_lines(ngx_http_request_t *r,
> -    ngx_table_elt_t *h, ngx_uint_t offset);
>  static ngx_int_t ngx_http_process_host(ngx_http_request_t *r,
>      ngx_table_elt_t *h, ngx_uint_t offset);
>  static ngx_int_t ngx_http_process_connection(ngx_http_request_t *r,
> @@ -164,7 +162,7 @@ ngx_http_header_t  ngx_http_headers_in[]
>  #if (NGX_HTTP_X_FORWARDED_FOR)
>      { ngx_string("X-Forwarded-For"),
>                   offsetof(ngx_http_headers_in_t, x_forwarded_for),
> -                 ngx_http_process_multi_header_lines },
> +                 ngx_http_process_header_line },
>  #endif
>  
>  #if (NGX_HTTP_REALIP)
> @@ -197,7 +195,7 @@ ngx_http_header_t  ngx_http_headers_in[]
>  #endif
>  
>      { ngx_string("Cookie"), offsetof(ngx_http_headers_in_t, cookie),
> -                 ngx_http_process_multi_header_lines },
> +                 ngx_http_process_header_line },
>  
>      { ngx_null_string, 0, NULL }
>  };
> @@ -1742,10 +1740,10 @@ ngx_http_process_header_line(ngx_http_re
>  
>      ph = (ngx_table_elt_t **) ((char *) &r->headers_in + offset);
>  
> -    if (*ph == NULL) {
> -        *ph = h;
> -        h->next = NULL;
> -    }
> +    while (*ph) { ph = &(*ph)->next; }
> +
> +    *ph = h;
> +    h->next = NULL;
>  
>      return NGX_OK;
>  }

Essentially, this makes the following headers multi-valuable,
which are specified (or otherwise implied) to have a single value:
Referer, Content-Type, Range, Keep-Alive, X-Real-IP,
also DAV headers (including Date) and User-Agent (as below).
This makes proxying comma-delimited header values to backend, such as:
  Date: Wed, 11 May 2022 19:29:54 GMT
  Date: Wed, 11 May 2022 19:29:55 GMT
->
  HTTP_DATE: Wed, 11 May 2022 19:29:54 GMT, Wed, 11 May 2022 19:29:55 GMT

Convert them to ngx_http_process_unique_header_line() ?

> @@ -1851,13 +1849,10 @@ ngx_http_process_user_agent(ngx_http_req
>  {
>      u_char  *user_agent, *msie;
>  
> -    if (r->headers_in.user_agent) {
> -        return NGX_OK;
> +    if (ngx_http_process_header_line(r, h, offset) != NGX_OK) {
> +        return NGX_ERROR;
>      }
>  
> -    r->headers_in.user_agent = h;
> -    h->next = NULL;
> -
>      /* check some widespread browsers while the header is in CPU cache */
>  
>      user_agent = h->value.data;
> @@ -1919,23 +1914,6 @@ ngx_http_process_user_agent(ngx_http_req
>  }
>  
>  
> -static ngx_int_t
> -ngx_http_process_multi_header_lines(ngx_http_request_t *r, ngx_table_elt_t *h,
> -    ngx_uint_t offset)
> -{
> -    ngx_table_elt_t  **ph;
> -
> -    ph = (ngx_table_elt_t **) ((char *) &r->headers_in + offset);
> -
> -    while (*ph) { ph = &(*ph)->next; }
> -
> -    *ph = h;
> -    h->next = NULL;
> -
> -    return NGX_OK;
> -}
> -
> -
>  ngx_int_t
>  ngx_http_process_request_header(ngx_http_request_t *r)
>  {
> diff --git a/src/http/ngx_http_variables.c b/src/http/ngx_http_variables.c
> --- a/src/http/ngx_http_variables.c
> +++ b/src/http/ngx_http_variables.c
> @@ -27,8 +27,6 @@ static ngx_int_t ngx_http_variable_heade
>  
>  static ngx_int_t ngx_http_variable_cookies(ngx_http_request_t *r,
>      ngx_http_variable_value_t *v, uintptr_t data);
> -static ngx_int_t ngx_http_variable_headers(ngx_http_request_t *r,
> -    ngx_http_variable_value_t *v, uintptr_t data);
>  static ngx_int_t ngx_http_variable_headers_internal(ngx_http_request_t *r,
>      ngx_http_variable_value_t *v, uintptr_t data, u_char sep);
>  
> @@ -178,7 +176,7 @@ static ngx_http_variable_t  ngx_http_cor
>  #endif
>  
>  #if (NGX_HTTP_X_FORWARDED_FOR)
> -    { ngx_string("http_x_forwarded_for"), NULL, ngx_http_variable_headers,
> +    { ngx_string("http_x_forwarded_for"), NULL, ngx_http_variable_header,
>        offsetof(ngx_http_request_t, headers_in.x_forwarded_for), 0, 0 },
>  #endif
>  
> @@ -327,10 +325,10 @@ static ngx_http_variable_t  ngx_http_cor
>      { ngx_string("sent_http_transfer_encoding"), NULL,
>        ngx_http_variable_sent_transfer_encoding, 0, 0, 0 },
>  
> -    { ngx_string("sent_http_cache_control"), NULL, ngx_http_variable_headers,
> +    { ngx_string("sent_http_cache_control"), NULL, ngx_http_variable_header,
>        offsetof(ngx_http_request_t, headers_out.cache_control), 0, 0 },
>  
> -    { ngx_string("sent_http_link"), NULL, ngx_http_variable_headers,
> +    { ngx_string("sent_http_link"), NULL, ngx_http_variable_header,
>        offsetof(ngx_http_request_t, headers_out.link), 0, 0 },
>  
>      { ngx_string("limit_rate"), ngx_http_variable_set_limit_rate,
> @@ -807,22 +805,7 @@ static ngx_int_t
>  ngx_http_variable_header(ngx_http_request_t *r, ngx_http_variable_value_t *v,
>      uintptr_t data)
>  {
> -    ngx_table_elt_t  *h;
> -
> -    h = *(ngx_table_elt_t **) ((char *) r + data);
> -
> -    if (h) {
> -        v->len = h->value.len;
> -        v->valid = 1;
> -        v->no_cacheable = 0;
> -        v->not_found = 0;
> -        v->data = h->value.data;
> -
> -    } else {
> -        v->not_found = 1;
> -    }
> -
> -    return NGX_OK;
> +    return ngx_http_variable_headers_internal(r, v, data, ',');
>  }
>  
>  
> @@ -835,14 +818,6 @@ ngx_http_variable_cookies(ngx_http_reque
>  
>  
>  static ngx_int_t
> -ngx_http_variable_headers(ngx_http_request_t *r,
> -    ngx_http_variable_value_t *v, uintptr_t data)
> -{
> -    return ngx_http_variable_headers_internal(r, v, data, ',');
> -}
> -
> -
> -static ngx_int_t
>  ngx_http_variable_headers_internal(ngx_http_request_t *r,
>      ngx_http_variable_value_t *v, uintptr_t data, u_char sep)
>  {
> 


From pluknet at nginx.com  Wed May 11 20:00:32 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Thu, 12 May 2022 00:00:32 +0400
Subject: [PATCH 11 of 20] Upstream: simplified Content-Encoding handling
In-Reply-To: <6441069e16a0c4755c66.1650493131@vm-bsd.mdounin.ru>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <6441069e16a0c4755c66.1650493131@vm-bsd.mdounin.ru>
Message-ID: <20220511200032.2mlm2ofg2ivgzuae@Y9MQ9X2QVV>

On Thu, Apr 21, 2022 at 01:18:51AM +0300, Maxim Dounin wrote:
> # HG changeset patch
> # User Maxim Dounin <mdounin at mdounin.ru>
> # Date 1650492330 -10800
> #      Thu Apr 21 01:05:30 2022 +0300
> # Node ID 6441069e16a0c4755c662bc07bdcb0960c9ac04a
> # Parent  ee1a8a4aa2c262d25a4aa871cda4f6c4515fc85c
> Upstream: simplified Content-Encoding handling.
> 
> Since introduction of offset handling in ngx_http_upstream_copy_header_line()
> in revision 573:58475592100c, the ngx_http_upstream_copy_content_encoding()
> function is no longer needed, as its behaviour is exactly equivalent to
> ngx_http_upstream_copy_header_line() with appropriate offset.  As such,
> the ngx_http_upstream_copy_content_encoding() function was removed.
> 
> Further, the u->headers_in.content_encoding field is not used anywhere,
> so it was removed as well.

jftr, it's used (seemingly incorrect) in ngx_http_redis for gzip decompression
https://github.com/onnimonni/redis-nginx-module/commit/cbe2630fd070
It might be better to improve gunzip filter itself as marked in TODO.

> 
> Further, Content-Encoding handling no longer depends on NGX_HTTP_GZIP,
> as it can be used even without any gzip handling compiled in (for example,
> in the charset filter).
> 
> diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
> --- a/src/http/ngx_http_upstream.c
> +++ b/src/http/ngx_http_upstream.c
> @@ -147,11 +147,6 @@ static ngx_int_t ngx_http_upstream_rewri
>  static ngx_int_t ngx_http_upstream_copy_allow_ranges(ngx_http_request_t *r,
>      ngx_table_elt_t *h, ngx_uint_t offset);
>  
> -#if (NGX_HTTP_GZIP)
> -static ngx_int_t ngx_http_upstream_copy_content_encoding(ngx_http_request_t *r,
> -    ngx_table_elt_t *h, ngx_uint_t offset);
> -#endif
> -
>  static ngx_int_t ngx_http_upstream_add_variables(ngx_conf_t *cf);
>  static ngx_int_t ngx_http_upstream_addr_variable(ngx_http_request_t *r,
>      ngx_http_variable_value_t *v, uintptr_t data);
> @@ -316,12 +311,10 @@ static ngx_http_upstream_header_t  ngx_h
>                   ngx_http_upstream_process_transfer_encoding, 0,
>                   ngx_http_upstream_ignore_header_line, 0, 0 },
>  
> -#if (NGX_HTTP_GZIP)
>      { ngx_string("Content-Encoding"),
> -                 ngx_http_upstream_process_header_line,
> -                 offsetof(ngx_http_upstream_headers_in_t, content_encoding),
> -                 ngx_http_upstream_copy_content_encoding, 0, 0 },
> -#endif
> +                 ngx_http_upstream_ignore_header_line, 0,
> +                 ngx_http_upstream_copy_header_line,
> +                 offsetof(ngx_http_headers_out_t, content_encoding), 0 },
>  
>      { ngx_null_string, NULL, 0, NULL, 0, 0 }
>  };
> @@ -5349,29 +5342,6 @@ ngx_http_upstream_copy_allow_ranges(ngx_
>  }
>  
>  
> -#if (NGX_HTTP_GZIP)
> -
> -static ngx_int_t
> -ngx_http_upstream_copy_content_encoding(ngx_http_request_t *r,
> -    ngx_table_elt_t *h, ngx_uint_t offset)
> -{
> -    ngx_table_elt_t  *ho;
> -
> -    ho = ngx_list_push(&r->headers_out.headers);
> -    if (ho == NULL) {
> -        return NGX_ERROR;
> -    }
> -
> -    *ho = *h;
> -
> -    r->headers_out.content_encoding = ho;
> -
> -    return NGX_OK;
> -}
> -
> -#endif
> -
> -
>  static ngx_int_t
>  ngx_http_upstream_add_variables(ngx_conf_t *cf)
>  {
> diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h
> --- a/src/http/ngx_http_upstream.h
> +++ b/src/http/ngx_http_upstream.h
> @@ -285,10 +285,6 @@ typedef struct {
>      ngx_table_elt_t                 *transfer_encoding;
>      ngx_table_elt_t                 *vary;
>  
> -#if (NGX_HTTP_GZIP)
> -    ngx_table_elt_t                 *content_encoding;
> -#endif
> -
>      ngx_table_elt_t                 *cache_control;
>      ngx_table_elt_t                 *set_cookie;
>  
> 


From pluknet at nginx.com  Wed May 11 20:26:37 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Thu, 12 May 2022 00:26:37 +0400
Subject: [PATCH 15 of 20] Upstream: header handlers can now return
 parsing errors
In-Reply-To: <ab424b5e32405aeec54c.1650493135@vm-bsd.mdounin.ru>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <ab424b5e32405aeec54c.1650493135@vm-bsd.mdounin.ru>
Message-ID: <20220511202637.4kawhatgl6xcofbt@Y9MQ9X2QVV>

On Thu, Apr 21, 2022 at 01:18:55AM +0300, Maxim Dounin wrote:
> # HG changeset patch
> # User Maxim Dounin <mdounin at mdounin.ru>
> # Date 1650492336 -10800
> #      Thu Apr 21 01:05:36 2022 +0300
> # Node ID ab424b5e32405aeec54ccdfe38e9408209209e0a
> # Parent  b110c54778e8f6af3ea402c0838a4f289dcd813e
> Upstream: header handlers can now return parsing errors.
> 
> With this change, duplicate Content-Length and Transfer-Encoding headers
> are now rejected.  Further, responses with invalid Content-Length or
> Transfer-Encoding headers are now rejected, as well as responses with both
> Content-Length and Transfer-Encoding.

jftr, various 3rd party modules that call header handlers:
mogilefs, passenger, ajp_module, nginx-clojure, srcache

> 
> diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c
> --- a/src/http/modules/ngx_http_fastcgi_module.c
> +++ b/src/http/modules/ngx_http_fastcgi_module.c
> @@ -2007,8 +2007,12 @@ ngx_http_fastcgi_process_header(ngx_http
>                  hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
>                                     h->lowcase_key, h->key.len);
>  
> -                if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
> -                    return NGX_ERROR;
> +                if (hh) {
> +                    rc = hh->handler(r, h, hh->offset);
> +
> +                    if (rc != NGX_OK) {
> +                        return rc;
> +                    }
>                  }
>  
>                  ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
> diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c
> --- a/src/http/modules/ngx_http_grpc_module.c
> +++ b/src/http/modules/ngx_http_grpc_module.c
> @@ -1891,8 +1891,12 @@ ngx_http_grpc_process_header(ngx_http_re
>                  hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
>                                     h->lowcase_key, h->key.len);
>  
> -                if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
> -                    return NGX_ERROR;
> +                if (hh) {
> +                    rc = hh->handler(r, h, hh->offset);
> +
> +                    if (rc != NGX_OK) {
> +                        return rc;
> +                    }
>                  }
>  
>                  continue;
> diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c
> --- a/src/http/modules/ngx_http_proxy_module.c
> +++ b/src/http/modules/ngx_http_proxy_module.c
> @@ -1930,8 +1930,12 @@ ngx_http_proxy_process_header(ngx_http_r
>              hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
>                                 h->lowcase_key, h->key.len);
>  
> -            if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
> -                return NGX_ERROR;
> +            if (hh) {
> +                rc = hh->handler(r, h, hh->offset);
> +
> +                if (rc != NGX_OK) {
> +                    return rc;
> +                }
>              }
>  
>              ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
> diff --git a/src/http/modules/ngx_http_scgi_module.c b/src/http/modules/ngx_http_scgi_module.c
> --- a/src/http/modules/ngx_http_scgi_module.c
> +++ b/src/http/modules/ngx_http_scgi_module.c
> @@ -1114,8 +1114,12 @@ ngx_http_scgi_process_header(ngx_http_re
>              hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
>                                 h->lowcase_key, h->key.len);
>  
> -            if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
> -                return NGX_ERROR;
> +            if (hh) {
> +                rc = hh->handler(r, h, hh->offset);
> +
> +                if (rc != NGX_OK) {
> +                    return rc;
> +                }
>              }
>  
>              ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
> diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c
> --- a/src/http/modules/ngx_http_uwsgi_module.c
> +++ b/src/http/modules/ngx_http_uwsgi_module.c
> @@ -1340,8 +1340,12 @@ ngx_http_uwsgi_process_header(ngx_http_r
>              hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
>                                 h->lowcase_key, h->key.len);
>  
> -            if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
> -                return NGX_ERROR;
> +            if (hh) {
> +                rc = hh->handler(r, h, hh->offset);
> +
> +                if (rc != NGX_OK) {
> +                    return rc;
> +                }
>              }
>  
>              ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
> diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
> --- a/src/http/ngx_http_upstream.c
> +++ b/src/http/ngx_http_upstream.c
> @@ -4633,10 +4633,34 @@ ngx_http_upstream_process_content_length
>  
>      u = r->upstream;
>  
> +    if (u->headers_in.content_length) {
> +        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> +                      "upstream sent duplicate header line: \"%V: %V\", "
> +                      "previous value: \"%V: %V\"",
> +                      &h->key, &h->value,
> +                      &u->headers_in.content_length->key,
> +                      &u->headers_in.content_length->value);
> +        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
> +    }
> +
> +    if (u->headers_in.transfer_encoding) {
> +        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> +                      "upstream sent \"Content-Length\" and "
> +                      "\"Transfer-Encoding\" headers at the same time");
> +        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
> +    }
> +
>      h->next = NULL;
>      u->headers_in.content_length = h;
>      u->headers_in.content_length_n = ngx_atoof(h->value.data, h->value.len);
>  
> +    if (u->headers_in.content_length_n == NGX_ERROR) {
> +        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> +                      "upstream sent invalid \"Content-Length\" header: "
> +                      "\"%V: %V\"", &h->key, &h->value);
> +        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
> +    }
> +
>      return NGX_OK;
>  }
>  
> @@ -5021,14 +5045,37 @@ ngx_http_upstream_process_transfer_encod
>      ngx_http_upstream_t  *u;
>  
>      u = r->upstream;
> +
> +    if (u->headers_in.transfer_encoding) {
> +        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> +                      "upstream sent duplicate header line: \"%V: %V\", "
> +                      "previous value: \"%V: %V\"",
> +                      &h->key, &h->value,
> +                      &u->headers_in.transfer_encoding->key,
> +                      &u->headers_in.transfer_encoding->value);
> +        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
> +    }
> +
> +    if (u->headers_in.content_length) {
> +        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> +                      "upstream sent \"Content-Length\" and "
> +                      "\"Transfer-Encoding\" headers at the same time");
> +        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
> +    }
> +
>      u->headers_in.transfer_encoding = h;
>      h->next = NULL;
>  
> -    if (ngx_strlcasestrn(h->value.data, h->value.data + h->value.len,
> -                         (u_char *) "chunked", 7 - 1)
> -        != NULL)
> +    if (h->value.len == 7
> +        && ngx_strncasecmp(h->value.data, (u_char *) "chunked", 7) == 0)
>      {
>          u->headers_in.chunked = 1;
> +
> +    } else {
> +        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> +                      "upstream sent unknown \"Transfer-Encoding\": \"%V\"",
> +                      &h->value);
> +        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
>      }
>  
>      return NGX_OK;

This is a subtle(?) change, which makes "chunked" the only valid header value.
OTOH, it looks fine since other values require explicit support as well.


From pluknet at nginx.com  Wed May 11 20:33:53 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Thu, 12 May 2022 00:33:53 +0400
Subject: [PATCH 16 of 20] Upstream: duplicate headers ignored or properly
 linked
In-Reply-To: <f460a2f9f88d264ef6c8.1650493136@vm-bsd.mdounin.ru>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <f460a2f9f88d264ef6c8.1650493136@vm-bsd.mdounin.ru>
Message-ID: <20220511203353.jvagykody7lfkhuv@Y9MQ9X2QVV>

On Thu, Apr 21, 2022 at 01:18:56AM +0300, Maxim Dounin wrote:
> # HG changeset patch
> # User Maxim Dounin <mdounin at mdounin.ru>
> # Date 1650492338 -10800
> #      Thu Apr 21 01:05:38 2022 +0300
> # Node ID f460a2f9f88d264ef6c8588eb37bcb85c48010db
> # Parent  ab424b5e32405aeec54ccdfe38e9408209209e0a
> Upstream: duplicate headers ignored or properly linked.
> 
> Most of the known duplicate upstream response headers are now ignored
> with a warning.
> 
> If syntax permits multiple headers, these are now properly linked to
> the lists, notably Vary and WWW-Authenticate.  This makes it possible
> to further handle such lists where it makes sense.

This reminds me of curl deficiency to handle WWW-Authenticate
as a comma-separate multi-value header (which nginx doesn't emit though):
https://curl.se/mail/lib-2020-01/0064.html
https://curl.se/docs/knownbugs.html

> 
> diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
> --- a/src/http/ngx_http_upstream.c
> +++ b/src/http/ngx_http_upstream.c
> @@ -101,6 +101,9 @@ static void ngx_http_upstream_finalize_r
>  
>  static ngx_int_t ngx_http_upstream_process_header_line(ngx_http_request_t *r,
>      ngx_table_elt_t *h, ngx_uint_t offset);
> +static ngx_int_t
> +    ngx_http_upstream_process_multi_header_lines(ngx_http_request_t *r,
> +    ngx_table_elt_t *h, ngx_uint_t offset);

I'd suggest renaming this pair to ngx_http_upstream_process_header_line /
ngx_http_upstream_process_unique_header_line to be on par with functions
in ngx_http_request.c (and after the 7th renaming patch of this series).

[..]


From pluknet at nginx.com  Wed May 11 20:44:52 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Thu, 12 May 2022 00:44:52 +0400
Subject: [PATCH 17 of 20] Upstream: handling of multiple Vary headers
 (ticket #1423)
In-Reply-To: <2027b85971d4b8a7e33c.1650493137@vm-bsd.mdounin.ru>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <2027b85971d4b8a7e33c.1650493137@vm-bsd.mdounin.ru>
Message-ID: <20220511204452.xycprosojrnmsmrd@Y9MQ9X2QVV>

On Thu, Apr 21, 2022 at 01:18:57AM +0300, Maxim Dounin wrote:
> # HG changeset patch
> # User Maxim Dounin <mdounin at mdounin.ru>
> # Date 1650492340 -10800
> #      Thu Apr 21 01:05:40 2022 +0300
> # Node ID 2027b85971d4b8a7e33c018548468057cb57eaf7
> # Parent  f460a2f9f88d264ef6c8588eb37bcb85c48010db
> Upstream: handling of multiple Vary headers (ticket #1423).
> 
> Previously, only the last header value was used when caching.
> 
> diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
> --- a/src/http/ngx_http_upstream.c
> +++ b/src/http/ngx_http_upstream.c
> @@ -5175,6 +5175,9 @@ static ngx_int_t
>  ngx_http_upstream_process_vary(ngx_http_request_t *r,
>      ngx_table_elt_t *h, ngx_uint_t offset)
>  {
> +    u_char                *p;
> +    size_t                 len;
> +    ngx_str_t              vary;
>      ngx_table_elt_t      **ph;
>      ngx_http_upstream_t   *u;
>  
> @@ -5192,17 +5195,47 @@ ngx_http_upstream_process_vary(ngx_http_
>          return NGX_OK;
>      }
>  
> -    if (r->cache == NULL) {
> +    if (r->cache == NULL || !u->cacheable) {
> +        return NGX_OK;
> +    }
> +
> +    if (h->value.len == 1 && h->value.data[0] == '*') {
> +        u->cacheable = 0;
>          return NGX_OK;
>      }
>  
> -    if (h->value.len > NGX_HTTP_CACHE_VARY_LEN
> -        || (h->value.len == 1 && h->value.data[0] == '*'))
> -    {
> +    if (u->headers_in.vary->next) {
> +
> +        len = 0;
> +
> +        for (h = u->headers_in.vary; h; h = h->next) {
> +            len += h->value.len + 2;
> +        }
> +
> +        len -= 2;
> +
> +        p = ngx_pnalloc(r->pool, len);
> +        if (p == NULL) {
> +            return NGX_ERROR;
> +        }
> +
> +        vary.len = len;
> +        vary.data = p;
> +
> +        for (h = u->headers_in.vary; h; h = h->next) {
> +            p = ngx_copy(p, h->value.data, h->value.len);

(h->next == NULL) check missing

> +            *p++ = ','; *p++ = ' ';
> +        }
> +
> +    } else {
> +        vary = h->value;
> +    }
> +
> +    if (vary.len > NGX_HTTP_CACHE_VARY_LEN) {
>          u->cacheable = 0;
>      }
>  
> -    r->cache->vary = h->value;
> +    r->cache->vary = vary;
>  
>  #endif
>  

Memory from the pool could be allocated just once among multiple values
if postpone it to somewhere else (but probably it's not worth it).

Combined patch on top off:

# HG changeset patch
# User Sergey Kandaurov <pluknet at nginx.com>
# Date 1652193477 -14400
#      Tue May 10 18:37:57 2022 +0400
# Node ID cffc4a23b8bb69143d4128b2249b0b87d5fd68cb
# Parent  facf698a696385b7b07abc0b2f7e5c1f6394a486
imported patch p17b

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -55,6 +55,8 @@ static ngx_int_t ngx_http_upstream_inter
 static ngx_int_t ngx_http_upstream_test_connect(ngx_connection_t *c);
 static ngx_int_t ngx_http_upstream_process_headers(ngx_http_request_t *r,
     ngx_http_upstream_t *u);
+static ngx_int_t ngx_http_upstream_process_vary_headers(ngx_http_request_t *r,
+    ngx_http_upstream_t *u);
 static ngx_int_t ngx_http_upstream_process_trailers(ngx_http_request_t *r,
     ngx_http_upstream_t *u);
 static void ngx_http_upstream_send_response(ngx_http_request_t *r,
@@ -2788,6 +2790,12 @@ ngx_http_upstream_process_headers(ngx_ht
 
     umcf = ngx_http_get_module_main_conf(r, ngx_http_upstream_module);
 
+    if (ngx_http_upstream_process_vary_headers(r, u) != NGX_OK) {
+        ngx_http_upstream_finalize_request(r, u,
+                                           NGX_HTTP_INTERNAL_SERVER_ERROR);
+        return NGX_DONE;
+    }
+
     if (u->headers_in.x_accel_redirect
         && !(u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_XA_REDIRECT))
     {
@@ -2928,6 +2936,73 @@ ngx_http_upstream_process_headers(ngx_ht
 
 
 static ngx_int_t
+ngx_http_upstream_process_vary_headers(ngx_http_request_t *r,
+    ngx_http_upstream_t *u)
+{
+    u_char           *p;
+    size_t            len;
+    ngx_str_t         vary;
+    ngx_table_elt_t  *h;
+
+#if (NGX_HTTP_CACHE)
+
+    if (u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_VARY) {
+        return NGX_OK;
+    }
+
+    if (r->cache == NULL || !u->cacheable) {
+        return NGX_OK;
+    }
+
+    if (u->headers_in.vary == NULL) {
+        return NGX_OK;
+    }
+
+    if (u->headers_in.vary->next) {
+
+        len = 0;
+
+        for (h = u->headers_in.vary; h; h = h->next) {
+            len += h->value.len + 2;
+        }
+
+        len -= 2;
+
+        p = ngx_pnalloc(r->pool, len);
+        if (p == NULL) {
+            return NGX_ERROR;
+        }
+
+        vary.len = len;
+        vary.data = p;
+
+        for (h = u->headers_in.vary; h; h = h->next) {
+            p = ngx_copy(p, h->value.data, h->value.len);
+
+            if (h->next == NULL) {
+                break;
+            }
+
+            *p++ = ','; *p++ = ' ';
+        }
+
+    } else {
+        vary = u->headers_in.vary->value;
+    }
+
+    if (vary.len > NGX_HTTP_CACHE_VARY_LEN) {
+        u->cacheable = 0;
+    }
+
+    r->cache->vary = vary;
+
+#endif
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
 ngx_http_upstream_process_trailers(ngx_http_request_t *r,
     ngx_http_upstream_t *u)
 {
@@ -5175,9 +5250,6 @@ static ngx_int_t
 ngx_http_upstream_process_vary(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset)
 {
-    u_char                *p;
-    size_t                 len;
-    ngx_str_t              vary;
     ngx_table_elt_t      **ph;
     ngx_http_upstream_t   *u;
 
@@ -5189,60 +5261,9 @@ ngx_http_upstream_process_vary(ngx_http_
     *ph = h;
     h->next = NULL;
 
-#if (NGX_HTTP_CACHE)
-
-    if (u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_VARY) {
-        return NGX_OK;
-    }
-
-    if (r->cache == NULL || !u->cacheable) {
-        return NGX_OK;
-    }
-
     if (h->value.len == 1 && h->value.data[0] == '*') {
         u->cacheable = 0;
-        return NGX_OK;
-    }
-
-    if (u->headers_in.vary->next) {
-
-        len = 0;
-
-        for (h = u->headers_in.vary; h; h = h->next) {
-            len += h->value.len + 2;
-        }
-
-        len -= 2;
-
-        p = ngx_pnalloc(r->pool, len);
-        if (p == NULL) {
-            return NGX_ERROR;
-        }
-
-        vary.len = len;
-        vary.data = p;
-
-        for (h = u->headers_in.vary; h; h = h->next) {
-            p = ngx_copy(p, h->value.data, h->value.len);
-
-            if (h->next == NULL) {
-                break;
-            }
-
-            *p++ = ','; *p++ = ' ';
-        }
-
-    } else {
-        vary = h->value;
-    }
-
-    if (vary.len > NGX_HTTP_CACHE_VARY_LEN) {
-        u->cacheable = 0;
-    }
-
-    r->cache->vary = vary;
-
-#endif
+    }
 
     return NGX_OK;
 }


From pluknet at nginx.com  Wed May 11 21:03:37 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Thu, 12 May 2022 01:03:37 +0400
Subject: [PATCH 18 of 20] Upstream: multiple WWW-Authenticate headers
 (ticket #485)
In-Reply-To: <fa9751ffe7723a11159c.1650493138@vm-bsd.mdounin.ru>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <fa9751ffe7723a11159c.1650493138@vm-bsd.mdounin.ru>
Message-ID: <20220511210337.nhxkr2alnri4immp@Y9MQ9X2QVV>

On Thu, Apr 21, 2022 at 01:18:58AM +0300, Maxim Dounin wrote:
> # HG changeset patch
> # User Maxim Dounin <mdounin at mdounin.ru>
> # Date 1650492341 -10800
> #      Thu Apr 21 01:05:41 2022 +0300
> # Node ID fa9751ffe7723a11159c158078e454671e81cb87
> # Parent  2027b85971d4b8a7e33c018548468057cb57eaf7
> Upstream: multiple WWW-Authenticate headers (ticket #485).
> 
> When using proxy_intercept_errors and an error page for error 401
> (Unauthorized), multiple WWW-Authenticate headers from the upstream server
> response are now properly copied to the response.
> 
> diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
> --- a/src/http/ngx_http_upstream.c
> +++ b/src/http/ngx_http_upstream.c
> @@ -2647,7 +2647,7 @@ ngx_http_upstream_intercept_errors(ngx_h
>  {
>      ngx_int_t                  status;
>      ngx_uint_t                 i;
> -    ngx_table_elt_t           *h;
> +    ngx_table_elt_t           *h, *ho, **ph;
>      ngx_http_err_page_t       *err_page;
>      ngx_http_core_loc_conf_t  *clcf;
>  
> @@ -2676,18 +2676,26 @@ ngx_http_upstream_intercept_errors(ngx_h
>              if (status == NGX_HTTP_UNAUTHORIZED
>                  && u->headers_in.www_authenticate)
>              {
> -                h = ngx_list_push(&r->headers_out.headers);
> -
> -                if (h == NULL) {
> -                    ngx_http_upstream_finalize_request(r, u,
> +                h = u->headers_in.www_authenticate;
> +                ph = &r->headers_out.www_authenticate;
> +
> +                while (h) {
> +                    ho = ngx_list_push(&r->headers_out.headers);
> +
> +                    if (ho == NULL) {
> +                        ngx_http_upstream_finalize_request(r, u,
>                                                 NGX_HTTP_INTERNAL_SERVER_ERROR);
> -                    return NGX_OK;
> +                        return NGX_OK;
> +                    }
> +
> +                    *ho = *h;
> +                    ho->next = NULL;
> +
> +                    *ph = ho;
> +                    ph = &ho->next;
> +
> +                    h = h->next;
>                  }
> -
> -                *h = *u->headers_in.www_authenticate;
> -                h->next = NULL;
> -
> -                r->headers_out.www_authenticate = h;
>              }
>  
>  #if (NGX_HTTP_CACHE)
> 

While the patch certainly looks correct,
I'm not sure about usefulness of r->headers_out.www_authenticate.
Is the header accessed directly through this pointer ever?
For read purposes I mean.  Header filters seem not.

The only place is in the auth_request module,
but it doesn't seem to make sense in subrequest.


From xeioex at nginx.com  Thu May 12 00:10:44 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Thu, 12 May 2022 00:10:44 +0000
Subject: [njs] Fixed ellipsis support.
Message-ID: <hg.34af2730bf97.1652314244.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/34af2730bf97
branches:  
changeset: 1853:34af2730bf97
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed May 11 16:33:46 2022 -0700
description:
Fixed ellipsis support.

As of now, ellispis syntax is supported for function declaration with
the rest arguments.

This closes #365 issue on Github.

diffstat:

 src/njs_parser.c         |  8 ++++++++
 src/test/njs_unit_test.c |  5 +++++
 2 files changed, 13 insertions(+), 0 deletions(-)

diffs (52 lines):

diff -r 5c7e02885c26 -r 34af2730bf97 src/njs_parser.c
--- a/src/njs_parser.c	Fri May 06 18:55:35 2022 -0700
+++ b/src/njs_parser.c	Wed May 11 16:33:46 2022 -0700
@@ -1664,12 +1664,18 @@ njs_parser_array_element_list(njs_parser
         return NJS_OK;
 
     case NJS_TOKEN_ELLIPSIS:
+#if 0
         njs_lexer_consume_token(parser->lexer, 1);
 
         njs_parser_next(parser, njs_parser_assignment_expression);
 
         return njs_parser_after(parser, current, array, 0,
                                 njs_parser_array_spread_element);
+#else
+        (void) njs_parser_array_spread_element;
+        return njs_parser_failed(parser);
+#endif
+
     default:
         break;
     }
@@ -2862,9 +2868,11 @@ njs_parser_argument_list(njs_parser_t *p
      * ArgumentList , ... AssignmentExpression
      */
 
+#if 0 /* TODO. */
     if (token->type == NJS_TOKEN_ELLIPSIS) {
         njs_lexer_consume_token(parser->lexer, 1);
     }
+#endif
 
     njs_parser_next(parser, njs_parser_assignment_expression);
 
diff -r 5c7e02885c26 -r 34af2730bf97 src/test/njs_unit_test.c
--- a/src/test/njs_unit_test.c	Fri May 06 18:55:35 2022 -0700
+++ b/src/test/njs_unit_test.c	Wed May 11 16:33:46 2022 -0700
@@ -18041,9 +18041,14 @@ static njs_unit_test_t  njs_test[] =
     { njs_str("[(]"),
       njs_str("SyntaxError: Unexpected token \"]\" in 1") },
 
+#if 0 /* TODO spreading support. */
     { njs_str("[...]"),
       njs_str("SyntaxError: Unexpected token \"]\" in 1") },
 
+    { njs_str("var id = (x) => x, x = id(...[1,2,3]); typeof x"),
+      njs_str("number") },
+#endif
+
     { njs_str("switch () {}"),
       njs_str("SyntaxError: Unexpected token \")\" in 1") },
 


From xeioex at nginx.com  Thu May 12 04:09:12 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Thu, 12 May 2022 04:09:12 +0000
Subject: [njs] Leaving consistency checks as asserts for njs_scope_index().
Message-ID: <hg.4c8487fb0ca8.1652328552.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/4c8487fb0ca8
branches:  
changeset: 1854:4c8487fb0ca8
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed May 11 17:51:17 2022 -0700
description:
Leaving consistency checks as asserts for njs_scope_index().

diffstat:

 src/njs_scope.h |  7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diffs (17 lines):

diff -r 34af2730bf97 -r 4c8487fb0ca8 src/njs_scope.h
--- a/src/njs_scope.h	Wed May 11 16:33:46 2022 -0700
+++ b/src/njs_scope.h	Wed May 11 17:51:17 2022 -0700
@@ -30,9 +30,10 @@ njs_inline njs_index_t
 njs_scope_index(njs_scope_t scope, njs_index_t index, njs_level_type_t type,
                 njs_variable_type_t var_type)
 {
-    if (index > NJS_SCOPE_VALUE_MAX || type >= NJS_LEVEL_MAX
-        || (scope != NJS_SCOPE_GLOBAL && scope != NJS_SCOPE_FUNCTION))
-    {
+    njs_assert(type < NJS_LEVEL_MAX);
+    njs_assert(scope == NJS_SCOPE_GLOBAL || scope == NJS_SCOPE_FUNCTION);
+
+    if (index > NJS_SCOPE_VALUE_MAX) {
         return NJS_INDEX_ERROR;
     }
 


From xeioex at nginx.com  Thu May 12 04:09:14 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Thu, 12 May 2022 04:09:14 +0000
Subject: [njs] Making function expression binding immutable according the
 specs.
Message-ID: <hg.6a28cdbc8cb6.1652328554.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/6a28cdbc8cb6
branches:  
changeset: 1855:6a28cdbc8cb6
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed May 11 21:08:21 2022 -0700
description:
Making function expression binding immutable according the specs.

This closes #56 issue on Github.

diffstat:

 src/njs_parser.c         |  7 ++++++-
 src/test/njs_unit_test.c |  6 ++++++
 2 files changed, 12 insertions(+), 1 deletions(-)

diffs (34 lines):

diff -r 4c8487fb0ca8 -r 6a28cdbc8cb6 src/njs_parser.c
--- a/src/njs_parser.c	Wed May 11 17:51:17 2022 -0700
+++ b/src/njs_parser.c	Wed May 11 21:08:21 2022 -0700
@@ -6954,8 +6954,13 @@ njs_parser_function_expression_after(njs
 
     var = (njs_variable_t *) parser->target;
 
+    if (var->self) {
+        var->init = 1;
+        var->type = NJS_VARIABLE_CONST;
+    }
+
     var->index = njs_scope_index(var->scope->type, var->scope->items,
-                                 NJS_LEVEL_LOCAL, NJS_VARIABLE_VAR);
+                                 NJS_LEVEL_LOCAL, var->type);
     var->scope->items++;
 
     if (var->self) {
diff -r 4c8487fb0ca8 -r 6a28cdbc8cb6 src/test/njs_unit_test.c
--- a/src/test/njs_unit_test.c	Wed May 11 17:51:17 2022 -0700
+++ b/src/test/njs_unit_test.c	Wed May 11 21:08:21 2022 -0700
@@ -197,6 +197,12 @@ static njs_unit_test_t  njs_test[] =
     { njs_str("var func = function x(x) {return x}; func()"),
       njs_str("undefined") },
 
+    { njs_str("var func = function f() {f = null; return f;}; func()"),
+      njs_str("TypeError: assignment to constant variable") },
+
+    { njs_str("var func = function f() {let f = null; return f;}; func()"),
+      njs_str("null") },
+
 #if 0 /* TODO */
     { njs_str("var a; Object.getOwnPropertyDescriptor(this, 'a').value"),
       njs_str("undefined") },


From P.Pautov at F5.com  Thu May 12 05:36:33 2022
From: P.Pautov at F5.com (Pavel Pautov)
Date: Thu, 12 May 2022 05:36:33 +0000
Subject: [PATCH] Upstream keepalive: remove extra SSL shutdown processing.
Message-ID:  <SJ0PR01MB65104E4B12176B44A1076851B1CB9@SJ0PR01MB6510.prod.exchangelabs.com>

Looks like, with both no_wait_shutdown and no_send_shutdown flags set, ngx_ssl_shutdown() is supposed to be synchronous.
There is similar ngx_ssl_shutdown() usage in ngx_http_upstream_next().

-------------- next part --------------
A non-text attachment was scrubbed...
Name: keepalive_ssl.patch
Type: application/octet-stream
Size: 868 bytes
Desc: keepalive_ssl.patch
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220512/21e076c7/attachment.obj>

From mdounin at mdounin.ru  Thu May 12 22:08:55 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 13 May 2022 01:08:55 +0300
Subject: [PATCH] Upstream keepalive: remove extra SSL shutdown processing.
In-Reply-To: <SJ0PR01MB65104E4B12176B44A1076851B1CB9@SJ0PR01MB6510.prod.exchangelabs.com>
References: <SJ0PR01MB65104E4B12176B44A1076851B1CB9@SJ0PR01MB6510.prod.exchangelabs.com>
Message-ID: <Yn2Fd21aJRG7cYIC@mdounin.ru>

Hello!

On Thu, May 12, 2022 at 05:36:33AM +0000, Pavel Pautov via nginx-devel wrote:

> Looks like, with both no_wait_shutdown and no_send_shutdown flags set, ngx_ssl_shutdown() is supposed to be synchronous.
> There is similar ngx_ssl_shutdown() usage in ngx_http_upstream_next().
> 

> # HG changeset patch
> # User Pavel Pautov <p.pautov at f5.com>
> # Date 1652332850 25200
> #      Wed May 11 22:20:50 2022 -0700
> # Node ID f9cca9c9a0aab58b4d16260c589a6142a1d52b95
> # Parent  35afae4b3dffff6718c0cab3ceb16b9de207c20a
> Upstream keepalive: remove extra SSL shutdown processing.
> 
> diff --git a/src/http/modules/ngx_http_upstream_keepalive_module.c b/src/http/modules/ngx_http_upstream_keepalive_module.c
> --- a/src/http/modules/ngx_http_upstream_keepalive_module.c
> +++ b/src/http/modules/ngx_http_upstream_keepalive_module.c
> @@ -471,10 +471,7 @@ ngx_http_upstream_keepalive_close(ngx_co
>          c->ssl->no_wait_shutdown = 1;
>          c->ssl->no_send_shutdown = 1;
>  
> -        if (ngx_ssl_shutdown(c) == NGX_AGAIN) {
> -            c->ssl->handler = ngx_http_upstream_keepalive_close;
> -            return;
> -        }
> +        (void) ngx_ssl_shutdown(c);
>      }
>  
>  #endif

No, thanks.

-- 
Maxim Dounin
http://mdounin.ru/


From P.Pautov at F5.com  Thu May 12 22:27:10 2022
From: P.Pautov at F5.com (Pavel Pautov)
Date: Thu, 12 May 2022 22:27:10 +0000
Subject: [PATCH] Upstream keepalive: remove extra SSL shutdown processing.
In-Reply-To: <Yn2Fd21aJRG7cYIC@mdounin.ru>
References:  <SJ0PR01MB65104E4B12176B44A1076851B1CB9@SJ0PR01MB6510.prod.exchangelabs.com>
 <Yn2Fd21aJRG7cYIC@mdounin.ru>
Message-ID:  <SJ0PR01MB65109FD0AB33F15D47C2D31FB1CB9@SJ0PR01MB6510.prod.exchangelabs.com>

Hello,

> -----Original Message-----
> From: Maxim Dounin <mdounin at mdounin.ru>
> ...
> > +++ b/src/http/modules/ngx_http_upstream_keepalive_module.c
> > @@ -471,10 +471,7 @@ ngx_http_upstream_keepalive_close(ngx_co
> >          c->ssl->no_wait_shutdown = 1;
> >          c->ssl->no_send_shutdown = 1;
> >
> > -        if (ngx_ssl_shutdown(c) == NGX_AGAIN) {
> > -            c->ssl->handler = ngx_http_upstream_keepalive_close;
> > -            return;
> > -        }
> > +        (void) ngx_ssl_shutdown(c);
> >      }
> >
> >  #endif
> 
> No, thanks.

So, is it possible for ngx_ssl_shutdown() to return NGX_AGAIN here (and in ngx_http_upstream_next)?


From mdounin at mdounin.ru  Thu May 12 22:32:16 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 13 May 2022 01:32:16 +0300
Subject: [PATCH 02 of 20] FastCGI: combining headers with identical names
 (ticket #1724)
In-Reply-To: <20220511153232.c3fd4i573xt64hnc@Y9MQ9X2QVV>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <61b29233a55216c6fa72.1650493122@vm-bsd.mdounin.ru>
 <20220511153232.c3fd4i573xt64hnc@Y9MQ9X2QVV>
Message-ID: <Yn2K8MLJpK+KasJt@mdounin.ru>

Hello!

On Wed, May 11, 2022 at 07:32:32PM +0400, Sergey Kandaurov wrote:

> On Thu, Apr 21, 2022 at 01:18:42AM +0300, Maxim Dounin wrote:
> > # HG changeset patch
> > # User Maxim Dounin <mdounin at mdounin.ru>
> > # Date 1650492316 -10800
> > #      Thu Apr 21 01:05:16 2022 +0300
> > # Node ID 61b29233a55216c6fa72e23b93a4a28d76a9fb94
> > # Parent  e70fb0fdfbc0fb7b7e9f493cc2eb65de617b115a
> > FastCGI: combining headers with identical names (ticket #1724).
> > 
> > FastCGI responder is expected to receive CGI/1.1 environment variables
> > in the parameters (see section "6.2 Responder" of the FastCGI specification).
> > Obviously enough, there cannot be multiple environment variables with
> > the same name.
> > 
> > Further, CGI specification (RFC 3875, section "4.1.18. Protocol-Specific
> > Meta-Variables") explicitly requires to combine headers: "If multiple
> > header fields with the same field-name are received then the server MUST
> > rewrite them as a single value having the same semantics".
> > 
> > diff --git a/src/core/ngx_hash.h b/src/core/ngx_hash.h
> > --- a/src/core/ngx_hash.h
> > +++ b/src/core/ngx_hash.h
> > @@ -89,12 +89,15 @@ typedef struct {
> >  } ngx_hash_keys_arrays_t;
> >  
> >  
> > -typedef struct {
> > +typedef struct ngx_table_elt_s  ngx_table_elt_t;
> > +
> > +struct ngx_table_elt_s {
> >      ngx_uint_t        hash;
> >      ngx_str_t         key;
> >      ngx_str_t         value;
> >      u_char           *lowcase_key;
> > -} ngx_table_elt_t;
> > +    ngx_table_elt_t  *next;
> > +};
> >  
> >  
> >  void *ngx_hash_find(ngx_hash_t *hash, ngx_uint_t key, u_char *name, size_t len);
> > diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c
> > --- a/src/http/modules/ngx_http_fastcgi_module.c
> > +++ b/src/http/modules/ngx_http_fastcgi_module.c
> > @@ -835,14 +835,14 @@ static ngx_int_t
> >  ngx_http_fastcgi_create_request(ngx_http_request_t *r)
> >  {
> >      off_t                         file_pos;
> > -    u_char                        ch, *pos, *lowcase_key;
> > +    u_char                        ch, sep, *pos, *lowcase_key;
> >      size_t                        size, len, key_len, val_len, padding,
> >                                    allocated;
> >      ngx_uint_t                    i, n, next, hash, skip_empty, header_params;
> >      ngx_buf_t                    *b;
> >      ngx_chain_t                  *cl, *body;
> >      ngx_list_part_t              *part;
> > -    ngx_table_elt_t              *header, **ignored;
> > +    ngx_table_elt_t              *header, *hn, **ignored;
> >      ngx_http_upstream_t          *u;
> >      ngx_http_script_code_pt       code;
> >      ngx_http_script_engine_t      e, le;
> > @@ -900,7 +900,11 @@ ngx_http_fastcgi_create_request(ngx_http
> >          allocated = 0;
> >          lowcase_key = NULL;
> >  
> > -        if (params->number) {
> > +        if (ngx_http_link_multi_headers(r) != NGX_OK) {
> > +            return NGX_ERROR;
> > +        }
> > +
> > +        if (params->number || r->headers_in.multi) {
> >              n = 0;
> >              part = &r->headers_in.headers.part;
> >  
> > @@ -930,6 +934,12 @@ ngx_http_fastcgi_create_request(ngx_http
> >                  i = 0;
> >              }
> >  
> > +            for (n = 0; n < header_params; n++) {
> > +                if (&header[i] == ignored[n]) {
> > +                    goto next_length;
> > +                }
> > +            }
> > +
> >              if (params->number) {
> >                  if (allocated < header[i].key.len) {
> >                      allocated = header[i].key.len + 16;
> > @@ -959,15 +969,23 @@ ngx_http_fastcgi_create_request(ngx_http
> >                      ignored[header_params++] = &header[i];
> >                      continue;
> >                  }
> > -
> > -                n += sizeof("HTTP_") - 1;
> > -
> > -            } else {
> > -                n = sizeof("HTTP_") - 1 + header[i].key.len;
> >              }
> >  
> > -            len += ((n > 127) ? 4 : 1) + ((header[i].value.len > 127) ? 4 : 1)
> > -                + n + header[i].value.len;
> > +            key_len = sizeof("HTTP_") - 1 + header[i].key.len;
> > +
> > +            val_len = header[i].value.len;
> > +
> > +            for (hn = header[i].next; hn; hn = hn->next) {
> > +                val_len += hn->value.len + 2;
> > +                ignored[header_params++] = hn;
> > +            }
> > +
> > +            len += ((key_len > 127) ? 4 : 1) + key_len
> > +                   + ((val_len > 127) ? 4 : 1) + val_len;
> > +
> > +        next_length:
> > +
> > +            continue;
> >          }
> >      }
> >  
> > @@ -1109,7 +1127,7 @@ ngx_http_fastcgi_create_request(ngx_http
> >  
> >              for (n = 0; n < header_params; n++) {
> >                  if (&header[i] == ignored[n]) {
> > -                    goto next;
> > +                    goto next_value;
> >                  }
> >              }
> >  
> > @@ -1125,6 +1143,11 @@ ngx_http_fastcgi_create_request(ngx_http
> >              }
> >  
> >              val_len = header[i].value.len;
> > +
> > +            for (hn = header[i].next; hn; hn = hn->next) {
> > +                val_len += hn->value.len + 2;
> > +            }
> > +
> >              if (val_len > 127) {
> >                  *b->last++ = (u_char) (((val_len >> 24) & 0x7f) | 0x80);
> >                  *b->last++ = (u_char) ((val_len >> 16) & 0xff);
> > @@ -1150,13 +1173,34 @@ ngx_http_fastcgi_create_request(ngx_http
> >                  *b->last++ = ch;
> >              }
> >  
> > -            b->last = ngx_copy(b->last, header[i].value.data, val_len);
> > +            b->last = ngx_copy(b->last, header[i].value.data,
> > +                               header[i].value.len);
> > +
> > +            if (header[i].next) {
> > +
> > +                if (header[i].key.len == sizeof("Cookie") - 1
> > +                    && ngx_strncasecmp(header[i].key.data, (u_char *) "Cookie",
> > +                                       sizeof("Cookie") - 1)
> > +                       == 0)
> > +                {
> > +                    sep = ';';
> > +
> > +                } else {
> > +                    sep = ',';
> > +                }
> > +
> > +                for (hn = header[i].next; hn; hn = hn->next) {
> > +                    *b->last++ = sep;
> > +                    *b->last++ = ' ';
> > +                    b->last = ngx_copy(b->last, hn->value.data, hn->value.len);
> > +                }
> > +            }
> >  
> >              ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
> >                             "fastcgi param: \"%*s: %*s\"",
> >                             key_len, b->last - (key_len + val_len),
> >                             val_len, b->last - val_len);
> > -        next:
> > +        next_value:
> >  
> >              continue;
> >          }
> 
> Overall, fastcgi/uwsgi/scgi parts look good, some observations below:
> 
> - fastcgi_param (still) overrides client headers with the same name
> e.g., "fastcgi_param header value;" overrides client header "header: value"

That's how it's expected to work (with HTTP_<HEADER> fastcgi 
param).

> - multiple directives itself aren't linked emitting separate params:
> fastcgi_param FOO BAR;
> fastcgi_param FOO BAZ;
> 
> I don't think it's a major point though, it can be handled
> in configuration in principle.

I don't think this needs to be changed, or at least I don't think 
this needs to be addressed in this patch series.  If we'll decide 
to address this somehow, rejecting such configurations might be a 
way to go.

> BTW, what about proxy modifications? they aren't required but could be useful

I don't think that's needed.  Unless required, I would rather 
preserve original request headers.

> > diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
> > --- a/src/http/ngx_http_core_module.c
> > +++ b/src/http/ngx_http_core_module.c
> > @@ -2802,6 +2802,78 @@ ngx_http_get_forwarded_addr_internal(ngx
> >  }
> >  
> >  
> > +ngx_int_t
> > +ngx_http_link_multi_headers(ngx_http_request_t *r)
> > +{
> > +    ngx_uint_t        i, j;
> > +    ngx_list_part_t  *part, *ppart;
> > +    ngx_table_elt_t  *header, *pheader, **ph;
> > +
> > +    if (r->headers_in.multi_linked) {
> > +        return NGX_OK;
> > +    }
> 
> multi_linked is never set, was the intension to avoid doing the work
> twice on repetitive invocation? it doesn't seem to be possible.
> I'd just axe it.

Yes, thanks, missed this somehow.  It should be set right after 
the test, updated with the following change:

diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c
+++ b/src/http/ngx_http_core_module.c
@@ -2813,6 +2813,8 @@ ngx_http_link_multi_headers(ngx_http_req
         return NGX_OK;
     }
 
+    r->headers_in.multi_linked = 1;
+
     part = &r->headers_in.headers.part;
     header = part->elts;
 

Not sure what do you mean by "doesn't seem to be possible".  The 
request headers are never changed after parsing, so it is enough 
to link them just once.

> > +
> > +    part = &r->headers_in.headers.part;
> > +    header = part->elts;
> > +
> > +    for (i = 0; /* void */; i++) {
> > +
> > +        if (i >= part->nelts) {
> > +            if (part->next == NULL) {
> > +                break;
> > +            }
> > +
> > +            part = part->next;
> > +            header = part->elts;
> > +            i = 0;
> > +        }
> > +
> > +        header[i].next = NULL;
> > +
> > +        /*
> > +         * search for previous headers with the same name;
> > +         * if there are any, link to them
> > +         */
> > +
> > +        ppart = &r->headers_in.headers.part;
> > +        pheader = part->elts;
> 
> pheader = ppart->elts;

Fixed, thanks.

> > +
> > +        for (j = 0; /* void */; j++) {
> > +
> > +            if (j >= ppart->nelts) {
> > +                if (ppart->next == NULL) {
> > +                    break;
> > +                }
> > +
> > +                ppart = ppart->next;
> > +                pheader = ppart->elts;
> > +                i = 0;
> 
> j = 0;

Fixed, thanks.

> > +            }
> > +
> > +            if (part == ppart && i == j) {
> > +                break;
> > +            }
> > +
> > +            if (header[i].key.len == pheader[j].key.len
> > +                && ngx_strncasecmp(header[i].key.data, pheader[j].key.data,
> > +                                   header[i].key.len)
> > +                   == 0)
> > +            {
> > +                ph = &pheader[j].next;
> > +                while (*ph) { ph = &(*ph)->next; }
> > +                *ph = &header[i];
> > +
> > +                r->headers_in.multi = 1;
> > +
> > +                break;
> > +            }
> > +        }
> > +    }
> > +
> > +    return NGX_OK;
> > +}
> > +
> > +
> >  static char *
> >  ngx_http_core_server(ngx_conf_t *cf, ngx_command_t *cmd, void *dummy)
> >  {
> > diff --git a/src/http/ngx_http_core_module.h b/src/http/ngx_http_core_module.h
> > --- a/src/http/ngx_http_core_module.h
> > +++ b/src/http/ngx_http_core_module.h
> > @@ -532,6 +532,8 @@ ngx_int_t ngx_http_get_forwarded_addr(ng
> >      ngx_array_t *headers, ngx_str_t *value, ngx_array_t *proxies,
> >      int recursive);
> >  
> > +ngx_int_t ngx_http_link_multi_headers(ngx_http_request_t *r);
> > +
> >  
> >  extern ngx_module_t  ngx_http_core_module;
> >  
> > diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
> > --- a/src/http/ngx_http_request.h
> > +++ b/src/http/ngx_http_request.h
> > @@ -242,6 +242,8 @@ typedef struct {
> >  
> >      unsigned                          connection_type:2;
> >      unsigned                          chunked:1;
> > +    unsigned                          multi:1;
> > +    unsigned                          multi_linked:1;
> >      unsigned                          msie:1;
> >      unsigned                          msie6:1;
> >      unsigned                          opera:1;
> > 
> > _______________________________________________
> > nginx-devel mailing list -- nginx-devel at nginx.org
> > To unsubscribe send an email to nginx-devel-leave at nginx.org
> _______________________________________________
> nginx-devel mailing list -- nginx-devel at nginx.org
> To unsubscribe send an email to nginx-devel-leave at nginx.org

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Thu May 12 23:15:17 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 13 May 2022 02:15:17 +0300
Subject: [PATCH 05 of 20] Combining unknown headers during variables
 lookup (ticket #1316)
In-Reply-To: <20220511160910.pycqf36eqvpyf66c@Y9MQ9X2QVV>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <2ea48b5e4643a818cd81.1650493125@vm-bsd.mdounin.ru>
 <20220511160910.pycqf36eqvpyf66c@Y9MQ9X2QVV>
Message-ID: <Yn2VBSd0hnHoU0V2@mdounin.ru>

Hello!

On Wed, May 11, 2022 at 08:09:10PM +0400, Sergey Kandaurov wrote:

> On Thu, Apr 21, 2022 at 01:18:45AM +0300, Maxim Dounin wrote:
> > # HG changeset patch
> > # User Maxim Dounin <mdounin at mdounin.ru>
> > # Date 1650492321 -10800
> > #      Thu Apr 21 01:05:21 2022 +0300
> > # Node ID 2ea48b5e4643a818cd81179f040f0b36be9050d6
> > # Parent  3618f35ac3c8833cfd4a4c80295ab94eb924a9a7
> > Combining unknown headers during variables lookup (ticket #1316).
> > 
> > Previously, $http_*, $sent_http_*, $upstream_http_*, and $upstream_trailer_*
> > variables returned only the first header (with a few specially handled
> > exceptions: $http_cookie, $http_x_forwarded_for, $sent_http_cache_control,
> > $sent_http_link).
> 
> forgot to mention "sent_trailer_"

Yes, thanks, added to the list.

> jftr, this is an API change, used at least in mod_zip for for "upstream_http_"

In the particular case this rather looks like a misuse of the 
ngx_http_variable_unknown_header() function in the mod_zip module, 
as it clearly does not use it to lookup the $upstream_http_* 
variable, but instead tries to emulate such a lookup.

Either way, this series introduces quite a few API changes, so it 
doesn't really matter.

> > 
> > With this change, all headers are returned, combined together.  For
> > example, $http_foo variable will be "a, b" if there are "Foo: a" and
> > "Foo: b" headers in the request.
> > 
> > Note that $upstream_http_set_cookie will also return all "Set-Cookie"
> > headers (ticket #1843), though this might not be what one want, since
> > the "Set-Cookie" header does not follow the list syntax (see RFC 7230,
> > section 3.2.2).
> > 
> > diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
> > --- a/src/http/ngx_http_upstream.c
> > +++ b/src/http/ngx_http_upstream.c
> > @@ -5703,7 +5703,7 @@ ngx_http_upstream_header_variable(ngx_ht
> >          return NGX_OK;
> >      }
> >  
> > -    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
> > +    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
> >                                           &r->upstream->headers_in.headers.part,
> >                                           sizeof("upstream_http_") - 1);
> >  }
> > @@ -5718,7 +5718,7 @@ ngx_http_upstream_trailer_variable(ngx_h
> >          return NGX_OK;
> >      }
> >  
> > -    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
> > +    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
> >                                          &r->upstream->headers_in.trailers.part,
> >                                          sizeof("upstream_trailer_") - 1);
> >  }
> > diff --git a/src/http/ngx_http_variables.c b/src/http/ngx_http_variables.c
> > --- a/src/http/ngx_http_variables.c
> > +++ b/src/http/ngx_http_variables.c
> > @@ -919,7 +919,7 @@ static ngx_int_t
> >  ngx_http_variable_unknown_header_in(ngx_http_request_t *r,
> >      ngx_http_variable_value_t *v, uintptr_t data)
> >  {
> > -    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
> > +    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
> >                                              &r->headers_in.headers.part,
> >                                              sizeof("http_") - 1);
> >  }
> > @@ -929,7 +929,7 @@ static ngx_int_t
> >  ngx_http_variable_unknown_header_out(ngx_http_request_t *r,
> >      ngx_http_variable_value_t *v, uintptr_t data)
> >  {
> > -    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
> > +    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
> >                                              &r->headers_out.headers.part,
> >                                              sizeof("sent_http_") - 1);
> >  }
> > @@ -939,19 +939,26 @@ static ngx_int_t
> >  ngx_http_variable_unknown_trailer_out(ngx_http_request_t *r,
> >      ngx_http_variable_value_t *v, uintptr_t data)
> >  {
> > -    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
> > +    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
> >                                              &r->headers_out.trailers.part,
> >                                              sizeof("sent_trailer_") - 1);
> >  }
> >  
> >  
> >  ngx_int_t
> > -ngx_http_variable_unknown_header(ngx_http_variable_value_t *v, ngx_str_t *var,
> > +ngx_http_variable_unknown_header(ngx_http_request_t *r,
> > +    ngx_http_variable_value_t *v, ngx_str_t *var,
> >      ngx_list_part_t *part, size_t prefix)
> >  {
> > -    u_char            ch;
> > +    u_char           *p, ch;
> > +    size_t            len;
> >      ngx_uint_t        i, n;
> > -    ngx_table_elt_t  *header;
> > +    ngx_table_elt_t  *header, *h, **ph;
> > +
> > +    ph = &h;
> > +#if (NGX_SUPPRESS_WARN)
> > +    len = 0;
> > +#endif
> >  
> >      header = part->elts;
> >  
> > @@ -971,7 +978,11 @@ ngx_http_variable_unknown_header(ngx_htt
> >              continue;
> >          }
> >  
> > -        for (n = 0; n + prefix < var->len && n < header[i].key.len; n++) {
> > +        if (header[i].key.len != var->len - prefix) {
> > +            continue;
> > +        }
> > +
> > +        for (n = 0; n < var->len - prefix; n++) {
> >              ch = header[i].key.data[n];
> >  
> >              if (ch >= 'A' && ch <= 'Z') {
> > @@ -986,18 +997,59 @@ ngx_http_variable_unknown_header(ngx_htt
> >              }
> >          }
> >  
> > -        if (n + prefix == var->len && n == header[i].key.len) {
> > -            v->len = header[i].value.len;
> > -            v->valid = 1;
> > -            v->no_cacheable = 0;
> > -            v->not_found = 0;
> > -            v->data = header[i].value.data;
> > -
> > -            return NGX_OK;
> > +        if (n != var->len - prefix) {
> > +            continue;
> >          }
> > +
> > +        len += header[i].value.len + 2;
> > +
> > +        *ph = &header[i];
> > +        ph = &header[i].next;
> >      }
> >  
> > -    v->not_found = 1;
> > +    *ph = NULL;
> > +
> > +    if (h == NULL) {
> > +        v->not_found = 1;
> > +        return NGX_OK;
> > +    }
> > +
> > +    len -= 2;
> > +
> > +    if (h->next == NULL) {
> > +
> > +        v->len = h->value.len;
> 
> could be just len for consistency with common case

I think it's better to keep it straight: if there are no 
additional headers, we just return the only header.  And this 
matches the code in ngx_http_variable_headers_internal().

> 
> > +        v->valid = 1;
> > +        v->no_cacheable = 0;
> > +        v->not_found = 0;
> > +        v->data = h->value.data;
> > +
> > +        return NGX_OK;
> > +    }
> > +
> > +    p = ngx_pnalloc(r->pool, len);
> > +    if (p == NULL) {
> > +        return NGX_ERROR;
> > +    }
> > +
> > +    v->len = len;
> > +    v->valid = 1;
> > +    v->no_cacheable = 0;
> > +    v->not_found = 0;
> > +    v->data = p;
> > +
> > +    for ( ;; ) {
> > +
> > +        p = ngx_copy(p, h->value.data, h->value.len);
> > +
> > +        if (h->next == NULL) {
> > +            break;
> > +        }
> > +
> > +        *p++ = ','; *p++ = ' ';
> 
> what about Set-Cookie separator ";" support?
> looks like can be easly integrated (on top off):

There is no valid separator for the Set-Cookie header, multiple 
Set-Cookie headers cannot be represented as a single string.  The 
";" character works for the Cookie header, but not for Set-Cookie, 
which uses ";" to separate attribute-value pairs (as already 
mentioned in the commit log, see RFC 7230, section 3.2.2).
 
[...]

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Thu May 12 23:42:20 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 13 May 2022 02:42:20 +0300
Subject: [PATCH 06 of 20] Reworked multi headers to use linked lists
In-Reply-To: <20220511192115.h3openel4fzpzikd@Y9MQ9X2QVV>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <50fe52f516ff9c148aa9.1650493126@vm-bsd.mdounin.ru>
 <20220511192115.h3openel4fzpzikd@Y9MQ9X2QVV>
Message-ID: <Yn2bXPrcbYHHSHB6@mdounin.ru>

Hello!

On Wed, May 11, 2022 at 11:21:15PM +0400, Sergey Kandaurov wrote:

> On Thu, Apr 21, 2022 at 01:18:46AM +0300, Maxim Dounin wrote:
> > # HG changeset patch
> > # User Maxim Dounin <mdounin at mdounin.ru>
> > # Date 1650492323 -10800
> > #      Thu Apr 21 01:05:23 2022 +0300
> > # Node ID 50fe52f516ff9c148aa9e7dfcc1c31cc6a4929ae
> > # Parent  2ea48b5e4643a818cd81179f040f0b36be9050d6
> > Reworked multi headers to use linked lists.
> > 
> > Multi headers are now using linked lists instead of arrays.  Notably,
> > the following fields were changed: r->headers_in.cookies (renamed
> > to r->headers_in.cookie), r->headers_in.x_forwarded_for,
> > r->headers_out.cache_control, r->headers_out.link, u->headers_in.cache_control
> > u->headers_in.cookies (renamed to u->headers_in.set_cookie).
> 
> jftr, this implies updating a bunch of 3rd party modules, including njs, lua,
> naxsi, nginx-clojure, headers-more, geoip2_module, ngx_http_sticky_module

Sure.

> > 
> > The r->headers_in.cookies and u->headers_in.cookies fields were renamed
> > to r->headers_in.cookie and u->headers_in.set_cookie to match header names.
> > 
> > The ngx_http_parse_multi_header_lines() and ngx_http_parse_set_cookie_lines()
> > functions were changed accordingly.
> > 
> > With this change, multi headers are now essentially equivalent to normal
> > headers, and following changes will further make them equivalent.
> > 
> > diff --git a/src/http/modules/ngx_http_geo_module.c b/src/http/modules/ngx_http_geo_module.c
> > --- a/src/http/modules/ngx_http_geo_module.c
> > +++ b/src/http/modules/ngx_http_geo_module.c
> > @@ -327,15 +327,15 @@ static ngx_int_t
> >  ngx_http_geo_addr(ngx_http_request_t *r, ngx_http_geo_ctx_t *ctx,
> >      ngx_addr_t *addr)
> >  {
> > -    ngx_array_t  *xfwd;
> > +    ngx_table_elt_t  *xfwd;
> >  
> >      if (ngx_http_geo_real_addr(r, ctx, addr) != NGX_OK) {
> >          return NGX_ERROR;
> >      }
> >  
> > -    xfwd = &r->headers_in.x_forwarded_for;
> > +    xfwd = r->headers_in.x_forwarded_for;
> >  
> > -    if (xfwd->nelts > 0 && ctx->proxies != NULL) {
> > +    if (xfwd != NULL && ctx->proxies != NULL) {
> >          (void) ngx_http_get_forwarded_addr(r, addr, xfwd, NULL,
> >                                             ctx->proxies, ctx->proxy_recursive);
> >      }
> > diff --git a/src/http/modules/ngx_http_geoip_module.c b/src/http/modules/ngx_http_geoip_module.c
> > --- a/src/http/modules/ngx_http_geoip_module.c
> > +++ b/src/http/modules/ngx_http_geoip_module.c
> > @@ -240,16 +240,16 @@ static u_long
> >  ngx_http_geoip_addr(ngx_http_request_t *r, ngx_http_geoip_conf_t *gcf)
> >  {
> >      ngx_addr_t           addr;
> > -    ngx_array_t         *xfwd;
> > +    ngx_table_elt_t     *xfwd;
> >      struct sockaddr_in  *sin;
> >  
> >      addr.sockaddr = r->connection->sockaddr;
> >      addr.socklen = r->connection->socklen;
> >      /* addr.name = r->connection->addr_text; */
> >  
> > -    xfwd = &r->headers_in.x_forwarded_for;
> > +    xfwd = r->headers_in.x_forwarded_for;
> >  
> > -    if (xfwd->nelts > 0 && gcf->proxies != NULL) {
> > +    if (xfwd != NULL && gcf->proxies != NULL) {
> >          (void) ngx_http_get_forwarded_addr(r, &addr, xfwd, NULL,
> >                                             gcf->proxies, gcf->proxy_recursive);
> >      }
> > @@ -292,7 +292,7 @@ static geoipv6_t
> >  ngx_http_geoip_addr_v6(ngx_http_request_t *r, ngx_http_geoip_conf_t *gcf)
> >  {
> >      ngx_addr_t            addr;
> > -    ngx_array_t          *xfwd;
> > +    ngx_table_elt_t      *xfwd;
> >      in_addr_t             addr4;
> >      struct in6_addr       addr6;
> >      struct sockaddr_in   *sin;
> > @@ -302,9 +302,9 @@ ngx_http_geoip_addr_v6(ngx_http_request_
> >      addr.socklen = r->connection->socklen;
> >      /* addr.name = r->connection->addr_text; */
> >  
> > -    xfwd = &r->headers_in.x_forwarded_for;
> > +    xfwd = r->headers_in.x_forwarded_for;
> >  
> > -    if (xfwd->nelts > 0 && gcf->proxies != NULL) {
> > +    if (xfwd != NULL && gcf->proxies != NULL) {
> >          (void) ngx_http_get_forwarded_addr(r, &addr, xfwd, NULL,
> >                                             gcf->proxies, gcf->proxy_recursive);
> >      }
> > diff --git a/src/http/modules/ngx_http_headers_filter_module.c b/src/http/modules/ngx_http_headers_filter_module.c
> > --- a/src/http/modules/ngx_http_headers_filter_module.c
> > +++ b/src/http/modules/ngx_http_headers_filter_module.c
> > @@ -329,8 +329,7 @@ ngx_http_set_expires(ngx_http_request_t 
> >      time_t               now, expires_time, max_age;
> >      ngx_str_t            value;
> >      ngx_int_t            rc;
> > -    ngx_uint_t           i;
> > -    ngx_table_elt_t     *e, *cc, **ccp;
> > +    ngx_table_elt_t     *e, *cc;
> >      ngx_http_expires_t   expires;
> >  
> >      expires = conf->expires;
> > @@ -371,38 +370,28 @@ ngx_http_set_expires(ngx_http_request_t 
> >      len = sizeof("Mon, 28 Sep 1970 06:00:00 GMT");
> >      e->value.len = len - 1;
> >  
> > -    ccp = r->headers_out.cache_control.elts;
> > -
> > -    if (ccp == NULL) {
> > +    cc = r->headers_out.cache_control;
> >  
> > -        if (ngx_array_init(&r->headers_out.cache_control, r->pool,
> > -                           1, sizeof(ngx_table_elt_t *))
> > -            != NGX_OK)
> > -        {
> > -            return NGX_ERROR;
> > -        }
> > +    if (cc == NULL) {
> >  
> >          cc = ngx_list_push(&r->headers_out.headers);
> >          if (cc == NULL) {
> >              return NGX_ERROR;
> >          }
> >  
> > +        r->headers_out.cache_control = cc;
> > +        cc->next = NULL;
> > +
> >          cc->hash = 1;
> >          ngx_str_set(&cc->key, "Cache-Control");
> >  
> > -        ccp = ngx_array_push(&r->headers_out.cache_control);
> > -        if (ccp == NULL) {
> > -            return NGX_ERROR;
> > +    } else {
> > +        for (cc = cc->next; cc; cc = cc->next) {
> > +            cc->hash = 0;
> >          }
> >  
> > -        *ccp = cc;
> > -
> > -    } else {
> > -        for (i = 1; i < r->headers_out.cache_control.nelts; i++) {
> > -            ccp[i]->hash = 0;
> > -        }
> > -
> > -        cc = ccp[0];
> > +        cc = r->headers_out.cache_control;
> > +        cc->next = NULL;
> >      }
> >  
> >      if (expires == NGX_HTTP_EXPIRES_EPOCH) {
> > @@ -564,22 +553,12 @@ static ngx_int_t
> >  ngx_http_add_multi_header_lines(ngx_http_request_t *r,
> >      ngx_http_header_val_t *hv, ngx_str_t *value)
> >  {
> > -    ngx_array_t      *pa;
> >      ngx_table_elt_t  *h, **ph;
> >  
> >      if (value->len == 0) {
> >          return NGX_OK;
> >      }
> >  
> > -    pa = (ngx_array_t *) ((char *) &r->headers_out + hv->offset);
> > -
> > -    if (pa->elts == NULL) {
> > -        if (ngx_array_init(pa, r->pool, 1, sizeof(ngx_table_elt_t *)) != NGX_OK)
> > -        {
> > -            return NGX_ERROR;
> > -        }
> > -    }
> > -
> >      h = ngx_list_push(&r->headers_out.headers);
> >      if (h == NULL) {
> >          return NGX_ERROR;
> > @@ -589,12 +568,12 @@ ngx_http_add_multi_header_lines(ngx_http
> >      h->key = hv->key;
> >      h->value = *value;
> >  
> > -    ph = ngx_array_push(pa);
> > -    if (ph == NULL) {
> > -        return NGX_ERROR;
> > -    }
> > +    ph = (ngx_table_elt_t **) ((char *) &r->headers_out + hv->offset);
> > +
> > +    while (*ph) { ph = &(*ph)->next; }
> >  
> >      *ph = h;
> > +    h->next = NULL;
> >  
> >      return NGX_OK;
> >  }
> > diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c
> > --- a/src/http/modules/ngx_http_proxy_module.c
> > +++ b/src/http/modules/ngx_http_proxy_module.c
> > @@ -2559,22 +2559,20 @@ static ngx_int_t
> >  ngx_http_proxy_add_x_forwarded_for_variable(ngx_http_request_t *r,
> >      ngx_http_variable_value_t *v, uintptr_t data)
> >  {
> > -    size_t             len;
> > -    u_char            *p;
> > -    ngx_uint_t         i, n;
> > -    ngx_table_elt_t  **h;
> > +    size_t            len;
> > +    u_char           *p;
> > +    ngx_table_elt_t  *h, *xfwd;
> >  
> >      v->valid = 1;
> >      v->no_cacheable = 0;
> >      v->not_found = 0;
> >  
> > -    n = r->headers_in.x_forwarded_for.nelts;
> > -    h = r->headers_in.x_forwarded_for.elts;
> > +    xfwd = r->headers_in.x_forwarded_for;
> >  
> >      len = 0;
> >  
> > -    for (i = 0; i < n; i++) {
> > -        len += h[i]->value.len + sizeof(", ") - 1;
> > +    for (h = xfwd; h; h = h->next) {
> > +        len += h->value.len + sizeof(", ") - 1;
> >      }
> >  
> >      if (len == 0) {
> 
> The only minor point in a whole patch (which I won't insist):
> a special case for xfwd absense can be slightly reshuffled now
> to avoid a do-nothing loop.
> 
> diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c
> --- a/src/http/modules/ngx_http_proxy_module.c
> +++ b/src/http/modules/ngx_http_proxy_module.c
> @@ -2569,18 +2569,18 @@ ngx_http_proxy_add_x_forwarded_for_varia
>  
>      xfwd = r->headers_in.x_forwarded_for;
>  
> +    if (xfwd == NULL) {
> +        v->len = r->connection->addr_text.len;
> +        v->data = r->connection->addr_text.data;
> +        return NGX_OK;
> +    }
> +
>      len = 0;
>  
>      for (h = xfwd; h; h = h->next) {
>          len += h->value.len + sizeof(", ") - 1;
>      }
>  
> -    if (len == 0) {
> -        v->len = r->connection->addr_text.len;
> -        v->data = r->connection->addr_text.data;
> -        return NGX_OK;
> -    }
> -
>      len += r->connection->addr_text.len;
>  
>      p = ngx_pnalloc(r->pool, len);
> 
> Otherwise, looks good.

Such an optimization (if at all) is completely orthogonal and 
available regardless of the r->headers_in.x_forwarded_for format 
(that is, nothing stops you from testing "n == 0" in the original 
code).

Either way, I don't think such a change belongs to this patch 
and/or this patch series.

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Thu May 12 23:54:21 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 13 May 2022 02:54:21 +0300
Subject: [PATCH 07 of 20] All non-unique input headers are now linked
 lists
In-Reply-To: <20220511194048.df6wpfybtpn4dxjg@Y9MQ9X2QVV>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <238d1f5f438735432822.1650493127@vm-bsd.mdounin.ru>
 <20220511194048.df6wpfybtpn4dxjg@Y9MQ9X2QVV>
Message-ID: <Yn2eLVRmDzq13xqU@mdounin.ru>

Hello!

On Wed, May 11, 2022 at 11:40:48PM +0400, Sergey Kandaurov wrote:

> On Thu, Apr 21, 2022 at 01:18:47AM +0300, Maxim Dounin wrote:
> > # HG changeset patch
> > # User Maxim Dounin <mdounin at mdounin.ru>
> > # Date 1650492324 -10800
> > #      Thu Apr 21 01:05:24 2022 +0300
> > # Node ID 238d1f5f438735432822689605fa37b1b0e01517
> > # Parent  50fe52f516ff9c148aa9e7dfcc1c31cc6a4929ae
> > All non-unique input headers are now linked lists.
> > 
> > The ngx_http_process_multi_header_lines() function is removed, as it is
> > exactly equivalent to ngx_http_process_header_line().  Similarly,
> > ngx_http_variable_header() is used instead of ngx_http_variable_headers().
> > 
> > diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
> > --- a/src/http/ngx_http_request.c
> > +++ b/src/http/ngx_http_request.c
> > @@ -22,8 +22,6 @@ static ngx_int_t ngx_http_process_header
> >      ngx_table_elt_t *h, ngx_uint_t offset);
> >  static ngx_int_t ngx_http_process_unique_header_line(ngx_http_request_t *r,
> >      ngx_table_elt_t *h, ngx_uint_t offset);
> > -static ngx_int_t ngx_http_process_multi_header_lines(ngx_http_request_t *r,
> > -    ngx_table_elt_t *h, ngx_uint_t offset);
> >  static ngx_int_t ngx_http_process_host(ngx_http_request_t *r,
> >      ngx_table_elt_t *h, ngx_uint_t offset);
> >  static ngx_int_t ngx_http_process_connection(ngx_http_request_t *r,
> > @@ -164,7 +162,7 @@ ngx_http_header_t  ngx_http_headers_in[]
> >  #if (NGX_HTTP_X_FORWARDED_FOR)
> >      { ngx_string("X-Forwarded-For"),
> >                   offsetof(ngx_http_headers_in_t, x_forwarded_for),
> > -                 ngx_http_process_multi_header_lines },
> > +                 ngx_http_process_header_line },
> >  #endif
> >  
> >  #if (NGX_HTTP_REALIP)
> > @@ -197,7 +195,7 @@ ngx_http_header_t  ngx_http_headers_in[]
> >  #endif
> >  
> >      { ngx_string("Cookie"), offsetof(ngx_http_headers_in_t, cookie),
> > -                 ngx_http_process_multi_header_lines },
> > +                 ngx_http_process_header_line },
> >  
> >      { ngx_null_string, 0, NULL }
> >  };
> > @@ -1742,10 +1740,10 @@ ngx_http_process_header_line(ngx_http_re
> >  
> >      ph = (ngx_table_elt_t **) ((char *) &r->headers_in + offset);
> >  
> > -    if (*ph == NULL) {
> > -        *ph = h;
> > -        h->next = NULL;
> > -    }
> > +    while (*ph) { ph = &(*ph)->next; }
> > +
> > +    *ph = h;
> > +    h->next = NULL;
> >  
> >      return NGX_OK;
> >  }
> 
> Essentially, this makes the following headers multi-valuable,
> which are specified (or otherwise implied) to have a single value:
> Referer, Content-Type, Range, Keep-Alive, X-Real-IP,
> also DAV headers (including Date) and User-Agent (as below).
> This makes proxying comma-delimited header values to backend, such as:
>   Date: Wed, 11 May 2022 19:29:54 GMT
>   Date: Wed, 11 May 2022 19:29:55 GMT
> ->
>   HTTP_DATE: Wed, 11 May 2022 19:29:54 GMT, Wed, 11 May 2022 19:29:55 GMT
> 
> Convert them to ngx_http_process_unique_header_line() ?

I don't think this change affects proxying: both previously and 
with the patch all headers were proxied.  The difference is that 
now nginx itself can access all the headers provided if it needs 
to (or reject particular requests if it the particular header 
might be important).

While rejecting requests with such duplicate headers might be 
correct thing to do, it might not be the optimal approach from the 
robustness point of view. 

> 
> > @@ -1851,13 +1849,10 @@ ngx_http_process_user_agent(ngx_http_req
> >  {
> >      u_char  *user_agent, *msie;
> >  
> > -    if (r->headers_in.user_agent) {
> > -        return NGX_OK;
> > +    if (ngx_http_process_header_line(r, h, offset) != NGX_OK) {
> > +        return NGX_ERROR;
> >      }
> >  
> > -    r->headers_in.user_agent = h;
> > -    h->next = NULL;
> > -
> >      /* check some widespread browsers while the header is in CPU cache */
> >  
> >      user_agent = h->value.data;
> > @@ -1919,23 +1914,6 @@ ngx_http_process_user_agent(ngx_http_req
> >  }
> >  
> >  
> > -static ngx_int_t
> > -ngx_http_process_multi_header_lines(ngx_http_request_t *r, ngx_table_elt_t *h,
> > -    ngx_uint_t offset)
> > -{
> > -    ngx_table_elt_t  **ph;
> > -
> > -    ph = (ngx_table_elt_t **) ((char *) &r->headers_in + offset);
> > -
> > -    while (*ph) { ph = &(*ph)->next; }
> > -
> > -    *ph = h;
> > -    h->next = NULL;
> > -
> > -    return NGX_OK;
> > -}
> > -
> > -
> >  ngx_int_t
> >  ngx_http_process_request_header(ngx_http_request_t *r)
> >  {
> > diff --git a/src/http/ngx_http_variables.c b/src/http/ngx_http_variables.c
> > --- a/src/http/ngx_http_variables.c
> > +++ b/src/http/ngx_http_variables.c
> > @@ -27,8 +27,6 @@ static ngx_int_t ngx_http_variable_heade
> >  
> >  static ngx_int_t ngx_http_variable_cookies(ngx_http_request_t *r,
> >      ngx_http_variable_value_t *v, uintptr_t data);
> > -static ngx_int_t ngx_http_variable_headers(ngx_http_request_t *r,
> > -    ngx_http_variable_value_t *v, uintptr_t data);
> >  static ngx_int_t ngx_http_variable_headers_internal(ngx_http_request_t *r,
> >      ngx_http_variable_value_t *v, uintptr_t data, u_char sep);
> >  
> > @@ -178,7 +176,7 @@ static ngx_http_variable_t  ngx_http_cor
> >  #endif
> >  
> >  #if (NGX_HTTP_X_FORWARDED_FOR)
> > -    { ngx_string("http_x_forwarded_for"), NULL, ngx_http_variable_headers,
> > +    { ngx_string("http_x_forwarded_for"), NULL, ngx_http_variable_header,
> >        offsetof(ngx_http_request_t, headers_in.x_forwarded_for), 0, 0 },
> >  #endif
> >  
> > @@ -327,10 +325,10 @@ static ngx_http_variable_t  ngx_http_cor
> >      { ngx_string("sent_http_transfer_encoding"), NULL,
> >        ngx_http_variable_sent_transfer_encoding, 0, 0, 0 },
> >  
> > -    { ngx_string("sent_http_cache_control"), NULL, ngx_http_variable_headers,
> > +    { ngx_string("sent_http_cache_control"), NULL, ngx_http_variable_header,
> >        offsetof(ngx_http_request_t, headers_out.cache_control), 0, 0 },
> >  
> > -    { ngx_string("sent_http_link"), NULL, ngx_http_variable_headers,
> > +    { ngx_string("sent_http_link"), NULL, ngx_http_variable_header,
> >        offsetof(ngx_http_request_t, headers_out.link), 0, 0 },
> >  
> >      { ngx_string("limit_rate"), ngx_http_variable_set_limit_rate,
> > @@ -807,22 +805,7 @@ static ngx_int_t
> >  ngx_http_variable_header(ngx_http_request_t *r, ngx_http_variable_value_t *v,
> >      uintptr_t data)
> >  {
> > -    ngx_table_elt_t  *h;
> > -
> > -    h = *(ngx_table_elt_t **) ((char *) r + data);
> > -
> > -    if (h) {
> > -        v->len = h->value.len;
> > -        v->valid = 1;
> > -        v->no_cacheable = 0;
> > -        v->not_found = 0;
> > -        v->data = h->value.data;
> > -
> > -    } else {
> > -        v->not_found = 1;
> > -    }
> > -
> > -    return NGX_OK;
> > +    return ngx_http_variable_headers_internal(r, v, data, ',');
> >  }
> >  
> >  
> > @@ -835,14 +818,6 @@ ngx_http_variable_cookies(ngx_http_reque
> >  
> >  
> >  static ngx_int_t
> > -ngx_http_variable_headers(ngx_http_request_t *r,
> > -    ngx_http_variable_value_t *v, uintptr_t data)
> > -{
> > -    return ngx_http_variable_headers_internal(r, v, data, ',');
> > -}
> > -
> > -
> > -static ngx_int_t
> >  ngx_http_variable_headers_internal(ngx_http_request_t *r,
> >      ngx_http_variable_value_t *v, uintptr_t data, u_char sep)
> >  {
> > 
> _______________________________________________
> nginx-devel mailing list -- nginx-devel at nginx.org
> To unsubscribe send an email to nginx-devel-leave at nginx.org

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Fri May 13 00:18:13 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 13 May 2022 03:18:13 +0300
Subject: [PATCH 11 of 20] Upstream: simplified Content-Encoding handling
In-Reply-To: <20220511200032.2mlm2ofg2ivgzuae@Y9MQ9X2QVV>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <6441069e16a0c4755c66.1650493131@vm-bsd.mdounin.ru>
 <20220511200032.2mlm2ofg2ivgzuae@Y9MQ9X2QVV>
Message-ID: <Yn2jxSlbbMiv2L9V@mdounin.ru>

Hello!

On Thu, May 12, 2022 at 12:00:32AM +0400, Sergey Kandaurov wrote:

> On Thu, Apr 21, 2022 at 01:18:51AM +0300, Maxim Dounin wrote:
> > # HG changeset patch
> > # User Maxim Dounin <mdounin at mdounin.ru>
> > # Date 1650492330 -10800
> > #      Thu Apr 21 01:05:30 2022 +0300
> > # Node ID 6441069e16a0c4755c662bc07bdcb0960c9ac04a
> > # Parent  ee1a8a4aa2c262d25a4aa871cda4f6c4515fc85c
> > Upstream: simplified Content-Encoding handling.
> > 
> > Since introduction of offset handling in ngx_http_upstream_copy_header_line()
> > in revision 573:58475592100c, the ngx_http_upstream_copy_content_encoding()
> > function is no longer needed, as its behaviour is exactly equivalent to
> > ngx_http_upstream_copy_header_line() with appropriate offset.  As such,
> > the ngx_http_upstream_copy_content_encoding() function was removed.
> > 
> > Further, the u->headers_in.content_encoding field is not used anywhere,
> > so it was removed as well.
> 
> jftr, it's used (seemingly incorrect) in ngx_http_redis for gzip decompression
> https://github.com/onnimonni/redis-nginx-module/commit/cbe2630fd070

This commit emulates previously existing processing of the 
"Content-Encoding: gzip" header.  Just removing the 
"u->headers_in.content_encoding = h;" line will suffice.

(Ideally, it should call hh->handler() instead, but this will 
require a lot more changes, notably compiling headers hash.  Just 
in case, in the memcached module similar code simply add r->headers_out 
header - which is also wrong, but won't somewhat simpler.)

> It might be better to improve gunzip filter itself as marked in TODO.

No changes in gunzip filter are needed for this.  Further, it 
doesn't need gunzip at all (assuming it is ok to ignore clients 
without gzip support): the code simply provides the 
"Content-Encoding: gzip" header in responses.

If you mean the "always gunzip" TODO, it is irrelevant here: in 
this case not all responses needs to be gunzipped (even assuming 
all responses are gzipped, which is not true, for example, in 
memcached, where gzipping is signalled by a flag stored in 
memcached).

> > 
> > Further, Content-Encoding handling no longer depends on NGX_HTTP_GZIP,
> > as it can be used even without any gzip handling compiled in (for example,
> > in the charset filter).
> > 
> > diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
> > --- a/src/http/ngx_http_upstream.c
> > +++ b/src/http/ngx_http_upstream.c
> > @@ -147,11 +147,6 @@ static ngx_int_t ngx_http_upstream_rewri
> >  static ngx_int_t ngx_http_upstream_copy_allow_ranges(ngx_http_request_t *r,
> >      ngx_table_elt_t *h, ngx_uint_t offset);
> >  
> > -#if (NGX_HTTP_GZIP)
> > -static ngx_int_t ngx_http_upstream_copy_content_encoding(ngx_http_request_t *r,
> > -    ngx_table_elt_t *h, ngx_uint_t offset);
> > -#endif
> > -
> >  static ngx_int_t ngx_http_upstream_add_variables(ngx_conf_t *cf);
> >  static ngx_int_t ngx_http_upstream_addr_variable(ngx_http_request_t *r,
> >      ngx_http_variable_value_t *v, uintptr_t data);
> > @@ -316,12 +311,10 @@ static ngx_http_upstream_header_t  ngx_h
> >                   ngx_http_upstream_process_transfer_encoding, 0,
> >                   ngx_http_upstream_ignore_header_line, 0, 0 },
> >  
> > -#if (NGX_HTTP_GZIP)
> >      { ngx_string("Content-Encoding"),
> > -                 ngx_http_upstream_process_header_line,
> > -                 offsetof(ngx_http_upstream_headers_in_t, content_encoding),
> > -                 ngx_http_upstream_copy_content_encoding, 0, 0 },
> > -#endif
> > +                 ngx_http_upstream_ignore_header_line, 0,
> > +                 ngx_http_upstream_copy_header_line,
> > +                 offsetof(ngx_http_headers_out_t, content_encoding), 0 },
> >  
> >      { ngx_null_string, NULL, 0, NULL, 0, 0 }
> >  };
> > @@ -5349,29 +5342,6 @@ ngx_http_upstream_copy_allow_ranges(ngx_
> >  }
> >  
> >  
> > -#if (NGX_HTTP_GZIP)
> > -
> > -static ngx_int_t
> > -ngx_http_upstream_copy_content_encoding(ngx_http_request_t *r,
> > -    ngx_table_elt_t *h, ngx_uint_t offset)
> > -{
> > -    ngx_table_elt_t  *ho;
> > -
> > -    ho = ngx_list_push(&r->headers_out.headers);
> > -    if (ho == NULL) {
> > -        return NGX_ERROR;
> > -    }
> > -
> > -    *ho = *h;
> > -
> > -    r->headers_out.content_encoding = ho;
> > -
> > -    return NGX_OK;
> > -}
> > -
> > -#endif
> > -
> > -
> >  static ngx_int_t
> >  ngx_http_upstream_add_variables(ngx_conf_t *cf)
> >  {
> > diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h
> > --- a/src/http/ngx_http_upstream.h
> > +++ b/src/http/ngx_http_upstream.h
> > @@ -285,10 +285,6 @@ typedef struct {
> >      ngx_table_elt_t                 *transfer_encoding;
> >      ngx_table_elt_t                 *vary;
> >  
> > -#if (NGX_HTTP_GZIP)
> > -    ngx_table_elt_t                 *content_encoding;
> > -#endif
> > -
> >      ngx_table_elt_t                 *cache_control;
> >      ngx_table_elt_t                 *set_cookie;
> >  
> > 
> 
> _______________________________________________
> nginx-devel mailing list -- nginx-devel at nginx.org
> To unsubscribe send an email to nginx-devel-leave at nginx.org

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Fri May 13 00:24:25 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 13 May 2022 03:24:25 +0300
Subject: [PATCH 15 of 20] Upstream: header handlers can now return
 parsing errors
In-Reply-To: <20220511202637.4kawhatgl6xcofbt@Y9MQ9X2QVV>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <ab424b5e32405aeec54c.1650493135@vm-bsd.mdounin.ru>
 <20220511202637.4kawhatgl6xcofbt@Y9MQ9X2QVV>
Message-ID: <Yn2lOVADjuo8VxH1@mdounin.ru>

Hello!

On Thu, May 12, 2022 at 12:26:37AM +0400, Sergey Kandaurov wrote:

> On Thu, Apr 21, 2022 at 01:18:55AM +0300, Maxim Dounin wrote:
> > # HG changeset patch
> > # User Maxim Dounin <mdounin at mdounin.ru>
> > # Date 1650492336 -10800
> > #      Thu Apr 21 01:05:36 2022 +0300
> > # Node ID ab424b5e32405aeec54ccdfe38e9408209209e0a
> > # Parent  b110c54778e8f6af3ea402c0838a4f289dcd813e
> > Upstream: header handlers can now return parsing errors.
> > 
> > With this change, duplicate Content-Length and Transfer-Encoding headers
> > are now rejected.  Further, responses with invalid Content-Length or
> > Transfer-Encoding headers are now rejected, as well as responses with both
> > Content-Length and Transfer-Encoding.
> 
> jftr, various 3rd party modules that call header handlers:
> mogilefs, passenger, ajp_module, nginx-clojure, srcache

This should degrade nicely to a generic NGX_ERROR.  Slightly less 
user-friendly, though shouldn't be a big issue even if duplicate 
headers are returned.

> 
> > 
> > diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c
> > --- a/src/http/modules/ngx_http_fastcgi_module.c
> > +++ b/src/http/modules/ngx_http_fastcgi_module.c
> > @@ -2007,8 +2007,12 @@ ngx_http_fastcgi_process_header(ngx_http
> >                  hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
> >                                     h->lowcase_key, h->key.len);
> >  
> > -                if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
> > -                    return NGX_ERROR;
> > +                if (hh) {
> > +                    rc = hh->handler(r, h, hh->offset);
> > +
> > +                    if (rc != NGX_OK) {
> > +                        return rc;
> > +                    }
> >                  }
> >  
> >                  ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
> > diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c
> > --- a/src/http/modules/ngx_http_grpc_module.c
> > +++ b/src/http/modules/ngx_http_grpc_module.c
> > @@ -1891,8 +1891,12 @@ ngx_http_grpc_process_header(ngx_http_re
> >                  hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
> >                                     h->lowcase_key, h->key.len);
> >  
> > -                if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
> > -                    return NGX_ERROR;
> > +                if (hh) {
> > +                    rc = hh->handler(r, h, hh->offset);
> > +
> > +                    if (rc != NGX_OK) {
> > +                        return rc;
> > +                    }
> >                  }
> >  
> >                  continue;
> > diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c
> > --- a/src/http/modules/ngx_http_proxy_module.c
> > +++ b/src/http/modules/ngx_http_proxy_module.c
> > @@ -1930,8 +1930,12 @@ ngx_http_proxy_process_header(ngx_http_r
> >              hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
> >                                 h->lowcase_key, h->key.len);
> >  
> > -            if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
> > -                return NGX_ERROR;
> > +            if (hh) {
> > +                rc = hh->handler(r, h, hh->offset);
> > +
> > +                if (rc != NGX_OK) {
> > +                    return rc;
> > +                }
> >              }
> >  
> >              ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
> > diff --git a/src/http/modules/ngx_http_scgi_module.c b/src/http/modules/ngx_http_scgi_module.c
> > --- a/src/http/modules/ngx_http_scgi_module.c
> > +++ b/src/http/modules/ngx_http_scgi_module.c
> > @@ -1114,8 +1114,12 @@ ngx_http_scgi_process_header(ngx_http_re
> >              hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
> >                                 h->lowcase_key, h->key.len);
> >  
> > -            if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
> > -                return NGX_ERROR;
> > +            if (hh) {
> > +                rc = hh->handler(r, h, hh->offset);
> > +
> > +                if (rc != NGX_OK) {
> > +                    return rc;
> > +                }
> >              }
> >  
> >              ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
> > diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c
> > --- a/src/http/modules/ngx_http_uwsgi_module.c
> > +++ b/src/http/modules/ngx_http_uwsgi_module.c
> > @@ -1340,8 +1340,12 @@ ngx_http_uwsgi_process_header(ngx_http_r
> >              hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
> >                                 h->lowcase_key, h->key.len);
> >  
> > -            if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
> > -                return NGX_ERROR;
> > +            if (hh) {
> > +                rc = hh->handler(r, h, hh->offset);
> > +
> > +                if (rc != NGX_OK) {
> > +                    return rc;
> > +                }
> >              }
> >  
> >              ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
> > diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
> > --- a/src/http/ngx_http_upstream.c
> > +++ b/src/http/ngx_http_upstream.c
> > @@ -4633,10 +4633,34 @@ ngx_http_upstream_process_content_length
> >  
> >      u = r->upstream;
> >  
> > +    if (u->headers_in.content_length) {
> > +        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> > +                      "upstream sent duplicate header line: \"%V: %V\", "
> > +                      "previous value: \"%V: %V\"",
> > +                      &h->key, &h->value,
> > +                      &u->headers_in.content_length->key,
> > +                      &u->headers_in.content_length->value);
> > +        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
> > +    }
> > +
> > +    if (u->headers_in.transfer_encoding) {
> > +        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> > +                      "upstream sent \"Content-Length\" and "
> > +                      "\"Transfer-Encoding\" headers at the same time");
> > +        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
> > +    }
> > +
> >      h->next = NULL;
> >      u->headers_in.content_length = h;
> >      u->headers_in.content_length_n = ngx_atoof(h->value.data, h->value.len);
> >  
> > +    if (u->headers_in.content_length_n == NGX_ERROR) {
> > +        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> > +                      "upstream sent invalid \"Content-Length\" header: "
> > +                      "\"%V: %V\"", &h->key, &h->value);
> > +        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
> > +    }
> > +
> >      return NGX_OK;
> >  }
> >  
> > @@ -5021,14 +5045,37 @@ ngx_http_upstream_process_transfer_encod
> >      ngx_http_upstream_t  *u;
> >  
> >      u = r->upstream;
> > +
> > +    if (u->headers_in.transfer_encoding) {
> > +        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> > +                      "upstream sent duplicate header line: \"%V: %V\", "
> > +                      "previous value: \"%V: %V\"",
> > +                      &h->key, &h->value,
> > +                      &u->headers_in.transfer_encoding->key,
> > +                      &u->headers_in.transfer_encoding->value);
> > +        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
> > +    }
> > +
> > +    if (u->headers_in.content_length) {
> > +        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> > +                      "upstream sent \"Content-Length\" and "
> > +                      "\"Transfer-Encoding\" headers at the same time");
> > +        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
> > +    }
> > +
> >      u->headers_in.transfer_encoding = h;
> >      h->next = NULL;
> >  
> > -    if (ngx_strlcasestrn(h->value.data, h->value.data + h->value.len,
> > -                         (u_char *) "chunked", 7 - 1)
> > -        != NULL)
> > +    if (h->value.len == 7
> > +        && ngx_strncasecmp(h->value.data, (u_char *) "chunked", 7) == 0)
> >      {
> >          u->headers_in.chunked = 1;
> > +
> > +    } else {
> > +        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> > +                      "upstream sent unknown \"Transfer-Encoding\": \"%V\"",
> > +                      &h->value);
> > +        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
> >      }
> >  
> >      return NGX_OK;
> 
> This is a subtle(?) change, which makes "chunked" the only valid header value.
> OTOH, it looks fine since other values require explicit support as well.

Note "responses with invalid ... Transfer-Encoding headers are 
now rejected" in the commit log.

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Fri May 13 00:34:16 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 13 May 2022 03:34:16 +0300
Subject: [PATCH 16 of 20] Upstream: duplicate headers ignored or properly
 linked
In-Reply-To: <20220511203353.jvagykody7lfkhuv@Y9MQ9X2QVV>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <f460a2f9f88d264ef6c8.1650493136@vm-bsd.mdounin.ru>
 <20220511203353.jvagykody7lfkhuv@Y9MQ9X2QVV>
Message-ID: <Yn2niHSSrLV6S8zb@mdounin.ru>

Hello!

On Thu, May 12, 2022 at 12:33:53AM +0400, Sergey Kandaurov wrote:

> On Thu, Apr 21, 2022 at 01:18:56AM +0300, Maxim Dounin wrote:
> > # HG changeset patch
> > # User Maxim Dounin <mdounin at mdounin.ru>
> > # Date 1650492338 -10800
> > #      Thu Apr 21 01:05:38 2022 +0300
> > # Node ID f460a2f9f88d264ef6c8588eb37bcb85c48010db
> > # Parent  ab424b5e32405aeec54ccdfe38e9408209209e0a
> > Upstream: duplicate headers ignored or properly linked.
> > 
> > Most of the known duplicate upstream response headers are now ignored
> > with a warning.
> > 
> > If syntax permits multiple headers, these are now properly linked to
> > the lists, notably Vary and WWW-Authenticate.  This makes it possible
> > to further handle such lists where it makes sense.
> 
> This reminds me of curl deficiency to handle WWW-Authenticate
> as a comma-separate multi-value header (which nginx doesn't emit though):
> https://curl.se/mail/lib-2020-01/0064.html
> https://curl.se/docs/knownbugs.html
> 
> > 
> > diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
> > --- a/src/http/ngx_http_upstream.c
> > +++ b/src/http/ngx_http_upstream.c
> > @@ -101,6 +101,9 @@ static void ngx_http_upstream_finalize_r
> >  
> >  static ngx_int_t ngx_http_upstream_process_header_line(ngx_http_request_t *r,
> >      ngx_table_elt_t *h, ngx_uint_t offset);
> > +static ngx_int_t
> > +    ngx_http_upstream_process_multi_header_lines(ngx_http_request_t *r,
> > +    ngx_table_elt_t *h, ngx_uint_t offset);
> 
> I'd suggest renaming this pair to ngx_http_upstream_process_header_line /
> ngx_http_upstream_process_unique_header_line to be on par with functions
> in ngx_http_request.c (and after the 7th renaming patch of this series).

The ngx_http_upstream_process_unique_header_line() implies that an 
attempt to provide a duplicate header will result in an error.  In 
contrast, ngx_http_upstream_process_header_line() simply ignores 
extra header lines (with a warning), so the name is intentionally 
preserved as is.

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Fri May 13 00:50:02 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 13 May 2022 03:50:02 +0300
Subject: [PATCH 17 of 20] Upstream: handling of multiple Vary headers
 (ticket #1423)
In-Reply-To: <20220511204452.xycprosojrnmsmrd@Y9MQ9X2QVV>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <2027b85971d4b8a7e33c.1650493137@vm-bsd.mdounin.ru>
 <20220511204452.xycprosojrnmsmrd@Y9MQ9X2QVV>
Message-ID: <Yn2rOuQXCOg/Wa9j@mdounin.ru>

Hello!

On Thu, May 12, 2022 at 12:44:52AM +0400, Sergey Kandaurov wrote:

> On Thu, Apr 21, 2022 at 01:18:57AM +0300, Maxim Dounin wrote:
> > # HG changeset patch
> > # User Maxim Dounin <mdounin at mdounin.ru>
> > # Date 1650492340 -10800
> > #      Thu Apr 21 01:05:40 2022 +0300
> > # Node ID 2027b85971d4b8a7e33c018548468057cb57eaf7
> > # Parent  f460a2f9f88d264ef6c8588eb37bcb85c48010db
> > Upstream: handling of multiple Vary headers (ticket #1423).
> > 
> > Previously, only the last header value was used when caching.
> > 
> > diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
> > --- a/src/http/ngx_http_upstream.c
> > +++ b/src/http/ngx_http_upstream.c
> > @@ -5175,6 +5175,9 @@ static ngx_int_t
> >  ngx_http_upstream_process_vary(ngx_http_request_t *r,
> >      ngx_table_elt_t *h, ngx_uint_t offset)
> >  {
> > +    u_char                *p;
> > +    size_t                 len;
> > +    ngx_str_t              vary;
> >      ngx_table_elt_t      **ph;
> >      ngx_http_upstream_t   *u;
> >  
> > @@ -5192,17 +5195,47 @@ ngx_http_upstream_process_vary(ngx_http_
> >          return NGX_OK;
> >      }
> >  
> > -    if (r->cache == NULL) {
> > +    if (r->cache == NULL || !u->cacheable) {
> > +        return NGX_OK;
> > +    }
> > +
> > +    if (h->value.len == 1 && h->value.data[0] == '*') {
> > +        u->cacheable = 0;
> >          return NGX_OK;
> >      }
> >  
> > -    if (h->value.len > NGX_HTTP_CACHE_VARY_LEN
> > -        || (h->value.len == 1 && h->value.data[0] == '*'))
> > -    {
> > +    if (u->headers_in.vary->next) {
> > +
> > +        len = 0;
> > +
> > +        for (h = u->headers_in.vary; h; h = h->next) {
> > +            len += h->value.len + 2;
> > +        }
> > +
> > +        len -= 2;
> > +
> > +        p = ngx_pnalloc(r->pool, len);
> > +        if (p == NULL) {
> > +            return NGX_ERROR;
> > +        }
> > +
> > +        vary.len = len;
> > +        vary.data = p;
> > +
> > +        for (h = u->headers_in.vary; h; h = h->next) {
> > +            p = ngx_copy(p, h->value.data, h->value.len);
> 
> (h->next == NULL) check missing

Thanks, fixed.

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -5224,6 +5224,11 @@ ngx_http_upstream_process_vary(ngx_http_
 
         for (h = u->headers_in.vary; h; h = h->next) {
             p = ngx_copy(p, h->value.data, h->value.len);
+
+            if (h->next == NULL) {
+                break;
+            }
+
             *p++ = ','; *p++ = ' ';
         }
 


> 
> > +            *p++ = ','; *p++ = ' ';
> > +        }
> > +
> > +    } else {
> > +        vary = h->value;
> > +    }
> > +
> > +    if (vary.len > NGX_HTTP_CACHE_VARY_LEN) {
> >          u->cacheable = 0;
> >      }
> >  
> > -    r->cache->vary = h->value;
> > +    r->cache->vary = vary;
> >  
> >  #endif
> >  
> 
> Memory from the pool could be allocated just once among multiple values
> if postpone it to somewhere else (but probably it's not worth it).

Yes, I've considered it, yet it looks much more readable to 
preserve handling as is.

This shouldn't be an issue in valid cases, as there shouldn't 
more than a couple of Vary header lines.  And assuming an attempt 
to consume as much memory as it can, this will be something like 
(2 + 4 + 6 + ... + 128) = 32 * 130 = ~4k, which isn't significant.

[...]

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Fri May 13 01:57:14 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 13 May 2022 04:57:14 +0300
Subject: [PATCH 18 of 20] Upstream: multiple WWW-Authenticate headers
 (ticket #485)
In-Reply-To: <20220511210337.nhxkr2alnri4immp@Y9MQ9X2QVV>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <fa9751ffe7723a11159c.1650493138@vm-bsd.mdounin.ru>
 <20220511210337.nhxkr2alnri4immp@Y9MQ9X2QVV>
Message-ID: <Yn26+uz/ffNpUZ7l@mdounin.ru>

Hello!

On Thu, May 12, 2022 at 01:03:37AM +0400, Sergey Kandaurov wrote:

> On Thu, Apr 21, 2022 at 01:18:58AM +0300, Maxim Dounin wrote:
> > # HG changeset patch
> > # User Maxim Dounin <mdounin at mdounin.ru>
> > # Date 1650492341 -10800
> > #      Thu Apr 21 01:05:41 2022 +0300
> > # Node ID fa9751ffe7723a11159c158078e454671e81cb87
> > # Parent  2027b85971d4b8a7e33c018548468057cb57eaf7
> > Upstream: multiple WWW-Authenticate headers (ticket #485).
> > 
> > When using proxy_intercept_errors and an error page for error 401
> > (Unauthorized), multiple WWW-Authenticate headers from the upstream server
> > response are now properly copied to the response.
> > 
> > diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
> > --- a/src/http/ngx_http_upstream.c
> > +++ b/src/http/ngx_http_upstream.c
> > @@ -2647,7 +2647,7 @@ ngx_http_upstream_intercept_errors(ngx_h
> >  {
> >      ngx_int_t                  status;
> >      ngx_uint_t                 i;
> > -    ngx_table_elt_t           *h;
> > +    ngx_table_elt_t           *h, *ho, **ph;
> >      ngx_http_err_page_t       *err_page;
> >      ngx_http_core_loc_conf_t  *clcf;
> >  
> > @@ -2676,18 +2676,26 @@ ngx_http_upstream_intercept_errors(ngx_h
> >              if (status == NGX_HTTP_UNAUTHORIZED
> >                  && u->headers_in.www_authenticate)
> >              {
> > -                h = ngx_list_push(&r->headers_out.headers);
> > -
> > -                if (h == NULL) {
> > -                    ngx_http_upstream_finalize_request(r, u,
> > +                h = u->headers_in.www_authenticate;
> > +                ph = &r->headers_out.www_authenticate;
> > +
> > +                while (h) {
> > +                    ho = ngx_list_push(&r->headers_out.headers);
> > +
> > +                    if (ho == NULL) {
> > +                        ngx_http_upstream_finalize_request(r, u,
> >                                                 NGX_HTTP_INTERNAL_SERVER_ERROR);
> > -                    return NGX_OK;
> > +                        return NGX_OK;
> > +                    }
> > +
> > +                    *ho = *h;
> > +                    ho->next = NULL;
> > +
> > +                    *ph = ho;
> > +                    ph = &ho->next;
> > +
> > +                    h = h->next;
> >                  }
> > -
> > -                *h = *u->headers_in.www_authenticate;
> > -                h->next = NULL;
> > -
> > -                r->headers_out.www_authenticate = h;
> >              }
> >  
> >  #if (NGX_HTTP_CACHE)
> > 
> 
> While the patch certainly looks correct,
> I'm not sure about usefulness of r->headers_out.www_authenticate.
> Is the header accessed directly through this pointer ever?
> For read purposes I mean.  Header filters seem not.
> 
> The only place is in the auth_request module,
> but it doesn't seem to make sense in subrequest.

The auth request module uses it to copy the WWW-Authenticate 
headers from the subrequest response to the main request.  For 
example, in a configuration like:

    location / { auth_request /auth; ... }
    location /auth { auth_basic foo; ... }

While such configurations are unlikely, as using auth_basic 
directly should be easier, these are certainly possible.

Similarly, r->headers_out.www_authenticate can be used with 
proxy_intercept_errors + error_page 401 in the subrequest (since 
r->upstream can be cleared by the error page).

BTW, looking at this once again, I tend to think we also need 
ngx_http_core_access_phase() changes for proper handling of 
multiple WWW-Authenticate headers.  It might not be a good idea to 
commit it right now though, see commit log.

On the other hand, breaking it all at once might be less painful 
(after all, the auth_request change will already made such modules 
to segfault when used with auth_request; though this is unlikely, 
since such configurations hardly make sense).

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1652405911 -10800
#      Fri May 13 04:38:31 2022 +0300
# Node ID 0c2cf997bde390c9be27e0685ee2a00024c57e4e
# Parent  c53cb6f31b9855020d85476fa98ef1cdca809aed
Multiple WWW-Authenticate headers with "satisfy any;".

If a module adds multiple WWW-Authenticate headers (ticket #485) to the
response, linked in r->headers_out.www_authenticate, all headers are now
cleared if another module later allows access.

This change is a nop for standard modules, since the only access module which
can add multiple WWW-Authenticate headers is the auth request module, and
it is checked after other standard access modules.  Though this might
affect some third party access modules.

Note that if a 3rd party module adds a single WWW-Authenticate header
and not yet modified to set the header's next pointer to NULL, attempt to
clear such a header with this change will result in a segmentation fault.

diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c
+++ b/src/http/ngx_http_core_module.c
@@ -1088,6 +1088,7 @@ ngx_int_t
 ngx_http_core_access_phase(ngx_http_request_t *r, ngx_http_phase_handler_t *ph)
 {
     ngx_int_t                  rc;
+    ngx_table_elt_t           *h;
     ngx_http_core_loc_conf_t  *clcf;
 
     if (r != r->main) {
@@ -1122,8 +1123,8 @@ ngx_http_core_access_phase(ngx_http_requ
         if (rc == NGX_OK) {
             r->access_code = 0;
 
-            if (r->headers_out.www_authenticate) {
-                r->headers_out.www_authenticate->hash = 0;
+            for (h = r->headers_out.www_authenticate; h; h = h->next) {
+                h->hash = 0;
             }
 
             r->phase_handler = ph->next;

-- 
Maxim Dounin
http://mdounin.ru/


From pluknet at nginx.com  Fri May 13 14:01:28 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Fri, 13 May 2022 18:01:28 +0400
Subject: [PATCH 02 of 20] FastCGI: combining headers with identical names
 (ticket #1724)
In-Reply-To: <Yn2K8MLJpK+KasJt@mdounin.ru>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <61b29233a55216c6fa72.1650493122@vm-bsd.mdounin.ru>
 <20220511153232.c3fd4i573xt64hnc@Y9MQ9X2QVV> <Yn2K8MLJpK+KasJt@mdounin.ru>
Message-ID: <EA0501DD-9B2F-49BD-9BFD-1C970E6E1568@nginx.com>



> On 13 May 2022, at 02:32, Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> Hello!
> 
> On Wed, May 11, 2022 at 07:32:32PM +0400, Sergey Kandaurov wrote:
> 
>> On Thu, Apr 21, 2022 at 01:18:42AM +0300, Maxim Dounin wrote:
>>> # HG changeset patch
>>> # User Maxim Dounin <mdounin at mdounin.ru>
>>> # Date 1650492316 -10800
>>> #      Thu Apr 21 01:05:16 2022 +0300
>>> # Node ID 61b29233a55216c6fa72e23b93a4a28d76a9fb94
>>> # Parent  e70fb0fdfbc0fb7b7e9f493cc2eb65de617b115a
>>> FastCGI: combining headers with identical names (ticket #1724).

[..]

>>> diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
>>> --- a/src/http/ngx_http_core_module.c
>>> +++ b/src/http/ngx_http_core_module.c
>>> @@ -2802,6 +2802,78 @@ ngx_http_get_forwarded_addr_internal(ngx
>>> }
>>> 
>>> 
>>> +ngx_int_t
>>> +ngx_http_link_multi_headers(ngx_http_request_t *r)
>>> +{
>>> +    ngx_uint_t        i, j;
>>> +    ngx_list_part_t  *part, *ppart;
>>> +    ngx_table_elt_t  *header, *pheader, **ph;
>>> +
>>> +    if (r->headers_in.multi_linked) {
>>> +        return NGX_OK;
>>> +    }
>> 
>> multi_linked is never set, was the intension to avoid doing the work
>> twice on repetitive invocation? it doesn't seem to be possible.
>> I'd just axe it.
> 
> Yes, thanks, missed this somehow.  It should be set right after 
> the test, updated with the following change:
> 
> diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
> --- a/src/http/ngx_http_core_module.c
> +++ b/src/http/ngx_http_core_module.c
> @@ -2813,6 +2813,8 @@ ngx_http_link_multi_headers(ngx_http_req
>         return NGX_OK;
>     }
> 
> +    r->headers_in.multi_linked = 1;
> +
>     part = &r->headers_in.headers.part;
>     header = part->elts;
> 
> 
> Not sure what do you mean by "doesn't seem to be possible".  The 
> request headers are never changed after parsing, so it is enough 
> to link them just once.

Nevermind, the point was the function itself is called just once.
But it's not so as part of internal redirect to another upstream.

[..]

-- 
Sergey Kandaurov


From pluknet at nginx.com  Fri May 13 14:02:39 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Fri, 13 May 2022 18:02:39 +0400
Subject: [PATCH 02 of 20] FastCGI: combining headers with identical names
 (ticket #1724)
In-Reply-To: <Yn2K8MLJpK+KasJt@mdounin.ru>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <61b29233a55216c6fa72.1650493122@vm-bsd.mdounin.ru>
 <20220511153232.c3fd4i573xt64hnc@Y9MQ9X2QVV> <Yn2K8MLJpK+KasJt@mdounin.ru>
Message-ID: <B5846A81-C61F-4B5F-A652-BC75BEFBD12B@nginx.com>



> On 13 May 2022, at 02:32, Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> Hello!
> 
> On Wed, May 11, 2022 at 07:32:32PM +0400, Sergey Kandaurov wrote:
> 
>> On Thu, Apr 21, 2022 at 01:18:42AM +0300, Maxim Dounin wrote:
>>> # HG changeset patch
>>> # User Maxim Dounin <mdounin at mdounin.ru>
>>> # Date 1650492316 -10800
>>> #      Thu Apr 21 01:05:16 2022 +0300
>>> # Node ID 61b29233a55216c6fa72e23b93a4a28d76a9fb94
>>> # Parent  e70fb0fdfbc0fb7b7e9f493cc2eb65de617b115a
>>> FastCGI: combining headers with identical names (ticket #1724).

[..]

>>> diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
>>> --- a/src/http/ngx_http_core_module.c
>>> +++ b/src/http/ngx_http_core_module.c
>>> @@ -2802,6 +2802,78 @@ ngx_http_get_forwarded_addr_internal(ngx
>>> }
>>> 
>>> 
>>> +ngx_int_t
>>> +ngx_http_link_multi_headers(ngx_http_request_t *r)
>>> +{
>>> +    ngx_uint_t        i, j;
>>> +    ngx_list_part_t  *part, *ppart;
>>> +    ngx_table_elt_t  *header, *pheader, **ph;
>>> +
>>> +    if (r->headers_in.multi_linked) {
>>> +        return NGX_OK;
>>> +    }
>> 
>> multi_linked is never set, was the intension to avoid doing the work
>> twice on repetitive invocation? it doesn't seem to be possible.
>> I'd just axe it.
> 
> Yes, thanks, missed this somehow.  It should be set right after 
> the test, updated with the following change:
> 
> diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
> --- a/src/http/ngx_http_core_module.c
> +++ b/src/http/ngx_http_core_module.c
> @@ -2813,6 +2813,8 @@ ngx_http_link_multi_headers(ngx_http_req
>        return NGX_OK;
>    }
> 
> +    r->headers_in.multi_linked = 1;
> +
>    part = &r->headers_in.headers.part;
>    header = part->elts;
> 
> 
> Not sure what do you mean by "doesn't seem to be possible".  The 
> request headers are never changed after parsing, so it is enough 
> to link them just once.

Nevermind, the point was the function itself is called just once.
But it's not so as part of internal redirect to another upstream.

[..]

-- 
Sergey Kandaurov


From xeioex at nginx.com  Tue May 17 22:54:04 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 17 May 2022 22:54:04 +0000
Subject: [njs] Fixed compatibility with BoringSSL.
Message-ID: <hg.30e03dbc6baf.1652828044.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/30e03dbc6baf
branches:  
changeset: 1856:30e03dbc6baf
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon May 16 22:57:27 2022 -0700
description:
Fixed compatibility with BoringSSL.

The fix is to use more conventional API when dealing with HMAC
algorithm.

This closes #455 issue on Github.

diffstat:

 external/njs_openssl.h          |   1 +
 external/njs_webcrypto_module.c |  83 ++++++++++------------------------------
 2 files changed, 23 insertions(+), 61 deletions(-)

diffs (182 lines):

diff -r 6a28cdbc8cb6 -r 30e03dbc6baf external/njs_openssl.h
--- a/external/njs_openssl.h	Wed May 11 21:08:21 2022 -0700
+++ b/external/njs_openssl.h	Mon May 16 22:57:27 2022 -0700
@@ -19,6 +19,7 @@
 #include <openssl/rsa.h>
 #include <openssl/err.h>
 #include <openssl/rand.h>
+#include <openssl/hmac.h>
 #include <openssl/crypto.h>
 
 #ifdef EVP_PKEY_HKDF
diff -r 6a28cdbc8cb6 -r 30e03dbc6baf external/njs_webcrypto_module.c
--- a/external/njs_webcrypto_module.c	Wed May 11 21:08:21 2022 -0700
+++ b/external/njs_webcrypto_module.c	Mon May 16 22:57:27 2022 -0700
@@ -1254,14 +1254,12 @@ njs_ext_derive(njs_vm_t *vm, njs_value_t
     u_char                     *k;
     size_t                     olen;
     int64_t                    iterations, length;
-    EVP_PKEY                   *pkey;
     unsigned                   usage, mask;
     njs_int_t                  ret;
     njs_str_t                  salt, info;
     njs_value_t                value, *aobject, *dobject;
     const EVP_MD               *md;
     EVP_PKEY_CTX               *pctx;
-    njs_mp_cleanup_t           *cln;
     njs_webcrypto_key_t        *key, *dkey;
     njs_webcrypto_hash_t       hash;
     njs_webcrypto_algorithm_t  *alg, *dalg;
@@ -1544,29 +1542,11 @@ free:
             if (njs_slow_path(ret == NJS_ERROR)) {
                 goto fail;
             }
-
-            pkey = EVP_PKEY_new_mac_key(EVP_PKEY_HMAC, NULL, k, length);
-            if (njs_slow_path(pkey == NULL)) {
-                njs_webcrypto_error(vm, "EVP_PKEY_new_mac_key() failed");
-                goto fail;
-            }
-
-            cln = njs_mp_cleanup_add(njs_vm_memory_pool(vm), 0);
-            if (cln == NULL) {
-                njs_memory_error(vm);
-                goto fail;
-            }
-
-            cln->handler = njs_webcrypto_cleanup_pkey;
-            cln->data = key;
-
-            dkey->pkey = pkey;
-
-        } else {
-            dkey->raw.start = k;
-            dkey->raw.length = length;
         }
 
+        dkey->raw.start = k;
+        dkey->raw.length = length;
+
         ret = njs_vm_external_create(vm, &value,
                                      njs_webcrypto_crypto_key_proto_id,
                                      dkey, 0);
@@ -1856,21 +1836,12 @@ njs_ext_import_key(njs_vm_t *vm, njs_val
         break;
 
     case NJS_ALGORITHM_HMAC:
-        pkey = EVP_PKEY_new_mac_key(EVP_PKEY_HMAC, NULL, key_data.start,
-                                    key_data.length);
-        if (njs_slow_path(pkey == NULL)) {
-            njs_webcrypto_error(vm, "EVP_PKEY_new_mac_key() failed");
-            goto fail;
-        }
-
         ret = njs_algorithm_hash(vm, options, &key->hash);
         if (njs_slow_path(ret == NJS_ERROR)) {
             goto fail;
         }
 
-        key->pkey = pkey;
-
-        break;
+        /* Fall through. */
 
     case NJS_ALGORITHM_AES_GCM:
     case NJS_ALGORITHM_AES_CTR:
@@ -1968,7 +1939,7 @@ static njs_int_t
 njs_ext_sign(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
     njs_index_t verify)
 {
-    u_char                     *dst;
+    u_char                     *dst, *p;
     size_t                     olen, outlen;
     unsigned                   mask, m_len;
     njs_int_t                  ret;
@@ -2030,12 +2001,6 @@ njs_ext_sign(njs_vm_t *vm, njs_value_t *
         }
     }
 
-    mctx = njs_evp_md_ctx_new();
-    if (njs_slow_path(mctx == NULL)) {
-        njs_webcrypto_error(vm, "njs_evp_md_ctx_new() failed");
-        goto fail;
-    }
-
     if (alg->type == NJS_ALGORITHM_ECDSA) {
         ret = njs_algorithm_hash(vm, options, &hash);
         if (njs_slow_path(ret == NJS_ERROR)) {
@@ -2052,22 +2017,10 @@ njs_ext_sign(njs_vm_t *vm, njs_value_t *
 
     switch (alg->type) {
     case NJS_ALGORITHM_HMAC:
-        ret = EVP_DigestSignInit(mctx, NULL, md, NULL, key->pkey);
-        if (njs_slow_path(ret <= 0)) {
-            njs_webcrypto_error(vm, "EVP_DigestSignInit() failed");
-            goto fail;
-        }
-
-        ret = EVP_DigestSignUpdate(mctx, data.start, data.length);
-        if (njs_slow_path(ret <= 0)) {
-            njs_webcrypto_error(vm, "EVP_DigestSignUpdate() failed");
-            goto fail;
-        }
-
-        olen = EVP_MD_size(md);
+        m_len = EVP_MD_size(md);
 
         if (!verify) {
-            dst = njs_mp_zalloc(njs_vm_memory_pool(vm), olen);
+            dst = njs_mp_alloc(njs_vm_memory_pool(vm), m_len);
             if (njs_slow_path(dst == NULL)) {
                 njs_memory_error(vm);
                 goto fail;
@@ -2077,11 +2030,13 @@ njs_ext_sign(njs_vm_t *vm, njs_value_t *
             dst = (u_char *) &m[0];
         }
 
-        outlen = olen;
-
-        ret = EVP_DigestSignFinal(mctx, dst, &outlen);
-        if (njs_slow_path(ret <= 0 || olen != outlen)) {
-            njs_webcrypto_error(vm, "EVP_DigestSignFinal() failed");
+        outlen = m_len;
+
+        p = HMAC(md, key->raw.start, key->raw.length, data.start, data.length,
+                 dst, &m_len);
+
+        if (njs_slow_path(p == NULL || m_len != outlen)) {
+            njs_webcrypto_error(vm, "HMAC() failed");
             goto fail;
         }
 
@@ -2095,6 +2050,12 @@ njs_ext_sign(njs_vm_t *vm, njs_value_t *
     case NJS_ALGORITHM_RSA_PSS:
     case NJS_ALGORITHM_ECDSA:
     default:
+        mctx = njs_evp_md_ctx_new();
+        if (njs_slow_path(mctx == NULL)) {
+            njs_webcrypto_error(vm, "njs_evp_md_ctx_new() failed");
+            goto fail;
+        }
+
         ret = EVP_DigestInit_ex(mctx, md, NULL);
         if (njs_slow_path(ret <= 0)) {
             njs_webcrypto_error(vm, "EVP_DigestInit_ex() failed");
@@ -2168,6 +2129,8 @@ njs_ext_sign(njs_vm_t *vm, njs_value_t *
             }
         }
 
+        njs_evp_md_ctx_free(mctx);
+
         EVP_PKEY_CTX_free(pctx);
 
         break;
@@ -2183,8 +2146,6 @@ njs_ext_sign(njs_vm_t *vm, njs_value_t *
         njs_set_boolean(&value, ret != 0);
     }
 
-    njs_evp_md_ctx_free(mctx);
-
     return njs_webcrypto_result(vm, &value, NJS_OK);
 
 fail:


From arut at nginx.com  Wed May 18 06:57:13 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Wed, 18 May 2022 10:57:13 +0400
Subject: [PATCH 0 of 2] QUIC connection reuse
Message-ID: <patchbomb.1652857033@arut-laptop>

- patch #1 adds support for reusable and idle modes for the main QUIC
  conneciton.  They allow to close QUIC connections that have no active
  request streams when connections are needed for a new client or when nginx
  worker is shutting down.

- patch #2 allows to switch main QUIC connection to reusable and idle modes
  after handshake and util streams are created.  A new callback init_streams()
  is added for this.


From arut at nginx.com  Wed May 18 06:57:14 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Wed, 18 May 2022 10:57:14 +0400
Subject: [PATCH 1 of 2] QUIC: reusable and idle modes for main connection
In-Reply-To: <patchbomb.1652857033@arut-laptop>
References: <patchbomb.1652857033@arut-laptop>
Message-ID: <67ae4b649f2e38a44b24.1652857034@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1652852691 -14400
#      Wed May 18 09:44:51 2022 +0400
# Branch quic
# Node ID 67ae4b649f2e38a44b245b7a842cf396c8250f02
# Parent  c2f5d79cde64457f1fa7344c56a5248a677a7e46
QUIC: reusable and idle modes for main connection.

The modes are controlled by application layer.  For HTTP/3, they are enabled
when there are no active request streams.

diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c
--- a/src/event/quic/ngx_event_quic.c
+++ b/src/event/quic/ngx_event_quic.c
@@ -414,8 +414,8 @@ ngx_quic_input_handler(ngx_event_t *rev)
     }
 
     if (c->close) {
-        qc->error_reason = "graceful shutdown";
-        ngx_quic_close_connection(c, NGX_OK);
+        qc->error = NGX_QUIC_ERR_NO_ERROR;
+        ngx_quic_close_connection(c, NGX_ERROR);
         return;
     }
 
diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c
--- a/src/event/quic/ngx_event_quic_streams.c
+++ b/src/event/quic/ngx_event_quic_streams.c
@@ -161,6 +161,7 @@ ngx_quic_close_streams(ngx_connection_t 
     ngx_pool_t         *pool;
     ngx_queue_t        *q;
     ngx_rbtree_t       *tree;
+    ngx_connection_t   *sc;
     ngx_rbtree_node_t  *node;
     ngx_quic_stream_t  *qs;
 
@@ -185,24 +186,41 @@ ngx_quic_close_streams(ngx_connection_t 
         return NGX_OK;
     }
 
-#if (NGX_DEBUG)
-    ns = 0;
-#endif
-
     node = ngx_rbtree_min(tree->root, tree->sentinel);
 
     while (node) {
         qs = (ngx_quic_stream_t *) node;
         node = ngx_rbtree_next(tree, node);
+        sc = qs->connection;
+
+        if (sc == NULL) {
+            ngx_quic_close_stream(qs);
+            continue;
+        }
+
+        if (c->close || sc->idle || sc->reusable) {
+            sc->close = 1;
+            sc->read->handler(sc->read);
+        }
+    }
+
+    if (tree->root == tree->sentinel) {
+        return NGX_OK;
+    }
+
+#if (NGX_DEBUG)
+    ns = 0;
+#endif
+
+    for (node = ngx_rbtree_min(tree->root, tree->sentinel);
+         node;
+         node = ngx_rbtree_next(tree, node))
+    {
+        qs = (ngx_quic_stream_t *) node;
 
         qs->recv_state = NGX_QUIC_STREAM_RECV_RESET_RECVD;
         qs->send_state = NGX_QUIC_STREAM_SEND_RESET_SENT;
 
-        if (qs->connection == NULL) {
-            ngx_quic_close_stream(qs);
-            continue;
-        }
-
         ngx_quic_set_event(qs->connection->read);
         ngx_quic_set_event(qs->connection->write);
 
@@ -587,6 +605,7 @@ ngx_quic_create_stream(ngx_connection_t 
 {
     ngx_log_t              *log;
     ngx_pool_t             *pool;
+    ngx_uint_t              reusable;
     ngx_queue_t            *q;
     ngx_connection_t       *sc;
     ngx_quic_stream_t      *qs;
@@ -639,7 +658,13 @@ ngx_quic_create_stream(ngx_connection_t 
     *log = *c->log;
     pool->log = log;
 
+    reusable = c->reusable;
+    ngx_reusable_connection(c, 0);
+
     sc = ngx_get_connection(c->fd, log);
+
+    ngx_reusable_connection(c, reusable);
+
     if (sc == NULL) {
         ngx_destroy_pool(pool);
         ngx_queue_insert_tail(&qc->streams.free, &qs->queue);
diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c
--- a/src/http/v3/ngx_http_v3_request.c
+++ b/src/http/v3/ngx_http_v3_request.c
@@ -210,6 +210,11 @@ ngx_http_v3_init_request_stream(ngx_conn
 
     h3c = ngx_http_v3_get_session(c);
 
+    if (ngx_terminate || ngx_exiting) {
+        h3c->goaway = 1;
+        (void) ngx_http_v3_send_goaway(c, c->quic->id);
+    }
+
     if (h3c->goaway) {
         c->close = 1;
         ngx_http_close_connection(c);
@@ -258,7 +263,7 @@ ngx_http_v3_wait_request_handler(ngx_eve
     size_t                     size;
     ssize_t                    n;
     ngx_buf_t                 *b;
-    ngx_connection_t          *c;
+    ngx_connection_t          *c, *pc;
     ngx_pool_cleanup_t        *cln;
     ngx_http_request_t        *r;
     ngx_http_connection_t     *hc;
@@ -385,6 +390,10 @@ ngx_http_v3_wait_request_handler(ngx_eve
     h3c = ngx_http_v3_get_session(c);
     h3c->nrequests++;
 
+    pc = c->quic->parent;
+    pc->idle = 0;
+    ngx_reusable_connection(pc, 0);
+
     if (h3c->keepalive.timer_set) {
         ngx_del_timer(&h3c->keepalive);
     }
@@ -430,7 +439,7 @@ ngx_http_v3_cleanup_request(void *data)
 {
     ngx_http_request_t  *r = data;
 
-    ngx_connection_t          *c;
+    ngx_connection_t          *c, *pc;
     ngx_http_v3_session_t     *h3c;
     ngx_http_core_loc_conf_t  *clcf;
 
@@ -443,6 +452,16 @@ ngx_http_v3_cleanup_request(void *data)
     h3c = ngx_http_v3_get_session(c);
 
     if (--h3c->nrequests == 0) {
+        pc = c->quic->parent;
+
+        if (ngx_terminate || ngx_exiting) {
+            ngx_quic_finalize_connection(pc, NGX_HTTP_V3_ERR_NO_ERROR, NULL);
+            return;
+        }
+
+        pc->idle = 1;
+        ngx_reusable_connection(pc, 1);
+
         clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module);
         ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout);
     }
diff --git a/src/http/v3/ngx_http_v3_uni.c b/src/http/v3/ngx_http_v3_uni.c
--- a/src/http/v3/ngx_http_v3_uni.c
+++ b/src/http/v3/ngx_http_v3_uni.c
@@ -182,6 +182,11 @@ ngx_http_v3_uni_read_handler(ngx_event_t
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 read handler");
 
+    if (c->close) {
+        ngx_http_v3_close_uni_stream(c);
+        return;
+    }
+
     ngx_memzero(&b, sizeof(ngx_buf_t));
 
     while (rev->ready) {
@@ -262,6 +267,11 @@ ngx_http_v3_uni_dummy_read_handler(ngx_e
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 dummy read handler");
 
+    if (c->close) {
+        ngx_http_v3_close_uni_stream(c);
+        return;
+    }
+
     if (rev->ready) {
         if (c->recv(c, &ch, 1) != 0) {
             ngx_http_v3_finalize_connection(c, NGX_HTTP_V3_ERR_NO_ERROR, NULL);


From arut at nginx.com  Wed May 18 06:57:15 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Wed, 18 May 2022 10:57:15 +0400
Subject: [PATCH 2 of 2] QUIC: init_streams() callback
In-Reply-To: <patchbomb.1652857033@arut-laptop>
References: <patchbomb.1652857033@arut-laptop>
Message-ID: <a0f2d69f1fe43dfc7182.1652857035@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1652856132 -14400
#      Wed May 18 10:42:12 2022 +0400
# Branch quic
# Node ID a0f2d69f1fe43dfc718262235bf04d7b05f1fd68
# Parent  67ae4b649f2e38a44b245b7a842cf396c8250f02
QUIC: init_streams() callback.

It's called after handshake completion to initialize application-level data
prior to creating streams.

HTTP/3 callback implementation switches main QUIC connection to idle and
reusable modes and sets keepalive timer.

diff --git a/src/event/quic/ngx_event_quic.h b/src/event/quic/ngx_event_quic.h
--- a/src/event/quic/ngx_event_quic.h
+++ b/src/event/quic/ngx_event_quic.h
@@ -28,6 +28,9 @@
 #define NGX_QUIC_STREAM_UNIDIRECTIONAL       0x02
 
 
+typedef ngx_int_t (*ngx_quic_init_streams_pt)(ngx_connection_t *c);
+
+
 typedef enum {
     NGX_QUIC_STREAM_SEND_READY = 0,
     NGX_QUIC_STREAM_SEND_SEND,
@@ -74,6 +77,8 @@ typedef struct {
     ngx_int_t                      stream_reject_code_uni;
     ngx_int_t                      stream_reject_code_bidi;
 
+    ngx_quic_init_streams_pt       init_streams;
+
     u_char                         av_token_key[NGX_QUIC_AV_KEY_LEN];
     u_char                         sr_token_key[NGX_QUIC_SR_KEY_LEN];
 } ngx_quic_conf_t;
diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c
--- a/src/event/quic/ngx_event_quic_streams.c
+++ b/src/event/quic/ngx_event_quic_streams.c
@@ -21,6 +21,7 @@ static ngx_quic_stream_t *ngx_quic_get_s
 static ngx_int_t ngx_quic_reject_stream(ngx_connection_t *c, uint64_t id);
 static void ngx_quic_init_stream_handler(ngx_event_t *ev);
 static void ngx_quic_init_streams_handler(ngx_connection_t *c);
+static ngx_int_t ngx_quic_do_init_streams(ngx_connection_t *c);
 static ngx_quic_stream_t *ngx_quic_create_stream(ngx_connection_t *c,
     uint64_t id);
 static void ngx_quic_empty_handler(ngx_event_t *ev);
@@ -571,15 +572,22 @@ ngx_quic_init_streams(ngx_connection_t *
         return NGX_OK;
     }
 
-    ngx_quic_init_streams_handler(c);
-
-    return NGX_OK;
+    return ngx_quic_do_init_streams(c);
 }
 
 
 static void
 ngx_quic_init_streams_handler(ngx_connection_t *c)
 {
+    if (ngx_quic_do_init_streams(c) != NGX_OK) {
+        ngx_quic_close_connection(c, NGX_ERROR);
+    }
+}
+
+
+static ngx_int_t
+ngx_quic_do_init_streams(ngx_connection_t *c)
+{
     ngx_queue_t            *q;
     ngx_quic_stream_t      *qs;
     ngx_quic_connection_t  *qc;
@@ -588,6 +596,12 @@ ngx_quic_init_streams_handler(ngx_connec
 
     qc = ngx_quic_get_connection(c);
 
+    if (qc->conf->init_streams) {
+        if (qc->conf->init_streams(c) != NGX_OK) {
+            return NGX_ERROR;
+        }
+    }
+
     for (q = ngx_queue_head(&qc->streams.uninitialized);
          q != ngx_queue_sentinel(&qc->streams.uninitialized);
          q = ngx_queue_next(q))
@@ -597,6 +611,8 @@ ngx_quic_init_streams_handler(ngx_connec
     }
 
     qc->streams.initialized = 1;
+
+    return NGX_OK;
 }
 
 
diff --git a/src/http/v3/ngx_http_v3.c b/src/http/v3/ngx_http_v3.c
--- a/src/http/v3/ngx_http_v3.c
+++ b/src/http/v3/ngx_http_v3.c
@@ -17,21 +17,15 @@ static void ngx_http_v3_cleanup_session(
 ngx_int_t
 ngx_http_v3_init_session(ngx_connection_t *c)
 {
-    ngx_connection_t       *pc;
     ngx_pool_cleanup_t     *cln;
     ngx_http_connection_t  *hc;
     ngx_http_v3_session_t  *h3c;
 
-    pc = c->quic->parent;
-    hc = pc->data;
-
-    if (hc->v3_session) {
-        return NGX_OK;
-    }
+    hc = c->data;
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 init session");
 
-    h3c = ngx_pcalloc(pc->pool, sizeof(ngx_http_v3_session_t));
+    h3c = ngx_pcalloc(c->pool, sizeof(ngx_http_v3_session_t));
     if (h3c == NULL) {
         goto failed;
     }
@@ -42,16 +36,16 @@ ngx_http_v3_init_session(ngx_connection_
     ngx_queue_init(&h3c->blocked);
     ngx_queue_init(&h3c->pushing);
 
-    h3c->keepalive.log = pc->log;
-    h3c->keepalive.data = pc;
+    h3c->keepalive.log = c->log;
+    h3c->keepalive.data = c;
     h3c->keepalive.handler = ngx_http_v3_keepalive_handler;
     h3c->keepalive.cancelable = 1;
 
-    h3c->table.send_insert_count.log = pc->log;
-    h3c->table.send_insert_count.data = pc;
+    h3c->table.send_insert_count.log = c->log;
+    h3c->table.send_insert_count.data = c;
     h3c->table.send_insert_count.handler = ngx_http_v3_inc_insert_count_handler;
 
-    cln = ngx_pool_cleanup_add(pc->pool, 0);
+    cln = ngx_pool_cleanup_add(c->pool, 0);
     if (cln == NULL) {
         goto failed;
     }
diff --git a/src/http/v3/ngx_http_v3.h b/src/http/v3/ngx_http_v3.h
--- a/src/http/v3/ngx_http_v3.h
+++ b/src/http/v3/ngx_http_v3.h
@@ -153,6 +153,7 @@ struct ngx_http_v3_session_s {
 
 void ngx_http_v3_init(ngx_connection_t *c);
 void ngx_http_v3_reset_connection(ngx_connection_t *c);
+ngx_int_t ngx_http_v3_init_streams(ngx_connection_t *c);
 ngx_int_t ngx_http_v3_init_session(ngx_connection_t *c);
 ngx_int_t ngx_http_v3_check_flood(ngx_connection_t *c);
 
diff --git a/src/http/v3/ngx_http_v3_module.c b/src/http/v3/ngx_http_v3_module.c
--- a/src/http/v3/ngx_http_v3_module.c
+++ b/src/http/v3/ngx_http_v3_module.c
@@ -249,6 +249,8 @@ ngx_http_v3_create_srv_conf(ngx_conf_t *
     h3scf->quic.stream_reject_code_bidi = NGX_HTTP_V3_ERR_REQUEST_REJECTED;
     h3scf->quic.active_connection_id_limit = NGX_CONF_UNSET_UINT;
 
+    h3scf->quic.init_streams = ngx_http_v3_init_streams;
+
     return h3scf;
 }
 
diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c
--- a/src/http/v3/ngx_http_v3_request.c
+++ b/src/http/v3/ngx_http_v3_request.c
@@ -93,11 +93,6 @@ ngx_http_v3_init(ngx_connection_t *c)
     }
 #endif
 
-    if (ngx_http_v3_init_session(c) != NGX_OK) {
-        ngx_http_close_connection(c);
-        return;
-    }
-
     if (c->quic->id & NGX_QUIC_STREAM_UNIDIRECTIONAL) {
         ngx_http_v3_init_uni_stream(c);
 
@@ -107,6 +102,43 @@ ngx_http_v3_init(ngx_connection_t *c)
 }
 
 
+ngx_int_t
+ngx_http_v3_init_streams(ngx_connection_t *c)
+{
+    ngx_http_v3_session_t     *h3c;
+    ngx_http_connection_t     *hc;
+    ngx_http_v3_srv_conf_t    *h3scf;
+    ngx_http_core_loc_conf_t  *clcf;
+
+    if (ngx_terminate || ngx_exiting) {
+        return NGX_ERROR;
+    }
+
+    hc = c->data;
+
+    h3scf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_v3_module);
+
+#if (NGX_HTTP_V3_HQ)
+    if (h3scf->hq) {
+        return NGX_OK;
+    }
+#endif
+
+    if (ngx_http_v3_init_session(c) != NGX_OK) {
+        return NGX_ERROR;
+    }
+
+    c->idle = 1;
+    ngx_reusable_connection(c, 1);
+
+    h3c = ngx_http_v3_get_session(c);
+    clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module);
+    ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout);
+
+    return NGX_OK;
+}
+
+
 #if (NGX_HTTP_V3_HQ)
 
 static void


From xeioex at nginx.com  Wed May 18 07:02:35 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 18 May 2022 07:02:35 +0000
Subject: [njs] Improved OPCODE debug.
Message-ID: <hg.3e754413e907.1652857355.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/3e754413e907
branches:  
changeset: 1857:3e754413e907
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue May 17 23:26:05 2022 -0700
description:
Improved OPCODE debug.

diffstat:

 src/njs_vmcode.c |  31 -------------------------------
 src/njs_vmcode.h |  13 +++++++++++++
 2 files changed, 13 insertions(+), 31 deletions(-)

diffs (115 lines):

diff -r 30e03dbc6baf -r 3e754413e907 src/njs_vmcode.c
--- a/src/njs_vmcode.c	Mon May 16 22:57:27 2022 -0700
+++ b/src/njs_vmcode.c	Tue May 17 23:26:05 2022 -0700
@@ -79,11 +79,6 @@ static njs_jump_off_t njs_function_frame
     } while (0)
 
 
-#ifdef NJS_OPCODE_DEBUG
-void njs_vmcode_debug(njs_vm_t *vm, u_char *pc, const char *prefix);
-#endif
-
-
 njs_int_t
 njs_vmcode_interpreter(njs_vm_t *vm, u_char *pc, void *promise_cap,
     void *async_ctx)
@@ -122,9 +117,7 @@ njs_vmcode_interpreter(njs_vm_t *vm, u_c
     njs_vmcode_try_trampoline_t  *try_trampoline;
     njs_vmcode_function_frame_t  *function_frame;
 
-#ifdef NJS_OPCODE_DEBUG
     njs_vmcode_debug(vm, pc, "ENTER");
-#endif
 
 next:
 
@@ -663,9 +656,7 @@ next:
                 njs_vmcode_operand(vm, (njs_index_t) value2, value2);
                 vm->retval = *value2;
 
-#ifdef NJS_OPCODE_DEBUG
                 njs_vmcode_debug(vm, pc, "EXIT STOP");
-#endif
 
                 return NJS_OK;
 
@@ -741,9 +732,7 @@ next:
             case NJS_VMCODE_RETURN:
                 njs_vmcode_operand(vm, (njs_index_t) value2, value2);
 
-#ifdef NJS_OPCODE_DEBUG
                 njs_vmcode_debug(vm, pc, "EXIT RETURN");
-#endif
 
                 return njs_vmcode_return(vm, NULL, value2);
 
@@ -865,9 +854,7 @@ next:
             case NJS_VMCODE_AWAIT:
                 await = (njs_vmcode_await_t *) pc;
 
-#ifdef NJS_OPCODE_DEBUG
                 njs_vmcode_debug(vm, pc, "EXIT AWAIT");
-#endif
 
                 return njs_vmcode_await(vm, await, promise_cap, async_ctx);
 
@@ -933,9 +920,7 @@ next:
                 switch (ret) {
                 case NJS_OK:
 
-#ifdef NJS_OPCODE_DEBUG
                     njs_vmcode_debug(vm, pc, "EXIT FINALLY");
-#endif
 
                     return NJS_OK;
                 case NJS_ERROR:
@@ -1053,9 +1038,7 @@ error:
         }
     }
 
-#ifdef NJS_OPCODE_DEBUG
     njs_vmcode_debug(vm, pc, "EXIT ERROR");
-#endif
 
     return NJS_ERROR;
 }
@@ -2180,17 +2163,3 @@ njs_vmcode_error(njs_vm_t *vm, u_char *p
         njs_error_fmt_new(vm, &vm->retval, err->type, "%V", &err->u.message);
     }
 }
-
-
-#ifdef NJS_OPCODE_DEBUG
-void
-njs_vmcode_debug(njs_vm_t *vm, u_char *pc, const char *prefix)
-{
-    njs_vm_code_t  *code;
-
-    code = njs_lookup_code(vm, pc);
-
-    njs_printf("%s %V\n", prefix,
-               (code != NULL) ? &code->name : &njs_entry_unknown);
-}
-#endif
diff -r 30e03dbc6baf -r 3e754413e907 src/njs_vmcode.h
--- a/src/njs_vmcode.h	Mon May 16 22:57:27 2022 -0700
+++ b/src/njs_vmcode.h	Tue May 17 23:26:05 2022 -0700
@@ -450,5 +450,18 @@ njs_int_t njs_vmcode_interpreter(njs_vm_
 
 njs_object_t *njs_function_new_object(njs_vm_t *vm, njs_value_t *constructor);
 
+#ifdef NJS_OPCODE_DEBUG
+#define njs_vmcode_debug(vm, pc, prefix) {                                    \
+    do {                                                                      \
+        njs_vm_code_t  *code;                                                 \
+                                                                              \
+        code = njs_lookup_code(vm, pc);                                       \
+                                                                              \
+        njs_printf("%s %V\n", prefix,                                         \
+                   (code != NULL) ? &code->name : &njs_entry_unknown);        \
+    } while (0)
+#else
+#define njs_vmcode_debug(vm, pc, prefix)
+#endif
 
 #endif /* _NJS_VMCODE_H_INCLUDED_ */


From xeioex at nginx.com  Wed May 18 07:02:37 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 18 May 2022 07:02:37 +0000
Subject: [njs] Added generator debug.
Message-ID: <hg.c48ecb7b24d7.1652857357.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/c48ecb7b24d7
branches:  
changeset: 1858:c48ecb7b24d7
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue May 17 23:26:09 2022 -0700
description:
Added generator debug.

diffstat:

 src/njs_generator.c |  22 +++++++++++++++-------
 1 files changed, 15 insertions(+), 7 deletions(-)

diffs (74 lines):

diff -r 3e754413e907 -r c48ecb7b24d7 src/njs_generator.c
--- a/src/njs_generator.c	Tue May 17 23:26:05 2022 -0700
+++ b/src/njs_generator.c	Tue May 17 23:26:09 2022 -0700
@@ -427,6 +427,16 @@ static njs_int_t njs_generate_index_rele
                           ##__VA_ARGS__)
 
 
+#ifdef NJS_GENERATOR_DEBUG
+#define njs_generator_debug(msg, ...) njs_printf(msg "\n", ##__VA_ARGS__)
+#define njs_generator_debug_code(code)                                        \
+        njs_disassemble((u_char *) code, NULL, 1, NULL)
+#else
+#define njs_generator_debug(msg, ...)
+#define njs_generator_debug_code(code)
+#endif
+
+
 static const njs_str_t  no_label     = njs_str("");
 static const njs_str_t  return_label = njs_str("@return");
 /* GCC and Clang complain about NULL argument passed to memcmp(). */
@@ -725,7 +735,6 @@ njs_generate(njs_vm_t *vm, njs_generator
         return njs_generate_await(vm, generator, node);
 
     default:
-        njs_thread_log_debug("unknown token: %d", node->token);
         njs_internal_error(vm, "Generator failed: unknown token");
 
         return NJS_ERROR;
@@ -3368,8 +3377,7 @@ njs_generate_3addr_operation_end(njs_vm_
 
     code->dst = node->index;
 
-    njs_thread_log_debug("CODE3  %p, %p, %p",
-                         code->dst, code->src1, code->src2);
+    njs_generator_debug_code(code);
 
     return njs_generator_stack_pop(vm, generator, generator->context);
 }
@@ -3404,7 +3412,7 @@ njs_generate_2addr_operation_end(njs_vm_
 
     code->dst = node->index;
 
-    njs_thread_log_debug("CODE2  %p, %p", code->dst, code->src);
+    njs_generator_debug_code(code);
 
     return njs_generator_stack_pop(vm, generator, NULL);
 }
@@ -3453,7 +3461,7 @@ njs_generate_typeof_operation_end(njs_vm
 
     code->dst = node->index;
 
-    njs_thread_log_debug("CODE2  %p, %p", code->dst, code->src);
+    njs_generator_debug_code(code);
 
     return njs_generator_stack_pop(vm, generator, NULL);
 }
@@ -4898,7 +4906,7 @@ njs_generate_temp_index_get(njs_vm_t *vm
     if (cache != NULL && cache->items != 0) {
         last = njs_arr_remove_last(cache);
 
-        njs_thread_log_debug("CACHE %p", *last);
+        njs_generator_debug("INDEX REUSE  %04Xz", (size_t) *last);
 
         return *last;
     }
@@ -4964,7 +4972,7 @@ njs_generate_index_release(njs_vm_t *vm,
     njs_arr_t    *cache;
     njs_index_t  *last;
 
-    njs_thread_log_debug("RELEASE %p", index);
+    njs_generator_debug("INDEX RELEASE %04Xz", (size_t) index);
 
     cache = generator->index_cache;
 


From xeioex at nginx.com  Wed May 18 07:02:38 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 18 May 2022 07:02:38 +0000
Subject: [njs] Getting rid of excessive NJS_LEVEL_TEMP.
Message-ID: <hg.aa27056f4bc9.1652857358.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/aa27056f4bc9
branches:  
changeset: 1859:aa27056f4bc9
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed May 18 00:01:05 2022 -0700
description:
Getting rid of excessive NJS_LEVEL_TEMP.

diffstat:

 src/njs_async.c     |   5 +----
 src/njs_function.c  |  37 +++++++++++++------------------------
 src/njs_function.h  |  13 -------------
 src/njs_generator.c |   3 +--
 src/njs_parser.c    |   4 +++-
 src/njs_parser.h    |   1 -
 src/njs_scope.c     |   2 +-
 src/njs_vm.c        |  12 ------------
 src/njs_vm.h        |   1 -
 9 files changed, 19 insertions(+), 59 deletions(-)

diffs (304 lines):

diff -r c48ecb7b24d7 -r aa27056f4bc9 src/njs_async.c
--- a/src/njs_async.c	Tue May 17 23:26:09 2022 -0700
+++ b/src/njs_async.c	Wed May 18 00:01:05 2022 -0700
@@ -59,7 +59,7 @@ njs_await_fulfilled(njs_vm_t *vm, njs_va
     njs_index_t unused)
 {
     njs_int_t           ret;
-    njs_value_t         **cur_local, **cur_closures, **cur_temp, *value;
+    njs_value_t         **cur_local, **cur_closures, *value;
     njs_frame_t         *frame, *async_frame;
     njs_async_ctx_t     *ctx;
     njs_native_frame_t  *top, *async;
@@ -77,13 +77,11 @@ njs_await_fulfilled(njs_vm_t *vm, njs_va
 
     cur_local = vm->levels[NJS_LEVEL_LOCAL];
     cur_closures = vm->levels[NJS_LEVEL_CLOSURE];
-    cur_temp = vm->levels[NJS_LEVEL_TEMP];
     top = vm->top_frame;
     frame = vm->active_frame;
 
     vm->levels[NJS_LEVEL_LOCAL] = async->local;
     vm->levels[NJS_LEVEL_CLOSURE] = njs_function_closures(async->function);
-    vm->levels[NJS_LEVEL_TEMP] = async->temp;
 
     vm->top_frame = async;
     vm->active_frame = async_frame;
@@ -97,7 +95,6 @@ njs_await_fulfilled(njs_vm_t *vm, njs_va
 
     vm->levels[NJS_LEVEL_LOCAL] = cur_local;
     vm->levels[NJS_LEVEL_CLOSURE] = cur_closures;
-    vm->levels[NJS_LEVEL_TEMP] = cur_temp;
 
     vm->top_frame = top;
     vm->active_frame = frame;
diff -r c48ecb7b24d7 -r aa27056f4bc9 src/njs_function.c
--- a/src/njs_function.c	Tue May 17 23:26:09 2022 -0700
+++ b/src/njs_function.c	Wed May 18 00:01:05 2022 -0700
@@ -424,8 +424,8 @@ njs_function_lambda_frame(njs_vm_t *vm, 
     njs_bool_t ctor)
 {
     size_t                 n, frame_size;
-    uint32_t               args_count, value_count, value_size, temp_size;
-    njs_value_t            *value, *bound, **new, **temp;
+    uint32_t               args_count, value_count, value_size;
+    njs_value_t            *value, *bound, **new;
     njs_frame_t            *frame;
     njs_function_t         *target;
     njs_native_frame_t     *native_frame;
@@ -458,10 +458,8 @@ njs_function_lambda_frame(njs_vm_t *vm, 
     value_count = args_count + njs_max(args_count, lambda->nlocal);
 
     value_size = value_count * sizeof(njs_value_t *);
-    temp_size = lambda->temp * sizeof(njs_value_t *);
 
-    frame_size = value_size + temp_size
-                        + ((value_count + lambda->temp) * sizeof(njs_value_t));
+    frame_size = value_size + (value_count * sizeof(njs_value_t));
 
     native_frame = njs_function_frame_alloc(vm, NJS_FRAME_SIZE + frame_size);
     if (njs_slow_path(native_frame == NULL)) {
@@ -471,9 +469,9 @@ njs_function_lambda_frame(njs_vm_t *vm, 
     /* Local */
 
     new = (njs_value_t **) ((u_char *) native_frame + NJS_FRAME_SIZE);
-    value = (njs_value_t *) ((u_char *) new + value_size + temp_size);
+    value = (njs_value_t *) ((u_char *) new + value_size);
 
-    n = value_count + lambda->temp;
+    n = value_count;
 
     while (n != 0) {
         n--;
@@ -481,15 +479,9 @@ njs_function_lambda_frame(njs_vm_t *vm, 
         njs_set_invalid(new[n]);
     }
 
-    /* Temp */
-
-    temp = (njs_value_t **) ((u_char *) native_frame + NJS_FRAME_SIZE
-                                                     + value_size);
-
     native_frame->arguments = value;
     native_frame->arguments_offset = value + (function->args_offset - 1);
     native_frame->local = new + args_count;
-    native_frame->temp = temp;
     native_frame->function = target;
     native_frame->nargs = nargs;
     native_frame->ctor = ctor;
@@ -614,7 +606,7 @@ njs_function_lambda_call(njs_vm_t *vm, v
     njs_int_t              ret;
     njs_frame_t            *frame;
     njs_value_t            *args, **local, *value;
-    njs_value_t            **cur_local, **cur_closures, **cur_temp;
+    njs_value_t            **cur_local, **cur_closures;
     njs_function_t         *function;
     njs_declaration_t      *declr;
     njs_function_lambda_t  *lambda;
@@ -650,13 +642,11 @@ njs_function_lambda_call(njs_vm_t *vm, v
 
     cur_local = vm->levels[NJS_LEVEL_LOCAL];
     cur_closures = vm->levels[NJS_LEVEL_CLOSURE];
-    cur_temp = vm->levels[NJS_LEVEL_TEMP];
 
     /* Replace current level. */
 
     vm->levels[NJS_LEVEL_LOCAL] = vm->top_frame->local;
     vm->levels[NJS_LEVEL_CLOSURE] = njs_function_closures(function);
-    vm->levels[NJS_LEVEL_TEMP] = frame->native.temp;
 
     if (lambda->rest_parameters) {
         ret = njs_function_rest_parameters_init(vm, &frame->native);
@@ -705,7 +695,6 @@ njs_function_lambda_call(njs_vm_t *vm, v
     /* Restore current level. */
     vm->levels[NJS_LEVEL_LOCAL] = cur_local;
     vm->levels[NJS_LEVEL_CLOSURE] = cur_closures;
-    vm->levels[NJS_LEVEL_TEMP] = cur_temp;
 
     return ret;
 }
@@ -805,9 +794,10 @@ njs_function_frame_free(njs_vm_t *vm, nj
 njs_int_t
 njs_function_frame_save(njs_vm_t *vm, njs_frame_t *frame, u_char *pc)
 {
-    size_t              value_count, n;
+    size_t              args_count, value_count, n;
     njs_value_t         *start, *end, *p, **new, *value, **local;
     njs_function_t      *function;
+    njs_function_lambda_t  *lambda;
     njs_native_frame_t  *active, *native;
 
     *frame = *vm->active_frame;
@@ -820,19 +810,18 @@ njs_function_frame_save(njs_vm_t *vm, nj
     native->free_size = 0;
 
     active = &vm->active_frame->native;
-    value_count = njs_function_frame_value_count(active);
+    function = active->function;
+    lambda = function->u.lambda;
 
-    function = active->function;
+    args_count = function->args_offset + njs_max(native->nargs, lambda->nargs);
+    value_count = args_count + njs_max(args_count, lambda->nlocal);
 
     new = (njs_value_t **) ((u_char *) native + NJS_FRAME_SIZE);
-    value = (njs_value_t *) (new + value_count
-                             + function->u.lambda->temp);
-
+    value = (njs_value_t *) (new + value_count);
 
     native->arguments = value;
     native->arguments_offset = value + (function->args_offset - 1);
     native->local = new + njs_function_frame_args_count(active);
-    native->temp = new + value_count;
     native->pc = pc;
 
     start = njs_function_frame_values(active, &end);
diff -r c48ecb7b24d7 -r aa27056f4bc9 src/njs_function.h
--- a/src/njs_function.h	Tue May 17 23:26:09 2022 -0700
+++ b/src/njs_function.h	Wed May 18 00:01:05 2022 -0700
@@ -12,7 +12,6 @@ struct njs_function_lambda_s {
     njs_index_t                    *closures;
     uint32_t                       nclosures;
     uint32_t                       nlocal;
-    uint32_t                       temp;
 
     njs_declaration_t              *declarations;
     uint32_t                       ndeclarations;
@@ -50,7 +49,6 @@ struct njs_native_frame_s {
     njs_object_t                   *arguments_object;
     njs_value_t                    *arguments_offset;
     njs_value_t                    **local;
-    njs_value_t                    **temp;
 
     uint32_t                       size;
     uint32_t                       free_size;
@@ -229,17 +227,6 @@ njs_function_frame_args_count(njs_native
 }
 
 
-njs_inline size_t
-njs_function_frame_value_count(njs_native_frame_t *frame)
-{
-    uintptr_t  start;
-
-    start = (uintptr_t) ((u_char *) frame + NJS_FRAME_SIZE);
-
-    return ((uintptr_t) frame->temp - start) / sizeof(njs_value_t *);
-}
-
-
 njs_inline njs_value_t *
 njs_function_frame_values(njs_native_frame_t *frame, njs_value_t **end)
 {
diff -r c48ecb7b24d7 -r aa27056f4bc9 src/njs_generator.c
--- a/src/njs_generator.c	Tue May 17 23:26:09 2022 -0700
+++ b/src/njs_generator.c	Wed May 18 00:01:05 2022 -0700
@@ -3698,7 +3698,6 @@ njs_generate_function_scope(njs_vm_t *vm
     lambda->closures = generator.closures->start;
     lambda->nclosures = generator.closures->items;
     lambda->nlocal = node->scope->items;
-    lambda->temp = node->scope->temp;
 
     arr = node->scope->declarations;
     lambda->declarations = (arr != NULL) ? arr->start : NULL;
@@ -4916,7 +4915,7 @@ njs_generate_temp_index_get(njs_vm_t *vm
         return NJS_ERROR;
     }
 
-    return njs_scope_index(scope->type, scope->temp++, NJS_LEVEL_TEMP,
+    return njs_scope_index(scope->type, scope->items++, NJS_LEVEL_LOCAL,
                            NJS_VARIABLE_VAR);
 }
 
diff -r c48ecb7b24d7 -r aa27056f4bc9 src/njs_parser.c
--- a/src/njs_parser.c	Tue May 17 23:26:09 2022 -0700
+++ b/src/njs_parser.c	Wed May 18 00:01:05 2022 -0700
@@ -562,7 +562,6 @@ njs_parser(njs_vm_t *vm, njs_parser_t *p
         }
 
     } else {
-        parser->scope->temp = 0;
         parser->scope->top = NULL;
         parser->node = NULL;
         parser->ret = NJS_OK;
@@ -1357,6 +1356,8 @@ njs_parser_template_literal(njs_parser_t
             return NJS_ERROR;
         }
 
+        node->temporary = 1;
+
         template->right = node;
         temp->right = node;
 
@@ -1370,6 +1371,7 @@ njs_parser_template_literal(njs_parser_t
         temp->right = template;
     }
 
+    temp->temporary = 1;
     temp->left = template;
     temp->index = index;
 
diff -r c48ecb7b24d7 -r aa27056f4bc9 src/njs_parser.h
--- a/src/njs_parser.h	Tue May 17 23:26:09 2022 -0700
+++ b/src/njs_parser.h	Wed May 18 00:01:05 2022 -0700
@@ -20,7 +20,6 @@ struct njs_parser_scope_s {
     njs_arr_t                       *closures;
     njs_arr_t                       *declarations;
 
-    uint32_t                        temp;
     uint32_t                        items;
 
     njs_scope_t                     type:8;
diff -r c48ecb7b24d7 -r aa27056f4bc9 src/njs_scope.c
--- a/src/njs_scope.c	Tue May 17 23:26:09 2022 -0700
+++ b/src/njs_scope.c	Wed May 18 00:01:05 2022 -0700
@@ -20,7 +20,7 @@ njs_scope_temp_index(njs_parser_scope_t 
         return NJS_INDEX_ERROR;
     }
 
-    return njs_scope_index(NJS_SCOPE_GLOBAL, scope->temp++, NJS_LEVEL_TEMP,
+    return njs_scope_index(scope->type, scope->items++, NJS_LEVEL_LOCAL,
                            NJS_VARIABLE_VAR);
 }
 
diff -r c48ecb7b24d7 -r aa27056f4bc9 src/njs_vm.c
--- a/src/njs_vm.c	Tue May 17 23:26:09 2022 -0700
+++ b/src/njs_vm.c	Wed May 18 00:01:05 2022 -0700
@@ -226,17 +226,6 @@ njs_vm_compile(njs_vm_t *vm, u_char **st
     vm->variables_hash = &scope->variables;
     vm->global_items = scope->items;
 
-    vm->levels[NJS_LEVEL_TEMP] = NULL;
-
-    if (scope->temp != 0) {
-        new = njs_scope_make(vm, scope->temp);
-        if (njs_slow_path(new == NULL)) {
-            return ret;
-        }
-
-        vm->levels[NJS_LEVEL_TEMP] = new;
-    }
-
     if (vm->options.disassemble) {
         njs_disassembler(vm);
     }
@@ -305,7 +294,6 @@ njs_vm_compile_module(njs_vm_t *vm, njs_
 
     lambda->start = generator.code_start;
     lambda->nlocal = scope->items;
-    lambda->temp = scope->temp;
 
     arr = scope->declarations;
     lambda->declarations = (arr != NULL) ? arr->start : NULL;
diff -r c48ecb7b24d7 -r aa27056f4bc9 src/njs_vm.h
--- a/src/njs_vm.h	Tue May 17 23:26:09 2022 -0700
+++ b/src/njs_vm.h	Wed May 18 00:01:05 2022 -0700
@@ -122,7 +122,6 @@ typedef enum {
     NJS_LEVEL_CLOSURE,
     NJS_LEVEL_GLOBAL,
     NJS_LEVEL_STATIC,
-    NJS_LEVEL_TEMP,
     NJS_LEVEL_MAX
 } njs_level_type_t;
 


From P.Pautov at F5.com  Wed May 18 07:20:51 2022
From: P.Pautov at F5.com (Pavel Pautov)
Date: Wed, 18 May 2022 07:20:51 +0000
Subject: SSL contexts reuse across locations
Message-ID:  <SJ0PR01MB6510467ED91D5036DAA3B5DFB1D19@SJ0PR01MB6510.prod.exchangelabs.com>

Hello,

Attaching POC patch for https://trac.nginx.org/nginx/ticket/1234.

At very least, ngx_http_proxy_set_ssl() needs to be converted into ngx_http_proxy_create_ssl(). But there are also a couple of things to discuss:

1. Patch uses pretty straightforward reuse criteria (absence of directives), but shall we go further, say, compare directive arguments (with special treatment of complex values with variables)?

2. Since similar change also makes sense for "grpc", "uwsgi" (and may be "stream proxy") modules, perhaps it's time to factor out SSL upstream settings code for all these modules to avoid copypasting of above patch? We can introduce something like "ngx_ssl_upstream_conf_t" to keep shared SSL settings and unite ngx_http_(proxy|grpc|uwsgi)_set_ssl functions. Config merge logic (together with attached patch) can be moved to something like ngx_ssl_upstream_conf_merge. Optionally, ngx_http_upstream_conf_t can be updated to contain ngx_ssl_upstream_conf_t.

Thanks,
Pavel

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ssl_ctx_reuse.patch
Type: application/octet-stream
Size: 2758 bytes
Desc: ssl_ctx_reuse.patch
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220518/a51214ce/attachment.obj>

From vvidovic at croz.net  Wed May 18 12:26:33 2022
From: vvidovic at croz.net (Vedran Vidovic)
Date: Wed, 18 May 2022 14:26:33 +0200
Subject: ssl_verify_partial_chain
Message-ID: <OF9D08BED8.6466852E-ONC1258846.00444E76-C1258846.00445982@LocalDomain>

An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220518/91efc462/attachment.htm>

From vvidovic at croz.net  Wed May 18 12:28:28 2022
From: vvidovic at croz.net (Vedran Vidovic)
Date: Wed, 18 May 2022 14:28:28 +0200
Subject: ssl_verify_partial_chain
Message-ID: <OF048EDAB4.3ACE898B-ONC1258846.00447C38-C1258846.00448643@LocalDomain>

An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220518/100854c1/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: http.server.verify_partial_chain.patchbomb
Type: application/octet-stream
Size: 8187 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220518/100854c1/attachment.obj>

From mdounin at mdounin.ru  Wed May 18 18:31:54 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 18 May 2022 21:31:54 +0300
Subject: SSL contexts reuse across locations
In-Reply-To: <SJ0PR01MB6510467ED91D5036DAA3B5DFB1D19@SJ0PR01MB6510.prod.exchangelabs.com>
References: <SJ0PR01MB6510467ED91D5036DAA3B5DFB1D19@SJ0PR01MB6510.prod.exchangelabs.com>
Message-ID: <YoU7mt+Mmi/2my8k@mdounin.ru>

Hello!

On Wed, May 18, 2022 at 07:20:51AM +0000, Pavel Pautov via nginx-devel wrote:

> Hello,
> 
> Attaching POC patch for 
> https://trac.nginx.org/nginx/ticket/1234.
> 
> At very least, ngx_http_proxy_set_ssl() needs to be converted 
> into ngx_http_proxy_create_ssl().

You may want to focus on actually making the code more readable 
and abstracting it into ngx_http_proxy_set_ssl() instead.  
Something like ngx_http_upstream_hide_headers_hash() might be a 
good example on how to do it properly.

Also, it should be a good idea to avoid creating SSL contexts if 
there is no SSL proxying configured.  Or, at very least, make sure 
only one context at the http level is used in such cases, so 
configurations with many servers won't suddenly blow up.

> But there are also a couple of things to discuss:
> 
> 1. Patch uses pretty straightforward reuse criteria (absence of 
> directives), but shall we go further, say, compare directive 
> arguments (with special treatment of complex values with 
> variables)?

Just checking if the relevant directives were inherited from the 
previous level should be enough, as it will allow creating 
memory-effective configurations.

> 2. Since similar change also makes sense for "grpc", "uwsgi" 
> (and may be "stream proxy") modules, perhaps it's time to factor 
> out SSL upstream settings code for all these modules to avoid 
> copypasting of above patch? We can introduce something like 
> "ngx_ssl_upstream_conf_t" to keep shared SSL settings and unite 
> ngx_http_(proxy|grpc|uwsgi)_set_ssl functions. Config merge 
> logic (together with attached patch) can be moved to something 
> like ngx_ssl_upstream_conf_merge. Optionally, 
> ngx_http_upstream_conf_t can be updated to contain 
> ngx_ssl_upstream_conf_t.

I don't think it the effort.  Further, there are protocol-specific 
differences, such as ALPN in gRPC proxy, so you can't fully 
abstract it anyway.

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Thu May 19 02:23:58 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 19 May 2022 05:23:58 +0300
Subject: ssl_verify_partial_chain
In-Reply-To: <OF048EDAB4.3ACE898B-ONC1258846.00447C38-C1258846.00448643@LocalDomain>
References: <OF048EDAB4.3ACE898B-ONC1258846.00447C38-C1258846.00448643@LocalDomain>
Message-ID: <YoWqPrfP2DsLVenu@mdounin.ru>

Hello!

On Wed, May 18, 2022 at 02:28:28PM +0200, Vedran Vidovic wrote:

>    We would like to be able to configure the mutual TLS client
>    authentication by:
>    - adding intermediate CA certificates
>    - without adding the root CA certificate for each intermediate
>    certificate
> 
>    If we add CA as a trusted issuer, we shouldn't need to add its issuer
>    to
>    the truststore (ssl_client_certificate).
> 
>    I propose a backward compatible solution to add a new configuration
>    option ssl_verify_partial_chain that can be turned on if the behaviour
>    described above is desired. This option enables the openssl library
>    partial_chain verification.

(First of all, just to make sure it's understood and this isn't 
something you are trying to do.  Note that if one want to limit 
access, it might be a good idea to use some actual authorization 
checks in additional to PKI, which essentially provides 
authentication.  Using narrow trust as a poor man's authorization 
checks is not the way to go.)

After reading https://github.com/openssl/openssl/issues/7871 I 
tend to think that a better solution might be to explicitly 
configure trust on the certificates if such configuration is 
needed.

Something like:

$ openssl x509 -in cert.pem -out trust.pem -trustout -addtrust anyExtendedKeyUsage

will do the trick.

For ssl_trusted_certificate / proxy_ssl_trusted_certificate this 
works out of the box (seems to work at least since OpenSSL 1.0.2, 
the same version where X509_V_FLAG_PARTIAL_CHAIN was introduced).

For ssl_client_certificate it needs some additional cert in the
file to work, as SSL_load_client_CA_file() is not able to parse
certificates with trust data.  (And such certificates won't be
advertized during SSL handshakes.)  Not sure if it's practical
problem, but if it is, it should be possible to adjust
SSL_load_client_CA_file() and/or switch to a different way to
create the CA list for SSL_CTX_set_client_CA_list().

[...]

> @@ -874,6 +874,25 @@
>  
>      SSL_CTX_set_verify_depth(ssl->ctx, depth);
>  
> +    if (partial_chain == 1) {
> +      X509_VERIFY_PARAM *param = X509_VERIFY_PARAM_new();;
> +      if (param) {
> +        X509_VERIFY_PARAM_set_flags(param, X509_V_FLAG_PARTIAL_CHAIN);
> +        if (SSL_CTX_set1_param(ssl->ctx, param) == 0) {

Just in case, setting flags via X509_STORE_set_flags(), much like 
ngx_ssl_crl() does, should be much easier.

-- 
Maxim Dounin
http://mdounin.ru/


From xeioex at nginx.com  Thu May 19 23:50:26 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Thu, 19 May 2022 23:50:26 +0000
Subject: [njs] Fixed redefinition of special props in Object.defineProperty().
Message-ID: <hg.beb180165976.1653004226.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/beb180165976
branches:  
changeset: 1860:beb180165976
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Thu May 19 16:41:08 2022 -0700
description:
Fixed redefinition of special props in Object.defineProperty().

Previously, when NJS_PROPERTY_HANDLER property was updated it might be
left in inconsistent state.  Namely, prop->type was left unchanged, but
prop->value did not have an expected property handler.  As a result
consecutive reference to the property may result in a segment violation.

The fix is to update the prop->type during redefinition.

This closes #504 issue on Github.

diffstat:

 src/njs_object.h         |   4 +++-
 src/njs_object_prop.c    |  11 ++++++++++-
 src/njs_value.c          |   1 +
 src/njs_value.h          |   2 ++
 src/test/njs_unit_test.c |   6 ++++++
 5 files changed, 22 insertions(+), 2 deletions(-)

diffs (88 lines):

diff -r aa27056f4bc9 -r beb180165976 src/njs_object.h
--- a/src/njs_object.h	Wed May 18 00:01:05 2022 -0700
+++ b/src/njs_object.h	Thu May 19 16:41:08 2022 -0700
@@ -88,7 +88,9 @@ njs_int_t njs_object_prop_init(njs_vm_t 
 njs_inline njs_bool_t
 njs_is_data_descriptor(njs_object_prop_t *prop)
 {
-    return prop->writable != NJS_ATTRIBUTE_UNSET || njs_is_valid(&prop->value);
+    return njs_is_valid(&prop->value) ||
+           prop->writable != NJS_ATTRIBUTE_UNSET ||
+           prop->type == NJS_PROPERTY_HANDLER;
 }
 
 
diff -r aa27056f4bc9 -r beb180165976 src/njs_object_prop.c
--- a/src/njs_object_prop.c	Wed May 18 00:01:05 2022 -0700
+++ b/src/njs_object_prop.c	Thu May 19 16:41:08 2022 -0700
@@ -364,6 +364,15 @@ set_prop:
          * the property's attributes to their default values.
          */
 
+        if (pq.temp) {
+            pq.lhq.value = NULL;
+            prop->configurable = prev->configurable;
+            prop->enumerable = prev->enumerable;
+            goto set_prop;
+        }
+
+        prev->type = prop->type;
+
         if (njs_is_data_descriptor(prev)) {
             njs_set_undefined(&prev->getter);
             njs_set_undefined(&prev->setter);
@@ -414,7 +423,7 @@ set_prop:
 
 done:
 
-    if (njs_is_valid(&prop->value) || njs_is_accessor_descriptor(prop)) {
+    if (njs_is_valid(&prop->value)) {
         if (prev->type == NJS_PROPERTY_HANDLER) {
             if (prev->writable) {
                 ret = prev->value.data.u.prop_handler(vm, prev, object,
diff -r aa27056f4bc9 -r beb180165976 src/njs_value.c
--- a/src/njs_value.c	Wed May 18 00:01:05 2022 -0700
+++ b/src/njs_value.c	Thu May 19 16:41:08 2022 -0700
@@ -942,6 +942,7 @@ njs_external_property_query(njs_vm_t *vm
         return NJS_DECLINED;
     }
 
+    pq->temp = 1;
     prop = &pq->scratch;
 
     njs_memzero(prop, sizeof(njs_object_prop_t));
diff -r aa27056f4bc9 -r beb180165976 src/njs_value.h
--- a/src/njs_value.h	Wed May 18 00:01:05 2022 -0700
+++ b/src/njs_value.h	Thu May 19 16:41:08 2022 -0700
@@ -374,6 +374,7 @@ typedef struct {
     njs_object_prop_t           *own_whiteout;
     uint8_t                     query;
     uint8_t                     shared;
+    uint8_t                     temp;
     uint8_t                     own;
 } njs_property_query_t;
 
@@ -1030,6 +1031,7 @@ njs_set_object_value(njs_value_t *value,
         (pq)->query = _query;                                                 \
         (pq)->shared = 0;                                                     \
         (pq)->own = _own;                                                     \
+        (pq)->temp = 0;                                                       \
     } while (0)
 
 
diff -r aa27056f4bc9 -r beb180165976 src/test/njs_unit_test.c
--- a/src/test/njs_unit_test.c	Wed May 18 00:01:05 2022 -0700
+++ b/src/test/njs_unit_test.c	Thu May 19 16:41:08 2022 -0700
@@ -9681,6 +9681,12 @@ static njs_unit_test_t  njs_test[] =
               "fn(1); arr"),
       njs_str("1,2,3,4,5,6") },
 
+    { njs_str("function f(){};"
+              "Object.defineProperty(f, 'length', {set: () => {}});"
+              "Object.defineProperty(f, 'length', {value: 42});"
+              "f.length"),
+      njs_str("42") },
+
     /* Function nesting depth. */
 
     { njs_str("() => () => () => () => () => () => () => () => () => () => () =>"


From P.Pautov at F5.com  Fri May 20 06:52:54 2022
From: P.Pautov at F5.com (Pavel Pautov)
Date: Fri, 20 May 2022 06:52:54 +0000
Subject: SSL contexts reuse across locations
In-Reply-To: <YoU7mt+Mmi/2my8k@mdounin.ru>
References:  <SJ0PR01MB6510467ED91D5036DAA3B5DFB1D19@SJ0PR01MB6510.prod.exchangelabs.com>
 <YoU7mt+Mmi/2my8k@mdounin.ru>
Message-ID:  <SJ0PR01MB65107F530D6D429E69595DFFB1D39@SJ0PR01MB6510.prod.exchangelabs.com>

Hi,

> -----Original Message-----
> From: Maxim Dounin <mdounin at mdounin.ru>
> Sent: Wednesday, May 18, 2022 11:32
[..]
> > At very least, ngx_http_proxy_set_ssl() needs to be converted
> > into ngx_http_proxy_create_ssl().
> 
> You may want to focus on actually making the code more readable
> and abstracting it into ngx_http_proxy_set_ssl() instead.
> Something like ngx_http_upstream_hide_headers_hash() might be a
> good example on how to do it properly.

Do you suggest to make ngx_http_proxy_set_ssl() responsible for merging context related settings? I guess, we can do that, but something like ngx_http_proxy_create_ssl() might be still beneficial.

> Also, it should be a good idea to avoid creating SSL contexts if
> there is no SSL proxying configured.  Or, at very least, make sure
> only one context at the http level is used in such cases, so
> configurations with many servers won't suddenly blow up.

The patch shouldn't cause a blow up to my understanding, expect perhaps for very specific configs like:
server {
  proxy_ssl_session_reuse off;
  location / {
     proxy_ssl_session_reuse on;
      proxy_pass https://backend;
  }
}
Basically, one have to have "proxy_ssl_*" directives scattered across hierarchy of locations with a single terminal "proxy_pass https". If there are many terminal locations with different proxy_passes, then increase shouldn't be that big. If there is no terminal "proxy_pass https", then why are these proxy_ssl_* options even there?
It might be interesting to measure  the memory cost of "default" ssl context per server. I assume, unless user loads big certificate bundle, the impact should be tolerable.

For configs where proxy_ssl_* directives are grouped at server level or absent altogether there is no impact with current patch.

That being said, I think, we can actually satisfy "no SSL contexts without https" requirement by linking location configs and traversing them backwards in a search for location with "proxy_ssl_*" directives. I'd update patch with that approach in mind.

I'd like to avoid forcing users into creating wrapper locations with dummy "proxy_pass https" for the sake of runtime optimization, i.e. just moving "proxy_ssl_*" directives to the server level should be enough.

> > But there are also a couple of things to discuss:
> >
> > 1. Patch uses pretty straightforward reuse criteria (absence of
> > directives), but shall we go further, say, compare directive
> > arguments (with special treatment of complex values with
> > variables)?
> 
> Just checking if the relevant directives were inherited from the
> previous level should be enough, as it will allow creating
> memory-effective configurations.
> 
> > 2. Since similar change also makes sense for "grpc", "uwsgi"
> > (and may be "stream proxy") modules, perhaps it's time to factor
> > out SSL upstream settings code for all these modules to avoid
> > copypasting of above patch? We can introduce something like
> > "ngx_ssl_upstream_conf_t" to keep shared SSL settings and unite
> > ngx_http_(proxy|grpc|uwsgi)_set_ssl functions. Config merge
> > logic (together with attached patch) can be moved to something
> > like ngx_ssl_upstream_conf_merge. Optionally,
> > ngx_http_upstream_conf_t can be updated to contain
> > ngx_ssl_upstream_conf_t.
> 
> I don't think it the effort.  Further, there are protocol-specific
> differences, such as ALPN in gRPC proxy, so you can't fully
> abstract it anyway.

Worth the effort? It doesn't look like a lot of effort to me compared with copy-pasting (and later maintaining) the optimized merge logic everywhere. I've noted the ALPN related peace in gRPC, but we can just leave it in the gRPC module. Modules still could have some specific settings in addition to the standard set.
Anyway, we need to agree on the solution for proxy module first.


From pluknet at nginx.com  Fri May 20 13:51:19 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Fri, 20 May 2022 17:51:19 +0400
Subject: [PATCH 18 of 20] Upstream: multiple WWW-Authenticate headers
 (ticket #485)
In-Reply-To: <Yn26+uz/ffNpUZ7l@mdounin.ru>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <fa9751ffe7723a11159c.1650493138@vm-bsd.mdounin.ru>
 <20220511210337.nhxkr2alnri4immp@Y9MQ9X2QVV> <Yn26+uz/ffNpUZ7l@mdounin.ru>
Message-ID: <016FFADB-29E1-4761-B50F-2F5A98160A22@nginx.com>


> On 13 May 2022, at 05:57, Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> Hello!
> 
> On Thu, May 12, 2022 at 01:03:37AM +0400, Sergey Kandaurov wrote:
> 
>> On Thu, Apr 21, 2022 at 01:18:58AM +0300, Maxim Dounin wrote:
>>> # HG changeset patch
>>> # User Maxim Dounin <mdounin at mdounin.ru>
>>> # Date 1650492341 -10800
>>> #      Thu Apr 21 01:05:41 2022 +0300
>>> # Node ID fa9751ffe7723a11159c158078e454671e81cb87
>>> # Parent  2027b85971d4b8a7e33c018548468057cb57eaf7
>>> Upstream: multiple WWW-Authenticate headers (ticket #485).
>>> 
>>> When using proxy_intercept_errors and an error page for error 401
>>> (Unauthorized), multiple WWW-Authenticate headers from the upstream server
>>> response are now properly copied to the response.
>>> 
>>> diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
>>> --- a/src/http/ngx_http_upstream.c
>>> +++ b/src/http/ngx_http_upstream.c
>>> @@ -2647,7 +2647,7 @@ ngx_http_upstream_intercept_errors(ngx_h
>>> {
>>>     ngx_int_t                  status;
>>>     ngx_uint_t                 i;
>>> -    ngx_table_elt_t           *h;
>>> +    ngx_table_elt_t           *h, *ho, **ph;
>>>     ngx_http_err_page_t       *err_page;
>>>     ngx_http_core_loc_conf_t  *clcf;
>>> 
>>> @@ -2676,18 +2676,26 @@ ngx_http_upstream_intercept_errors(ngx_h
>>>             if (status == NGX_HTTP_UNAUTHORIZED
>>>                 && u->headers_in.www_authenticate)
>>>             {
>>> -                h = ngx_list_push(&r->headers_out.headers);
>>> -
>>> -                if (h == NULL) {
>>> -                    ngx_http_upstream_finalize_request(r, u,
>>> +                h = u->headers_in.www_authenticate;
>>> +                ph = &r->headers_out.www_authenticate;
>>> +
>>> +                while (h) {
>>> +                    ho = ngx_list_push(&r->headers_out.headers);
>>> +
>>> +                    if (ho == NULL) {
>>> +                        ngx_http_upstream_finalize_request(r, u,
>>>                                                NGX_HTTP_INTERNAL_SERVER_ERROR);
>>> -                    return NGX_OK;
>>> +                        return NGX_OK;
>>> +                    }
>>> +
>>> +                    *ho = *h;
>>> +                    ho->next = NULL;
>>> +
>>> +                    *ph = ho;
>>> +                    ph = &ho->next;
>>> +
>>> +                    h = h->next;
>>>                 }
>>> -
>>> -                *h = *u->headers_in.www_authenticate;
>>> -                h->next = NULL;
>>> -
>>> -                r->headers_out.www_authenticate = h;
>>>             }
>>> 
>>> #if (NGX_HTTP_CACHE)
>>> 
>> 
>> While the patch certainly looks correct,
>> I'm not sure about usefulness of r->headers_out.www_authenticate.
>> Is the header accessed directly through this pointer ever?
>> For read purposes I mean.  Header filters seem not.
>> 
>> The only place is in the auth_request module,
>> but it doesn't seem to make sense in subrequest.
> 
> The auth request module uses it to copy the WWW-Authenticate 
> headers from the subrequest response to the main request.  For 
> example, in a configuration like:
> 
>    location / { auth_request /auth; ... }
>    location /auth { auth_basic foo; ... }
> 
> While such configurations are unlikely, as using auth_basic 
> directly should be easier, these are certainly possible.

I doubt access phase handlers are executed in subrequests at all,
see the "r != r->main" check in ngx_http_core_access_phase().
So it doesn't look feasible with auth_basic (or another auth module).

> 
> Similarly, r->headers_out.www_authenticate can be used with 
> proxy_intercept_errors + error_page 401 in the subrequest (since 
> r->upstream can be cleared by the error page).

That's true, tnx for the point.
Indeed, with such setup, after internal redirect while in subrequest,
headers are already there in (s)r->headers_out.www_authenticate.

> 
> BTW, looking at this once again, I tend to think we also need 
> ngx_http_core_access_phase() changes for proper handling of 
> multiple WWW-Authenticate headers.  It might not be a good idea to 
> commit it right now though, see commit log.

auth_spnego is such an outstanding example.
As said, with the patch applied, when it comes to clear headers with
"satisfy any;", this leads to segfault on h->next.
Even with h->next initialized, and since the module allows to insert
two headers at once, the first one gets lost tracking to be cleared
(which is also true now).
So, the module needs more work to gain from properly linked headers.

> 
> On the other hand, breaking it all at once might be less painful 
> (after all, the auth_request change will already made such modules 
> to segfault when used with auth_request; though this is unlikely, 
> since such configurations hardly make sense).

Still not clear how's that possible in subrequests, see above.

> 
> # HG changeset patch
> # User Maxim Dounin <mdounin at mdounin.ru>
> # Date 1652405911 -10800
> #      Fri May 13 04:38:31 2022 +0300
> # Node ID 0c2cf997bde390c9be27e0685ee2a00024c57e4e
> # Parent  c53cb6f31b9855020d85476fa98ef1cdca809aed
> Multiple WWW-Authenticate headers with "satisfy any;".
> 
> If a module adds multiple WWW-Authenticate headers (ticket #485) to the
> response, linked in r->headers_out.www_authenticate, all headers are now
> cleared if another module later allows access.
> 
> This change is a nop for standard modules, since the only access module which
> can add multiple WWW-Authenticate headers is the auth request module, and
> it is checked after other standard access modules.  Though this might
> affect some third party access modules.
> 
> Note that if a 3rd party module adds a single WWW-Authenticate header
> and not yet modified to set the header's next pointer to NULL, attempt to
> clear such a header with this change will result in a segmentation fault.
> 
> diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
> --- a/src/http/ngx_http_core_module.c
> +++ b/src/http/ngx_http_core_module.c
> @@ -1088,6 +1088,7 @@ ngx_int_t
> ngx_http_core_access_phase(ngx_http_request_t *r, ngx_http_phase_handler_t *ph)
> {
>     ngx_int_t                  rc;
> +    ngx_table_elt_t           *h;
>     ngx_http_core_loc_conf_t  *clcf;
> 
>     if (r != r->main) {
> @@ -1122,8 +1123,8 @@ ngx_http_core_access_phase(ngx_http_requ
>         if (rc == NGX_OK) {
>             r->access_code = 0;
> 
> -            if (r->headers_out.www_authenticate) {
> -                r->headers_out.www_authenticate->hash = 0;
> +            for (h = r->headers_out.www_authenticate; h; h = h->next) {
> +                h->hash = 0;
>             }
> 
>             r->phase_handler = ph->next;
> 

I think it is good from an architectural point of view.

-- 
Sergey Kandaurov


From vvidovic at croz.net  Fri May 20 13:55:47 2022
From: vvidovic at croz.net (Vedran Vidovic)
Date: Fri, 20 May 2022 15:55:47 +0200
Subject: ssl_verify_partial_chain
In-Reply-To: <YoWqPrfP2DsLVenu@mdounin.ru>
Message-ID: <OFB386D432.ED1CB321-ONC1258847.00230236-C1258848.004C84E7@LocalDomain>

An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220520/e0813471/attachment.htm>

From vvidovic at croz.net  Fri May 20 14:02:08 2022
From: vvidovic at croz.net (Vedran Vidovic)
Date: Fri, 20 May 2022 16:02:08 +0200
Subject: ssl_verify_partial_chain
In-Reply-To: <OFB386D432.ED1CB321-ONC1258847.00230236-C1258848.004C84E7@LocalDomain>
References: <OFB386D432.ED1CB321-ONC1258847.00230236-C1258848.004C84E7@LocalDomain>
Message-ID: <OF9D901334.FB4FEAB9-ONC1258848.004D1999-C1258848.004D199B@LocalDomain>

Hello,
 
thanks for the extensive answer.
 
I will try to pursue a solution similar to your suggestion:
- convert intermediate CA certificate to trusted certificate
- validate client certificate using trusted certificate
 
I still believe that addition of the new configuration option
"ssl_verify_partial_chain" would benefit nginx because if
we configure it using the "ssl_trusted_certificate" it doesn't
send a list of allowed CAs to the client. We just can't cover
the case when we want to send a list of allowed issuers
(without their root certs) without changes to nginx.
 
 
In my view, if I configure a certificate I1 as a trusted issuer, I should not be enforced
to add its issuer to list of trusted certificates. I would like to tell my server to trust all
certificates issued by certificate I1 (and not by it's root issuer). Similar functionality
is available in some other products I used and it seems natural to me but people
with different background can (of course) disagree.
 
---
Use Case 1 for such an approach:
For example, we could even have the following hierarchy of certificates:
- R (root CA)
  - L0n (client leaf cert 00, 01,..)
  - I1 (intermediate CA)
    - L1n (client leaf cert 10, 11)
  - I2 (intermediate CA)
    - L2n (client leaf cert 20, 21,..)
 
We would want to trust certs issued by intermediate CA I1 but not trust
certs L0n or L2n.
 
Without the possibility to trust only the I1 and not the R we can't make sure
that someone won't call us with L0n certs.
 
---
Use Case 2 for such an approach (a real use case):
On a more practical note, we need to trust all certs issued by any of the
issuer certs from the EU trusted certificates list site and root certs are
not published there.
 
 
 
 
Kind regards
Vedran Vidovic

Odricanje od odgovornosti - disclaimer   


From mdounin at mdounin.ru  Fri May 20 21:07:27 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sat, 21 May 2022 00:07:27 +0300
Subject: [PATCH 18 of 20] Upstream: multiple WWW-Authenticate headers
 (ticket #485)
In-Reply-To: <016FFADB-29E1-4761-B50F-2F5A98160A22@nginx.com>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <fa9751ffe7723a11159c.1650493138@vm-bsd.mdounin.ru>
 <20220511210337.nhxkr2alnri4immp@Y9MQ9X2QVV>
 <Yn26+uz/ffNpUZ7l@mdounin.ru>
 <016FFADB-29E1-4761-B50F-2F5A98160A22@nginx.com>
Message-ID: <YogDD+/Ar2pF+rmf@mdounin.ru>

Hello!

On Fri, May 20, 2022 at 05:51:19PM +0400, Sergey Kandaurov wrote:

> > On 13 May 2022, at 05:57, Maxim Dounin <mdounin at mdounin.ru> wrote:
> > 
> > On Thu, May 12, 2022 at 01:03:37AM +0400, Sergey Kandaurov wrote:
> > 
> >> On Thu, Apr 21, 2022 at 01:18:58AM +0300, Maxim Dounin wrote:
> >>> # HG changeset patch
> >>> # User Maxim Dounin <mdounin at mdounin.ru>
> >>> # Date 1650492341 -10800
> >>> #      Thu Apr 21 01:05:41 2022 +0300
> >>> # Node ID fa9751ffe7723a11159c158078e454671e81cb87
> >>> # Parent  2027b85971d4b8a7e33c018548468057cb57eaf7
> >>> Upstream: multiple WWW-Authenticate headers (ticket #485).
> >>> 
> >>> When using proxy_intercept_errors and an error page for error 401
> >>> (Unauthorized), multiple WWW-Authenticate headers from the upstream server
> >>> response are now properly copied to the response.
> >>> 
> >>> diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
> >>> --- a/src/http/ngx_http_upstream.c
> >>> +++ b/src/http/ngx_http_upstream.c
> >>> @@ -2647,7 +2647,7 @@ ngx_http_upstream_intercept_errors(ngx_h
> >>> {
> >>>     ngx_int_t                  status;
> >>>     ngx_uint_t                 i;
> >>> -    ngx_table_elt_t           *h;
> >>> +    ngx_table_elt_t           *h, *ho, **ph;
> >>>     ngx_http_err_page_t       *err_page;
> >>>     ngx_http_core_loc_conf_t  *clcf;
> >>> 
> >>> @@ -2676,18 +2676,26 @@ ngx_http_upstream_intercept_errors(ngx_h
> >>>             if (status == NGX_HTTP_UNAUTHORIZED
> >>>                 && u->headers_in.www_authenticate)
> >>>             {
> >>> -                h = ngx_list_push(&r->headers_out.headers);
> >>> -
> >>> -                if (h == NULL) {
> >>> -                    ngx_http_upstream_finalize_request(r, u,
> >>> +                h = u->headers_in.www_authenticate;
> >>> +                ph = &r->headers_out.www_authenticate;
> >>> +
> >>> +                while (h) {
> >>> +                    ho = ngx_list_push(&r->headers_out.headers);
> >>> +
> >>> +                    if (ho == NULL) {
> >>> +                        ngx_http_upstream_finalize_request(r, u,
> >>>                                                NGX_HTTP_INTERNAL_SERVER_ERROR);
> >>> -                    return NGX_OK;
> >>> +                        return NGX_OK;
> >>> +                    }
> >>> +
> >>> +                    *ho = *h;
> >>> +                    ho->next = NULL;
> >>> +
> >>> +                    *ph = ho;
> >>> +                    ph = &ho->next;
> >>> +
> >>> +                    h = h->next;
> >>>                 }
> >>> -
> >>> -                *h = *u->headers_in.www_authenticate;
> >>> -                h->next = NULL;
> >>> -
> >>> -                r->headers_out.www_authenticate = h;
> >>>             }
> >>> 
> >>> #if (NGX_HTTP_CACHE)
> >>> 
> >> 
> >> While the patch certainly looks correct,
> >> I'm not sure about usefulness of r->headers_out.www_authenticate.
> >> Is the header accessed directly through this pointer ever?
> >> For read purposes I mean.  Header filters seem not.
> >> 
> >> The only place is in the auth_request module,
> >> but it doesn't seem to make sense in subrequest.
> > 
> > The auth request module uses it to copy the WWW-Authenticate 
> > headers from the subrequest response to the main request.  For 
> > example, in a configuration like:
> > 
> >    location / { auth_request /auth; ... }
> >    location /auth { auth_basic foo; ... }
> > 
> > While such configurations are unlikely, as using auth_basic 
> > directly should be easier, these are certainly possible.
> 
> I doubt access phase handlers are executed in subrequests at all,
> see the "r != r->main" check in ngx_http_core_access_phase().
> So it doesn't look feasible with auth_basic (or another auth module).

Yes, you're right, forgot about this check.  So it's probably only 
proxying then.

> > Similarly, r->headers_out.www_authenticate can be used with 
> > proxy_intercept_errors + error_page 401 in the subrequest (since 
> > r->upstream can be cleared by the error page).
> 
> That's true, tnx for the point.
> Indeed, with such setup, after internal redirect while in subrequest,
> headers are already there in (s)r->headers_out.www_authenticate.
> 
> > BTW, looking at this once again, I tend to think we also need 
> > ngx_http_core_access_phase() changes for proper handling of 
> > multiple WWW-Authenticate headers.  It might not be a good idea to 
> > commit it right now though, see commit log.
> 
> auth_spnego is such an outstanding example.
> As said, with the patch applied, when it comes to clear headers with
> "satisfy any;", this leads to segfault on h->next.
> Even with h->next initialized, and since the module allows to insert
> two headers at once, the first one gets lost tracking to be cleared
> (which is also true now).
> So, the module needs more work to gain from properly linked headers.

Yes, looks like a good example.  So, the module doesn't work 
correctly with "satisfy any;" now, and it cannot be fixed to work 
properly.  With the patch, it will be possible to fix the module 
to work correctly.

> > On the other hand, breaking it all at once might be less painful 
> > (after all, the auth_request change will already made such modules 
> > to segfault when used with auth_request; though this is unlikely, 
> > since such configurations hardly make sense).
> 
> Still not clear how's that possible in subrequests, see above.

I think you are right and it's not currently possible, see above.  
Yet I still tend to think that committing this with other changes 
would the right thing to do.

> > # HG changeset patch
> > # User Maxim Dounin <mdounin at mdounin.ru>
> > # Date 1652405911 -10800
> > #      Fri May 13 04:38:31 2022 +0300
> > # Node ID 0c2cf997bde390c9be27e0685ee2a00024c57e4e
> > # Parent  c53cb6f31b9855020d85476fa98ef1cdca809aed
> > Multiple WWW-Authenticate headers with "satisfy any;".
> > 
> > If a module adds multiple WWW-Authenticate headers (ticket #485) to the
> > response, linked in r->headers_out.www_authenticate, all headers are now
> > cleared if another module later allows access.
> > 
> > This change is a nop for standard modules, since the only access module which
> > can add multiple WWW-Authenticate headers is the auth request module, and
> > it is checked after other standard access modules.  Though this might
> > affect some third party access modules.
> > 
> > Note that if a 3rd party module adds a single WWW-Authenticate header
> > and not yet modified to set the header's next pointer to NULL, attempt to
> > clear such a header with this change will result in a segmentation fault.
> > 
> > diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
> > --- a/src/http/ngx_http_core_module.c
> > +++ b/src/http/ngx_http_core_module.c
> > @@ -1088,6 +1088,7 @@ ngx_int_t
> > ngx_http_core_access_phase(ngx_http_request_t *r, ngx_http_phase_handler_t *ph)
> > {
> >     ngx_int_t                  rc;
> > +    ngx_table_elt_t           *h;
> >     ngx_http_core_loc_conf_t  *clcf;
> > 
> >     if (r != r->main) {
> > @@ -1122,8 +1123,8 @@ ngx_http_core_access_phase(ngx_http_requ
> >         if (rc == NGX_OK) {
> >             r->access_code = 0;
> > 
> > -            if (r->headers_out.www_authenticate) {
> > -                r->headers_out.www_authenticate->hash = 0;
> > +            for (h = r->headers_out.www_authenticate; h; h = h->next) {
> > +                h->hash = 0;
> >             }
> > 
> >             r->phase_handler = ph->next;
> > 
> 
> I think it is good from an architectural point of view.

Yep, sure.  And I tend to think it worth committing, so it will be 
clear why and how modules adding WWW-Authenticate have to be 
modified.  Added this to the queue.

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Fri May 20 23:23:10 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sat, 21 May 2022 02:23:10 +0300
Subject: SSL contexts reuse across locations
In-Reply-To: <SJ0PR01MB65107F530D6D429E69595DFFB1D39@SJ0PR01MB6510.prod.exchangelabs.com>
References: <SJ0PR01MB6510467ED91D5036DAA3B5DFB1D19@SJ0PR01MB6510.prod.exchangelabs.com>
 <YoU7mt+Mmi/2my8k@mdounin.ru>
 <SJ0PR01MB65107F530D6D429E69595DFFB1D39@SJ0PR01MB6510.prod.exchangelabs.com>
Message-ID: <Yogi3ndHZFB1R6do@mdounin.ru>

Hello!

On Fri, May 20, 2022 at 06:52:54AM +0000, Pavel Pautov via nginx-devel wrote:

> > -----Original Message-----
> > From: Maxim Dounin <mdounin at mdounin.ru>
> > Sent: Wednesday, May 18, 2022 11:32
> [..]
> > > At very least, ngx_http_proxy_set_ssl() needs to be converted
> > > into ngx_http_proxy_create_ssl().
> > 
> > You may want to focus on actually making the code more readable
> > and abstracting it into ngx_http_proxy_set_ssl() instead.
> > Something like ngx_http_upstream_hide_headers_hash() might be a
> > good example on how to do it properly.
> 
> Do you suggest to make ngx_http_proxy_set_ssl() responsible for 
> merging context related settings? I guess, we can do that, but 
> something like ngx_http_proxy_create_ssl() might be still 
> beneficial.

No, I'm suggesting to use ngx_http_proxy_set_ssl() for what it 
currently does: create plcf->upstream.ssl.  And extend it to 
inherit plcf->upstream.ssl when possible.

> > Also, it should be a good idea to avoid creating SSL contexts if
> > there is no SSL proxying configured.  Or, at very least, make sure
> > only one context at the http level is used in such cases, so
> > configurations with many servers won't suddenly blow up.
> 

> The patch shouldn't cause a blow up to my understanding, expect perhaps for very specific configs like:
> server {
>   proxy_ssl_session_reuse off;
>   location / {
>      proxy_ssl_session_reuse on;
>       proxy_pass https://backend;
>   }
> }
> Basically, one have to have "proxy_ssl_*" directives scattered 
> across hierarchy of locations with a single terminal "proxy_pass 
> https". If there are many terminal locations with different 
> proxy_passes, then increase shouldn't be that big. If there is 
> no terminal "proxy_pass https", then why are these proxy_ssl_* 
> options even there?

Ah, it looks like I've misread your patch.  With the existing 
approach it is not clear how do you expect it to actually fix the 
original problem as in the ticket, that is, a configuration like:

http {
   proxy_ssl_trusted_certificate /etc/pki/tls/cert.pem;

   server {
      ...
      location ... { proxy_pass https://...; }
      location ... { proxy_pass https://...; }
      location ... { proxy_pass https://...; }

   }
}

With your patch, no SSL context will be created for the server{} 
block, so each location will have to create its own SSL context 
for proxying: exactly what we are trying to avoid.

[...]

> That being said, I think, we can actually satisfy "no SSL 
> contexts without https" requirement by linking location configs 
> and traversing them backwards in a search for location with 
> "proxy_ssl_*" directives. I'd update patch with that approach in 
> mind.

You may want to re-evaluate your understanding of the configuration 
merging process, and re-read the 
ngx_http_upstream_hide_headers_hash() function I've already 
mentioned above.

[...]

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Sat May 21 01:46:44 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sat, 21 May 2022 04:46:44 +0300
Subject: ssl_verify_partial_chain
In-Reply-To: <OF9D901334.FB4FEAB9-ONC1258848.004D1999-C1258848.004D199B@LocalDomain>
References: <OFB386D432.ED1CB321-ONC1258847.00230236-C1258848.004C84E7@LocalDomain>
 <OF9D901334.FB4FEAB9-ONC1258848.004D1999-C1258848.004D199B@LocalDomain>
Message-ID: <YohEhHjzr/LkpcKE@mdounin.ru>

Hello!

On Fri, May 20, 2022 at 04:02:08PM +0200, Vedran Vidovic wrote:

> Hello,
>  
> thanks for the extensive answer.
>  
> I will try to pursue a solution similar to your suggestion:
> - convert intermediate CA certificate to trusted certificate
> - validate client certificate using trusted certificate
>  
> I still believe that addition of the new configuration option
> "ssl_verify_partial_chain" would benefit nginx because if
> we configure it using the "ssl_trusted_certificate" it doesn't
> send a list of allowed CAs to the client. We just can't cover
> the case when we want to send a list of allowed issuers
> (without their root certs) without changes to nginx.

Sure, it could be beneficial to provide proper list of allowed 
certificate authorities during the SSL handshake.  But, as already 
suggested, it should be possible to solve this without introducing 
additional options.  Further, given the comments in the OpenSSL 
ticket, it might be something to address on the OpenSSL side if 
OpenSSL team is willing to simplify providing explicit trust 
information.

> In my view, if I configure a certificate I1 as a trusted issuer, I should not be enforced
> to add its issuer to list of trusted certificates. I would like to tell my server to trust all
> certificates issued by certificate I1 (and not by it's root issuer). Similar functionality
> is available in some other products I used and it seems natural to me but people
> with different background can (of course) disagree.

I agree that this approach is more natural and easier to 
understand than OpenSSL's default (and the only available till 
OpenSSL 1.0.2) "we have to build a chain up to a self-signed root 
CA" approach.

(Further, one of the practical use cases for this might be 
checking OCSP responses in OCSP Stapling, which are expected to be 
signed by the certificate issuer, so no additional certificates 
should be required for the OCSP response verification, which is 
not currently the case, see 
http://nginx.org/r/ssl_stapling_verify.)

What I don't like is the idea of introducing additional 
verification options, which will further complicate things.

We might actually consider enabling X509_V_FLAG_PARTIAL_CHAIN by 
default if available, though this might be something to carefully 
consider to ensure there will be no unwanted side-effects.

[...]

> Use Case 2 for such an approach (a real use case):
> On a more practical note, we need to trust all certs issued by any of the
> issuer certs from the EU trusted certificates list site and root certs are
> not published there.

Interesting, thanks for the details.

-- 
Maxim Dounin
http://mdounin.ru/


From lee.iitb at gmail.com  Mon May 23 03:14:00 2022
From: lee.iitb at gmail.com (Thomas Stephen Lee)
Date: Mon, 23 May 2022 08:44:00 +0530
Subject: EL 9 RPMs and SRPMs
Message-ID: <CAG7s96VWjL_PxcKaOSXR7sj9ADBFwAyeJ=E_xo9j_2=2X9FB_g@mail.gmail.com>

Hi,

Now that Red Hat has released EL 9, please provide SRPMs and RPMs for EL 9.
The Red Hat clones like Rocky and Amla will also be released soon.
We want to test the Nginx installation before putting it into production.

Thanks

---
Lee


From mths at google.com  Mon May 23 09:32:08 2022
From: mths at google.com (Mathias Bynens)
Date: Mon, 23 May 2022 11:32:08 +0200
Subject: [PATCH] Correct JavaScript MIME types + extensions per RFC 9239
In-Reply-To: <YnlczZbl0TtbIx8Y@mdounin.ru>
References: <CADizRgZC4NXLhjuBRyDq5hRCNNf_ctMye+ZqKN0-BWmXdbFF3w@mail.gmail.com>
 <YnlczZbl0TtbIx8Y@mdounin.ru>
Message-ID: <CADizRgYqFVMAtnwBiqDdcnU-w1U67JVVDnSkcjne4Ck5ZABnHQ@mail.gmail.com>

Thank you, Maxim. Is there anything else I can do to help move this
along? The change is rather small and clearly follows the latest RFC,
so I would expect it to be uncontroversial.

On Mon, May 9, 2022 at 8:26 PM Maxim Dounin <mdounin at mdounin.ru> wrote:
>
> Hello!
>
> On Mon, May 09, 2022 at 11:46:54AM +0200, Mathias Bynens via nginx-devel wrote:
>
> > # HG changeset patch
> > # User Mathias Bynens <mathias at qiwi.be>
> > # Date 1652088302 -7200
> > #      Mon May 09 11:25:02 2022 +0200
> > # Branch fix-js-mime-and-extensions
> > # Node ID 7944657540c64d92a5bdc3932710c977d984b54d
> > # Parent  35afae4b3dffff6718c0cab3ceb16b9de207c20a
> > Correct JavaScript MIME types + extensions per RFC 9239
> >
> > This patch updates the MIME type configuration per RFC 9239.
> > https://www.rfc-editor.org/rfc/rfc9239
> >
> > First, the recommended MIME type is now `text/javascript`:
> >
> > > The most widely supported media type in use is `text/javascript`; all
> > > others are considered historical and obsolete aliases of
> > `text/javascript`.
> >
> > Second, the `.mjs` extension is now explicitly registered:
> >
> > > The `.mjs` file extension signals that the file represents a JavaScript
> > > module. Execution environments that rely on file extensions to
> > > determine how to process inputs parse `.mjs` files using the Module
> > > grammar of [ECMA-262].
> >
> > IANA template: https://www.iana.org/assignments/media-types/text/javascript
> >
> > diff -r 35afae4b3dff -r 7944657540c6 conf/mime.types
> > --- a/conf/mime.types Fri Apr 29 17:38:01 2022 +0400
> > +++ b/conf/mime.types Mon May 09 11:25:02 2022 +0200
> > @@ -5,7 +5,7 @@
> >      text/xml                                         xml;
> >      image/gif                                        gif;
> >      image/jpeg                                       jpeg jpg;
> > -    application/javascript                           js;
> > +    text/javascript                                  js mjs;
> >      application/atom+xml                             atom;
> >      application/rss+xml                              rss;
> >
> > diff -r 35afae4b3dff -r 7944657540c6
> > src/http/modules/ngx_http_charset_filter_module.c
> > --- a/src/http/modules/ngx_http_charset_filter_module.c Fri Apr 29 17:38:01
> > 2022 +0400
> > +++ b/src/http/modules/ngx_http_charset_filter_module.c Mon May 09 11:25:02
> > 2022 +0200
> > @@ -128,7 +128,7 @@
> >      ngx_string("text/xml"),
> >      ngx_string("text/plain"),
> >      ngx_string("text/vnd.wap.wml"),
> > -    ngx_string("application/javascript"),
> > +    ngx_string("text/javascript"),
> >      ngx_string("application/rss+xml"),
> >      ngx_null_string
> >  };
>
> Thanks for the patch.
>
> I've added a link to the patch to the most relevant ticket
> (https://trac.nginx.org/nginx/ticket/2216).
>
> --
> Maxim Dounin
> http://mdounin.ru/


From thresh at nginx.com  Mon May 23 13:06:40 2022
From: thresh at nginx.com (Konstantin Pavlov)
Date: Mon, 23 May 2022 17:06:40 +0400
Subject: EL 9 RPMs and SRPMs
In-Reply-To: <CAG7s96VWjL_PxcKaOSXR7sj9ADBFwAyeJ=E_xo9j_2=2X9FB_g@mail.gmail.com>
References: <CAG7s96VWjL_PxcKaOSXR7sj9ADBFwAyeJ=E_xo9j_2=2X9FB_g@mail.gmail.com>
Message-ID: <63c98cf6-8109-a46e-1ca5-0149931e583f@nginx.com>

Hello!

On 23/05/2022 7:14 AM, Thomas Stephen Lee wrote:
> Hi,
> 
> Now that Red Hat has released EL 9, please provide SRPMs and RPMs for EL 9.
> The Red Hat clones like Rocky and Amla will also be released soon.
> We want to test the Nginx installation before putting it into production.

RHEL 9 packages are now published for both mainline and stable versions, 
enjoy:

stable: https://nginx.org/packages/rhel/9/
mainline: https://nginx.org/packages/mainline/rhel/9/

website docs on https://nginx.org/en/linux_packages.html will follow 
shortly.

Thanks,

-- 
Konstantin Pavlov
https://www.nginx.com


From dnj0496 at gmail.com  Mon May 23 19:24:16 2022
From: dnj0496 at gmail.com (Dk Jack)
Date: Mon, 23 May 2022 12:24:16 -0700
Subject: close request
Message-ID: <CAGATen6HyzkqBbD5MvFyk_iPFOgs38SYQH=mUdYrw4ua-Cg0fg@mail.gmail.com>

Hi,
What is the significance of this alert in ngx_http_close_request

    if (r->count == 0) {
        ngx_log_error(NGX_LOG_ALERT, c->log, 0, "http request count is
zero");
    }

I am seeing this alert in my logs in some cases.

Dk.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220523/57993323/attachment.htm>

From mdounin at mdounin.ru  Mon May 23 22:37:10 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 24 May 2022 01:37:10 +0300
Subject: close request
In-Reply-To: <CAGATen6HyzkqBbD5MvFyk_iPFOgs38SYQH=mUdYrw4ua-Cg0fg@mail.gmail.com>
References: <CAGATen6HyzkqBbD5MvFyk_iPFOgs38SYQH=mUdYrw4ua-Cg0fg@mail.gmail.com>
Message-ID: <YowMls422u5tNF73@mdounin.ru>

Hello!

On Mon, May 23, 2022 at 12:24:16PM -0700, Dk Jack wrote:

> Hi,
> What is the significance of this alert in ngx_http_close_request
> 
>     if (r->count == 0) {
>         ngx_log_error(NGX_LOG_ALERT, c->log, 0, "http request count is
> zero");
>     }
> 
> I am seeing this alert in my logs in some cases.

This alert means that request reference counting is incorrect, and 
this is serious enough to investigate as soon as possible.  
Depending on various other factors, this can also result in socket 
leaks, segmentation faults, or even remote code execution.

If you are seeing this with standard modules, please provide more 
details.  Ideally - steps how to reproduce this, though 
configuration and debug logs of a particular request should be 
enough.  If you are seeing this with your own or 3rd party 
modules, likely this indicate a bug in these modules.

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Tue May 24 03:17:59 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 24 May 2022 06:17:59 +0300
Subject: nginx-1.22.0 draft
Message-ID: <YoxOZ6jQ1xtIrVE3@mdounin.ru>

Hello!

Below are patches for the nginx-1.22.0 release, and corresponding 
changes to the site.

The nginx-1.22.0 release is based on nginx-1.21.6 with two 
additional changes grafted from the default branch: copyright year 
update and OpenSSL/zlib for win32 builds update.

(Note that applying these patches requires importing the second 
patch with "hg import --exact", to make sure it will use correct 
parent changeset and will use branch information.  Importing all 
patches with "--exact" will, however, fail, as export does not 
preserve graft information, and "--exact" will notice this.)

Comments are welcome.

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1653349909 -10800
#      Tue May 24 02:51:49 2022 +0300
# Node ID 8a54733c9d1290e6dc2f86af18e8a976a6352e4f
# Parent  35afae4b3dffff6718c0cab3ceb16b9de207c20a
Updated OpenSSL and zlib used for win32 builds.

diff --git a/misc/GNUmakefile b/misc/GNUmakefile
--- a/misc/GNUmakefile
+++ b/misc/GNUmakefile
@@ -6,8 +6,8 @@ TEMP =		tmp
 
 CC =		cl
 OBJS =		objs.msvc8
-OPENSSL =	openssl-1.1.1m
-ZLIB =		zlib-1.2.11
+OPENSSL =	openssl-1.1.1o
+ZLIB =		zlib-1.2.12
 PCRE =		pcre2-10.39
 
 
# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1653350129 -10800
#      Tue May 24 02:55:29 2022 +0300
# Branch stable-1.22
# Node ID c32b775633d37dffc3cfe4330549a85f21b47e3d
# Parent  56ead48cfe885e8b89b30017459bf621b21d95f5
Stable branch.

diff --git a/src/core/nginx.h b/src/core/nginx.h
--- a/src/core/nginx.h
+++ b/src/core/nginx.h
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1021006
-#define NGINX_VERSION      "1.21.6"
+#define nginx_version      1022000
+#define NGINX_VERSION      "1.22.0"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #ifdef NGX_BUILD
# HG changeset patch
# User Sergey Kandaurov <pluknet at nginx.com>
# Date 1643970571 -10800
#      Fri Feb 04 13:29:31 2022 +0300
# Branch stable-1.22
# Node ID dbf18f45a2b627944f79ec2e96859c7b6367d769
# Parent  c32b775633d37dffc3cfe4330549a85f21b47e3d
Year 2022.

diff --git a/docs/text/LICENSE b/docs/text/LICENSE
--- a/docs/text/LICENSE
+++ b/docs/text/LICENSE
@@ -1,6 +1,6 @@
 /* 
  * Copyright (C) 2002-2021 Igor Sysoev
- * Copyright (C) 2011-2021 Nginx, Inc.
+ * Copyright (C) 2011-2022 Nginx, Inc.
  * All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without
# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1653349909 -10800
#      Tue May 24 02:51:49 2022 +0300
# Branch stable-1.22
# Node ID adbfc4fb948c0f3043fdce2c52689293b96a6a65
# Parent  dbf18f45a2b627944f79ec2e96859c7b6367d769
Updated OpenSSL and zlib used for win32 builds.

diff --git a/misc/GNUmakefile b/misc/GNUmakefile
--- a/misc/GNUmakefile
+++ b/misc/GNUmakefile
@@ -6,8 +6,8 @@ TEMP =		tmp
 
 CC =		cl
 OBJS =		objs.msvc8
-OPENSSL =	openssl-1.1.1m
-ZLIB =		zlib-1.2.11
+OPENSSL =	openssl-1.1.1o
+ZLIB =		zlib-1.2.12
 PCRE =		pcre2-10.39
 
 
# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1653350358 -10800
#      Tue May 24 02:59:18 2022 +0300
# Branch stable-1.22
# Node ID f669c9c2a617d80daf753e012265ab5290df0d9b
# Parent  adbfc4fb948c0f3043fdce2c52689293b96a6a65
nginx-1.22.0-RELEASE

diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml
--- a/docs/xml/nginx/changes.xml
+++ b/docs/xml/nginx/changes.xml
@@ -5,6 +5,20 @@
 <change_log title="nginx">
 
 
+<changes ver="1.22.0" date="2022-05-24">
+
+<change>
+<para lang="ru">
+Стабильная ветка 1.22.x.
+</para>
+<para lang="en">
+1.22.x stable branch.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.21.6" date="2022-01-25">
 
 <change type="bugfix">
# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1653350359 -10800
#      Tue May 24 02:59:19 2022 +0300
# Branch stable-1.22
# Node ID 2d3ed138ce652d5fdb2403d996bdc05c3c638803
# Parent  f669c9c2a617d80daf753e012265ab5290df0d9b
release-1.22.0 tag

diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -467,3 +467,4 @@ 2217a9c1d0b86026f22700b3c089545db1964f55
 39be8a682c58308d9399cddd57e37f9fdb7bdf3e release-1.21.4
 d986378168fd4d70e0121cabac274c560cca9bdf release-1.21.5
 714eb4b2c09e712fb2572a2164ce2bf67638ccac release-1.21.6
+f669c9c2a617d80daf753e012265ab5290df0d9b release-1.22.0


Site changes:

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1653359312 -10800
#      Tue May 24 05:28:32 2022 +0300
# Node ID 8be8d7f16ce76dfcb8011ac69cd629f03478e321
# Parent  6cd5251fa7600f840ada2dd2a39c2aed6af29a02
nginx-1.22.0

diff --git a/text/en/CHANGES b/text/en/CHANGES-1.22
copy from text/en/CHANGES
copy to text/en/CHANGES-1.22
--- a/text/en/CHANGES
+++ b/text/en/CHANGES-1.22
@@ -1,4 +1,9 @@
 
+Changes with nginx 1.22.0                                        24 May 2022
+
+    *) 1.22.x stable branch.
+
+
 Changes with nginx 1.21.6                                        25 Jan 2022
 
     *) Bugfix: when using EPOLLEXCLUSIVE on Linux client connections were
diff --git a/text/ru/CHANGES.ru b/text/ru/CHANGES.ru-1.22
copy from text/ru/CHANGES.ru
copy to text/ru/CHANGES.ru-1.22
--- a/text/ru/CHANGES.ru
+++ b/text/ru/CHANGES.ru-1.22
@@ -1,4 +1,9 @@
 
+Изменения в nginx 1.22.0                                          24.05.2022
+
+    *) Стабильная ветка 1.22.x.
+
+
 Изменения в nginx 1.21.6                                          25.01.2022
 
     *) Исправление: при использование EPOLLEXCLUSIVE на Linux распределение
diff --git a/xml/index.xml b/xml/index.xml
--- a/xml/index.xml
+++ b/xml/index.xml
@@ -7,6 +7,29 @@
 
 <news name="nginx news" link="/" lang="en">
 
+<event date="2021-05-24">
+<para>
+<link doc="en/download.xml">nginx-1.22.0</link>
+stable version has been released,
+incorporating new features and bug fixes from the 1.21.x mainline branch &mdash;
+including
+hardening against potential requests smuggling
+and cross-protocol attacks,
+<link doc="en/docs/stream/ngx_stream_ssl_module.xml" id="ssl_alpn">ALPN
+support</link> in the stream module,
+better distribution of connections among worker processes on Linux,
+support for the PCRE2 library,
+support for OpenSSL 3.0 and <c-func>SSL_sendfile</c-func>,
+improved
+<link doc="en/docs/http/ngx_http_core_module.xml" id="sendfile">sendfile</link>
+handling on FreeBSD,
+the
+<link doc="en/docs/http/ngx_http_mp4_module.xml" id="mp4_start_key_frame">
+mp4_start_key_frame</link> directive,
+and more.
+</para>
+</event>
+
 <event date="2022-04-12">
 <para>
 <link doc="en/docs/njs/index.xml">njs-0.7.3</link>
diff --git a/xml/versions.xml b/xml/versions.xml
--- a/xml/versions.xml
+++ b/xml/versions.xml
@@ -10,6 +10,14 @@
 <download tag="mainline" changes="">
 
 <item ver="1.21.6" />
+
+</download>
+
+
+<download tag="stable" changes="1.22">
+
+<item ver="1.22.0" />
+<item ver="1.21.6" />
 <item ver="1.21.5" />
 <item ver="1.21.4" />
 <item ver="1.21.3" />
@@ -20,7 +28,7 @@
 </download>
 
 
-<download tag="stable" changes="1.20">
+<download tag="legacy" changes="1.20">
 
 <item ver="1.20.2" />
 <item ver="1.20.1" />

-- 
Maxim Dounin
http://mdounin.ru/


From lee.iitb at gmail.com  Tue May 24 04:40:00 2022
From: lee.iitb at gmail.com (Thomas Stephen Lee)
Date: Tue, 24 May 2022 10:10:00 +0530
Subject: EL 9 RPMs and SRPMs
In-Reply-To: <63c98cf6-8109-a46e-1ca5-0149931e583f@nginx.com>
References: <CAG7s96VWjL_PxcKaOSXR7sj9ADBFwAyeJ=E_xo9j_2=2X9FB_g@mail.gmail.com>
 <63c98cf6-8109-a46e-1ca5-0149931e583f@nginx.com>
Message-ID: <CAG7s96U99xyG5F=5z5qc_bVJAVeqT40O9R4hsSig-9_ugdZ3ng@mail.gmail.com>

Thanks a lot for the prompt action 😎.

---
Lee

On Mon, May 23, 2022 at 6:36 PM Konstantin Pavlov <thresh at nginx.com> wrote:
>
> Hello!
>
> On 23/05/2022 7:14 AM, Thomas Stephen Lee wrote:
> > Hi,
> >
> > Now that Red Hat has released EL 9, please provide SRPMs and RPMs for EL 9.
> > The Red Hat clones like Rocky and Amla will also be released soon.
> > We want to test the Nginx installation before putting it into production.
>
> RHEL 9 packages are now published for both mainline and stable versions,
> enjoy:
>
> stable: https://nginx.org/packages/rhel/9/
> mainline: https://nginx.org/packages/mainline/rhel/9/
>
> website docs on https://nginx.org/en/linux_packages.html will follow
> shortly.
>
> Thanks,
>
> --
> Konstantin Pavlov
> https://www.nginx.com


From xeioex at nginx.com  Tue May 24 05:27:36 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 24 May 2022 05:27:36 +0000
Subject: [njs] Fixed Object.prototype.constructor property handler with large
 heap.
Message-ID: <hg.f37972e95e17.1653370056.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/f37972e95e17
branches:  
changeset: 1861:f37972e95e17
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon May 23 22:26:35 2022 -0700
description:
Fixed Object.prototype.constructor property handler with large heap.

Found by Memory Sanitizer.

diffstat:

 src/njs_object.c |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (21 lines):

diff -r beb180165976 -r f37972e95e17 src/njs_object.c
--- a/src/njs_object.c	Thu May 19 16:41:08 2022 -0700
+++ b/src/njs_object.c	Mon May 23 22:26:35 2022 -0700
@@ -1849,7 +1849,7 @@ njs_int_t
 njs_object_prototype_create(njs_vm_t *vm, njs_object_prop_t *prop,
     njs_value_t *value, njs_value_t *setval, njs_value_t *retval)
 {
-    int32_t            index;
+    int64_t            index;
     njs_function_t     *function;
     const njs_value_t  *proto;
 
@@ -2194,7 +2194,7 @@ njs_int_t
 njs_object_prototype_create_constructor(njs_vm_t *vm, njs_object_prop_t *prop,
     njs_value_t *value, njs_value_t *setval, njs_value_t *retval)
 {
-    int32_t                 index;
+    int64_t                 index;
     njs_value_t             *cons, constructor;
     njs_object_t            *object;
     njs_object_prototype_t  *prototype;


From xeioex at nginx.com  Tue May 24 05:27:38 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 24 May 2022 05:27:38 +0000
Subject: [njs] Fixed use-of-uninitialized-value introduced in beb180165976.
Message-ID: <hg.98959158fe23.1653370058.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/98959158fe23
branches:  
changeset: 1862:98959158fe23
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon May 23 22:26:50 2022 -0700
description:
Fixed use-of-uninitialized-value introduced in beb180165976.

Found by Memory Sanitizer.

diffstat:

 src/njs_object.h |  7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diffs (17 lines):

diff -r f37972e95e17 -r 98959158fe23 src/njs_object.h
--- a/src/njs_object.h	Mon May 23 22:26:35 2022 -0700
+++ b/src/njs_object.h	Mon May 23 22:26:50 2022 -0700
@@ -88,9 +88,10 @@ njs_int_t njs_object_prop_init(njs_vm_t 
 njs_inline njs_bool_t
 njs_is_data_descriptor(njs_object_prop_t *prop)
 {
-    return njs_is_valid(&prop->value) ||
-           prop->writable != NJS_ATTRIBUTE_UNSET ||
-           prop->type == NJS_PROPERTY_HANDLER;
+    return prop->writable != NJS_ATTRIBUTE_UNSET
+           || njs_is_valid(&prop->value)
+           || prop->type == NJS_PROPERTY_HANDLER;
+
 }
 
 


From xeioex at nginx.com  Tue May 24 05:27:40 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 24 May 2022 05:27:40 +0000
Subject: [njs] Tests: adapted WebCrypto tests to RHEL9.
Message-ID: <hg.169f20b8a3e8.1653370060.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/169f20b8a3e8
branches:  
changeset: 1863:169f20b8a3e8
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon May 23 22:26:51 2022 -0700
description:
Tests: adapted WebCrypto tests to RHEL9.

diffstat:

 test/webcrypto/sign.t.js   |  12 +++++++++++-
 test/webcrypto/verify.t.js |  12 +++++++++++-
 2 files changed, 22 insertions(+), 2 deletions(-)

diffs (44 lines):

diff -r 98959158fe23 -r 169f20b8a3e8 test/webcrypto/sign.t.js
--- a/test/webcrypto/sign.t.js	Mon May 23 22:26:50 2022 -0700
+++ b/test/webcrypto/sign.t.js	Mon May 23 22:26:51 2022 -0700
@@ -11,7 +11,17 @@ async function test(params) {
                                                  false, [ "sign" ]);
 
     let sig = await crypto.subtle.sign(params.sign_alg, sign_key,
-                                     encoder.encode(params.text));
+                                     encoder.encode(params.text))
+                    .catch (e => {
+                        if (e.toString().startsWith("Error: EVP_PKEY_CTX_set_signature_md() failed")) {
+                            /* Red Hat Enterprise Linux: SHA-1 is disabled */
+                            return "SKIPPED";
+                        }
+                    });
+
+    if (sig == "SKIPPED") {
+        return sig;
+    }
 
     if (params.verify) {
         let verify_key = await crypto.subtle.importKey(params.verify_key.fmt,
diff -r 98959158fe23 -r 169f20b8a3e8 test/webcrypto/verify.t.js
--- a/test/webcrypto/verify.t.js	Mon May 23 22:26:50 2022 -0700
+++ b/test/webcrypto/verify.t.js	Mon May 23 22:26:51 2022 -0700
@@ -11,7 +11,17 @@ async function test(params) {
 
     let r = await crypto.subtle.verify(params.verify_alg,
                                        key, params.signature,
-                                       params.text);
+                                       params.text)
+                    .catch (e => {
+                        if (e.toString().startsWith("Error: EVP_PKEY_CTX_set_signature_md() failed")) {
+                            /* Red Hat Enterprise Linux: SHA-1 is disabled */
+                            return "SKIPPED";
+                        }
+                    });
+
+    if (r == "SKIPPED") {
+        return r;
+    }
 
     if (params.expected !== r) {
         throw Error(`${params.import_alg.name} failed expected: "${params.expected}" vs "${r}"`);


From maxim at nginx.com  Tue May 24 07:39:31 2022
From: maxim at nginx.com (Maxim Konovalov)
Date: Tue, 24 May 2022 11:39:31 +0400
Subject: nginx-1.22.0 draft
In-Reply-To: <YoxOZ6jQ1xtIrVE3@mdounin.ru>
References: <YoxOZ6jQ1xtIrVE3@mdounin.ru>
Message-ID: <0eee2e94-cdd3-ec39-7b79-dfe67b3c34db@nginx.com>

On 24.05.2022 07:17, Maxim Dounin wrote:
> Hello!
> 
> Below are patches for the nginx-1.22.0 release, and corresponding
> changes to the site.
> 
> The nginx-1.22.0 release is based on nginx-1.21.6 with two
> additional changes grafted from the default branch: copyright year
> update and OpenSSL/zlib for win32 builds update.
> 
> (Note that applying these patches requires importing the second
> patch with "hg import --exact", to make sure it will use correct
> parent changeset and will use branch information.  Importing all
> patches with "--exact" will, however, fail, as export does not
> preserve graft information, and "--exact" will notice this.)
> 
> Comments are welcome.
> 
> # HG changeset patch
> # User Maxim Dounin <mdounin at mdounin.ru>
> # Date 1653349909 -10800
> #      Tue May 24 02:51:49 2022 +0300
> # Node ID 8a54733c9d1290e6dc2f86af18e8a976a6352e4f
> # Parent  35afae4b3dffff6718c0cab3ceb16b9de207c20a
> Updated OpenSSL and zlib used for win32 builds.
> 

[...]

Looks good.

-- 
Maxim Konovalov


From P.Pautov at F5.com  Tue May 24 09:25:59 2022
From: P.Pautov at F5.com (Pavel Pautov)
Date: Tue, 24 May 2022 09:25:59 +0000
Subject: SSL contexts reuse across locations
In-Reply-To: <Yogi3ndHZFB1R6do@mdounin.ru>
References:  <SJ0PR01MB6510467ED91D5036DAA3B5DFB1D19@SJ0PR01MB6510.prod.exchangelabs.com>
 <YoU7mt+Mmi/2my8k@mdounin.ru>
 <SJ0PR01MB65107F530D6D429E69595DFFB1D39@SJ0PR01MB6510.prod.exchangelabs.com>
 <Yogi3ndHZFB1R6do@mdounin.ru>
Message-ID:  <SJ0PR01MB65100A5C153CF349E6EF3ECBB1D79@SJ0PR01MB6510.prod.exchangelabs.com>

Hi,

Attaching updated PoC patch, which also reuses contexts from 'http' level.

> -----Original Message-----
> From: Maxim Dounin <mdounin at mdounin.ru>
> Sent: Friday, May 20, 2022 16:23
> 
> Hello!
> 
> On Fri, May 20, 2022 at 06:52:54AM +0000, Pavel Pautov via nginx-devel wrote:
> 
> > > -----Original Message-----
> > > From: Maxim Dounin <mdounin at mdounin.ru>
> > > Sent: Wednesday, May 18, 2022 11:32
> > [..]
> > > > At very least, ngx_http_proxy_set_ssl() needs to be converted
> > > > into ngx_http_proxy_create_ssl().
> > >
> > > You may want to focus on actually making the code more readable
> > > and abstracting it into ngx_http_proxy_set_ssl() instead.
> > > Something like ngx_http_upstream_hide_headers_hash() might be a
> > > good example on how to do it properly.
> >
> > Do you suggest to make ngx_http_proxy_set_ssl() responsible for
> > merging context related settings? I guess, we can do that, but
> > something like ngx_http_proxy_create_ssl() might be still
> > beneficial.
> 
> No, I'm suggesting to use ngx_http_proxy_set_ssl() for what it
> currently does: create plcf->upstream.ssl.  And extend it to
> inherit plcf->upstream.ssl when possible.

I went with ngx_http_proxy_create_ssl() for now (see patch), as I need to call it twice in some cases.
However, contexts reuse logic can be moved into separate function, as well. Especially if we are going to reuse it across modules.

> > > Also, it should be a good idea to avoid creating SSL contexts if
> > > there is no SSL proxying configured.  Or, at very least, make sure
> > > only one context at the http level is used in such cases, so
> > > configurations with many servers won't suddenly blow up.
> >
> 
> > The patch shouldn't cause a blow up to my understanding, expect perhaps for
> very specific configs like:
> > server {
> >   proxy_ssl_session_reuse off;
> >   location / {
> >      proxy_ssl_session_reuse on;
> >       proxy_pass https://backend;
> >   }
> > }
> > Basically, one have to have "proxy_ssl_*" directives scattered
> > across hierarchy of locations with a single terminal "proxy_pass
> > https". If there are many terminal locations with different
> > proxy_passes, then increase shouldn't be that big. If there is
> > no terminal "proxy_pass https", then why are these proxy_ssl_*
> > options even there?
> 
> Ah, it looks like I've misread your patch.  With the existing
> approach it is not clear how do you expect it to actually fix the
> original problem as in the ticket, that is, a configuration like:
> 
> http {
>    proxy_ssl_trusted_certificate /etc/pki/tls/cert.pem;
> 
>    server {
>       ...
>       location ... { proxy_pass https://...; }
>       location ... { proxy_pass https://...; }
>       location ... { proxy_pass https://...; }
> 
>    }
> }
> 
> With your patch, no SSL context will be created for the server{}
> block, so each location will have to create its own SSL context
> for proxying: exactly what we are trying to avoid.

Indeed, somehow I've assumed there is going to be a merge into 'http' from some 'null' location.

> 
> [...]
> 
> > That being said, I think, we can actually satisfy "no SSL
> > contexts without https" requirement by linking location configs
> > and traversing them backwards in a search for location with
> > "proxy_ssl_*" directives. I'd update patch with that approach in
> > mind.
> 
> You may want to re-evaluate your understanding of the configuration
> merging process, and re-read the
> ngx_http_upstream_hide_headers_hash() function I've already
> mentioned above.

I did and I still find above a valid technique for achieving "no contexts without https proxy_pass" (not sure though, if it's still a requirement). There could be several layers of locations between "proxy_ssl_*" and proxy_pass and if we don't create context on the way down, we'll have to go up to create it for other nested locations to see it. Alternatively, I guess, we can use something like "ngx_ssl_t **" to have same placeholder for locations which would reuse context, and init it only if we hit "https proxy_pass".

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ssl_ctx_reuse.patch
Type: application/octet-stream
Size: 6615 bytes
Desc: ssl_ctx_reuse.patch
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220524/5a1c60c6/attachment.obj>

From mdounin at mdounin.ru  Tue May 24 13:55:13 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 24 May 2022 16:55:13 +0300
Subject: nginx-1.22.0 draft
In-Reply-To: <0eee2e94-cdd3-ec39-7b79-dfe67b3c34db@nginx.com>
References: <YoxOZ6jQ1xtIrVE3@mdounin.ru>
 <0eee2e94-cdd3-ec39-7b79-dfe67b3c34db@nginx.com>
Message-ID: <Yozjwab7QUotlVke@mdounin.ru>

Hello!

On Tue, May 24, 2022 at 11:39:31AM +0400, Maxim Konovalov wrote:

> On 24.05.2022 07:17, Maxim Dounin wrote:
> > Hello!
> > 
> > Below are patches for the nginx-1.22.0 release, and corresponding
> > changes to the site.
> > 
> > The nginx-1.22.0 release is based on nginx-1.21.6 with two
> > additional changes grafted from the default branch: copyright year
> > update and OpenSSL/zlib for win32 builds update.
> > 
> > (Note that applying these patches requires importing the second
> > patch with "hg import --exact", to make sure it will use correct
> > parent changeset and will use branch information.  Importing all
> > patches with "--exact" will, however, fail, as export does not
> > preserve graft information, and "--exact" will notice this.)
> > 
> > Comments are welcome.
> > 
> > # HG changeset patch
> > # User Maxim Dounin <mdounin at mdounin.ru>
> > # Date 1653349909 -10800
> > #      Tue May 24 02:51:49 2022 +0300
> > # Node ID 8a54733c9d1290e6dc2f86af18e8a976a6352e4f
> > # Parent  35afae4b3dffff6718c0cab3ceb16b9de207c20a
> > Updated OpenSSL and zlib used for win32 builds.
> > 
> 
> [...]
> 
> Looks good.

Pushed to:

http://mdounin.ru/hg/nginx
http://mdounin.ru/hg/nginx.org

Release files:

http://mdounin.ru/temp/nginx-1.22.0.tar.gz
http://mdounin.ru/temp/nginx-1.22.0.tar.gz.asc
http://mdounin.ru/temp/nginx-1.22.0.zip
http://mdounin.ru/temp/nginx-1.22.0.zip.asc

-- 
Maxim Dounin
http://mdounin.ru/


From thresh at nginx.com  Tue May 24 14:05:24 2022
From: thresh at nginx.com (Konstantin Pavlov)
Date: Tue, 24 May 2022 14:05:24 +0000
Subject: [nginx] Updated OpenSSL and zlib used for win32 builds.
Message-ID: <hg.8a54733c9d12.1653401124.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/8a54733c9d12
branches:  
changeset: 8011:8a54733c9d12
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue May 24 02:51:49 2022 +0300
description:
Updated OpenSSL and zlib used for win32 builds.

diffstat:

 misc/GNUmakefile |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff -r 35afae4b3dff -r 8a54733c9d12 misc/GNUmakefile
--- a/misc/GNUmakefile	Fri Apr 29 17:38:01 2022 +0400
+++ b/misc/GNUmakefile	Tue May 24 02:51:49 2022 +0300
@@ -6,8 +6,8 @@ TEMP =		tmp
 
 CC =		cl
 OBJS =		objs.msvc8
-OPENSSL =	openssl-1.1.1m
-ZLIB =		zlib-1.2.11
+OPENSSL =	openssl-1.1.1o
+ZLIB =		zlib-1.2.12
 PCRE =		pcre2-10.39
 
 


From thresh at nginx.com  Tue May 24 14:05:27 2022
From: thresh at nginx.com (Konstantin Pavlov)
Date: Tue, 24 May 2022 14:05:27 +0000
Subject: [nginx] Stable branch.
Message-ID: <hg.c32b775633d3.1653401127.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/c32b775633d3
branches:  stable-1.22
changeset: 8012:c32b775633d3
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue May 24 02:55:29 2022 +0300
description:
Stable branch.

diffstat:

 src/core/nginx.h |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff -r 56ead48cfe88 -r c32b775633d3 src/core/nginx.h
--- a/src/core/nginx.h	Tue Jan 25 18:03:52 2022 +0300
+++ b/src/core/nginx.h	Tue May 24 02:55:29 2022 +0300
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1021006
-#define NGINX_VERSION      "1.21.6"
+#define nginx_version      1022000
+#define NGINX_VERSION      "1.22.0"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #ifdef NGX_BUILD


From thresh at nginx.com  Tue May 24 14:05:29 2022
From: thresh at nginx.com (Konstantin Pavlov)
Date: Tue, 24 May 2022 14:05:29 +0000
Subject: [nginx] Year 2022.
Message-ID: <hg.dbf18f45a2b6.1653401129.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/dbf18f45a2b6
branches:  stable-1.22
changeset: 8013:dbf18f45a2b6
user:      Sergey Kandaurov <pluknet at nginx.com>
date:      Fri Feb 04 13:29:31 2022 +0300
description:
Year 2022.

diffstat:

 docs/text/LICENSE |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (11 lines):

diff -r c32b775633d3 -r dbf18f45a2b6 docs/text/LICENSE
--- a/docs/text/LICENSE	Tue May 24 02:55:29 2022 +0300
+++ b/docs/text/LICENSE	Fri Feb 04 13:29:31 2022 +0300
@@ -1,6 +1,6 @@
 /* 
  * Copyright (C) 2002-2021 Igor Sysoev
- * Copyright (C) 2011-2021 Nginx, Inc.
+ * Copyright (C) 2011-2022 Nginx, Inc.
  * All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without


From thresh at nginx.com  Tue May 24 14:05:32 2022
From: thresh at nginx.com (Konstantin Pavlov)
Date: Tue, 24 May 2022 14:05:32 +0000
Subject: [nginx] Updated OpenSSL and zlib used for win32 builds.
Message-ID: <hg.adbfc4fb948c.1653401132.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/adbfc4fb948c
branches:  stable-1.22
changeset: 8014:adbfc4fb948c
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue May 24 02:51:49 2022 +0300
description:
Updated OpenSSL and zlib used for win32 builds.

diffstat:

 misc/GNUmakefile |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff -r dbf18f45a2b6 -r adbfc4fb948c misc/GNUmakefile
--- a/misc/GNUmakefile	Fri Feb 04 13:29:31 2022 +0300
+++ b/misc/GNUmakefile	Tue May 24 02:51:49 2022 +0300
@@ -6,8 +6,8 @@ TEMP =		tmp
 
 CC =		cl
 OBJS =		objs.msvc8
-OPENSSL =	openssl-1.1.1m
-ZLIB =		zlib-1.2.11
+OPENSSL =	openssl-1.1.1o
+ZLIB =		zlib-1.2.12
 PCRE =		pcre2-10.39
 
 


From thresh at nginx.com  Tue May 24 14:05:35 2022
From: thresh at nginx.com (Konstantin Pavlov)
Date: Tue, 24 May 2022 14:05:35 +0000
Subject: [nginx] nginx-1.22.0-RELEASE
Message-ID: <hg.f669c9c2a617.1653401135.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/f669c9c2a617
branches:  stable-1.22
changeset: 8015:f669c9c2a617
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue May 24 02:59:18 2022 +0300
description:
nginx-1.22.0-RELEASE

diffstat:

 docs/xml/nginx/changes.xml |  14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

diffs (24 lines):

diff -r adbfc4fb948c -r f669c9c2a617 docs/xml/nginx/changes.xml
--- a/docs/xml/nginx/changes.xml	Tue May 24 02:51:49 2022 +0300
+++ b/docs/xml/nginx/changes.xml	Tue May 24 02:59:18 2022 +0300
@@ -5,6 +5,20 @@
 <change_log title="nginx">
 
 
+<changes ver="1.22.0" date="2022-05-24">
+
+<change>
+<para lang="ru">
+Стабильная ветка 1.22.x.
+</para>
+<para lang="en">
+1.22.x stable branch.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.21.6" date="2022-01-25">
 
 <change type="bugfix">

From thresh at nginx.com  Tue May 24 14:05:38 2022
From: thresh at nginx.com (Konstantin Pavlov)
Date: Tue, 24 May 2022 14:05:38 +0000
Subject: [nginx] release-1.22.0 tag
Message-ID: <hg.2d3ed138ce65.1653401138.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/2d3ed138ce65
branches:  stable-1.22
changeset: 8016:2d3ed138ce65
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue May 24 02:59:19 2022 +0300
description:
release-1.22.0 tag

diffstat:

 .hgtags |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (8 lines):

diff -r f669c9c2a617 -r 2d3ed138ce65 .hgtags
--- a/.hgtags	Tue May 24 02:59:18 2022 +0300
+++ b/.hgtags	Tue May 24 02:59:19 2022 +0300
@@ -467,3 +467,4 @@ 2217a9c1d0b86026f22700b3c089545db1964f55
 39be8a682c58308d9399cddd57e37f9fdb7bdf3e release-1.21.4
 d986378168fd4d70e0121cabac274c560cca9bdf release-1.21.5
 714eb4b2c09e712fb2572a2164ce2bf67638ccac release-1.21.6
+f669c9c2a617d80daf753e012265ab5290df0d9b release-1.22.0


From dnj0496 at gmail.com  Tue May 24 15:04:16 2022
From: dnj0496 at gmail.com (Dk Jack)
Date: Tue, 24 May 2022 08:04:16 -0700
Subject: close request
In-Reply-To: <CAGATen6HyzkqBbD5MvFyk_iPFOgs38SYQH=mUdYrw4ua-Cg0fg@mail.gmail.com>
References: <CAGATen6HyzkqBbD5MvFyk_iPFOgs38SYQH=mUdYrw4ua-Cg0fg@mail.gmail.com>
Message-ID: <CAGATen4rHby1inr5Wk9j-D8fLxG3hdq=uqi0OBoQVEPbtnwYGg@mail.gmail.com>

Maxim,
I am noticing this with our modules. Although, we do not do not message
with any nginx reference counting etc.
At most, we are sending 403-forbidden for some requests. Could you suggest
some ways about debugging this?
Thanks.

Bhasker.

------------------------------------------------------------------------
This alert means that request reference counting is incorrect, and
this is serious enough to investigate as soon as possible.
Depending on various other factors, this can also result in socket
leaks, segmentation faults, or even remote code execution.

If you are seeing this with standard modules, please provide more
details. Ideally - steps how to reproduce this, though
configuration and debug logs of a particular request should be
enough. If you are seeing this with your own or 3rd party
modules, likely this indicate a bug in these modules.

--
Maxim Dounin
http://mdounin.ru/
_______________________________________________
nginx-devel mailing list -- nginx-devel at nginx.org
To unsubscribe send an email to nginx-devel-leave at nginx.org

> Hi,
> What is the significance of this alert in ngx_http_close_request
>
>     if (r->count == 0) {
>         ngx_log_error(NGX_LOG_ALERT, c->log, 0, "http request count is
> zero");
>     }
>
> I am seeing this alert in my logs in some cases.
>
> Dk.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220524/b1efcbd8/attachment.htm>

From xeioex at nginx.com  Tue May 24 16:11:33 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 24 May 2022 16:11:33 +0000
Subject: [njs] Version 0.7.4.
Message-ID: <hg.b5198f7f11a3.1653408693.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/b5198f7f11a3
branches:  
changeset: 1864:b5198f7f11a3
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue May 24 09:07:58 2022 -0700
description:
Version 0.7.4.

diffstat:

 CHANGES |  45 +++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 45 insertions(+), 0 deletions(-)

diffs (52 lines):

diff -r 169f20b8a3e8 -r b5198f7f11a3 CHANGES
--- a/CHANGES	Mon May 23 22:26:51 2022 -0700
+++ b/CHANGES	Tue May 24 09:07:58 2022 -0700
@@ -1,3 +1,48 @@
+Changes with njs 0.7.4                                        24 May 2022
+
+    nginx modules:
+
+    *) Feature: added extended directive to configure Fetch API.
+       The following directives were added: "js_fetch_timeout",
+       "js_fetch_verify", "js_fetch_buffer_size",
+       "js_fetch_max_response_buffer_size".
+
+    *) Change: r.internalRedirect() now accepts escaped URIs.
+
+    *) Bugfix: fixed Response parsing with more than 8 headers
+       in Fetch API.
+
+    Core:
+
+    *) Feature: added njs.version_number property.
+
+    *) Feature: added compatibility with BoringSSL for
+       WebCrypto API.
+
+    *) Bugfix: fixed Array.prototype.sort() when arr size is changed
+       in a comparator.
+
+    *) Bugfix: fixed Array.prototype.slice() with slow "this" argument.
+
+    *) Bugfix: fixed aggregation methods of Promise ctor with
+       array-like object.
+
+    *) Bugfix: fixed Array.prototype.lastIndexOf() with unicode
+       string as "this".
+
+    *) Bugfix: fixed JSON.parse() when reviver function is provided.
+
+    *) Bugfix: fixed Object.defineProperty() when a recursive descriptor
+       is provided.
+
+    *) Bugfix: fixed Array.prototype.fill() for typed-arrays.
+
+    *) Bugfix: making function expression binding immutable according
+       the specs.
+
+    *) Bugfix: fixed redefinition of special props in
+       Object.defineProperty().
+
 Changes with njs 0.7.3                                        12 Apr 2022
 
     Core:


From xeioex at nginx.com  Tue May 24 16:11:35 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 24 May 2022 16:11:35 +0000
Subject: [njs] Added tag 0.7.4 for changeset b5198f7f11a3
Message-ID: <hg.640ed49b2dce.1653408695.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/640ed49b2dce
branches:  
changeset: 1865:640ed49b2dce
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue May 24 09:10:52 2022 -0700
description:
Added tag 0.7.4 for changeset b5198f7f11a3

diffstat:

 .hgtags |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (8 lines):

diff -r b5198f7f11a3 -r 640ed49b2dce .hgtags
--- a/.hgtags	Tue May 24 09:07:58 2022 -0700
+++ b/.hgtags	Tue May 24 09:10:52 2022 -0700
@@ -49,3 +49,4 @@ 8418bd4a4ce3114d57b4d75f913e8c4912bf4b5d
 35aca5cc5ea7582b80947caa1e3f4a4fb8ee232d 0.7.1
 3dd315b80bab10b6ac475ee25dd207d2eb759881 0.7.2
 f15d039cf625fb92e061f21c9f28a788032a0faa 0.7.3
+b5198f7f11a3b5e174f9e75a7bd50394fa354fb0 0.7.4


From A.Bavshin at F5.com  Tue May 24 16:37:11 2022
From: A.Bavshin at F5.com (Aleksei Bavshin)
Date: Tue, 24 May 2022 16:37:11 +0000
Subject: [PATCH] Stream: don't flush empty buffers created for read errors.
Message-ID:  <CO6PR01MB7482629728C2B522E90A6AECC0D79@CO6PR01MB7482.prod.exchangelabs.com>

In all honesty, the main motivation for the patch is to address a regression in UDP healthchecks caused by the behavior described below. It's already covered by tests, but the corresponding cases were disabled by default.

I considered
    cl->buf->flush = !src->read->error;
or even
    cl->buf->flush = !src->read->eof; 
but neither of those are exactly equivalent to the code below (with the latter having significantly different behavior), and IMO it would look less obvious. 

# HG changeset patch
# User Aleksei Bavshin <a.bavshin at f5.com>
# Date 1653330584 25200
#      Mon May 23 11:29:44 2022 -0700
# Branch se
# Node ID 5a98e9cb437f7719afa2bde62de68e174fd8e03e
# Parent  8902674cc7fe759cada415c10340f31ae4a90fba
Stream: don't flush empty buffers created for read errors.

When we generate the last_buf buffer for an UDP upstream recv error, it does
not contain any data from the wire. ngx_stream_write_filter attempts to forward
it anyways, which is incorrect (e.g., UDP upstream ECONNREFUSED will be
translated to an empty packet).

Reproduction:

stream {
    upstream unreachable {
        server     127.0.0.1:8880;
    }
    server {
        listen     127.0.0.1:8998 udp;
        proxy_pass unreachable;
    }
}

1 0.000000000    127.0.0.1 → 127.0.0.1    UDP 47 45588 → 8998 Len=5
2 0.000166300    127.0.0.1 → 127.0.0.1    UDP 47 51149 → 8880 Len=5
3 0.000172600    127.0.0.1 → 127.0.0.1    ICMP 75 Destination unreachable (Port
unreachable)
4 0.000202400    127.0.0.1 → 127.0.0.1    UDP 42 8998 → 45588 Len=0

Fixes d127837c714f.

diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c
--- a/src/stream/ngx_stream_proxy_module.c
+++ b/src/stream/ngx_stream_proxy_module.c
@@ -1676,7 +1676,7 @@ ngx_stream_proxy_process(ngx_stream_sess
     ssize_t                       n;
     ngx_buf_t                    *b;
     ngx_int_t                     rc;
-    ngx_uint_t                    flags, *packets;
+    ngx_uint_t                    flags, flush, *packets;
     ngx_msec_t                    delay;
     ngx_chain_t                  *cl, **ll, **out, **busy;
     ngx_connection_t             *c, *pc, *src, *dst;
@@ -1803,9 +1803,12 @@ ngx_stream_proxy_process(ngx_stream_sess
                 break;
             }
 
+            flush = 1;
+
             if (n == NGX_ERROR) {
                 src->read->eof = 1;
                 n = 0;
+                flush = 0;
             }
 
             if (n >= 0) {
@@ -1846,7 +1849,7 @@ ngx_stream_proxy_process(ngx_stream_sess
 
                 cl->buf->temporary = (n ? 1 : 0);
                 cl->buf->last_buf = src->read->eof;
-                cl->buf->flush = 1;
+                cl->buf->flush = flush;
 
                 (*packets)++;
                 *received += n;


From mdounin at mdounin.ru  Tue May 24 17:15:05 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 24 May 2022 20:15:05 +0300
Subject: close request
In-Reply-To: <CAGATen4rHby1inr5Wk9j-D8fLxG3hdq=uqi0OBoQVEPbtnwYGg@mail.gmail.com>
References: <CAGATen6HyzkqBbD5MvFyk_iPFOgs38SYQH=mUdYrw4ua-Cg0fg@mail.gmail.com>
 <CAGATen4rHby1inr5Wk9j-D8fLxG3hdq=uqi0OBoQVEPbtnwYGg@mail.gmail.com>
Message-ID: <Yo0Sma0IlV14o3iG@mdounin.ru>

Hello!

On Tue, May 24, 2022 at 08:04:16AM -0700, Dk Jack wrote:

> I am noticing this with our modules. Although, we do not do not message
> with any nginx reference counting etc.
> At most, we are sending 403-forbidden for some requests. Could you suggest
> some ways about debugging this?

Request reference counting automatically happens in various nginx 
functions, such as ngx_http_read_client_request_body() and 
ngx_http_finalize_request().  Incorrect reference counting 
suggests that your module does something wrong: for example, calls 
ngx_http_finalize_request() when it shouldn't.  Or returns an 
incorrect result from a handler, which causes 
ngx_http_finalize_request() to be called when it shouldn't.

The most straightforward way to debug such problems is to enable 
debug logging and check all reference counting step-by-step for a 
request which results in the alert.

-- 
Maxim Dounin
http://mdounin.ru/


From pluknet at nginx.com  Tue May 24 20:05:57 2022
From: pluknet at nginx.com (=?iso-8859-1?q?Sergey_Kandaurov?=)
Date: Wed, 25 May 2022 00:05:57 +0400
Subject: [PATCH] Upstream: handling of certificates specified as an empty
 string
Message-ID: <3bb1adbb74dfcd372f73.1653422757@enoparse.local>

# HG changeset patch
# User Sergey Kandaurov <pluknet at nginx.com>
# Date 1653422583 -14400
#      Wed May 25 00:03:03 2022 +0400
# Node ID 3bb1adbb74dfcd372f7369530967cfb415900778
# Parent  8a54733c9d1290e6dc2f86af18e8a976a6352e4f
Upstream: handling of certificates specified as an empty string.

Now, if the directive is given an empty string, such configuration cancels
loading of certificates should they be inherited from the previous level.
This restores a previous behaviour, before variables support in certificates
was introduced (3ab8e1e2f0f7).

diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c
--- a/src/http/modules/ngx_http_grpc_module.c
+++ b/src/http/modules/ngx_http_grpc_module.c
@@ -4921,7 +4921,7 @@ ngx_http_grpc_set_ssl(ngx_conf_t *cf, ng
                 return NGX_ERROR;
             }
 
-        } else {
+        } else if (glcf->upstream.ssl_certificate->value.len) {
             if (ngx_ssl_certificate(cf, glcf->upstream.ssl,
                                     &glcf->upstream.ssl_certificate->value,
                                     &glcf->upstream.ssl_certificate_key->value,
diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c
--- a/src/http/modules/ngx_http_proxy_module.c
+++ b/src/http/modules/ngx_http_proxy_module.c
@@ -4970,7 +4970,7 @@ ngx_http_proxy_set_ssl(ngx_conf_t *cf, n
                 return NGX_ERROR;
             }
 
-        } else {
+        } else if (plcf->upstream.ssl_certificate->value.len) {
             if (ngx_ssl_certificate(cf, plcf->upstream.ssl,
                                     &plcf->upstream.ssl_certificate->value,
                                     &plcf->upstream.ssl_certificate_key->value,
diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c
--- a/src/http/modules/ngx_http_uwsgi_module.c
+++ b/src/http/modules/ngx_http_uwsgi_module.c
@@ -2457,7 +2457,7 @@ ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, n
                 return NGX_ERROR;
             }
 
-        } else {
+        } else if (uwcf->upstream.ssl_certificate->value.len) {
             if (ngx_ssl_certificate(cf, uwcf->upstream.ssl,
                                     &uwcf->upstream.ssl_certificate->value,
                                     &uwcf->upstream.ssl_certificate_key->value,


From P.Pautov at F5.com  Wed May 25 06:15:35 2022
From: P.Pautov at F5.com (Pavel Pautov)
Date: Wed, 25 May 2022 06:15:35 +0000
Subject: SSL contexts reuse across locations
In-Reply-To:  <SJ0PR01MB65100A5C153CF349E6EF3ECBB1D79@SJ0PR01MB6510.prod.exchangelabs.com>
References:  <SJ0PR01MB6510467ED91D5036DAA3B5DFB1D19@SJ0PR01MB6510.prod.exchangelabs.com>
 <YoU7mt+Mmi/2my8k@mdounin.ru>
 <SJ0PR01MB65107F530D6D429E69595DFFB1D39@SJ0PR01MB6510.prod.exchangelabs.com>
 <Yogi3ndHZFB1R6do@mdounin.ru>
 <SJ0PR01MB65100A5C153CF349E6EF3ECBB1D79@SJ0PR01MB6510.prod.exchangelabs.com>
Message-ID:  <SJ0PR01MB651009C0172BEB76DEBB64EDB1D69@SJ0PR01MB6510.prod.exchangelabs.com>

Hi,

Here is another iteration of patch, now it completely avoids creating SSL contexts without SSL proxying.
Also changed some variable names and formatting in attempt to improve readability.
Not sure about moving of merge/reuse logic into ngx_http_proxy_set_ssl(). I assume, it could look like "ngx_http_proxy_set_ssl(cf, conf, prev, reuse_ssl)", which seems to widen its scope too much.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ssl_ctx_reuse_v2.patch
Type: application/octet-stream
Size: 3326 bytes
Desc: ssl_ctx_reuse_v2.patch
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220525/d0c55a7e/attachment.obj>

From mths at google.com  Wed May 25 07:25:01 2022
From: mths at google.com (Mathias Bynens)
Date: Wed, 25 May 2022 09:25:01 +0200
Subject: [PATCH] Correct JavaScript MIME types + extensions per RFC 9239
In-Reply-To: <CADizRgZC4NXLhjuBRyDq5hRCNNf_ctMye+ZqKN0-BWmXdbFF3w@mail.gmail.com>
References: <CADizRgZC4NXLhjuBRyDq5hRCNNf_ctMye+ZqKN0-BWmXdbFF3w@mail.gmail.com>
Message-ID: <CADizRgb828d6E8cfKRcV3xDr4zSh-trG4=18P8jR9W_tHsnkAw@mail.gmail.com>

Is there anything I can do to help move this along? The change is
rather small and clearly follows the latest RFC, so I would expect it
to be uncontroversial.

On Mon, May 9, 2022 at 11:46 AM Mathias Bynens <mths at google.com> wrote:
>
> # HG changeset patch
> # User Mathias Bynens <mathias at qiwi.be>
> # Date 1652088302 -7200
> #      Mon May 09 11:25:02 2022 +0200
> # Branch fix-js-mime-and-extensions
> # Node ID 7944657540c64d92a5bdc3932710c977d984b54d
> # Parent  35afae4b3dffff6718c0cab3ceb16b9de207c20a
> Correct JavaScript MIME types + extensions per RFC 9239
>
> This patch updates the MIME type configuration per RFC 9239.
> https://www.rfc-editor.org/rfc/rfc9239
>
> First, the recommended MIME type is now `text/javascript`:
>
> > The most widely supported media type in use is `text/javascript`; all
> > others are considered historical and obsolete aliases of `text/javascript`.
>
> Second, the `.mjs` extension is now explicitly registered:
>
> > The `.mjs` file extension signals that the file represents a JavaScript
> > module. Execution environments that rely on file extensions to
> > determine how to process inputs parse `.mjs` files using the Module
> > grammar of [ECMA-262].
>
> IANA template: https://www.iana.org/assignments/media-types/text/javascript
>
> diff -r 35afae4b3dff -r 7944657540c6 conf/mime.types
> --- a/conf/mime.types Fri Apr 29 17:38:01 2022 +0400
> +++ b/conf/mime.types Mon May 09 11:25:02 2022 +0200
> @@ -5,7 +5,7 @@
>      text/xml                                         xml;
>      image/gif                                        gif;
>      image/jpeg                                       jpeg jpg;
> -    application/javascript                           js;
> +    text/javascript                                  js mjs;
>      application/atom+xml                             atom;
>      application/rss+xml                              rss;
>
> diff -r 35afae4b3dff -r 7944657540c6 src/http/modules/ngx_http_charset_filter_module.c
> --- a/src/http/modules/ngx_http_charset_filter_module.c Fri Apr 29 17:38:01 2022 +0400
> +++ b/src/http/modules/ngx_http_charset_filter_module.c Mon May 09 11:25:02 2022 +0200
> @@ -128,7 +128,7 @@
>      ngx_string("text/xml"),
>      ngx_string("text/plain"),
>      ngx_string("text/vnd.wap.wml"),
> -    ngx_string("application/javascript"),
> +    ngx_string("text/javascript"),
>      ngx_string("application/rss+xml"),
>      ngx_null_string
>  };


From arut at nginx.com  Thu May 26 09:33:57 2022
From: arut at nginx.com (Roman Arutyunyan)
Date: Thu, 26 May 2022 13:33:57 +0400
Subject: [PATCH] Stream: don't flush empty buffers created for read
 errors.
In-Reply-To: <CO6PR01MB7482629728C2B522E90A6AECC0D79@CO6PR01MB7482.prod.exchangelabs.com>
References: <CO6PR01MB7482629728C2B522E90A6AECC0D79@CO6PR01MB7482.prod.exchangelabs.com>
Message-ID: <20220526093357.s2v5kgvlzunjpm5o@N00W24XTQX>

On Tue, May 24, 2022 at 04:37:11PM +0000, Aleksei Bavshin via nginx-devel wrote:
> In all honesty, the main motivation for the patch is to address a regression in UDP healthchecks caused by the behavior described below. It's already covered by tests, but the corresponding cases were disabled by default.
> 
> I considered
>     cl->buf->flush = !src->read->error;
> or even
>     cl->buf->flush = !src->read->eof; 
> but neither of those are exactly equivalent to the code below (with the latter having significantly different behavior), and IMO it would look less obvious. 
> 
> # HG changeset patch
> # User Aleksei Bavshin <a.bavshin at f5.com>
> # Date 1653330584 25200
> #      Mon May 23 11:29:44 2022 -0700
> # Branch se
> # Node ID 5a98e9cb437f7719afa2bde62de68e174fd8e03e
> # Parent  8902674cc7fe759cada415c10340f31ae4a90fba
> Stream: don't flush empty buffers created for read errors.
> 
> When we generate the last_buf buffer for an UDP upstream recv error, it does
> not contain any data from the wire. ngx_stream_write_filter attempts to forward
> it anyways, which is incorrect (e.g., UDP upstream ECONNREFUSED will be
> translated to an empty packet).
> 
> Reproduction:
> 
> stream {
>     upstream unreachable {
>         server     127.0.0.1:8880;
>     }
>     server {
>         listen     127.0.0.1:8998 udp;
>         proxy_pass unreachable;
>     }
> }
> 
> 1 0.000000000    127.0.0.1 → 127.0.0.1    UDP 47 45588 → 8998 Len=5
> 2 0.000166300    127.0.0.1 → 127.0.0.1    UDP 47 51149 → 8880 Len=5
> 3 0.000172600    127.0.0.1 → 127.0.0.1    ICMP 75 Destination unreachable (Port
> unreachable)
> 4 0.000202400    127.0.0.1 → 127.0.0.1    UDP 42 8998 → 45588 Len=0
> 
> Fixes d127837c714f.
> 
> diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c
> --- a/src/stream/ngx_stream_proxy_module.c
> +++ b/src/stream/ngx_stream_proxy_module.c
> @@ -1676,7 +1676,7 @@ ngx_stream_proxy_process(ngx_stream_sess
>      ssize_t                       n;
>      ngx_buf_t                    *b;
>      ngx_int_t                     rc;
> -    ngx_uint_t                    flags, *packets;
> +    ngx_uint_t                    flags, flush, *packets;
>      ngx_msec_t                    delay;
>      ngx_chain_t                  *cl, **ll, **out, **busy;
>      ngx_connection_t             *c, *pc, *src, *dst;
> @@ -1803,9 +1803,12 @@ ngx_stream_proxy_process(ngx_stream_sess
>                  break;
>              }
>  
> +            flush = 1;
> +
>              if (n == NGX_ERROR) {
>                  src->read->eof = 1;
>                  n = 0;
> +                flush = 0;
>              }
>  
>              if (n >= 0) {
> @@ -1846,7 +1849,7 @@ ngx_stream_proxy_process(ngx_stream_sess
>  
>                  cl->buf->temporary = (n ? 1 : 0);
>                  cl->buf->last_buf = src->read->eof;
> -                cl->buf->flush = 1;
> +                cl->buf->flush = flush;
>  
>                  (*packets)++;
>                  *received += n;
> _______________________________________________
> nginx-devel mailing list -- nginx-devel at nginx.org
> To unsubscribe send an email to nginx-devel-leave at nginx.org

Looks good


From arut at nginx.com  Thu May 26 11:45:55 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Thu, 26 May 2022 15:45:55 +0400
Subject: [PATCH] HTTP/3: require that field section base index is not negative
Message-ID: <9feb0196a87189b57375.1653565555@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1653564901 -14400
#      Thu May 26 15:35:01 2022 +0400
# Branch quic
# Node ID 9feb0196a87189b573757cc00a15ff0acf4d5f8c
# Parent  c2f5d79cde64457f1fa7344c56a5248a677a7e46
HTTP/3: require that field section base index is not negative.

RFC 9204 explicitly requires that.

diff --git a/src/http/v3/ngx_http_v3_parse.c b/src/http/v3/ngx_http_v3_parse.c
--- a/src/http/v3/ngx_http_v3_parse.c
+++ b/src/http/v3/ngx_http_v3_parse.c
@@ -474,7 +474,12 @@ done:
     }
 
     if (st->sign) {
+        if (st->insert_count <= st->delta_base) {
+            return NGX_HTTP_V3_ERR_DECOMPRESSION_FAILED;
+        }
+
         st->base = st->insert_count - st->delta_base - 1;
+
     } else {
         st->base = st->insert_count + st->delta_base;
     }


From pluknet at nginx.com  Thu May 26 13:04:21 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Thu, 26 May 2022 17:04:21 +0400
Subject: [PATCH] HTTP/3: require that field section base index is not
 negative
In-Reply-To: <9feb0196a87189b57375.1653565555@arut-laptop>
References: <9feb0196a87189b57375.1653565555@arut-laptop>
Message-ID: <AA8308F7-4C0E-4D2D-B5AF-9845A92DC118@nginx.com>


> On 26 May 2022, at 15:45, Roman Arutyunyan <arut at nginx.com> wrote:
> 
> # HG changeset patch
> # User Roman Arutyunyan <arut at nginx.com>
> # Date 1653564901 -14400
> #      Thu May 26 15:35:01 2022 +0400
> # Branch quic
> # Node ID 9feb0196a87189b573757cc00a15ff0acf4d5f8c
> # Parent  c2f5d79cde64457f1fa7344c56a5248a677a7e46
> HTTP/3: require that field section base index is not negative.
> 
> RFC 9204 explicitly requires that.
> 
> diff --git a/src/http/v3/ngx_http_v3_parse.c b/src/http/v3/ngx_http_v3_parse.c
> --- a/src/http/v3/ngx_http_v3_parse.c
> +++ b/src/http/v3/ngx_http_v3_parse.c
> @@ -474,7 +474,12 @@ done:
>     }
> 
>     if (st->sign) {
> +        if (st->insert_count <= st->delta_base) {
> +            return NGX_HTTP_V3_ERR_DECOMPRESSION_FAILED;
> +        }
> +
>         st->base = st->insert_count - st->delta_base - 1;
> +
>     } else {
>         st->base = st->insert_count + st->delta_base;
>     }
> 

I'd put something in logs to emphasize bad Base.
Otherwise looks good.

-- 
Sergey Kandaurov


From A.Bavshin at F5.com  Thu May 26 15:50:22 2022
From: A.Bavshin at F5.com (Aleksei Bavshin)
Date: Thu, 26 May 2022 15:50:22 +0000
Subject: [PATCH] Resolver: make TCP write timer event cancelable.
Message-ID:  <CO6PR01MB748214DB5F0F6740D093E4BCC0D99@CO6PR01MB7482.prod.exchangelabs.com>

# HG changeset patch
# User Aleksei Bavshin <a.bavshin at f5.com>
# Date 1653579686 25200
#      Thu May 26 08:41:26 2022 -0700
# Branch se
# Node ID afb3646a08b6ebcf46fdbd2e7bf97cdcccca774b
# Parent  5a98e9cb437f7719afa2bde62de68e174fd8e03e
Resolver: make TCP write timer event cancelable.

In certain circumstances, this timer could become the only remaining
non-cancelable timer in a worker, effectively delaying the worker shutdown.

We're keeping the resolver alive to avoid disrupting any leftover tasks in the
worker that may require up-to-date upstream peer data, but it is unnecessary
in absence of other connections. And any useful workload would have its own
non-cancelable timers, so it is safe to allow canceling this one.

The scenario in question is a periodic resolver task with a period less than
the TCP write timeout and responses large enough to warrant fallback to a TCP
connection - something that could be reproduced with a dynamic upstream
implementation. With each event loop wakeup, we either see a previously
set write timer instance or schedule a new one.

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -1624,6 +1624,7 @@ ngx_resolver_send_tcp_query(ngx_resolver
         }
 
         rec->tcp->data = rec;
+        rec->tcp->write->cancelable = 1;
         rec->tcp->write->handler = ngx_resolver_tcp_write;
         rec->tcp->read->handler = ngx_resolver_tcp_read;
         rec->tcp->read->resolver = 1;


From abeltran at linux.microsoft.com  Thu May 26 21:42:14 2022
From: abeltran at linux.microsoft.com (Andres Beltran)
Date: Thu, 26 May 2022 14:42:14 -0700
Subject: [PATCH] fix -Wsign-conversion warning with gcc 8.2
Message-ID: <807f506c-60bf-fc48-d3fe-3d26f9518987@linux.microsoft.com>

Hello,

Gently ping on this patch. Any comments?

Best,
Andres Beltran

On 3/14/2022 4:42 PM, Sinan Kaya wrote:
> Subject:
 > [PATCH] fix -Wsign-conversion warning with gcc 8.2
 > From:
 > Sinan Kaya <okaya at kernel.org>
 > Date:
 > 3/14/2022, 4:42 PM
 >
 > To:
 > nginx-devel at nginx.org
 >
 >
 >
 > # HG changeset patch
 > # User Sinan Kaya <sinan.kaya at microsoft.com>
 > # Date 1647289518 14400
 > #      Mon Mar 14 16:25:18 2022 -0400
 > # Node ID f22520b612969dbfa17205129510927519370000
 > # Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
 > fix -Wsign-conversion warning with gcc 8.2
 >
 > Getting compiler warning with -Wsign-conversion.
 >
 > /usr/include/nginx/core/ngx_crc32.h:31:47: warning: conversion to
 > 'uint32_t' {aka 'unsigned int'} from 'int' may change the sign of the
 > result [-Wsign-conversion]
 >    31 |         crc = ngx_crc32_table_short[(crc ^ (c >> 4)) & 0xf] ^
 > (crc >> 4);
 >
 > diff -r a736a7a613ea -r f22520b61296 src/core/ngx_crc32.h
 > --- a/src/core/ngx_crc32.h    Tue Feb 08 17:35:27 2022 +0300
 > +++ b/src/core/ngx_crc32.h    Mon Mar 14 16:25:18 2022 -0400
 > @@ -28,7 +28,8 @@
 >      while (len--) {
 >          c = *p++;
 >          crc = ngx_crc32_table_short[(crc ^ (c & 0xf)) & 0xf] ^ (crc >>
 > 4);
 > -        crc = ngx_crc32_table_short[(crc ^ (c >> 4)) & 0xf] ^ (crc 
 >> 4);
 > +        crc = ngx_crc32_table_short[(crc ^ (u_char)(c >> 4)) & 0xf];
 > +        crc = crc ^ (crc >> 4);
 >      }
 >
 >      return crc ^ 0xffffffff;
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220526/cc9647b8/attachment.htm>

From mdounin at mdounin.ru  Thu May 26 22:31:21 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 27 May 2022 01:31:21 +0300
Subject: [PATCH] Resolver: make TCP write timer event cancelable.
In-Reply-To: <CO6PR01MB748214DB5F0F6740D093E4BCC0D99@CO6PR01MB7482.prod.exchangelabs.com>
References: <CO6PR01MB748214DB5F0F6740D093E4BCC0D99@CO6PR01MB7482.prod.exchangelabs.com>
Message-ID: <Yo//uXBdGlRRmklX@mdounin.ru>

Hello!

On Thu, May 26, 2022 at 03:50:22PM +0000, Aleksei Bavshin via nginx-devel wrote:

> # HG changeset patch
> # User Aleksei Bavshin <a.bavshin at f5.com>
> # Date 1653579686 25200
> #      Thu May 26 08:41:26 2022 -0700
> # Branch se

Looks like a wrong branch to me.

> # Node ID afb3646a08b6ebcf46fdbd2e7bf97cdcccca774b
> # Parent  5a98e9cb437f7719afa2bde62de68e174fd8e03e
> Resolver: make TCP write timer event cancelable.
> 
> In certain circumstances, this timer could become the only remaining
> non-cancelable timer in a worker, effectively delaying the worker shutdown.
> 
> We're keeping the resolver alive to avoid disrupting any leftover tasks in the
> worker that may require up-to-date upstream peer data, but it is unnecessary
> in absence of other connections. And any useful workload would have its own
> non-cancelable timers, so it is safe to allow canceling this one.

This doesn't sound correct to me: when nginx is doing upstream 
dynamic name-to-address resolution during request processing (see 
ngx_http_upstream_init_request()), the request has no "own 
non-cancelable timers".

Still, there should be resolve task timer, and canceling the tcp 
timer should be safe, similarly to canceling the resolver resend 
timer, see 70e65bf8dfd7.  And, BTW, it might be a good idea to 
explicitly mention this changeset in the commit log.

> 
> The scenario in question is a periodic resolver task with a period less than
> the TCP write timeout and responses large enough to warrant fallback to a TCP
> connection - something that could be reproduced with a dynamic upstream
> implementation. With each event loop wakeup, we either see a previously
> set write timer instance or schedule a new one.
> 
> diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
> --- a/src/core/ngx_resolver.c
> +++ b/src/core/ngx_resolver.c
> @@ -1624,6 +1624,7 @@ ngx_resolver_send_tcp_query(ngx_resolver
>          }
>  
>          rec->tcp->data = rec;
> +        rec->tcp->write->cancelable = 1;
>          rec->tcp->write->handler = ngx_resolver_tcp_write;
>          rec->tcp->read->handler = ngx_resolver_tcp_read;
>          rec->tcp->read->resolver = 1;

>From style point of view, it would be a good idea to set flags 
after the handler.

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Thu May 26 23:18:13 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 27 May 2022 02:18:13 +0300
Subject: [PATCH] Correct JavaScript MIME types + extensions per RFC 9239
In-Reply-To: <CADizRgYqFVMAtnwBiqDdcnU-w1U67JVVDnSkcjne4Ck5ZABnHQ@mail.gmail.com>
References: <CADizRgZC4NXLhjuBRyDq5hRCNNf_ctMye+ZqKN0-BWmXdbFF3w@mail.gmail.com>
 <YnlczZbl0TtbIx8Y@mdounin.ru>
 <CADizRgYqFVMAtnwBiqDdcnU-w1U67JVVDnSkcjne4Ck5ZABnHQ@mail.gmail.com>
Message-ID: <YpAKtboxhn7As733@mdounin.ru>

Hello!

On Mon, May 23, 2022 at 11:32:08AM +0200, Mathias Bynens via nginx-devel wrote:

> Thank you, Maxim. Is there anything else I can do to help move this
> along? The change is rather small and clearly follows the latest RFC,
> so I would expect it to be uncontroversial.

On the other hand, the RFC itself is rather controversial, as it 
changes preferred MIME type for javascript for no apparent reason, 
and it adds the "mjs" extension, which is known to be mostly used 
in NodeJS-specific environments and was originally introduced with 
quite questionable claim that "on the web file extensions doesn't 
matter".

I don't think we are in position to add the mjs extension, it does 
not seem to qualify as something commonly used on the web now.  
The ticket referenced in the previous message can be used to track 
the request.

As for the MIME type change from "application/javascript" to 
"text/javascript", this is something we'll consider as the outcome 
of the change suggested by the RFC becomes more clear.  For now, 
it looks like this might need more compatibility shims (such 
as adding both "text/javascript" and "application/javascript" to 
the default list of types handled by the charset module) and more 
changes (such as change of the MIME type used by autoindex with 
JSONP format).

Hope this helps.

-- 
Maxim Dounin
http://mdounin.ru/


From A.Bavshin at F5.com  Thu May 26 23:39:36 2022
From: A.Bavshin at F5.com (Aleksei Bavshin)
Date: Thu, 26 May 2022 23:39:36 +0000
Subject: [PATCH] Resolver: make TCP write timer event cancelable.
In-Reply-To: <Yo//uXBdGlRRmklX@mdounin.ru>
References:  <CO6PR01MB748214DB5F0F6740D093E4BCC0D99@CO6PR01MB7482.prod.exchangelabs.com>
 <Yo//uXBdGlRRmklX@mdounin.ru>
Message-ID:  <CO6PR01MB74826457EE3031B7223747ABC0D99@CO6PR01MB7482.prod.exchangelabs.com>

> -----Original Message-----
> From: Maxim Dounin <mdounin at mdounin.ru>
> Sent: Thursday, May 26, 2022 3:31 PM
> To: Aleksei Bavshin via nginx-devel <nginx-devel at nginx.org>
> Subject: Re: [PATCH] Resolver: make TCP write timer event cancelable.
> 
> EXTERNAL MAIL: nginx-devel-bounces at nginx.org
> 
> Hello!
> 
> On Thu, May 26, 2022 at 03:50:22PM +0000, Aleksei Bavshin via nginx-devel
> wrote:
> 
> > # HG changeset patch
> > # User Aleksei Bavshin <a.bavshin at f5.com>
> > # Date 1653579686 25200
> > #      Thu May 26 08:41:26 2022 -0700
> > # Branch se
> 
> Looks like a wrong branch to me.

Yep, I was already informed about that. As the main reproducer for the problem is the upstream code from 'se', I had to use this branch for verification and forgot to rebase. Sorry ☹
The change is in the oss portion of the code though and applies cleanly. I'll make sure to rebase any further patches on the right branch.

> 
> > # Node ID afb3646a08b6ebcf46fdbd2e7bf97cdcccca774b
> > # Parent  5a98e9cb437f7719afa2bde62de68e174fd8e03e
> > Resolver: make TCP write timer event cancelable.
> >
> > In certain circumstances, this timer could become the only remaining
> > non-cancelable timer in a worker, effectively delaying the worker
> shutdown.
> >
> > We're keeping the resolver alive to avoid disrupting any leftover tasks in
> the
> > worker that may require up-to-date upstream peer data, but it is
> unnecessary
> > in absence of other connections. And any useful workload would have its
> own
> > non-cancelable timers, so it is safe to allow canceling this one.
> 
> This doesn't sound correct to me: when nginx is doing upstream
> dynamic name-to-address resolution during request processing (see
> ngx_http_upstream_init_request()), the request has no "own
> non-cancelable timers".

Sorry, I've likely used a wrong terminology here.

An upstream server with 'resolve' parameter monitors the address changes independently from http request processing.  There's an effort made to keep these resolver tasks cancelable (including 70e65bf8dfd7 you mentioned), while ensuring that the resolver connections are closed last - ngx_shutdown_timer_handler() ignores connections with resolver flag.
As I understand the intention, we wanted to keep updating the upstream servers until the last http (stream, mail, etc) connection is closed and only then allow the worker to shut down. The non-cancelable timer in resolver interferes with this last part.

I'll rewrite the description to make that clearer.

> Still, there should be resolve task timer, and canceling the tcp
> timer should be safe, similarly to canceling the resolver resend
> timer, see 70e65bf8dfd7.  And, BTW, it might be a good idea to
> explicitly mention this changeset in the commit log.
> 
> >
> > The scenario in question is a periodic resolver task with a period less than
> > the TCP write timeout and responses large enough to warrant fallback to a
> TCP
> > connection - something that could be reproduced with a dynamic upstream
> > implementation. With each event loop wakeup, we either see a previously
> > set write timer instance or schedule a new one.
> >
> > diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
> > --- a/src/core/ngx_resolver.c
> > +++ b/src/core/ngx_resolver.c
> > @@ -1624,6 +1624,7 @@ ngx_resolver_send_tcp_query(ngx_resolver
> >          }
> >
> >          rec->tcp->data = rec;
> > +        rec->tcp->write->cancelable = 1;
> >          rec->tcp->write->handler = ngx_resolver_tcp_write;
> >          rec->tcp->read->handler = ngx_resolver_tcp_read;
> >          rec->tcp->read->resolver = 1;
> 
> From style point of view, it would be a good idea to set flags
> after the handler.

Ok, will do in v2.

> 
> --
> Maxim Dounin
> https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmdo
> unin.ru%2F&amp;data=05%7C01%7CA.Bavshin%40F5.com%7C3c7d773d6815
> 4de9e63008da3f6790cd%7Cdd3dfd2f6a3b40d19be0bf8327d81c50%7C0%7C0
> %7C637892011303441223%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjA
> wMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C
> %7C&amp;sdata=3G6yY5Ryx6a%2BXsQowcL0V%2F%2F217ByGKWKHAiVgalH
> up0%3D&amp;reserved=0
> _______________________________________________
> nginx-devel mailing list -- nginx-devel at nginx.org
> To unsubscribe send an email to nginx-devel-leave at nginx.org


From mdounin at mdounin.ru  Fri May 27 02:42:05 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 27 May 2022 05:42:05 +0300
Subject: [PATCH] fix -Wsign-conversion warning with gcc 8.2
In-Reply-To: <807f506c-60bf-fc48-d3fe-3d26f9518987@linux.microsoft.com>
References: <807f506c-60bf-fc48-d3fe-3d26f9518987@linux.microsoft.com>
Message-ID: <YpA6fSYodp+xm4bH@mdounin.ru>

Hello!

On Thu, May 26, 2022 at 02:42:14PM -0700, Andres Beltran wrote:

> Gently ping on this patch. Any comments?
> 
> On 3/14/2022 4:42 PM, Sinan Kaya wrote:
> 
>  > # HG changeset patch
>  > # User Sinan Kaya <sinan.kaya at microsoft.com>
>  > # Date 1647289518 14400
>  > #      Mon Mar 14 16:25:18 2022 -0400
>  > # Node ID f22520b612969dbfa17205129510927519370000
>  > # Parent  a736a7a613ea6e182ff86fbadcb98bb0f8891c0b
>  > fix -Wsign-conversion warning with gcc 8.2
>  >
>  > Getting compiler warning with -Wsign-conversion.
>  >
>  > /usr/include/nginx/core/ngx_crc32.h:31:47: warning: conversion to
>  > 'uint32_t' {aka 'unsigned int'} from 'int' may change the sign of the
>  > result [-Wsign-conversion]
>  >    31 |         crc = ngx_crc32_table_short[(crc ^ (c >> 4)) & 0xf] ^
>  > (crc >> 4);
>  >
>  > diff -r a736a7a613ea -r f22520b61296 src/core/ngx_crc32.h
>  > --- a/src/core/ngx_crc32.h    Tue Feb 08 17:35:27 2022 +0300
>  > +++ b/src/core/ngx_crc32.h    Mon Mar 14 16:25:18 2022 -0400
>  > @@ -28,7 +28,8 @@
>  >      while (len--) {
>  >          c = *p++;
>  >          crc = ngx_crc32_table_short[(crc ^ (c & 0xf)) & 0xf] ^ (crc >>
>  > 4);
>  > -        crc = ngx_crc32_table_short[(crc ^ (c >> 4)) & 0xf] ^ (crc 
>  >> 4);
>  > +        crc = ngx_crc32_table_short[(crc ^ (u_char)(c >> 4)) & 0xf];
>  > +        crc = crc ^ (crc >> 4);
>  >      }
>  >
>  >      return crc ^ 0xffffffff;

Trying to compile nginx with -Wsign-conversion produces a lot more 
than this specific (mis)warning.  For more or less obvious reasons 
there are no plans to silence them: this is going to be a large 
work which is highly unlikely to catch any real bugs, yet will 
result in a lot of garbage in the code.

If for some reason you think this particular warning should be 
silenced, you may want to elaborate more on your reasons.

Hope this helps.

-- 
Maxim Dounin
http://mdounin.ru/


From P.Pautov at F5.com  Fri May 27 04:26:20 2022
From: P.Pautov at F5.com (Pavel Pautov)
Date: Fri, 27 May 2022 04:26:20 +0000
Subject: [PATCH] Upstream: handling of certificates specified as an empty
 string
In-Reply-To: <3bb1adbb74dfcd372f73.1653422757@enoparse.local>
References: <3bb1adbb74dfcd372f73.1653422757@enoparse.local>
Message-ID:  <MW4PR01MB6499737D56ED6863B0A9D3E5B1D89@MW4PR01MB6499.prod.exchangelabs.com>

Hi,

Shall we restore original behavior for stream proxy as well?


From mths at google.com  Fri May 27 07:23:47 2022
From: mths at google.com (Mathias Bynens)
Date: Fri, 27 May 2022 09:23:47 +0200
Subject: [PATCH] Correct JavaScript MIME types + extensions per RFC 9239
In-Reply-To: <YpAKtboxhn7As733@mdounin.ru>
References: <CADizRgZC4NXLhjuBRyDq5hRCNNf_ctMye+ZqKN0-BWmXdbFF3w@mail.gmail.com>
 <YnlczZbl0TtbIx8Y@mdounin.ru> <CADizRgYqFVMAtnwBiqDdcnU-w1U67JVVDnSkcjne4Ck5ZABnHQ@mail.gmail.com>
 <YpAKtboxhn7As733@mdounin.ru>
Message-ID: <CADizRgYm70VNiSKVvwX7cP0zOt4BJsyFPqdjM+_gV_JDeopbCg@mail.gmail.com>

On Fri, May 27, 2022 at 1:55 AM Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Mon, May 23, 2022 at 11:32:08AM +0200, Mathias Bynens via nginx-devel
> wrote:
>
> > Thank you, Maxim. Is there anything else I can do to help move this
> > along? The change is rather small and clearly follows the latest RFC,
> > so I would expect it to be uncontroversial.
>
> On the other hand, the RFC itself is rather controversial, as it
> changes preferred MIME type for javascript for no apparent reason,


Are you suggesting that RFCs can be published "for no apparent reason"? The
publication of this RFC ends a year-long cross-standard disagreement
between WHATWG/HTML & IETF/IANA. The controversy is finally over!

I don’t understand why nginx would willingly refuse to follow a now
universally-agreed up IETF standard. This patch has already landed in
apache/httpd, by the way.

and it adds the "mjs" extension, which is known to be mostly used
> in NodeJS-specific environments and was originally introduced with
> quite questionable claim that "on the web file extensions doesn't
> matter".
>

What do you think is questionable? It’s a fact that browsers do not care
about file extensions. (They care about MIME types and Content-Type HTTP
header.)

For example, https://mathiasbynens.be/ loads a JavaScript file at /js — no
extension. I could have named that file js.html or js.css or anything I
wanted, as long as I serve it with the proper Content-Type header, browsers
happily accept it.

Of course, file extensions are still useful for servers to send the proper
Content-Type header.


> I don't think we are in position to add the mjs extension, it does
> not seem to qualify as something commonly used on the web now.


What’s the harm in adding the extension mapping? If it’s not used, then
there’s no harm done.

The ticket referenced in the previous message can be used to track
> the request.
>
> As for the MIME type change from "application/javascript" to
> "text/javascript", this is something we'll consider as the outcome
> of the change suggested by the RFC becomes more clear.


What is unclear?


> For now,
> it looks like this might need more compatibility shims (such
> as adding both "text/javascript" and "application/javascript" to
> the default list of types handled by the charset module) and more
> changes (such as change of the MIME type used by autoindex with
> JSONP format).
>
> Hope this helps.
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx-devel mailing list -- nginx-devel at nginx.org
> To unsubscribe send an email to nginx-devel-leave at nginx.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220527/8bccec6f/attachment.htm>

From mdounin at mdounin.ru  Fri May 27 16:00:13 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 27 May 2022 19:00:13 +0300
Subject: [PATCH] Correct JavaScript MIME types + extensions per RFC 9239
In-Reply-To: <CADizRgYm70VNiSKVvwX7cP0zOt4BJsyFPqdjM+_gV_JDeopbCg@mail.gmail.com>
References: <CADizRgZC4NXLhjuBRyDq5hRCNNf_ctMye+ZqKN0-BWmXdbFF3w@mail.gmail.com>
 <YnlczZbl0TtbIx8Y@mdounin.ru>
 <CADizRgYqFVMAtnwBiqDdcnU-w1U67JVVDnSkcjne4Ck5ZABnHQ@mail.gmail.com>
 <YpAKtboxhn7As733@mdounin.ru>
 <CADizRgYm70VNiSKVvwX7cP0zOt4BJsyFPqdjM+_gV_JDeopbCg@mail.gmail.com>
Message-ID: <YpD1jdO4FIC9Vkib@mdounin.ru>

Hello!

On Fri, May 27, 2022 at 09:23:47AM +0200, Mathias Bynens via nginx-devel wrote:

> On Fri, May 27, 2022 at 1:55 AM Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> > On Mon, May 23, 2022 at 11:32:08AM +0200, Mathias Bynens via nginx-devel
> > wrote:
> >
> > > Thank you, Maxim. Is there anything else I can do to help move this
> > > along? The change is rather small and clearly follows the latest RFC,
> > > so I would expect it to be uncontroversial.
> >
> > On the other hand, the RFC itself is rather controversial, as it
> > changes preferred MIME type for javascript for no apparent reason,
> 
> 
> Are you suggesting that RFCs can be published "for no apparent reason"? The
> publication of this RFC ends a year-long cross-standard disagreement
> between WHATWG/HTML & IETF/IANA. The controversy is finally over!
> 
> I don’t understand why nginx would willingly refuse to follow a now
> universally-agreed up IETF standard. This patch has already landed in
> apache/httpd, by the way.

For sure there are reasons.  For example, RFC 2324 was also 
published for a reason.  The question is: how it aligns with the 
real world usage, and how it should be implemented, if at all.

In this particular case, the RFC itself talks about "text/*" being 
an incorrect, and only used because "the industry widely adopted 
text/* anyway".  On the other hand, httparchive.org data suggests 
that "application/javascript" is more popular now than 
"text/javascript":

Query:

SELECT
  mimeType,
  COUNT(distinct pageid) total_pages,
  COUNT(0) total_requests
FROM
  `httparchive.summary_requests.2022_03_01_desktop`
GROUP BY
  mimeType
ORDER BY
  total_requests DESC

Results:

Row	mimeType	total_pages	total_requests	
1	application/javascript 5346744 95187099
2	image/jpeg 4608713 78732193
3	image/png 5129850 58321805
4	text/css 5546313 56797104
5	text/html 5718485 41375375
6	text/javascript 4670822 36042510
...
11	application/json 3641048 16939555
12	application/x-javascript 2577400 14903323
...

>From the rationale provided in the RFC it is not clear why 
"application/javascript", which is more popular, was dropped in 
favor of the "text/javascript", which is incorrect and less 
popular.

> > and it adds the "mjs" extension, which is known to be mostly used
> > in NodeJS-specific environments and was originally introduced with
> > quite questionable claim that "on the web file extensions doesn't
> > matter".
> >
> 
> What do you think is questionable? It’s a fact that browsers do not care
> about file extensions. (They care about MIME types and Content-Type HTTP
> header.)
> 
> For example, https://mathiasbynens.be/ loads a JavaScript file at /js — no
> extension. I could have named that file js.html or js.css or anything I
> wanted, as long as I serve it with the proper Content-Type header, browsers
> happily accept it.
> 
> Of course, file extensions are still useful for servers to send the proper
> Content-Type header.

Your patch clearly shows that the file extension does matter on 
the web: servers use them to provide proper content type.  As 
such, adding arbitrary additional file extensions is going to hurt 
the web, since servers won't be able to provide appropriate 
content type.

> > I don't think we are in position to add the mjs extension, it does
> > not seem to qualify as something commonly used on the web now.
> 
> What’s the harm in adding the extension mapping? If it’s not used, then
> there’s no harm done.

Additional types and extensions imply additional resources on all 
MIME type lookups.  As such, nginx doesn't try to maintain all the 
mime types and extensions, but rather tries to supply types 
generally needed for a web server.

> > The ticket referenced in the previous message can be used to track
> > the request.
> >
> > As for the MIME type change from "application/javascript" to
> > "text/javascript", this is something we'll consider as the outcome
> > of the change suggested by the RFC becomes more clear.
> 
> What is unclear?

I believe I've provided a few things not considered by your patch 
in the following sentence, including the one which is a 
compatibility problem and can result in non-working sites due to 
an unexpected MIME type change, resulting in incorrect charset 
information.

-- 
Maxim Dounin
http://mdounin.ru/


From dimitri_emich at protonmail.com  Sat May 28 12:38:45 2022
From: dimitri_emich at protonmail.com (Dimitri)
Date: Sat, 28 May 2022 12:38:45 +0000
Subject: Wrong documentation about fastcgi_param...
Message-ID: <eDr_C3L051Ki1MYrY9iLXir6J-PUmJ_-2xHGT8OhH3r2WkXvvZkCgP7LOv2Z5oOzgoBmt9s14L5PIB1dw4WMav1sUUIRD18za7m7aB_m_Vg=@protonmail.com>

Hi,

for a few hours yesterday i've had the Problem "FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream".

I use PHP-FPM with chroot and know about the configuration-problems with wrong "fastcgi_param SCRIPT_FILENAME ...", which you can find on the internet.

But my settings seems right:

PHP:

> chroot = /test/website/example.com/Files

Nginx example.conf:

> root /test/website/example.com/Files;

> ...

> location ~ \.php$
> {
> try_files $uri =404;
> fastcgi_param SCRIPT_FILENAME $fastcgi_script_name;
> fastcgi_pass unix:/test/php/8.1.5/sockets/website.sock;}

But i get the Error "Primary script unknown...".
And in Browser i don't get the index.php, but only the text "File not found"

php access.log:

> - - 27/May/2022:14:05:02 +0000 "GET /index.php" 404

So i searched the internet, but without success...

Then i removed the SCRIPT_FILENAME-line in the example.conf nginx configuration file and
set the main SCRIPT_FILENAME-setting in my fastcgi.conf (which is included by nginx.conf) to
just "$fastcgi_script_name" (previously $request_filename) and all works as expectet.

The [Nginx-documentation](http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html#fastcgi_param) say:
These directives are inherited from the previous configuration level if and only if there are no fastcgi_param directives defined on the current level.

But these directive WAS SET in the example.conf, so why nginx does not behave as stated in the docs?
Why nginx uses the global SCRIPT_FILENAME-setting, although these setting is on the "current level" in example.conf?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220528/5a298533/attachment.htm>

From juliyvchirkov at gmail.com  Sat May 28 13:25:58 2022
From: juliyvchirkov at gmail.com (Juliy V. Chirkov)
Date: Sat, 28 May 2022 09:25:58 -0400
Subject: Server config analysis engine in Nginx Amplify fails to recognize
 regex pattern
Message-ID: <CAJJuwsyuszpQeCtWL8JghFX-AhiG9cFdp3nHx_MF79g4=QSPLw@mail.gmail.com>

Hi guys

Seems server config analysis engine in Nginx Amplify fails to recognize
common regex pattern

Server config analysis reports

——————
Regex location has no regex pattern

Regex location has a valid modifier, but does not have a regex pattern.
Performance-wise it is more efficient to configure exact or prefix matching
for locations that do not require regex matching. It is also less prone to
errors. Please refer to the documentation describing location directive to
learn more.

Check the following files:
/etc/nginx/conf.d/location.common.conf, line 11
-------------

While line 11 of that file indeed does contain regex pattern: location ~
"/\." {

I.e. any file or folder which name starts with dot. The goal is to protect
dot files like .env from public access. The complete rule follows

location ~ "/\." {
    return 444;
}

--
Juliy V. Chirkov
https://juliyvchirkov.github.io
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220528/6dd34381/attachment.htm>

From lukas at lihotzki.de  Sat May 28 16:29:32 2022
From: lukas at lihotzki.de (Lukas Lihotzki)
Date: Sat, 28 May 2022 18:29:32 +0200
Subject: [PATCH] Add ipv4=off option in resolver like ipv6=off (ticket
 #1330)
In-Reply-To: <20220223040934.GD18027@lo0.su>
References: <e9f06dc2d6a4a1aa61c1.1642618064@lukas-laptop>
 <20220216123055.GA18027@lo0.su> <20220223040934.GD18027@lo0.su>
Message-ID: <dfa23ec4-d893-5488-37a2-47a9eeb365e3@lihotzki.de>

This patch works great (both with and without Addon). Can it be upstreamed?


On 23.02.22 05:09, Ruslan Ermilov wrote:
> On Wed, Feb 16, 2022 at 03:30:55PM +0300, Ruslan Ermilov wrote:
>> Hi Lukas,
>>
>> On Wed, Jan 19, 2022 at 07:47:44PM +0100, Lukas Lihotzki via nginx-devel wrote:
>>> # HG changeset patch
>>> # User Lukas Lihotzki <lukas at lihotzki.de>
>>> # Date 1642618053 -3600
>>> #      Wed Jan 19 19:47:33 2022 +0100
>>> # Node ID e9f06dc2d6a4a1aa61c15009b84ceedcaf5983b2
>>> # Parent  aeab41dfd2606dd36cabbf01f1472726e27e8aea
>>> Add ipv4=off option in resolver like ipv6=off (ticket #1330).
>>>
>>> IPv6-only hosts (ticket #1330) and upstreams with IPv6 bind address
>>> (ticket #1535) need to disable resolving to IPv4 addresses.
>>>
>>> Ticket #1330 mentions ipv4=off is the proper fix.
>> There's a number of problems in your patch.  Please try this
>> one instead:
>>
>> # HG changeset patch
>> # User Ruslan Ermilov <ru at nginx.com>
>> # Date 1644873563 -10800
>> #      Tue Feb 15 00:19:23 2022 +0300
>> # Node ID 5d2cb60a78dd32a10a0010ccff39974fd7605867
>> # Parent  1add55d236522616ce34ffaa4dc697a76d3d41a4
>> The "ipv4=" parameter of the "resolver" directive (ticket #2196).
>>
>> When set to "off", only IPv6 addresses will be resolved, and no
>> A queries are ever sent.
>>
>> diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
>> --- a/src/core/ngx_resolver.c
>> +++ b/src/core/ngx_resolver.c
>> @@ -157,6 +157,8 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_
>>       cln->handler = ngx_resolver_cleanup;
>>       cln->data = r;
>>   
>> +    r->ipv4 = 1;
>> +
>>       ngx_rbtree_init(&r->name_rbtree, &r->name_sentinel,
>>                       ngx_resolver_rbtree_insert_value);
>>   
>> @@ -225,6 +227,23 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_
>>           }
>>   
>>   #if (NGX_HAVE_INET6)
>> +        if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) {
>> +
>> +            if (ngx_strcmp(&names[i].data[5], "on") == 0) {
>> +                r->ipv4 = 1;
>> +
>> +            } else if (ngx_strcmp(&names[i].data[5], "off") == 0) {
>> +                r->ipv4 = 0;
>> +
>> +            } else {
>> +                ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
>> +                                   "invalid parameter: %V", &names[i]);
>> +                return NULL;
>> +            }
>> +
>> +            continue;
>> +        }
>> +
>>           if (ngx_strncmp(names[i].data, "ipv6=", 5) == 0) {
>>   
>>               if (ngx_strcmp(&names[i].data[5], "on") == 0) {
> Addon to the patch:
>
> diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
> --- a/src/core/ngx_resolver.c
> +++ b/src/core/ngx_resolver.c
> @@ -229,10 +229,12 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_
>   #if (NGX_HAVE_INET6)
>           if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) {
>   
> -            if (ngx_strcmp(&names[i].data[5], "on") == 0) {
> +            if (ngx_strcasecmp(&names[i].data[5], (u_char *) "on") == 0) {
>                   r->ipv4 = 1;
>   
> -            } else if (ngx_strcmp(&names[i].data[5], "off") == 0) {
> +            } else if (ngx_strcasecmp(&names[i].data[5], (u_char *) "off")
> +                       == 0)
> +            {
>                   r->ipv4 = 0;
>   
>               } else {
> @@ -246,10 +248,12 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_
>   
>           if (ngx_strncmp(names[i].data, "ipv6=", 5) == 0) {
>   
> -            if (ngx_strcmp(&names[i].data[5], "on") == 0) {
> +            if (ngx_strcasecmp(&names[i].data[5], (u_char *) "on") == 0) {
>                   r->ipv6 = 1;
>   
> -            } else if (ngx_strcmp(&names[i].data[5], "off") == 0) {
> +            } else if (ngx_strcasecmp(&names[i].data[5], (u_char *) "off")
> +                       == 0)
> +            {
>                   r->ipv6 = 0;
>   
>               } else {
>
> And a full updated patch:
>
> # HG changeset patch
> # User Ruslan Ermilov <ru at nginx.com>
> # Date 1645589317 -10800
> #      Wed Feb 23 07:08:37 2022 +0300
> # Node ID 1c19779448db2309d607c74e2628ff98f84569ff
> # Parent  1add55d236522616ce34ffaa4dc697a76d3d41a4
> The "ipv4=" parameter of the "resolver" directive (ticket #2196).
>
> When set to "off", only IPv6 addresses will be resolved, and no
> A queries are ever sent.
>
> diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
> --- a/src/core/ngx_resolver.c
> +++ b/src/core/ngx_resolver.c
> @@ -157,6 +157,8 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_
>       cln->handler = ngx_resolver_cleanup;
>       cln->data = r;
>   
> +    r->ipv4 = 1;
> +
>       ngx_rbtree_init(&r->name_rbtree, &r->name_sentinel,
>                       ngx_resolver_rbtree_insert_value);
>   
> @@ -225,12 +227,33 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_
>           }
>   
>   #if (NGX_HAVE_INET6)
> +        if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) {
> +
> +            if (ngx_strcasecmp(&names[i].data[5], (u_char *) "on") == 0) {
> +                r->ipv4 = 1;
> +
> +            } else if (ngx_strcasecmp(&names[i].data[5], (u_char *) "off")
> +                       == 0)
> +            {
> +                r->ipv4 = 0;
> +
> +            } else {
> +                ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
> +                                   "invalid parameter: %V", &names[i]);
> +                return NULL;
> +            }
> +
> +            continue;
> +        }
> +
>           if (ngx_strncmp(names[i].data, "ipv6=", 5) == 0) {
>   
> -            if (ngx_strcmp(&names[i].data[5], "on") == 0) {
> +            if (ngx_strcasecmp(&names[i].data[5], (u_char *) "on") == 0) {
>                   r->ipv6 = 1;
>   
> -            } else if (ngx_strcmp(&names[i].data[5], "off") == 0) {
> +            } else if (ngx_strcasecmp(&names[i].data[5], (u_char *) "off")
> +                       == 0)
> +            {
>                   r->ipv6 = 0;
>   
>               } else {
> @@ -273,6 +296,14 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_
>           }
>       }
>   
> +#if (NGX_HAVE_INET6)
> +    if (r->ipv4 + r->ipv6 == 0) {
> +        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
> +                           "\"ipv4\" and \"ipv6\" cannot both be \"off\"");
> +        return NULL;
> +    }
> +#endif
> +
>       if (n && r->connections.nelts == 0) {
>           ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "no name servers defined");
>           return NULL;
> @@ -836,7 +867,7 @@ ngx_resolve_name_locked(ngx_resolver_t *
>           r->last_connection = 0;
>       }
>   
> -    rn->naddrs = (u_short) -1;
> +    rn->naddrs = r->ipv4 ? (u_short) -1 : 0;
>       rn->tcp = 0;
>   #if (NGX_HAVE_INET6)
>       rn->naddrs6 = r->ipv6 ? (u_short) -1 : 0;
> @@ -1263,7 +1294,7 @@ ngx_resolver_send_query(ngx_resolver_t *
>           rec->log.action = "resolving";
>       }
>   
> -    if (rn->naddrs == (u_short) -1) {
> +    if (rn->query && rn->naddrs == (u_short) -1) {
>           rc = rn->tcp ? ngx_resolver_send_tcp_query(r, rec, rn->query, rn->qlen)
>                        : ngx_resolver_send_udp_query(r, rec, rn->query, rn->qlen);
>   
> @@ -1764,10 +1795,13 @@ ngx_resolver_process_response(ngx_resolv
>                q = ngx_queue_next(q))
>           {
>               rn = ngx_queue_data(q, ngx_resolver_node_t, queue);
> -            qident = (rn->query[0] << 8) + rn->query[1];
> -
> -            if (qident == ident) {
> -                goto dns_error_name;
> +
> +            if (rn->query) {
> +                qident = (rn->query[0] << 8) + rn->query[1];
> +
> +                if (qident == ident) {
> +                    goto dns_error_name;
> +                }
>               }
>   
>   #if (NGX_HAVE_INET6)
> @@ -3644,7 +3678,7 @@ ngx_resolver_create_name_query(ngx_resol
>       len = sizeof(ngx_resolver_hdr_t) + nlen + sizeof(ngx_resolver_qs_t);
>   
>   #if (NGX_HAVE_INET6)
> -    p = ngx_resolver_alloc(r, r->ipv6 ? len * 2 : len);
> +    p = ngx_resolver_alloc(r, len * (r->ipv4 + r->ipv6));
>   #else
>       p = ngx_resolver_alloc(r, len);
>   #endif
> @@ -3653,23 +3687,28 @@ ngx_resolver_create_name_query(ngx_resol
>       }
>   
>       rn->qlen = (u_short) len;
> -    rn->query = p;
> +
> +    if (r->ipv4) {
> +        rn->query = p;
> +    }
>   
>   #if (NGX_HAVE_INET6)
>       if (r->ipv6) {
> -        rn->query6 = p + len;
> +        rn->query6 = r->ipv4 ? (p + len) : p;
>       }
>   #endif
>   
>       query = (ngx_resolver_hdr_t *) p;
>   
> -    ident = ngx_random();
> -
> -    ngx_log_debug2(NGX_LOG_DEBUG_CORE, r->log, 0,
> -                   "resolve: \"%V\" A %i", name, ident & 0xffff);
> -
> -    query->ident_hi = (u_char) ((ident >> 8) & 0xff);
> -    query->ident_lo = (u_char) (ident & 0xff);
> +    if (r->ipv4) {
> +        ident = ngx_random();
> +
> +        ngx_log_debug2(NGX_LOG_DEBUG_CORE, r->log, 0,
> +                       "resolve: \"%V\" A %i", name, ident & 0xffff);
> +
> +        query->ident_hi = (u_char) ((ident >> 8) & 0xff);
> +        query->ident_lo = (u_char) (ident & 0xff);
> +    }
>   
>       /* recursion query */
>       query->flags_hi = 1; query->flags_lo = 0;
> @@ -3730,7 +3769,9 @@ ngx_resolver_create_name_query(ngx_resol
>   
>       p = rn->query6;
>   
> -    ngx_memcpy(p, rn->query, rn->qlen);
> +    if (r->ipv4) {
> +        ngx_memcpy(p, rn->query, rn->qlen);
> +    }
>   
>       query = (ngx_resolver_hdr_t *) p;
>   
> diff --git a/src/core/ngx_resolver.h b/src/core/ngx_resolver.h
> --- a/src/core/ngx_resolver.h
> +++ b/src/core/ngx_resolver.h
> @@ -175,8 +175,10 @@ struct ngx_resolver_s {
>       ngx_queue_t               srv_expire_queue;
>       ngx_queue_t               addr_expire_queue;
>   
> +    unsigned                  ipv4:1;
> +
>   #if (NGX_HAVE_INET6)
> -    ngx_uint_t                ipv6;                 /* unsigned  ipv6:1; */
> +    unsigned                  ipv6:1;
>       ngx_rbtree_t              addr6_rbtree;
>       ngx_rbtree_node_t         addr6_sentinel;
>       ngx_queue_t               addr6_resend_queue;
> _______________________________________________
> nginx-devel mailing list -- nginx-devel at nginx.org
> To unsubscribe send an email to nginx-devel-leave at nginx.org


From mdounin at mdounin.ru  Sun May 29 01:45:06 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sun, 29 May 2022 04:45:06 +0300
Subject: Wrong documentation about fastcgi_param...
In-Reply-To: <eDr_C3L051Ki1MYrY9iLXir6J-PUmJ_-2xHGT8OhH3r2WkXvvZkCgP7LOv2Z5oOzgoBmt9s14L5PIB1dw4WMav1sUUIRD18za7m7aB_m_Vg=@protonmail.com>
References: <eDr_C3L051Ki1MYrY9iLXir6J-PUmJ_-2xHGT8OhH3r2WkXvvZkCgP7LOv2Z5oOzgoBmt9s14L5PIB1dw4WMav1sUUIRD18za7m7aB_m_Vg=@protonmail.com>
Message-ID: <YpLQIpdfLznvKoxj@mdounin.ru>

Hello!

On Sat, May 28, 2022 at 12:38:45PM +0000, Dimitri via nginx-devel wrote:

> Hi,
> 
> for a few hours yesterday i've had the Problem "FastCGI sent in 
> stderr: "Primary script unknown" while reading response header 
> from upstream".
> 
> I use PHP-FPM with chroot and know about the 
> configuration-problems with wrong "fastcgi_param SCRIPT_FILENAME 
> ...", which you can find on the internet.

[...]

> But these directive WAS SET in the example.conf, so why nginx 
> does not behave as stated in the docs?
> Why nginx uses the global SCRIPT_FILENAME-setting, although 
> these setting is on the "current level" in example.conf?

The nginx-devel@ mailing list is to discuss nginx development.  If 
don't want to suggest a patch, but rather need help with 
understanding what goes wrong in your nginx configuration and how 
to configure nginx properly, please use the nginx@ mailing list 
instead.

(Note though that you may want to provide more details - in 
particular, full configuration you are trying to use.  Note well 
that the location configuration you've tried to use will only send 
the SCRIPT_FILENAME param to PHP, without any other parameters.  
This will certainly break PHP-FPM, but will result in 200 with an 
empty body, not 404.  So probably there is more than one issue in 
your configuration.)

Thank you.

-- 
Maxim Dounin
http://mdounin.ru/


From dimitri_emich at protonmail.com  Sun May 29 10:21:36 2022
From: dimitri_emich at protonmail.com (Dimitri)
Date: Sun, 29 May 2022 10:21:36 +0000
Subject: Wrong documentation about fastcgi_param...
In-Reply-To: <YpLQIpdfLznvKoxj@mdounin.ru>
References: <eDr_C3L051Ki1MYrY9iLXir6J-PUmJ_-2xHGT8OhH3r2WkXvvZkCgP7LOv2Z5oOzgoBmt9s14L5PIB1dw4WMav1sUUIRD18za7m7aB_m_Vg=@protonmail.com> <YpLQIpdfLznvKoxj@mdounin.ru>
Message-ID: <wsbjF3tElaX3MK8ucMwcrBrGXWG9RQuxa8mwLqQ0ozaY7mLGHLrm47rUwISVK1BV_LnNJEMfZ1SYf_ROTg9DM7lfp15FTjLFjBkTNgYwQhE=@protonmail.com>

Hi Maxim,

yes sorry, it's the wrong mailing-list.

I've got it wrong.
I thought, that the current level only overwrite the specified parameters
and takes the other parameters from the global level(which makes more sense to me).

However, sorry for the misplaced post and thanks for the clarification.




Gesendet mittels einer sicheren E-Mail von Proton Mail.
------- Original Message -------
Maxim Dounin <mdounin at mdounin.ru> schrieb am Sonntag, 29. Mai 2022 um 03:45:


> Hello!
>
> On Sat, May 28, 2022 at 12:38:45PM +0000, Dimitri via nginx-devel wrote:
>
> > Hi,
> >
> > for a few hours yesterday i've had the Problem "FastCGI sent in
> > stderr: "Primary script unknown" while reading response header
> > from upstream".
> >
> > I use PHP-FPM with chroot and know about the
> > configuration-problems with wrong "fastcgi_param SCRIPT_FILENAME
> > ...", which you can find on the internet.
>
>
> [...]
>
> > But these directive WAS SET in the example.conf, so why nginx
> > does not behave as stated in the docs?
> > Why nginx uses the global SCRIPT_FILENAME-setting, although
> > these setting is on the "current level" in example.conf?
>
>
> The nginx-devel@ mailing list is to discuss nginx development. If
> don't want to suggest a patch, but rather need help with
> understanding what goes wrong in your nginx configuration and how
> to configure nginx properly, please use the nginx@ mailing list
> instead.
>
> (Note though that you may want to provide more details - in
> particular, full configuration you are trying to use. Note well
> that the location configuration you've tried to use will only send
> the SCRIPT_FILENAME param to PHP, without any other parameters.
> This will certainly break PHP-FPM, but will result in 200 with an
> empty body, not 404. So probably there is more than one issue in
> your configuration.)
>
> Thank you.
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx-devel mailing list -- nginx-devel at nginx.org
> To unsubscribe send an email to nginx-devel-leave at nginx.org


From Husain.Dalroti at mastercard.com  Mon May 30 07:30:10 2022
From: Husain.Dalroti at mastercard.com (Dalroti, Husain)
Date: Mon, 30 May 2022 07:30:10 +0000
Subject: Pass Lua variable in proxy pass directive
Message-ID: <SJ0PR14MB46804C2BD0CE9F1BB105FC73F2DD9@SJ0PR14MB4680.namprd14.prod.outlook.com>

Hi Team,

Is it possible to pass lua variable in proxy pass directive which we define using set_by_lua_file directive ?

Example :
set_by_lua_file $ref test.lua;
proxy_pass '$ref';

Below is lua code :

local function main()
    local host="https://example.com"
    return host
end

return main()

Thanks & Regards
Husain Dalroti
CONFIDENTIALITY NOTICE This e-mail message and any attachments are only for the use of the intended recipient and may contain information that is privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient, any disclosure, distribution or other use of this e-mail message or attachments is prohibited. If you have received this e-mail message in error, please delete and notify the sender immediately. Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220530/73d2982a/attachment.htm>

From pluknet at nginx.com  Mon May 30 10:13:11 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 10:13:11 +0000
Subject: [nginx] Version bump.
Message-ID: <hg.6206bca52d73.1653905591.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/6206bca52d73
branches:  
changeset: 8017:6206bca52d73
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 02:37:59 2022 +0300
description:
Version bump.

diffstat:

 src/core/nginx.h |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff -r 8a54733c9d12 -r 6206bca52d73 src/core/nginx.h
--- a/src/core/nginx.h	Tue May 24 02:51:49 2022 +0300
+++ b/src/core/nginx.h	Mon May 30 02:37:59 2022 +0300
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1021007
-#define NGINX_VERSION      "1.21.7"
+#define nginx_version      1023000
+#define NGINX_VERSION      "1.23.0"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #ifdef NGX_BUILD


From pluknet at nginx.com  Mon May 30 10:13:14 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 10:13:14 +0000
Subject: [nginx] Fixed runtime handling of systems without EPOLLRDHUP support.
Message-ID: <hg.5119c8150478.1653905594.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/5119c8150478
branches:  
changeset: 8018:5119c8150478
user:      Marcus Ball <marcus.ball at live.com>
date:      Mon May 30 02:38:07 2022 +0300
description:
Fixed runtime handling of systems without EPOLLRDHUP support.

In 7583:efd71d49bde0 (nginx 1.17.5) along with introduction of the
ioctl(FIONREAD) support proper handling of systems without EPOLLRDHUP
support in the kernel (but with EPOLLRDHUP in headers) was broken.

Before the change, rev->available was never set to 0 unless
ngx_use_epoll_rdhup was also set (that is, runtime test for EPOLLRDHUP
introduced in 6536:f7849bfb6d21 succeeded).  After the change,
rev->available might reach 0 on systems without runtime EPOLLRDHUP
support, stopping further reading in ngx_readv_chain() and ngx_unix_recv().
And, if EOF happened to be already reported along with the last event,
it is not reported again by epoll_wait(), leading to connection hangs
and timeouts on such systems.

This affects Linux kernels before 2.6.17 if nginx was compiled
with newer headers, and, more importantly, emulation layers, such as
DigitalOcean's App Platform's / gVisor's epoll emulation layer.

Fix is to explicitly check ngx_use_epoll_rdhup before the corresponding
rev->pending_eof tests in ngx_readv_chain() and ngx_unix_recv().

diffstat:

 src/os/unix/ngx_readv_chain.c |  4 +++-
 src/os/unix/ngx_recv.c        |  4 +++-
 2 files changed, 6 insertions(+), 2 deletions(-)

diffs (28 lines):

diff -r 6206bca52d73 -r 5119c8150478 src/os/unix/ngx_readv_chain.c
--- a/src/os/unix/ngx_readv_chain.c	Mon May 30 02:37:59 2022 +0300
+++ b/src/os/unix/ngx_readv_chain.c	Mon May 30 02:38:07 2022 +0300
@@ -55,7 +55,9 @@ ngx_readv_chain(ngx_connection_t *c, ngx
 
 #if (NGX_HAVE_EPOLLRDHUP)
 
-    if (ngx_event_flags & NGX_USE_EPOLL_EVENT) {
+    if ((ngx_event_flags & NGX_USE_EPOLL_EVENT)
+        && ngx_use_epoll_rdhup)
+    {
         ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0,
                        "readv: eof:%d, avail:%d",
                        rev->pending_eof, rev->available);
diff -r 6206bca52d73 -r 5119c8150478 src/os/unix/ngx_recv.c
--- a/src/os/unix/ngx_recv.c	Mon May 30 02:37:59 2022 +0300
+++ b/src/os/unix/ngx_recv.c	Mon May 30 02:38:07 2022 +0300
@@ -52,7 +52,9 @@ ngx_unix_recv(ngx_connection_t *c, u_cha
 
 #if (NGX_HAVE_EPOLLRDHUP)
 
-    if (ngx_event_flags & NGX_USE_EPOLL_EVENT) {
+    if ((ngx_event_flags & NGX_USE_EPOLL_EVENT)
+        && ngx_use_epoll_rdhup)
+    {
         ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0,
                        "recv: eof:%d, avail:%d",
                        rev->pending_eof, rev->available);


From yolkking at 126.com  Mon May 30 10:55:15 2022
From: yolkking at 126.com (=?GBK?B?1uzT7g==?=)
Date: Mon, 30 May 2022 18:55:15 +0800 (CST)
Subject: how to avoid new quic connection distributed to old workers when
 nginx-quic reload
Message-ID: <5aba58f8.4a6b.181149b1004.Coremail.yolkking@126.com>

Hi,


in "src/event/quic/bpf/ngx_quic_reuseport_helper.c",  if can not find socket by dcid (cookie), udp packet will be distributed by kernel. 


so when nginx-quic reload, how to avoid new quic connecion packet distributed to old workers which result in old worker processes can't exit?


thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220530/f9675323/attachment.htm>

From pluknet at nginx.com  Mon May 30 13:54:34 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 17:54:34 +0400
Subject: [PATCH 1 of 2] QUIC: reusable and idle modes for main connection
In-Reply-To: <67ae4b649f2e38a44b24.1652857034@arut-laptop>
References: <patchbomb.1652857033@arut-laptop>
 <67ae4b649f2e38a44b24.1652857034@arut-laptop>
Message-ID: <47D0EFBF-31D0-4C69-AC0F-271D92516717@nginx.com>


> On 18 May 2022, at 10:57, Roman Arutyunyan <arut at nginx.com> wrote:
> 
> # HG changeset patch
> # User Roman Arutyunyan <arut at nginx.com>
> # Date 1652852691 -14400
> #      Wed May 18 09:44:51 2022 +0400
> # Branch quic
> # Node ID 67ae4b649f2e38a44b245b7a842cf396c8250f02
> # Parent  c2f5d79cde64457f1fa7344c56a5248a677a7e46
> QUIC: reusable and idle modes for main connection.
> 
> The modes are controlled by application layer.  For HTTP/3, they are enabled
> when there are no active request streams.
> 
> diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c
> --- a/src/event/quic/ngx_event_quic.c
> +++ b/src/event/quic/ngx_event_quic.c
> @@ -414,8 +414,8 @@ ngx_quic_input_handler(ngx_event_t *rev)
>     }
> 
>     if (c->close) {
> -        qc->error_reason = "graceful shutdown";
> -        ngx_quic_close_connection(c, NGX_OK);
> +        qc->error = NGX_QUIC_ERR_NO_ERROR;
> +        ngx_quic_close_connection(c, NGX_ERROR);
>         return;
>     }
> 
> diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c
> --- a/src/event/quic/ngx_event_quic_streams.c
> +++ b/src/event/quic/ngx_event_quic_streams.c
> @@ -161,6 +161,7 @@ ngx_quic_close_streams(ngx_connection_t 
>     ngx_pool_t         *pool;
>     ngx_queue_t        *q;
>     ngx_rbtree_t       *tree;
> +    ngx_connection_t   *sc;
>     ngx_rbtree_node_t  *node;
>     ngx_quic_stream_t  *qs;
> 
> @@ -185,24 +186,41 @@ ngx_quic_close_streams(ngx_connection_t 
>         return NGX_OK;
>     }
> 
> -#if (NGX_DEBUG)
> -    ns = 0;
> -#endif
> -
>     node = ngx_rbtree_min(tree->root, tree->sentinel);
> 
>     while (node) {
>         qs = (ngx_quic_stream_t *) node;
>         node = ngx_rbtree_next(tree, node);
> +        sc = qs->connection;
> +
> +        if (sc == NULL) {
> +            ngx_quic_close_stream(qs);
> +            continue;
> +        }
> +
> +        if (c->close || sc->idle || sc->reusable) {
> +            sc->close = 1;
> +            sc->read->handler(sc->read);
> +        }
> +    }
> +
> +    if (tree->root == tree->sentinel) {
> +        return NGX_OK;
> +    }
> +
> +#if (NGX_DEBUG)
> +    ns = 0;
> +#endif
> +
> +    for (node = ngx_rbtree_min(tree->root, tree->sentinel);
> +         node;
> +         node = ngx_rbtree_next(tree, node))
> +    {
> +        qs = (ngx_quic_stream_t *) node;
> 
>         qs->recv_state = NGX_QUIC_STREAM_RECV_RESET_RECVD;
>         qs->send_state = NGX_QUIC_STREAM_SEND_RESET_SENT;
> 
> -        if (qs->connection == NULL) {
> -            ngx_quic_close_stream(qs);
> -            continue;
> -        }
> -
>         ngx_quic_set_event(qs->connection->read);
>         ngx_quic_set_event(qs->connection->write);
> 
> @@ -587,6 +605,7 @@ ngx_quic_create_stream(ngx_connection_t 
> {
>     ngx_log_t              *log;
>     ngx_pool_t             *pool;
> +    ngx_uint_t              reusable;
>     ngx_queue_t            *q;
>     ngx_connection_t       *sc;
>     ngx_quic_stream_t      *qs;
> @@ -639,7 +658,13 @@ ngx_quic_create_stream(ngx_connection_t 
>     *log = *c->log;
>     pool->log = log;
> 
> +    reusable = c->reusable;
> +    ngx_reusable_connection(c, 0);
> +
>     sc = ngx_get_connection(c->fd, log);
> +
> +    ngx_reusable_connection(c, reusable);
> +

jftr, this pollutes debug log a bit, but generally looks ok.

>     if (sc == NULL) {
>         ngx_destroy_pool(pool);
>         ngx_queue_insert_tail(&qc->streams.free, &qs->queue);
> diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c
> --- a/src/http/v3/ngx_http_v3_request.c
> +++ b/src/http/v3/ngx_http_v3_request.c
> @@ -210,6 +210,11 @@ ngx_http_v3_init_request_stream(ngx_conn
> 
>     h3c = ngx_http_v3_get_session(c);
> 
> +    if (ngx_terminate || ngx_exiting) {
> +        h3c->goaway = 1;
> +        (void) ngx_http_v3_send_goaway(c, c->quic->id);
> +    }
> +

This results in ever increasing GOAWAY identifiers if multiple
new client streams received after graceful shutdown initiated.
This is forbidden per rfc9114, 5.2. Connection Shutdown.

I wonder if it should move after the below condition.

>     if (h3c->goaway) {
>         c->close = 1;
>         ngx_http_close_connection(c);
> @@ -258,7 +263,7 @@ ngx_http_v3_wait_request_handler(ngx_eve
>     size_t                     size;
>     ssize_t                    n;
>     ngx_buf_t                 *b;
> -    ngx_connection_t          *c;
> +    ngx_connection_t          *c, *pc;
>     ngx_pool_cleanup_t        *cln;
>     ngx_http_request_t        *r;
>     ngx_http_connection_t     *hc;
> @@ -385,6 +390,10 @@ ngx_http_v3_wait_request_handler(ngx_eve
>     h3c = ngx_http_v3_get_session(c);
>     h3c->nrequests++;
> 
> +    pc = c->quic->parent;
> +    pc->idle = 0;
> +    ngx_reusable_connection(pc, 0);
> +
>     if (h3c->keepalive.timer_set) {
>         ngx_del_timer(&h3c->keepalive);
>     }
> @@ -430,7 +439,7 @@ ngx_http_v3_cleanup_request(void *data)
> {
>     ngx_http_request_t  *r = data;
> 
> -    ngx_connection_t          *c;
> +    ngx_connection_t          *c, *pc;
>     ngx_http_v3_session_t     *h3c;
>     ngx_http_core_loc_conf_t  *clcf;
> 
> @@ -443,6 +452,16 @@ ngx_http_v3_cleanup_request(void *data)
>     h3c = ngx_http_v3_get_session(c);
> 
>     if (--h3c->nrequests == 0) {
> +        pc = c->quic->parent;
> +
> +        if (ngx_terminate || ngx_exiting) {
> +            ngx_quic_finalize_connection(pc, NGX_HTTP_V3_ERR_NO_ERROR, NULL);
> +            return;
> +        }
> +
> +        pc->idle = 1;
> +        ngx_reusable_connection(pc, 1);
> +
>         clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module);
>         ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout);
>     }
> diff --git a/src/http/v3/ngx_http_v3_uni.c b/src/http/v3/ngx_http_v3_uni.c
> --- a/src/http/v3/ngx_http_v3_uni.c
> +++ b/src/http/v3/ngx_http_v3_uni.c
> @@ -182,6 +182,11 @@ ngx_http_v3_uni_read_handler(ngx_event_t
> 
>     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 read handler");
> 
> +    if (c->close) {
> +        ngx_http_v3_close_uni_stream(c);
> +        return;
> +    }
> +
>     ngx_memzero(&b, sizeof(ngx_buf_t));
> 
>     while (rev->ready) {
> @@ -262,6 +267,11 @@ ngx_http_v3_uni_dummy_read_handler(ngx_e
> 
>     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 dummy read handler");
> 
> +    if (c->close) {
> +        ngx_http_v3_close_uni_stream(c);
> +        return;
> +    }
> +
>     if (rev->ready) {
>         if (c->recv(c, &ch, 1) != 0) {
>             ngx_http_v3_finalize_connection(c, NGX_HTTP_V3_ERR_NO_ERROR, NULL);
> 

-- 
Sergey Kandaurov


From osa at freebsd.org.ru  Mon May 30 14:49:47 2022
From: osa at freebsd.org.ru (Sergey A. Osokin)
Date: Mon, 30 May 2022 17:49:47 +0300
Subject: Pass Lua variable in proxy pass directive
In-Reply-To: <SJ0PR14MB46804C2BD0CE9F1BB105FC73F2DD9@SJ0PR14MB4680.namprd14.prod.outlook.com>
References: <SJ0PR14MB46804C2BD0CE9F1BB105FC73F2DD9@SJ0PR14MB4680.namprd14.prod.outlook.com>
Message-ID: <YpTZiw7RT3mxR5dL@FreeBSD.org.ru>

Hi Husain,

On Mon, May 30, 2022 at 07:30:10AM +0000, Dalroti, Husain wrote:
> Hi Team,
> 
> Is it possible to pass lua variable in proxy pass directive which we define using set_by_lua_file directive ?
> 
> Example :
> set_by_lua_file $ref test.lua;
> proxy_pass '$ref';
> 
> Below is lua code :
> 
> local function main()
>     local host="https://example.com"
>     return host
> end
> 
> return main()

The nginx-devel@ mailing list is to discuss nginx development.  If
don't want to suggest a patch, but rather need help with
understanding what goes wrong in your nginx configuration and how
to configure nginx properly, please use the nginx@ mailing list
instead.

Also, the question is related to the third-party module,
developing by another team.  So, I'd recommend to address the
question to the corresponding mailing list.

Thank you.

-- 
Sergey A. Osokin


From vl at inspert.ru  Mon May 30 17:24:19 2022
From: vl at inspert.ru (Vladimir Homutov)
Date: Mon, 30 May 2022 20:24:19 +0300
Subject: how to avoid new quic connection distributed to old workers when
 nginx-quic reload
In-Reply-To: <5aba58f8.4a6b.181149b1004.Coremail.yolkking@126.com>
References: <5aba58f8.4a6b.181149b1004.Coremail.yolkking@126.com>
Message-ID: <YpT9w0ONmWpChCSI@vl.krasnogorsk.ru>

On Mon, May 30, 2022 at 06:55:15PM +0800, 朱宇 wrote:
> Hi,
>
>
> in "src/event/quic/bpf/ngx_quic_reuseport_helper.c",  if can not find socket by dcid (cookie), udp packet will be distributed by kernel.
>
>
> so when nginx-quic reload, how to avoid new quic connecion packet distributed to old workers which result in old worker processes can't exit?
>
>
> thanks

Old (exiting) workers stop accepting new connections and ignores such
packet or replies with 'retry' if configured. The sender assumes that the
packet was lost or gets the 'retry' packet and repeats send. Hopefully,
the next time old worker will already exit and the packet will be
delivered to new worker. This is not ideal, but this is how it works now.

See http://hg.nginx.org/nginx-quic/file/quic/src/event/quic/ngx_event_quic.c#l913


From mdounin at mdounin.ru  Mon May 30 21:05:13 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Tue, 31 May 2022 00:05:13 +0300
Subject: [PATCH] Mp4: fixed potential overflow in
 ngx_http_mp4_crop_stts_data()
Message-ID: <d5363be9fa61e0327574.1653944713@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1653942033 -10800
#      Mon May 30 23:20:33 2022 +0300
# Node ID d5363be9fa61e0327574b8aa1342c874efd027b0
# Parent  cd40709c91e245036e7f71a6c3e5190d9533be75
Mp4: fixed potential overflow in ngx_http_mp4_crop_stts_data().

Both "count" and "duration" variables are 32-bit, so their product might
potentially overflow.  It is used to reduce 64-bit start_time variable,
and with very large start_time this can result in incorrect seeking.

Found by Coverity (CID 1499904).

diff --git a/src/http/modules/ngx_http_mp4_module.c b/src/http/modules/ngx_http_mp4_module.c
--- a/src/http/modules/ngx_http_mp4_module.c
+++ b/src/http/modules/ngx_http_mp4_module.c
@@ -2331,7 +2331,7 @@ ngx_http_mp4_crop_stts_data(ngx_http_mp4
         }
 
         start_sample += count;
-        start_time -= count * duration;
+        start_time -= (uint64_t) count * duration;
         entries--;
         entry++;
     }


From pluknet at nginx.com  Mon May 30 21:53:08 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Tue, 31 May 2022 01:53:08 +0400
Subject: [PATCH 2 of 2] QUIC: init_streams() callback
In-Reply-To: <a0f2d69f1fe43dfc7182.1652857035@arut-laptop>
References: <patchbomb.1652857033@arut-laptop>
 <a0f2d69f1fe43dfc7182.1652857035@arut-laptop>
Message-ID: <22737FC4-820A-4DDD-870F-1C841999D822@nginx.com>


> On 18 May 2022, at 10:57, Roman Arutyunyan <arut at nginx.com> wrote:
> 
> # HG changeset patch
> # User Roman Arutyunyan <arut at nginx.com>
> # Date 1652856132 -14400
> #      Wed May 18 10:42:12 2022 +0400
> # Branch quic
> # Node ID a0f2d69f1fe43dfc718262235bf04d7b05f1fd68
> # Parent  67ae4b649f2e38a44b245b7a842cf396c8250f02
> QUIC: init_streams() callback.
> 
> It's called after handshake completion to initialize application-level data
> prior to creating streams.

See below for 0-RTT case.

> 
> HTTP/3 callback implementation switches main QUIC connection to idle and
> reusable modes and sets keepalive timer.
> 
> diff --git a/src/event/quic/ngx_event_quic.h b/src/event/quic/ngx_event_quic.h
> --- a/src/event/quic/ngx_event_quic.h
> +++ b/src/event/quic/ngx_event_quic.h
> @@ -28,6 +28,9 @@
> #define NGX_QUIC_STREAM_UNIDIRECTIONAL       0x02
> 
> 
> +typedef ngx_int_t (*ngx_quic_init_streams_pt)(ngx_connection_t *c);
> +
> +
> typedef enum {
>     NGX_QUIC_STREAM_SEND_READY = 0,
>     NGX_QUIC_STREAM_SEND_SEND,
> @@ -74,6 +77,8 @@ typedef struct {
>     ngx_int_t                      stream_reject_code_uni;
>     ngx_int_t                      stream_reject_code_bidi;
> 
> +    ngx_quic_init_streams_pt       init_streams;
> +
>     u_char                         av_token_key[NGX_QUIC_AV_KEY_LEN];
>     u_char                         sr_token_key[NGX_QUIC_SR_KEY_LEN];
> } ngx_quic_conf_t;
> diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c
> --- a/src/event/quic/ngx_event_quic_streams.c
> +++ b/src/event/quic/ngx_event_quic_streams.c
> @@ -21,6 +21,7 @@ static ngx_quic_stream_t *ngx_quic_get_s
> static ngx_int_t ngx_quic_reject_stream(ngx_connection_t *c, uint64_t id);
> static void ngx_quic_init_stream_handler(ngx_event_t *ev);
> static void ngx_quic_init_streams_handler(ngx_connection_t *c);
> +static ngx_int_t ngx_quic_do_init_streams(ngx_connection_t *c);
> static ngx_quic_stream_t *ngx_quic_create_stream(ngx_connection_t *c,
>     uint64_t id);
> static void ngx_quic_empty_handler(ngx_event_t *ev);
> @@ -571,15 +572,22 @@ ngx_quic_init_streams(ngx_connection_t *
>         return NGX_OK;
>     }
> 
> -    ngx_quic_init_streams_handler(c);
> -
> -    return NGX_OK;
> +    return ngx_quic_do_init_streams(c);
> }
> 
> 
> static void
> ngx_quic_init_streams_handler(ngx_connection_t *c)
> {
> +    if (ngx_quic_do_init_streams(c) != NGX_OK) {
> +        ngx_quic_close_connection(c, NGX_ERROR);
> +    }
> +}
> +
> +
> +static ngx_int_t
> +ngx_quic_do_init_streams(ngx_connection_t *c)
> +{
>     ngx_queue_t            *q;
>     ngx_quic_stream_t      *qs;
>     ngx_quic_connection_t  *qc;
> @@ -588,6 +596,12 @@ ngx_quic_init_streams_handler(ngx_connec
> 
>     qc = ngx_quic_get_connection(c);
> 
> +    if (qc->conf->init_streams) {
> +        if (qc->conf->init_streams(c) != NGX_OK) {
> +            return NGX_ERROR;
> +        }
> +    }
> +
>     for (q = ngx_queue_head(&qc->streams.uninitialized);
>          q != ngx_queue_sentinel(&qc->streams.uninitialized);
>          q = ngx_queue_next(q))
> @@ -597,6 +611,8 @@ ngx_quic_init_streams_handler(ngx_connec
>     }
> 
>     qc->streams.initialized = 1;
> +
> +    return NGX_OK;
> }
> 
> 
> diff --git a/src/http/v3/ngx_http_v3.c b/src/http/v3/ngx_http_v3.c
> --- a/src/http/v3/ngx_http_v3.c
> +++ b/src/http/v3/ngx_http_v3.c
> @@ -17,21 +17,15 @@ static void ngx_http_v3_cleanup_session(
> ngx_int_t
> ngx_http_v3_init_session(ngx_connection_t *c)
> {
> -    ngx_connection_t       *pc;
>     ngx_pool_cleanup_t     *cln;
>     ngx_http_connection_t  *hc;
>     ngx_http_v3_session_t  *h3c;
> 
> -    pc = c->quic->parent;
> -    hc = pc->data;
> -
> -    if (hc->v3_session) {
> -        return NGX_OK;
> -    }
> +    hc = c->data;
> 
>     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 init session");
> 
> -    h3c = ngx_pcalloc(pc->pool, sizeof(ngx_http_v3_session_t));
> +    h3c = ngx_pcalloc(c->pool, sizeof(ngx_http_v3_session_t));
>     if (h3c == NULL) {
>         goto failed;
>     }
> @@ -42,16 +36,16 @@ ngx_http_v3_init_session(ngx_connection_
>     ngx_queue_init(&h3c->blocked);
>     ngx_queue_init(&h3c->pushing);
> 
> -    h3c->keepalive.log = pc->log;
> -    h3c->keepalive.data = pc;
> +    h3c->keepalive.log = c->log;
> +    h3c->keepalive.data = c;
>     h3c->keepalive.handler = ngx_http_v3_keepalive_handler;
>     h3c->keepalive.cancelable = 1;
> 
> -    h3c->table.send_insert_count.log = pc->log;
> -    h3c->table.send_insert_count.data = pc;
> +    h3c->table.send_insert_count.log = c->log;
> +    h3c->table.send_insert_count.data = c;
>     h3c->table.send_insert_count.handler = ngx_http_v3_inc_insert_count_handler;
> 
> -    cln = ngx_pool_cleanup_add(pc->pool, 0);
> +    cln = ngx_pool_cleanup_add(c->pool, 0);
>     if (cln == NULL) {
>         goto failed;
>     }
> diff --git a/src/http/v3/ngx_http_v3.h b/src/http/v3/ngx_http_v3.h
> --- a/src/http/v3/ngx_http_v3.h
> +++ b/src/http/v3/ngx_http_v3.h
> @@ -153,6 +153,7 @@ struct ngx_http_v3_session_s {
> 
> void ngx_http_v3_init(ngx_connection_t *c);
> void ngx_http_v3_reset_connection(ngx_connection_t *c);
> +ngx_int_t ngx_http_v3_init_streams(ngx_connection_t *c);
> ngx_int_t ngx_http_v3_init_session(ngx_connection_t *c);
> ngx_int_t ngx_http_v3_check_flood(ngx_connection_t *c);
> 
> diff --git a/src/http/v3/ngx_http_v3_module.c b/src/http/v3/ngx_http_v3_module.c
> --- a/src/http/v3/ngx_http_v3_module.c
> +++ b/src/http/v3/ngx_http_v3_module.c
> @@ -249,6 +249,8 @@ ngx_http_v3_create_srv_conf(ngx_conf_t *
>     h3scf->quic.stream_reject_code_bidi = NGX_HTTP_V3_ERR_REQUEST_REJECTED;
>     h3scf->quic.active_connection_id_limit = NGX_CONF_UNSET_UINT;
> 
> +    h3scf->quic.init_streams = ngx_http_v3_init_streams;
> +
>     return h3scf;
> }
> 
> diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c
> --- a/src/http/v3/ngx_http_v3_request.c
> +++ b/src/http/v3/ngx_http_v3_request.c
> @@ -93,11 +93,6 @@ ngx_http_v3_init(ngx_connection_t *c)
>     }
> #endif
> 
> -    if (ngx_http_v3_init_session(c) != NGX_OK) {
> -        ngx_http_close_connection(c);
> -        return;
> -    }
> -
>     if (c->quic->id & NGX_QUIC_STREAM_UNIDIRECTIONAL) {
>         ngx_http_v3_init_uni_stream(c);
> 
> @@ -107,6 +102,43 @@ ngx_http_v3_init(ngx_connection_t *c)
> }
> 
> 
> +ngx_int_t
> +ngx_http_v3_init_streams(ngx_connection_t *c)
> +{
> +    ngx_http_v3_session_t     *h3c;
> +    ngx_http_connection_t     *hc;
> +    ngx_http_v3_srv_conf_t    *h3scf;
> +    ngx_http_core_loc_conf_t  *clcf;
> +
> +    if (ngx_terminate || ngx_exiting) {
> +        return NGX_ERROR;
> +    }
> +
> +    hc = c->data;
> +
> +    h3scf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_v3_module);
> +
> +#if (NGX_HTTP_V3_HQ)
> +    if (h3scf->hq) {
> +        return NGX_OK;
> +    }
> +#endif

Clang complains about unused variables:

diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c
--- a/src/http/v3/ngx_http_v3_request.c
+++ b/src/http/v3/ngx_http_v3_request.c
@@ -106,22 +106,26 @@ ngx_int_t
 ngx_http_v3_init_streams(ngx_connection_t *c)
 {
     ngx_http_v3_session_t     *h3c;
+#if (NGX_HTTP_V3_HQ)
     ngx_http_connection_t     *hc;
     ngx_http_v3_srv_conf_t    *h3scf;
+#endif
     ngx_http_core_loc_conf_t  *clcf;
 
     if (ngx_terminate || ngx_exiting) {
         return NGX_ERROR;
     }
 
+#if (NGX_HTTP_V3_HQ)
+
     hc = c->data;
 
     h3scf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_v3_module);
 
-#if (NGX_HTTP_V3_HQ)
     if (h3scf->hq) {
         return NGX_OK;
     }
+
 #endif
 
     if (ngx_http_v3_init_session(c) != NGX_OK) {

> +
> +    if (ngx_http_v3_init_session(c) != NGX_OK) {
> +        return NGX_ERROR;
> +    }

If 0-RTT data is accepted, ngx_http_v3_init_session() is called too early,
before SSL_do_handshake() is used to apply client transport parameters.
In particular that means inability to open a server unidirectional stream.

> +
> +    c->idle = 1;
> +    ngx_reusable_connection(c, 1);
> +
> +    h3c = ngx_http_v3_get_session(c);
> +    clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module);
> +    ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout);
> +
> +    return NGX_OK;
> +}
> +
> +
> #if (NGX_HTTP_V3_HQ)
> 
> static void
> 

-- 
Sergey Kandaurov


From pluknet at nginx.com  Mon May 30 22:32:10 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:10 +0000
Subject: [nginx] Perl: fixed $r->header_in("Connection").
Message-ID: <hg.0e562a332529.1653949930.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/0e562a332529
branches:  
changeset: 8019:0e562a332529
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:25 2022 +0300
description:
Perl: fixed $r->header_in("Connection").

Previously, the r->header_in->connection pointer was never set despite
being present in ngx_http_headers_in, resulting in incorrect value returned
by $r->header_in("Connection") in embedded perl.

diffstat:

 src/http/ngx_http_request.c |  4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diffs (14 lines):

diff -r 5119c8150478 -r 0e562a332529 src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c	Mon May 30 02:38:07 2022 +0300
+++ b/src/http/ngx_http_request.c	Mon May 30 21:25:25 2022 +0300
@@ -1827,6 +1827,10 @@ static ngx_int_t
 ngx_http_process_connection(ngx_http_request_t *r, ngx_table_elt_t *h,
     ngx_uint_t offset)
 {
+    if (ngx_http_process_header_line(r, h, offset) != NGX_OK) {
+        return NGX_ERROR;
+    }
+
     if (ngx_strcasestrn(h->value.data, "close", 5 - 1)) {
         r->headers_in.connection_type = NGX_HTTP_CONNECTION_CLOSE;
 


From pluknet at nginx.com  Mon May 30 22:32:13 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:13 +0000
Subject: [nginx] FastCGI: combining headers with identical names (ticket
 #1724).
Message-ID: <hg.f8f6b9fee66a.1653949933.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/f8f6b9fee66a
branches:  
changeset: 8020:f8f6b9fee66a
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:27 2022 +0300
description:
FastCGI: combining headers with identical names (ticket #1724).

FastCGI responder is expected to receive CGI/1.1 environment variables
in the parameters (see section "6.2 Responder" of the FastCGI specification).
Obviously enough, there cannot be multiple environment variables with
the same name.

Further, CGI specification (RFC 3875, section "4.1.18. Protocol-Specific
Meta-Variables") explicitly requires to combine headers: "If multiple
header fields with the same field-name are received then the server MUST
rewrite them as a single value having the same semantics".

diffstat:

 src/core/ngx_hash.h                        |   7 ++-
 src/http/modules/ngx_http_fastcgi_module.c |  70 +++++++++++++++++++++++-----
 src/http/ngx_http_core_module.c            |  74 ++++++++++++++++++++++++++++++
 src/http/ngx_http_core_module.h            |   2 +
 src/http/ngx_http_request.h                |   2 +
 5 files changed, 140 insertions(+), 15 deletions(-)

diffs (264 lines):

diff -r 0e562a332529 -r f8f6b9fee66a src/core/ngx_hash.h
--- a/src/core/ngx_hash.h	Mon May 30 21:25:25 2022 +0300
+++ b/src/core/ngx_hash.h	Mon May 30 21:25:27 2022 +0300
@@ -89,12 +89,15 @@ typedef struct {
 } ngx_hash_keys_arrays_t;
 
 
-typedef struct {
+typedef struct ngx_table_elt_s  ngx_table_elt_t;
+
+struct ngx_table_elt_s {
     ngx_uint_t        hash;
     ngx_str_t         key;
     ngx_str_t         value;
     u_char           *lowcase_key;
-} ngx_table_elt_t;
+    ngx_table_elt_t  *next;
+};
 
 
 void *ngx_hash_find(ngx_hash_t *hash, ngx_uint_t key, u_char *name, size_t len);
diff -r 0e562a332529 -r f8f6b9fee66a src/http/modules/ngx_http_fastcgi_module.c
--- a/src/http/modules/ngx_http_fastcgi_module.c	Mon May 30 21:25:25 2022 +0300
+++ b/src/http/modules/ngx_http_fastcgi_module.c	Mon May 30 21:25:27 2022 +0300
@@ -835,14 +835,14 @@ static ngx_int_t
 ngx_http_fastcgi_create_request(ngx_http_request_t *r)
 {
     off_t                         file_pos;
-    u_char                        ch, *pos, *lowcase_key;
+    u_char                        ch, sep, *pos, *lowcase_key;
     size_t                        size, len, key_len, val_len, padding,
                                   allocated;
     ngx_uint_t                    i, n, next, hash, skip_empty, header_params;
     ngx_buf_t                    *b;
     ngx_chain_t                  *cl, *body;
     ngx_list_part_t              *part;
-    ngx_table_elt_t              *header, **ignored;
+    ngx_table_elt_t              *header, *hn, **ignored;
     ngx_http_upstream_t          *u;
     ngx_http_script_code_pt       code;
     ngx_http_script_engine_t      e, le;
@@ -900,7 +900,11 @@ ngx_http_fastcgi_create_request(ngx_http
         allocated = 0;
         lowcase_key = NULL;
 
-        if (params->number) {
+        if (ngx_http_link_multi_headers(r) != NGX_OK) {
+            return NGX_ERROR;
+        }
+
+        if (params->number || r->headers_in.multi) {
             n = 0;
             part = &r->headers_in.headers.part;
 
@@ -930,6 +934,12 @@ ngx_http_fastcgi_create_request(ngx_http
                 i = 0;
             }
 
+            for (n = 0; n < header_params; n++) {
+                if (&header[i] == ignored[n]) {
+                    goto next_length;
+                }
+            }
+
             if (params->number) {
                 if (allocated < header[i].key.len) {
                     allocated = header[i].key.len + 16;
@@ -959,15 +969,23 @@ ngx_http_fastcgi_create_request(ngx_http
                     ignored[header_params++] = &header[i];
                     continue;
                 }
-
-                n += sizeof("HTTP_") - 1;
-
-            } else {
-                n = sizeof("HTTP_") - 1 + header[i].key.len;
             }
 
-            len += ((n > 127) ? 4 : 1) + ((header[i].value.len > 127) ? 4 : 1)
-                + n + header[i].value.len;
+            key_len = sizeof("HTTP_") - 1 + header[i].key.len;
+
+            val_len = header[i].value.len;
+
+            for (hn = header[i].next; hn; hn = hn->next) {
+                val_len += hn->value.len + 2;
+                ignored[header_params++] = hn;
+            }
+
+            len += ((key_len > 127) ? 4 : 1) + key_len
+                   + ((val_len > 127) ? 4 : 1) + val_len;
+
+        next_length:
+
+            continue;
         }
     }
 
@@ -1109,7 +1127,7 @@ ngx_http_fastcgi_create_request(ngx_http
 
             for (n = 0; n < header_params; n++) {
                 if (&header[i] == ignored[n]) {
-                    goto next;
+                    goto next_value;
                 }
             }
 
@@ -1125,6 +1143,11 @@ ngx_http_fastcgi_create_request(ngx_http
             }
 
             val_len = header[i].value.len;
+
+            for (hn = header[i].next; hn; hn = hn->next) {
+                val_len += hn->value.len + 2;
+            }
+
             if (val_len > 127) {
                 *b->last++ = (u_char) (((val_len >> 24) & 0x7f) | 0x80);
                 *b->last++ = (u_char) ((val_len >> 16) & 0xff);
@@ -1150,13 +1173,34 @@ ngx_http_fastcgi_create_request(ngx_http
                 *b->last++ = ch;
             }
 
-            b->last = ngx_copy(b->last, header[i].value.data, val_len);
+            b->last = ngx_copy(b->last, header[i].value.data,
+                               header[i].value.len);
+
+            if (header[i].next) {
+
+                if (header[i].key.len == sizeof("Cookie") - 1
+                    && ngx_strncasecmp(header[i].key.data, (u_char *) "Cookie",
+                                       sizeof("Cookie") - 1)
+                       == 0)
+                {
+                    sep = ';';
+
+                } else {
+                    sep = ',';
+                }
+
+                for (hn = header[i].next; hn; hn = hn->next) {
+                    *b->last++ = sep;
+                    *b->last++ = ' ';
+                    b->last = ngx_copy(b->last, hn->value.data, hn->value.len);
+                }
+            }
 
             ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                            "fastcgi param: \"%*s: %*s\"",
                            key_len, b->last - (key_len + val_len),
                            val_len, b->last - val_len);
-        next:
+        next_value:
 
             continue;
         }
diff -r 0e562a332529 -r f8f6b9fee66a src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c	Mon May 30 21:25:25 2022 +0300
+++ b/src/http/ngx_http_core_module.c	Mon May 30 21:25:27 2022 +0300
@@ -2802,6 +2802,80 @@ ngx_http_get_forwarded_addr_internal(ngx
 }
 
 
+ngx_int_t
+ngx_http_link_multi_headers(ngx_http_request_t *r)
+{
+    ngx_uint_t        i, j;
+    ngx_list_part_t  *part, *ppart;
+    ngx_table_elt_t  *header, *pheader, **ph;
+
+    if (r->headers_in.multi_linked) {
+        return NGX_OK;
+    }
+
+    r->headers_in.multi_linked = 1;
+
+    part = &r->headers_in.headers.part;
+    header = part->elts;
+
+    for (i = 0; /* void */; i++) {
+
+        if (i >= part->nelts) {
+            if (part->next == NULL) {
+                break;
+            }
+
+            part = part->next;
+            header = part->elts;
+            i = 0;
+        }
+
+        header[i].next = NULL;
+
+        /*
+         * search for previous headers with the same name;
+         * if there are any, link to them
+         */
+
+        ppart = &r->headers_in.headers.part;
+        pheader = ppart->elts;
+
+        for (j = 0; /* void */; j++) {
+
+            if (j >= ppart->nelts) {
+                if (ppart->next == NULL) {
+                    break;
+                }
+
+                ppart = ppart->next;
+                pheader = ppart->elts;
+                j = 0;
+            }
+
+            if (part == ppart && i == j) {
+                break;
+            }
+
+            if (header[i].key.len == pheader[j].key.len
+                && ngx_strncasecmp(header[i].key.data, pheader[j].key.data,
+                                   header[i].key.len)
+                   == 0)
+            {
+                ph = &pheader[j].next;
+                while (*ph) { ph = &(*ph)->next; }
+                *ph = &header[i];
+
+                r->headers_in.multi = 1;
+
+                break;
+            }
+        }
+    }
+
+    return NGX_OK;
+}
+
+
 static char *
 ngx_http_core_server(ngx_conf_t *cf, ngx_command_t *cmd, void *dummy)
 {
diff -r 0e562a332529 -r f8f6b9fee66a src/http/ngx_http_core_module.h
--- a/src/http/ngx_http_core_module.h	Mon May 30 21:25:25 2022 +0300
+++ b/src/http/ngx_http_core_module.h	Mon May 30 21:25:27 2022 +0300
@@ -532,6 +532,8 @@ ngx_int_t ngx_http_get_forwarded_addr(ng
     ngx_array_t *headers, ngx_str_t *value, ngx_array_t *proxies,
     int recursive);
 
+ngx_int_t ngx_http_link_multi_headers(ngx_http_request_t *r);
+
 
 extern ngx_module_t  ngx_http_core_module;
 
diff -r 0e562a332529 -r f8f6b9fee66a src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h	Mon May 30 21:25:25 2022 +0300
+++ b/src/http/ngx_http_request.h	Mon May 30 21:25:27 2022 +0300
@@ -242,6 +242,8 @@ typedef struct {
 
     unsigned                          connection_type:2;
     unsigned                          chunked:1;
+    unsigned                          multi:1;
+    unsigned                          multi_linked:1;
     unsigned                          msie:1;
     unsigned                          msie6:1;
     unsigned                          opera:1;


From pluknet at nginx.com  Mon May 30 22:32:16 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:16 +0000
Subject: [nginx] SCGI: combining headers with identical names (ticket #1724).
Message-ID: <hg.75af96daee97.1653949936.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/75af96daee97
branches:  
changeset: 8021:75af96daee97
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:28 2022 +0300
description:
SCGI: combining headers with identical names (ticket #1724).

SCGI specification explicitly forbids headers with duplicate names
(section "3. Request Format"): "Duplicate names are not allowed in
the headers".

Further, provided headers are expected to follow CGI specification,
which also requires to combine headers (RFC 3875, section "4.1.18.
Protocol-Specific Meta-Variables"): "If multiple header fields with
the same field-name are received then the server MUST rewrite them
as a single value having the same semantics".

diffstat:

 src/http/modules/ngx_http_scgi_module.c |  50 +++++++++++++++++++++++++++++---
 1 files changed, 45 insertions(+), 5 deletions(-)

diffs (106 lines):

diff -r f8f6b9fee66a -r 75af96daee97 src/http/modules/ngx_http_scgi_module.c
--- a/src/http/modules/ngx_http_scgi_module.c	Mon May 30 21:25:27 2022 +0300
+++ b/src/http/modules/ngx_http_scgi_module.c	Mon May 30 21:25:28 2022 +0300
@@ -633,14 +633,14 @@ static ngx_int_t
 ngx_http_scgi_create_request(ngx_http_request_t *r)
 {
     off_t                         content_length_n;
-    u_char                        ch, *key, *val, *lowcase_key;
+    u_char                        ch, sep, *key, *val, *lowcase_key;
     size_t                        len, key_len, val_len, allocated;
     ngx_buf_t                    *b;
     ngx_str_t                     content_length;
     ngx_uint_t                    i, n, hash, skip_empty, header_params;
     ngx_chain_t                  *cl, *body;
     ngx_list_part_t              *part;
-    ngx_table_elt_t              *header, **ignored;
+    ngx_table_elt_t              *header, *hn, **ignored;
     ngx_http_scgi_params_t       *params;
     ngx_http_script_code_pt       code;
     ngx_http_script_engine_t      e, le;
@@ -707,7 +707,11 @@ ngx_http_scgi_create_request(ngx_http_re
         allocated = 0;
         lowcase_key = NULL;
 
-        if (params->number) {
+        if (ngx_http_link_multi_headers(r) != NGX_OK) {
+            return NGX_ERROR;
+        }
+
+        if (params->number || r->headers_in.multi) {
             n = 0;
             part = &r->headers_in.headers.part;
 
@@ -737,6 +741,12 @@ ngx_http_scgi_create_request(ngx_http_re
                 i = 0;
             }
 
+            for (n = 0; n < header_params; n++) {
+                if (&header[i] == ignored[n]) {
+                    goto next_length;
+                }
+            }
+
             if (params->number) {
                 if (allocated < header[i].key.len) {
                     allocated = header[i].key.len + 16;
@@ -770,6 +780,15 @@ ngx_http_scgi_create_request(ngx_http_re
 
             len += sizeof("HTTP_") - 1 + header[i].key.len + 1
                 + header[i].value.len + 1;
+
+            for (hn = header[i].next; hn; hn = hn->next) {
+                len += hn->value.len + 2;
+                ignored[header_params++] = hn;
+            }
+
+        next_length:
+
+            continue;
         }
     }
 
@@ -869,7 +888,7 @@ ngx_http_scgi_create_request(ngx_http_re
 
             for (n = 0; n < header_params; n++) {
                 if (&header[i] == ignored[n]) {
-                    goto next;
+                    goto next_value;
                 }
             }
 
@@ -893,12 +912,33 @@ ngx_http_scgi_create_request(ngx_http_re
 
             val = b->last;
             b->last = ngx_copy(val, header[i].value.data, header[i].value.len);
+
+            if (header[i].next) {
+
+                if (header[i].key.len == sizeof("Cookie") - 1
+                    && ngx_strncasecmp(header[i].key.data, (u_char *) "Cookie",
+                                       sizeof("Cookie") - 1)
+                       == 0)
+                {
+                    sep = ';';
+
+                } else {
+                    sep = ',';
+                }
+
+                for (hn = header[i].next; hn; hn = hn->next) {
+                    *b->last++ = sep;
+                    *b->last++ = ' ';
+                    b->last = ngx_copy(b->last, hn->value.data, hn->value.len);
+                }
+            }
+
             *b->last++ = (u_char) 0;
 
             ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                            "scgi param: \"%s: %s\"", key, val);
 
-        next:
+        next_value:
 
             continue;
         }


From pluknet at nginx.com  Mon May 30 22:32:19 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:19 +0000
Subject: [nginx] Uwsgi: combining headers with identical names (ticket #1724).
Message-ID: <hg.8b7a96fdd54c.1653949939.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/8b7a96fdd54c
branches:  
changeset: 8022:8b7a96fdd54c
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:30 2022 +0300
description:
Uwsgi: combining headers with identical names (ticket #1724).

The uwsgi specification states that "The uwsgi block vars represent a
dictionary/hash".  This implies that no duplicate headers are expected.

Further, provided headers are expected to follow CGI specification,
which also requires to combine headers (RFC 3875, section "4.1.18.
Protocol-Specific Meta-Variables"): "If multiple header fields with
the same field-name are received then the server MUST rewrite them
as a single value having the same semantics".

diffstat:

 src/http/modules/ngx_http_uwsgi_module.c |  57 ++++++++++++++++++++++++++++---
 1 files changed, 51 insertions(+), 6 deletions(-)

diffs (114 lines):

diff -r 75af96daee97 -r 8b7a96fdd54c src/http/modules/ngx_http_uwsgi_module.c
--- a/src/http/modules/ngx_http_uwsgi_module.c	Mon May 30 21:25:28 2022 +0300
+++ b/src/http/modules/ngx_http_uwsgi_module.c	Mon May 30 21:25:30 2022 +0300
@@ -845,13 +845,13 @@ ngx_http_uwsgi_create_key(ngx_http_reque
 static ngx_int_t
 ngx_http_uwsgi_create_request(ngx_http_request_t *r)
 {
-    u_char                        ch, *lowcase_key;
+    u_char                        ch, sep, *lowcase_key;
     size_t                        key_len, val_len, len, allocated;
     ngx_uint_t                    i, n, hash, skip_empty, header_params;
     ngx_buf_t                    *b;
     ngx_chain_t                  *cl, *body;
     ngx_list_part_t              *part;
-    ngx_table_elt_t              *header, **ignored;
+    ngx_table_elt_t              *header, *hn, **ignored;
     ngx_http_uwsgi_params_t      *params;
     ngx_http_script_code_pt       code;
     ngx_http_script_engine_t      e, le;
@@ -905,7 +905,11 @@ ngx_http_uwsgi_create_request(ngx_http_r
         allocated = 0;
         lowcase_key = NULL;
 
-        if (params->number) {
+        if (ngx_http_link_multi_headers(r) != NGX_OK) {
+            return NGX_ERROR;
+        }
+
+        if (params->number || r->headers_in.multi) {
             n = 0;
             part = &r->headers_in.headers.part;
 
@@ -935,6 +939,12 @@ ngx_http_uwsgi_create_request(ngx_http_r
                 i = 0;
             }
 
+            for (n = 0; n < header_params; n++) {
+                if (&header[i] == ignored[n]) {
+                    goto next_length;
+                }
+            }
+
             if (params->number) {
                 if (allocated < header[i].key.len) {
                     allocated = header[i].key.len + 16;
@@ -968,6 +978,15 @@ ngx_http_uwsgi_create_request(ngx_http_r
 
             len += 2 + sizeof("HTTP_") - 1 + header[i].key.len
                  + 2 + header[i].value.len;
+
+            for (hn = header[i].next; hn; hn = hn->next) {
+                len += hn->value.len + 2;
+                ignored[header_params++] = hn;
+            }
+
+        next_length:
+
+            continue;
         }
     }
 
@@ -1086,7 +1105,7 @@ ngx_http_uwsgi_create_request(ngx_http_r
 
             for (n = 0; n < header_params; n++) {
                 if (&header[i] == ignored[n]) {
-                    goto next;
+                    goto next_value;
                 }
             }
 
@@ -1109,15 +1128,41 @@ ngx_http_uwsgi_create_request(ngx_http_r
             }
 
             val_len = header[i].value.len;
+
+            for (hn = header[i].next; hn; hn = hn->next) {
+                val_len += hn->value.len + 2;
+            }
+
             *b->last++ = (u_char) (val_len & 0xff);
             *b->last++ = (u_char) ((val_len >> 8) & 0xff);
-            b->last = ngx_copy(b->last, header[i].value.data, val_len);
+            b->last = ngx_copy(b->last, header[i].value.data,
+                               header[i].value.len);
+
+            if (header[i].next) {
+
+                if (header[i].key.len == sizeof("Cookie") - 1
+                    && ngx_strncasecmp(header[i].key.data, (u_char *) "Cookie",
+                                       sizeof("Cookie") - 1)
+                       == 0)
+                {
+                    sep = ';';
+
+                } else {
+                    sep = ',';
+                }
+
+                for (hn = header[i].next; hn; hn = hn->next) {
+                    *b->last++ = sep;
+                    *b->last++ = ' ';
+                    b->last = ngx_copy(b->last, hn->value.data, hn->value.len);
+                }
+            }
 
             ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                            "uwsgi param: \"%*s: %*s\"",
                            key_len, b->last - (key_len + 2 + val_len),
                            val_len, b->last - val_len);
-        next:
+        next_value:
 
             continue;
         }


From pluknet at nginx.com  Mon May 30 22:32:22 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:22 +0000
Subject: [nginx] Combining unknown headers during variables lookup (ticket
 #1316).
Message-ID: <hg.08b3ea81ff5f.1653949942.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/08b3ea81ff5f
branches:  
changeset: 8023:08b3ea81ff5f
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:32 2022 +0300
description:
Combining unknown headers during variables lookup (ticket #1316).

Previously, $http_*, $sent_http_*, $sent_trailer_*, $upstream_http_*,
and $upstream_trailer_* variables returned only the first header (with
a few specially handled exceptions: $http_cookie, $http_x_forwarded_for,
$sent_http_cache_control, $sent_http_link).

With this change, all headers are returned, combined together.  For
example, $http_foo variable will be "a, b" if there are "Foo: a" and
"Foo: b" headers in the request.

Note that $upstream_http_set_cookie will also return all "Set-Cookie"
headers (ticket #1843), though this might not be what one want, since
the "Set-Cookie" header does not follow the list syntax (see RFC 7230,
section 3.2.2).

diffstat:

 src/http/ngx_http_upstream.c  |   4 +-
 src/http/ngx_http_variables.c |  86 ++++++++++++++++++++++++++++++++++--------
 src/http/ngx_http_variables.h |   5 +-
 3 files changed, 74 insertions(+), 21 deletions(-)

diffs (179 lines):

diff -r 8b7a96fdd54c -r 08b3ea81ff5f src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Mon May 30 21:25:30 2022 +0300
+++ b/src/http/ngx_http_upstream.c	Mon May 30 21:25:32 2022 +0300
@@ -5703,7 +5703,7 @@ ngx_http_upstream_header_variable(ngx_ht
         return NGX_OK;
     }
 
-    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
+    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
                                          &r->upstream->headers_in.headers.part,
                                          sizeof("upstream_http_") - 1);
 }
@@ -5718,7 +5718,7 @@ ngx_http_upstream_trailer_variable(ngx_h
         return NGX_OK;
     }
 
-    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
+    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
                                         &r->upstream->headers_in.trailers.part,
                                         sizeof("upstream_trailer_") - 1);
 }
diff -r 8b7a96fdd54c -r 08b3ea81ff5f src/http/ngx_http_variables.c
--- a/src/http/ngx_http_variables.c	Mon May 30 21:25:30 2022 +0300
+++ b/src/http/ngx_http_variables.c	Mon May 30 21:25:32 2022 +0300
@@ -919,7 +919,7 @@ static ngx_int_t
 ngx_http_variable_unknown_header_in(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data)
 {
-    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
+    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
                                             &r->headers_in.headers.part,
                                             sizeof("http_") - 1);
 }
@@ -929,7 +929,7 @@ static ngx_int_t
 ngx_http_variable_unknown_header_out(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data)
 {
-    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
+    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
                                             &r->headers_out.headers.part,
                                             sizeof("sent_http_") - 1);
 }
@@ -939,19 +939,26 @@ static ngx_int_t
 ngx_http_variable_unknown_trailer_out(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data)
 {
-    return ngx_http_variable_unknown_header(v, (ngx_str_t *) data,
+    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
                                             &r->headers_out.trailers.part,
                                             sizeof("sent_trailer_") - 1);
 }
 
 
 ngx_int_t
-ngx_http_variable_unknown_header(ngx_http_variable_value_t *v, ngx_str_t *var,
+ngx_http_variable_unknown_header(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, ngx_str_t *var,
     ngx_list_part_t *part, size_t prefix)
 {
-    u_char            ch;
+    u_char           *p, ch;
+    size_t            len;
     ngx_uint_t        i, n;
-    ngx_table_elt_t  *header;
+    ngx_table_elt_t  *header, *h, **ph;
+
+    ph = &h;
+#if (NGX_SUPPRESS_WARN)
+    len = 0;
+#endif
 
     header = part->elts;
 
@@ -971,7 +978,11 @@ ngx_http_variable_unknown_header(ngx_htt
             continue;
         }
 
-        for (n = 0; n + prefix < var->len && n < header[i].key.len; n++) {
+        if (header[i].key.len != var->len - prefix) {
+            continue;
+        }
+
+        for (n = 0; n < var->len - prefix; n++) {
             ch = header[i].key.data[n];
 
             if (ch >= 'A' && ch <= 'Z') {
@@ -986,18 +997,59 @@ ngx_http_variable_unknown_header(ngx_htt
             }
         }
 
-        if (n + prefix == var->len && n == header[i].key.len) {
-            v->len = header[i].value.len;
-            v->valid = 1;
-            v->no_cacheable = 0;
-            v->not_found = 0;
-            v->data = header[i].value.data;
-
-            return NGX_OK;
+        if (n != var->len - prefix) {
+            continue;
         }
+
+        len += header[i].value.len + 2;
+
+        *ph = &header[i];
+        ph = &header[i].next;
     }
 
-    v->not_found = 1;
+    *ph = NULL;
+
+    if (h == NULL) {
+        v->not_found = 1;
+        return NGX_OK;
+    }
+
+    len -= 2;
+
+    if (h->next == NULL) {
+
+        v->len = h->value.len;
+        v->valid = 1;
+        v->no_cacheable = 0;
+        v->not_found = 0;
+        v->data = h->value.data;
+
+        return NGX_OK;
+    }
+
+    p = ngx_pnalloc(r->pool, len);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
+
+    v->len = len;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+    v->data = p;
+
+    for ( ;; ) {
+
+        p = ngx_copy(p, h->value.data, h->value.len);
+
+        if (h->next == NULL) {
+            break;
+        }
+
+        *p++ = ','; *p++ = ' ';
+
+        h = h->next;
+    }
 
     return NGX_OK;
 }
@@ -1879,7 +1931,7 @@ ngx_http_variable_sent_location(ngx_http
 
     ngx_str_set(&name, "sent_http_location");
 
-    return ngx_http_variable_unknown_header(v, &name,
+    return ngx_http_variable_unknown_header(r, v, &name,
                                             &r->headers_out.headers.part,
                                             sizeof("sent_http_") - 1);
 }
diff -r 8b7a96fdd54c -r 08b3ea81ff5f src/http/ngx_http_variables.h
--- a/src/http/ngx_http_variables.h	Mon May 30 21:25:30 2022 +0300
+++ b/src/http/ngx_http_variables.h	Mon May 30 21:25:32 2022 +0300
@@ -57,8 +57,9 @@ ngx_http_variable_value_t *ngx_http_get_
 ngx_http_variable_value_t *ngx_http_get_variable(ngx_http_request_t *r,
     ngx_str_t *name, ngx_uint_t key);
 
-ngx_int_t ngx_http_variable_unknown_header(ngx_http_variable_value_t *v,
-    ngx_str_t *var, ngx_list_part_t *part, size_t prefix);
+ngx_int_t ngx_http_variable_unknown_header(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, ngx_str_t *var, ngx_list_part_t *part,
+    size_t prefix);
 
 
 #if (NGX_PCRE)


From pluknet at nginx.com  Mon May 30 22:32:25 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:25 +0000
Subject: [nginx] Reworked multi headers to use linked lists.
Message-ID: <hg.ef6a3a99a81a.1653949945.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/ef6a3a99a81a
branches:  
changeset: 8024:ef6a3a99a81a
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:33 2022 +0300
description:
Reworked multi headers to use linked lists.

Multi headers are now using linked lists instead of arrays.  Notably,
the following fields were changed: r->headers_in.cookies (renamed
to r->headers_in.cookie), r->headers_in.x_forwarded_for,
r->headers_out.cache_control, r->headers_out.link, u->headers_in.cache_control
u->headers_in.cookies (renamed to u->headers_in.set_cookie).

The r->headers_in.cookies and u->headers_in.cookies fields were renamed
to r->headers_in.cookie and u->headers_in.set_cookie to match header names.

The ngx_http_parse_multi_header_lines() and ngx_http_parse_set_cookie_lines()
functions were changed accordingly.

With this change, multi headers are now essentially equivalent to normal
headers, and following changes will further make them equivalent.

diffstat:

 src/http/modules/ngx_http_geo_module.c            |   6 +-
 src/http/modules/ngx_http_geoip_module.c          |  12 +-
 src/http/modules/ngx_http_headers_filter_module.c |  51 ++++-----------
 src/http/modules/ngx_http_proxy_module.c          |  18 ++---
 src/http/modules/ngx_http_realip_module.c         |   7 +-
 src/http/modules/ngx_http_userid_filter_module.c  |  19 ++---
 src/http/modules/perl/nginx.xs                    |  22 ++----
 src/http/ngx_http.h                               |   8 +-
 src/http/ngx_http_core_module.c                   |  54 ++++++++++------
 src/http/ngx_http_core_module.h                   |   2 +-
 src/http/ngx_http_parse.c                         |  72 ++++++++++------------
 src/http/ngx_http_request.c                       |  28 +++-----
 src/http/ngx_http_request.h                       |  12 +-
 src/http/ngx_http_upstream.c                      |  60 ++++--------------
 src/http/ngx_http_upstream.h                      |   4 +-
 src/http/ngx_http_variables.c                     |  49 ++++++--------
 src/http/v2/ngx_http_v2_filter_module.c           |  28 ++++----
 17 files changed, 191 insertions(+), 261 deletions(-)

diffs (truncated from 1018 to 1000 lines):

diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/modules/ngx_http_geo_module.c
--- a/src/http/modules/ngx_http_geo_module.c	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/modules/ngx_http_geo_module.c	Mon May 30 21:25:33 2022 +0300
@@ -327,15 +327,15 @@ static ngx_int_t
 ngx_http_geo_addr(ngx_http_request_t *r, ngx_http_geo_ctx_t *ctx,
     ngx_addr_t *addr)
 {
-    ngx_array_t  *xfwd;
+    ngx_table_elt_t  *xfwd;
 
     if (ngx_http_geo_real_addr(r, ctx, addr) != NGX_OK) {
         return NGX_ERROR;
     }
 
-    xfwd = &r->headers_in.x_forwarded_for;
+    xfwd = r->headers_in.x_forwarded_for;
 
-    if (xfwd->nelts > 0 && ctx->proxies != NULL) {
+    if (xfwd != NULL && ctx->proxies != NULL) {
         (void) ngx_http_get_forwarded_addr(r, addr, xfwd, NULL,
                                            ctx->proxies, ctx->proxy_recursive);
     }
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/modules/ngx_http_geoip_module.c
--- a/src/http/modules/ngx_http_geoip_module.c	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/modules/ngx_http_geoip_module.c	Mon May 30 21:25:33 2022 +0300
@@ -240,16 +240,16 @@ static u_long
 ngx_http_geoip_addr(ngx_http_request_t *r, ngx_http_geoip_conf_t *gcf)
 {
     ngx_addr_t           addr;
-    ngx_array_t         *xfwd;
+    ngx_table_elt_t     *xfwd;
     struct sockaddr_in  *sin;
 
     addr.sockaddr = r->connection->sockaddr;
     addr.socklen = r->connection->socklen;
     /* addr.name = r->connection->addr_text; */
 
-    xfwd = &r->headers_in.x_forwarded_for;
+    xfwd = r->headers_in.x_forwarded_for;
 
-    if (xfwd->nelts > 0 && gcf->proxies != NULL) {
+    if (xfwd != NULL && gcf->proxies != NULL) {
         (void) ngx_http_get_forwarded_addr(r, &addr, xfwd, NULL,
                                            gcf->proxies, gcf->proxy_recursive);
     }
@@ -292,7 +292,7 @@ static geoipv6_t
 ngx_http_geoip_addr_v6(ngx_http_request_t *r, ngx_http_geoip_conf_t *gcf)
 {
     ngx_addr_t            addr;
-    ngx_array_t          *xfwd;
+    ngx_table_elt_t      *xfwd;
     in_addr_t             addr4;
     struct in6_addr       addr6;
     struct sockaddr_in   *sin;
@@ -302,9 +302,9 @@ ngx_http_geoip_addr_v6(ngx_http_request_
     addr.socklen = r->connection->socklen;
     /* addr.name = r->connection->addr_text; */
 
-    xfwd = &r->headers_in.x_forwarded_for;
+    xfwd = r->headers_in.x_forwarded_for;
 
-    if (xfwd->nelts > 0 && gcf->proxies != NULL) {
+    if (xfwd != NULL && gcf->proxies != NULL) {
         (void) ngx_http_get_forwarded_addr(r, &addr, xfwd, NULL,
                                            gcf->proxies, gcf->proxy_recursive);
     }
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/modules/ngx_http_headers_filter_module.c
--- a/src/http/modules/ngx_http_headers_filter_module.c	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/modules/ngx_http_headers_filter_module.c	Mon May 30 21:25:33 2022 +0300
@@ -329,8 +329,7 @@ ngx_http_set_expires(ngx_http_request_t 
     time_t               now, expires_time, max_age;
     ngx_str_t            value;
     ngx_int_t            rc;
-    ngx_uint_t           i;
-    ngx_table_elt_t     *e, *cc, **ccp;
+    ngx_table_elt_t     *e, *cc;
     ngx_http_expires_t   expires;
 
     expires = conf->expires;
@@ -371,38 +370,28 @@ ngx_http_set_expires(ngx_http_request_t 
     len = sizeof("Mon, 28 Sep 1970 06:00:00 GMT");
     e->value.len = len - 1;
 
-    ccp = r->headers_out.cache_control.elts;
-
-    if (ccp == NULL) {
+    cc = r->headers_out.cache_control;
 
-        if (ngx_array_init(&r->headers_out.cache_control, r->pool,
-                           1, sizeof(ngx_table_elt_t *))
-            != NGX_OK)
-        {
-            return NGX_ERROR;
-        }
+    if (cc == NULL) {
 
         cc = ngx_list_push(&r->headers_out.headers);
         if (cc == NULL) {
             return NGX_ERROR;
         }
 
+        r->headers_out.cache_control = cc;
+        cc->next = NULL;
+
         cc->hash = 1;
         ngx_str_set(&cc->key, "Cache-Control");
 
-        ccp = ngx_array_push(&r->headers_out.cache_control);
-        if (ccp == NULL) {
-            return NGX_ERROR;
+    } else {
+        for (cc = cc->next; cc; cc = cc->next) {
+            cc->hash = 0;
         }
 
-        *ccp = cc;
-
-    } else {
-        for (i = 1; i < r->headers_out.cache_control.nelts; i++) {
-            ccp[i]->hash = 0;
-        }
-
-        cc = ccp[0];
+        cc = r->headers_out.cache_control;
+        cc->next = NULL;
     }
 
     if (expires == NGX_HTTP_EXPIRES_EPOCH) {
@@ -564,22 +553,12 @@ static ngx_int_t
 ngx_http_add_multi_header_lines(ngx_http_request_t *r,
     ngx_http_header_val_t *hv, ngx_str_t *value)
 {
-    ngx_array_t      *pa;
     ngx_table_elt_t  *h, **ph;
 
     if (value->len == 0) {
         return NGX_OK;
     }
 
-    pa = (ngx_array_t *) ((char *) &r->headers_out + hv->offset);
-
-    if (pa->elts == NULL) {
-        if (ngx_array_init(pa, r->pool, 1, sizeof(ngx_table_elt_t *)) != NGX_OK)
-        {
-            return NGX_ERROR;
-        }
-    }
-
     h = ngx_list_push(&r->headers_out.headers);
     if (h == NULL) {
         return NGX_ERROR;
@@ -589,12 +568,12 @@ ngx_http_add_multi_header_lines(ngx_http
     h->key = hv->key;
     h->value = *value;
 
-    ph = ngx_array_push(pa);
-    if (ph == NULL) {
-        return NGX_ERROR;
-    }
+    ph = (ngx_table_elt_t **) ((char *) &r->headers_out + hv->offset);
+
+    while (*ph) { ph = &(*ph)->next; }
 
     *ph = h;
+    h->next = NULL;
 
     return NGX_OK;
 }
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/modules/ngx_http_proxy_module.c
--- a/src/http/modules/ngx_http_proxy_module.c	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/modules/ngx_http_proxy_module.c	Mon May 30 21:25:33 2022 +0300
@@ -2559,22 +2559,20 @@ static ngx_int_t
 ngx_http_proxy_add_x_forwarded_for_variable(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data)
 {
-    size_t             len;
-    u_char            *p;
-    ngx_uint_t         i, n;
-    ngx_table_elt_t  **h;
+    size_t            len;
+    u_char           *p;
+    ngx_table_elt_t  *h, *xfwd;
 
     v->valid = 1;
     v->no_cacheable = 0;
     v->not_found = 0;
 
-    n = r->headers_in.x_forwarded_for.nelts;
-    h = r->headers_in.x_forwarded_for.elts;
+    xfwd = r->headers_in.x_forwarded_for;
 
     len = 0;
 
-    for (i = 0; i < n; i++) {
-        len += h[i]->value.len + sizeof(", ") - 1;
+    for (h = xfwd; h; h = h->next) {
+        len += h->value.len + sizeof(", ") - 1;
     }
 
     if (len == 0) {
@@ -2593,8 +2591,8 @@ ngx_http_proxy_add_x_forwarded_for_varia
     v->len = len;
     v->data = p;
 
-    for (i = 0; i < n; i++) {
-        p = ngx_copy(p, h[i]->value.data, h[i]->value.len);
+    for (h = xfwd; h; h = h->next) {
+        p = ngx_copy(p, h->value.data, h->value.len);
         *p++ = ','; *p++ = ' ';
     }
 
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/modules/ngx_http_realip_module.c
--- a/src/http/modules/ngx_http_realip_module.c	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/modules/ngx_http_realip_module.c	Mon May 30 21:25:33 2022 +0300
@@ -134,9 +134,8 @@ ngx_http_realip_handler(ngx_http_request
     ngx_str_t                   *value;
     ngx_uint_t                   i, hash;
     ngx_addr_t                   addr;
-    ngx_array_t                 *xfwd;
     ngx_list_part_t             *part;
-    ngx_table_elt_t             *header;
+    ngx_table_elt_t             *header, *xfwd;
     ngx_connection_t            *c;
     ngx_http_realip_ctx_t       *ctx;
     ngx_http_realip_loc_conf_t  *rlcf;
@@ -168,9 +167,9 @@ ngx_http_realip_handler(ngx_http_request
 
     case NGX_HTTP_REALIP_XFWD:
 
-        xfwd = &r->headers_in.x_forwarded_for;
+        xfwd = r->headers_in.x_forwarded_for;
 
-        if (xfwd->elts == NULL) {
+        if (xfwd == NULL) {
             return NGX_DECLINED;
         }
 
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/modules/ngx_http_userid_filter_module.c
--- a/src/http/modules/ngx_http_userid_filter_module.c	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/modules/ngx_http_userid_filter_module.c	Mon May 30 21:25:33 2022 +0300
@@ -319,10 +319,9 @@ ngx_http_userid_set_variable(ngx_http_re
 static ngx_http_userid_ctx_t *
 ngx_http_userid_get_uid(ngx_http_request_t *r, ngx_http_userid_conf_t *conf)
 {
-    ngx_int_t                n;
-    ngx_str_t                src, dst;
-    ngx_table_elt_t        **cookies;
-    ngx_http_userid_ctx_t   *ctx;
+    ngx_str_t               src, dst;
+    ngx_table_elt_t        *cookie;
+    ngx_http_userid_ctx_t  *ctx;
 
     ctx = ngx_http_get_module_ctx(r, ngx_http_userid_filter_module);
 
@@ -339,9 +338,9 @@ ngx_http_userid_get_uid(ngx_http_request
         ngx_http_set_ctx(r, ctx, ngx_http_userid_filter_module);
     }
 
-    n = ngx_http_parse_multi_header_lines(&r->headers_in.cookies, &conf->name,
-                                          &ctx->cookie);
-    if (n == NGX_DECLINED) {
+    cookie = ngx_http_parse_multi_header_lines(r, r->headers_in.cookie,
+                                               &conf->name, &ctx->cookie);
+    if (cookie == NULL) {
         return ctx;
     }
 
@@ -349,10 +348,9 @@ ngx_http_userid_get_uid(ngx_http_request
                    "uid cookie: \"%V\"", &ctx->cookie);
 
     if (ctx->cookie.len < 22) {
-        cookies = r->headers_in.cookies.elts;
         ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                       "client sent too short userid cookie \"%V\"",
-                      &cookies[n]->value);
+                      &cookie->value);
         return ctx;
     }
 
@@ -370,10 +368,9 @@ ngx_http_userid_get_uid(ngx_http_request
     dst.data = (u_char *) ctx->uid_got;
 
     if (ngx_decode_base64(&dst, &src) == NGX_ERROR) {
-        cookies = r->headers_in.cookies.elts;
         ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                       "client sent invalid userid cookie \"%V\"",
-                      &cookies[n]->value);
+                      &cookie->value);
         return ctx;
     }
 
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/modules/perl/nginx.xs
--- a/src/http/modules/perl/nginx.xs	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/modules/perl/nginx.xs	Mon May 30 21:25:33 2022 +0300
@@ -302,7 +302,7 @@ header_in(r, key)
 
     if (hh) {
 
-        if (hh->offset == offsetof(ngx_http_headers_in_t, cookies)) {
+        if (hh->offset == offsetof(ngx_http_headers_in_t, cookie)) {
             sep = ';';
             goto multi;
         }
@@ -327,17 +327,13 @@ header_in(r, key)
 
         /* Cookie, X-Forwarded-For */
 
-        a = (ngx_array_t *) ((char *) &r->headers_in + hh->offset);
+        ph = (ngx_table_elt_t **) ((char *) &r->headers_in + hh->offset);
 
-        n = a->nelts;
-
-        if (n == 0) {
+        if (*ph == NULL) {
             XSRETURN_UNDEF;
         }
 
-        ph = a->elts;
-
-        if (n == 1) {
+        if ((*ph)->next == NULL) {
             ngx_http_perl_set_targ((*ph)->value.data, (*ph)->value.len);
 
             goto done;
@@ -345,8 +341,8 @@ header_in(r, key)
 
         size = - (ssize_t) (sizeof("; ") - 1);
 
-        for (i = 0; i < n; i++) {
-            size += ph[i]->value.len + sizeof("; ") - 1;
+        for (h = *ph; h; h = h->next) {
+            size += h->value.len + sizeof("; ") - 1;
         }
 
         value = ngx_pnalloc(r->pool, size);
@@ -357,10 +353,10 @@ header_in(r, key)
 
         p = value;
 
-        for (i = 0; /* void */ ; i++) {
-            p = ngx_copy(p, ph[i]->value.data, ph[i]->value.len);
+        for (h = *ph; h; h = h->next) {
+            p = ngx_copy(p, h->value.data, h->value.len);
 
-            if (i == n - 1) {
+            if (h->next == NULL) {
                 break;
             }
 
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/ngx_http.h
--- a/src/http/ngx_http.h	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/ngx_http.h	Mon May 30 21:25:33 2022 +0300
@@ -103,10 +103,10 @@ ngx_int_t ngx_http_parse_unsafe_uri(ngx_
     ngx_str_t *args, ngx_uint_t *flags);
 ngx_int_t ngx_http_parse_header_line(ngx_http_request_t *r, ngx_buf_t *b,
     ngx_uint_t allow_underscores);
-ngx_int_t ngx_http_parse_multi_header_lines(ngx_array_t *headers,
-    ngx_str_t *name, ngx_str_t *value);
-ngx_int_t ngx_http_parse_set_cookie_lines(ngx_array_t *headers,
-    ngx_str_t *name, ngx_str_t *value);
+ngx_table_elt_t *ngx_http_parse_multi_header_lines(ngx_http_request_t *r,
+    ngx_table_elt_t *headers, ngx_str_t *name, ngx_str_t *value);
+ngx_table_elt_t *ngx_http_parse_set_cookie_lines(ngx_http_request_t *r,
+    ngx_table_elt_t *headers, ngx_str_t *name, ngx_str_t *value);
 ngx_int_t ngx_http_arg(ngx_http_request_t *r, u_char *name, size_t len,
     ngx_str_t *value);
 void ngx_http_split_args(ngx_http_request_t *r, ngx_str_t *uri,
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/ngx_http_core_module.c	Mon May 30 21:25:33 2022 +0300
@@ -2024,8 +2024,7 @@ ngx_http_gzip_ok(ngx_http_request_t *r)
 {
     time_t                     date, expires;
     ngx_uint_t                 p;
-    ngx_array_t               *cc;
-    ngx_table_elt_t           *e, *d, *ae;
+    ngx_table_elt_t           *e, *d, *ae, *cc;
     ngx_http_core_loc_conf_t  *clcf;
 
     r->gzip_tested = 1;
@@ -2118,30 +2117,30 @@ ngx_http_gzip_ok(ngx_http_request_t *r)
         return NGX_DECLINED;
     }
 
-    cc = &r->headers_out.cache_control;
-
-    if (cc->elts) {
+    cc = r->headers_out.cache_control;
+
+    if (cc) {
 
         if ((p & NGX_HTTP_GZIP_PROXIED_NO_CACHE)
-            && ngx_http_parse_multi_header_lines(cc, &ngx_http_gzip_no_cache,
+            && ngx_http_parse_multi_header_lines(r, cc, &ngx_http_gzip_no_cache,
                                                  NULL)
-               >= 0)
+               != NULL)
         {
             goto ok;
         }
 
         if ((p & NGX_HTTP_GZIP_PROXIED_NO_STORE)
-            && ngx_http_parse_multi_header_lines(cc, &ngx_http_gzip_no_store,
+            && ngx_http_parse_multi_header_lines(r, cc, &ngx_http_gzip_no_store,
                                                  NULL)
-               >= 0)
+               != NULL)
         {
             goto ok;
         }
 
         if ((p & NGX_HTTP_GZIP_PROXIED_PRIVATE)
-            && ngx_http_parse_multi_header_lines(cc, &ngx_http_gzip_private,
+            && ngx_http_parse_multi_header_lines(r, cc, &ngx_http_gzip_private,
                                                  NULL)
-               >= 0)
+               != NULL)
         {
             goto ok;
         }
@@ -2712,12 +2711,12 @@ ngx_http_set_disable_symlinks(ngx_http_r
 
 ngx_int_t
 ngx_http_get_forwarded_addr(ngx_http_request_t *r, ngx_addr_t *addr,
-    ngx_array_t *headers, ngx_str_t *value, ngx_array_t *proxies,
+    ngx_table_elt_t *headers, ngx_str_t *value, ngx_array_t *proxies,
     int recursive)
 {
-    ngx_int_t          rc;
-    ngx_uint_t         i, found;
-    ngx_table_elt_t  **h;
+    ngx_int_t         rc;
+    ngx_uint_t        found;
+    ngx_table_elt_t  *h, *next;
 
     if (headers == NULL) {
         return ngx_http_get_forwarded_addr_internal(r, addr, value->data,
@@ -2725,16 +2724,23 @@ ngx_http_get_forwarded_addr(ngx_http_req
                                                     recursive);
     }
 
-    i = headers->nelts;
-    h = headers->elts;
+    /* revert headers order */
+
+    for (h = headers, headers = NULL; h; h = next) {
+        next = h->next;
+        h->next = headers;
+        headers = h;
+    }
+
+    /* iterate over all headers in reverse order */
 
     rc = NGX_DECLINED;
 
     found = 0;
 
-    while (i-- > 0) {
-        rc = ngx_http_get_forwarded_addr_internal(r, addr, h[i]->value.data,
-                                                  h[i]->value.len, proxies,
+    for (h = headers; h; h = h->next) {
+        rc = ngx_http_get_forwarded_addr_internal(r, addr, h->value.data,
+                                                  h->value.len, proxies,
                                                   recursive);
 
         if (!recursive) {
@@ -2753,6 +2759,14 @@ ngx_http_get_forwarded_addr(ngx_http_req
         found = 1;
     }
 
+    /* restore headers order */
+
+    for (h = headers, headers = NULL; h; h = next) {
+        next = h->next;
+        h->next = headers;
+        headers = h;
+    }
+
     return rc;
 }
 
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/ngx_http_core_module.h
--- a/src/http/ngx_http_core_module.h	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/ngx_http_core_module.h	Mon May 30 21:25:33 2022 +0300
@@ -529,7 +529,7 @@ ngx_int_t ngx_http_set_disable_symlinks(
     ngx_http_core_loc_conf_t *clcf, ngx_str_t *path, ngx_open_file_info_t *of);
 
 ngx_int_t ngx_http_get_forwarded_addr(ngx_http_request_t *r, ngx_addr_t *addr,
-    ngx_array_t *headers, ngx_str_t *value, ngx_array_t *proxies,
+    ngx_table_elt_t *headers, ngx_str_t *value, ngx_array_t *proxies,
     int recursive);
 
 ngx_int_t ngx_http_link_multi_headers(ngx_http_request_t *r);
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/ngx_http_parse.c
--- a/src/http/ngx_http_parse.c	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/ngx_http_parse.c	Mon May 30 21:25:33 2022 +0300
@@ -1960,27 +1960,24 @@ unsafe:
 }
 
 
-ngx_int_t
-ngx_http_parse_multi_header_lines(ngx_array_t *headers, ngx_str_t *name,
-    ngx_str_t *value)
+ngx_table_elt_t *
+ngx_http_parse_multi_header_lines(ngx_http_request_t *r,
+    ngx_table_elt_t *headers, ngx_str_t *name, ngx_str_t *value)
 {
-    ngx_uint_t         i;
-    u_char            *start, *last, *end, ch;
-    ngx_table_elt_t  **h;
-
-    h = headers->elts;
-
-    for (i = 0; i < headers->nelts; i++) {
-
-        ngx_log_debug2(NGX_LOG_DEBUG_HTTP, headers->pool->log, 0,
-                       "parse header: \"%V: %V\"", &h[i]->key, &h[i]->value);
-
-        if (name->len > h[i]->value.len) {
+    u_char           *start, *last, *end, ch;
+    ngx_table_elt_t  *h;
+
+    for (h = headers; h; h = h->next) {
+
+        ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
+                       "parse header: \"%V: %V\"", &h->key, &h->value);
+
+        if (name->len > h->value.len) {
             continue;
         }
 
-        start = h[i]->value.data;
-        end = h[i]->value.data + h[i]->value.len;
+        start = h->value.data;
+        end = h->value.data + h->value.len;
 
         while (start < end) {
 
@@ -1994,7 +1991,7 @@ ngx_http_parse_multi_header_lines(ngx_ar
 
             if (value == NULL) {
                 if (start == end || *start == ',') {
-                    return i;
+                    return h;
                 }
 
                 goto skip;
@@ -2014,7 +2011,7 @@ ngx_http_parse_multi_header_lines(ngx_ar
             value->len = last - start;
             value->data = start;
 
-            return i;
+            return h;
 
         skip:
 
@@ -2029,31 +2026,28 @@ ngx_http_parse_multi_header_lines(ngx_ar
         }
     }
 
-    return NGX_DECLINED;
+    return NULL;
 }
 
 
-ngx_int_t
-ngx_http_parse_set_cookie_lines(ngx_array_t *headers, ngx_str_t *name,
-    ngx_str_t *value)
+ngx_table_elt_t *
+ngx_http_parse_set_cookie_lines(ngx_http_request_t *r,
+    ngx_table_elt_t *headers, ngx_str_t *name, ngx_str_t *value)
 {
-    ngx_uint_t         i;
-    u_char            *start, *last, *end;
-    ngx_table_elt_t  **h;
-
-    h = headers->elts;
-
-    for (i = 0; i < headers->nelts; i++) {
-
-        ngx_log_debug2(NGX_LOG_DEBUG_HTTP, headers->pool->log, 0,
-                       "parse header: \"%V: %V\"", &h[i]->key, &h[i]->value);
-
-        if (name->len >= h[i]->value.len) {
+    u_char           *start, *last, *end;
+    ngx_table_elt_t  *h;
+
+    for (h = headers; h; h = h->next) {
+
+        ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
+                       "parse header: \"%V: %V\"", &h->key, &h->value);
+
+        if (name->len >= h->value.len) {
             continue;
         }
 
-        start = h[i]->value.data;
-        end = h[i]->value.data + h[i]->value.len;
+        start = h->value.data;
+        end = h->value.data + h->value.len;
 
         if (ngx_strncasecmp(start, name->data, name->len) != 0) {
             continue;
@@ -2077,10 +2071,10 @@ ngx_http_parse_set_cookie_lines(ngx_arra
         value->len = last - start;
         value->data = start;
 
-        return i;
+        return h;
     }
 
-    return NGX_DECLINED;
+    return NULL;
 }
 
 
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/ngx_http_request.c	Mon May 30 21:25:33 2022 +0300
@@ -196,7 +196,7 @@ ngx_http_header_t  ngx_http_headers_in[]
                  ngx_http_process_header_line },
 #endif
 
-    { ngx_string("Cookie"), offsetof(ngx_http_headers_in_t, cookies),
+    { ngx_string("Cookie"), offsetof(ngx_http_headers_in_t, cookie),
                  ngx_http_process_multi_header_lines },
 
     { ngx_null_string, 0, NULL }
@@ -1744,6 +1744,7 @@ ngx_http_process_header_line(ngx_http_re
 
     if (*ph == NULL) {
         *ph = h;
+        h->next = NULL;
     }
 
     return NGX_OK;
@@ -1760,6 +1761,7 @@ ngx_http_process_unique_header_line(ngx_
 
     if (*ph == NULL) {
         *ph = h;
+        h->next = NULL;
         return NGX_OK;
     }
 
@@ -1792,6 +1794,7 @@ ngx_http_process_host(ngx_http_request_t
     }
 
     r->headers_in.host = h;
+    h->next = NULL;
 
     host = h->value;
 
@@ -1853,6 +1856,7 @@ ngx_http_process_user_agent(ngx_http_req
     }
 
     r->headers_in.user_agent = h;
+    h->next = NULL;
 
     /* check some widespread browsers while the header is in CPU cache */
 
@@ -1919,27 +1923,15 @@ static ngx_int_t
 ngx_http_process_multi_header_lines(ngx_http_request_t *r, ngx_table_elt_t *h,
     ngx_uint_t offset)
 {
-    ngx_array_t       *headers;
     ngx_table_elt_t  **ph;
 
-    headers = (ngx_array_t *) ((char *) &r->headers_in + offset);
-
-    if (headers->elts == NULL) {
-        if (ngx_array_init(headers, r->pool, 1, sizeof(ngx_table_elt_t *))
-            != NGX_OK)
-        {
-            ngx_http_close_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
-            return NGX_ERROR;
-        }
-    }
-
-    ph = ngx_array_push(headers);
-    if (ph == NULL) {
-        ngx_http_close_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
-        return NGX_ERROR;
-    }
+    ph = (ngx_table_elt_t **) ((char *) &r->headers_in + offset);
+
+    while (*ph) { ph = &(*ph)->next; }
 
     *ph = h;
+    h->next = NULL;
+
     return NGX_OK;
 }
 
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/ngx_http_request.h	Mon May 30 21:25:33 2022 +0300
@@ -212,7 +212,7 @@ typedef struct {
     ngx_table_elt_t                  *keep_alive;
 
 #if (NGX_HTTP_X_FORWARDED_FOR)
-    ngx_array_t                       x_forwarded_for;
+    ngx_table_elt_t                  *x_forwarded_for;
 #endif
 
 #if (NGX_HTTP_REALIP)
@@ -231,11 +231,11 @@ typedef struct {
     ngx_table_elt_t                  *date;
 #endif
 
+    ngx_table_elt_t                  *cookie;
+
     ngx_str_t                         user;
     ngx_str_t                         passwd;
 
-    ngx_array_t                       cookies;
-
     ngx_str_t                         server;
     off_t                             content_length_n;
     time_t                            keep_alive_n;
@@ -274,6 +274,9 @@ typedef struct {
     ngx_table_elt_t                  *expires;
     ngx_table_elt_t                  *etag;
 
+    ngx_table_elt_t                  *cache_control;
+    ngx_table_elt_t                  *link;
+
     ngx_str_t                        *override_charset;
 
     size_t                            content_type_len;
@@ -282,9 +285,6 @@ typedef struct {
     u_char                           *content_type_lowcase;
     ngx_uint_t                        content_type_hash;
 
-    ngx_array_t                       cache_control;
-    ngx_array_t                       link;
-
     off_t                             content_length_n;
     off_t                             content_offset;
     time_t                            date_time;
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/ngx_http_upstream.c	Mon May 30 21:25:33 2022 +0300
@@ -246,7 +246,7 @@ static ngx_http_upstream_header_t  ngx_h
 
     { ngx_string("Set-Cookie"),
                  ngx_http_upstream_process_set_cookie,
-                 offsetof(ngx_http_upstream_headers_in_t, cookies),
+                 offsetof(ngx_http_upstream_headers_in_t, set_cookie),
                  ngx_http_upstream_rewrite_set_cookie, 0, 1 },
 
     { ngx_string("Content-Disposition"),
@@ -4666,26 +4666,16 @@ static ngx_int_t
 ngx_http_upstream_process_set_cookie(ngx_http_request_t *r, ngx_table_elt_t *h,
     ngx_uint_t offset)
 {
-    ngx_array_t           *pa;
     ngx_table_elt_t      **ph;
     ngx_http_upstream_t   *u;
 
     u = r->upstream;
-    pa = &u->headers_in.cookies;
-
-    if (pa->elts == NULL) {
-        if (ngx_array_init(pa, r->pool, 1, sizeof(ngx_table_elt_t *)) != NGX_OK)
-        {
-            return NGX_ERROR;
-        }
-    }
-
-    ph = ngx_array_push(pa);
-    if (ph == NULL) {
-        return NGX_ERROR;
-    }
+    ph = &u->headers_in.set_cookie;
+
+    while (*ph) { ph = &(*ph)->next; }
 
     *ph = h;
+    h->next = NULL;
 
 #if (NGX_HTTP_CACHE)
     if (!(u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_SET_COOKIE)) {
@@ -4701,26 +4691,16 @@ static ngx_int_t
 ngx_http_upstream_process_cache_control(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset)
 {
-    ngx_array_t          *pa;
     ngx_table_elt_t     **ph;
     ngx_http_upstream_t  *u;
 
     u = r->upstream;
-    pa = &u->headers_in.cache_control;
-
-    if (pa->elts == NULL) {
-        if (ngx_array_init(pa, r->pool, 2, sizeof(ngx_table_elt_t *)) != NGX_OK)
-        {
-            return NGX_ERROR;
-        }
-    }
-
-    ph = ngx_array_push(pa);
-    if (ph == NULL) {
-        return NGX_ERROR;
-    }
+    ph = &u->headers_in.cache_control;
+
+    while (*ph) { ph = &(*ph)->next; }
 
     *ph = h;
+    h->next = NULL;
 
 #if (NGX_HTTP_CACHE)
     {
@@ -5103,18 +5083,8 @@ static ngx_int_t
 ngx_http_upstream_copy_multi_header_lines(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset)
 {
-    ngx_array_t      *pa;
     ngx_table_elt_t  *ho, **ph;
 
-    pa = (ngx_array_t *) ((char *) &r->headers_out + offset);
-
-    if (pa->elts == NULL) {
-        if (ngx_array_init(pa, r->pool, 2, sizeof(ngx_table_elt_t *)) != NGX_OK)
-        {
-            return NGX_ERROR;
-        }
-    }
-
     ho = ngx_list_push(&r->headers_out.headers);
     if (ho == NULL) {
         return NGX_ERROR;
@@ -5122,12 +5092,12 @@ ngx_http_upstream_copy_multi_header_line
 
     *ho = *h;
 
-    ph = ngx_array_push(pa);
-    if (ph == NULL) {
-        return NGX_ERROR;
-    }
+    ph = (ngx_table_elt_t **) ((char *) &r->headers_out + offset);
+
+    while (*ph) { ph = &(*ph)->next; }
 
     *ph = ho;
+    ho->next = NULL;
 
     return NGX_OK;
 }
@@ -5740,9 +5710,9 @@ ngx_http_upstream_cookie_variable(ngx_ht
     s.len = name->len - (sizeof("upstream_cookie_") - 1);
     s.data = name->data + sizeof("upstream_cookie_") - 1;
 
-    if (ngx_http_parse_set_cookie_lines(&r->upstream->headers_in.cookies,
+    if (ngx_http_parse_set_cookie_lines(r, r->upstream->headers_in.set_cookie,
                                         &s, &cookie)
-        == NGX_DECLINED)
+        == NULL)
     {
         v->not_found = 1;
         return NGX_OK;
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/ngx_http_upstream.h	Mon May 30 21:25:33 2022 +0300
@@ -289,8 +289,8 @@ typedef struct {
     ngx_table_elt_t                 *content_encoding;
 #endif
 
-    ngx_array_t                      cache_control;
-    ngx_array_t                      cookies;
+    ngx_table_elt_t                 *cache_control;
+    ngx_table_elt_t                 *set_cookie;
 
     off_t                            content_length_n;
     time_t                           last_modified_time;
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/ngx_http_variables.c
--- a/src/http/ngx_http_variables.c	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/ngx_http_variables.c	Mon May 30 21:25:33 2022 +0300
@@ -183,7 +183,7 @@ static ngx_http_variable_t  ngx_http_cor
 #endif
 
     { ngx_string("http_cookie"), NULL, ngx_http_variable_cookies,
-      offsetof(ngx_http_request_t, headers_in.cookies), 0, 0 },
+      offsetof(ngx_http_request_t, headers_in.cookie), 0, 0 },
 
     { ngx_string("content_length"), NULL, ngx_http_variable_content_length,
       0, 0, 0 },
@@ -846,26 +846,21 @@ static ngx_int_t
 ngx_http_variable_headers_internal(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data, u_char sep)
 {
-    size_t             len;
-    u_char            *p, *end;
-    ngx_uint_t         i, n;
-    ngx_array_t       *a;
-    ngx_table_elt_t  **h;
-
-    a = (ngx_array_t *) ((char *) r + data);
-
-    n = a->nelts;
-    h = a->elts;
+    size_t            len;
+    u_char           *p;
+    ngx_table_elt_t  *h, *th;
+
+    h = *(ngx_table_elt_t **) ((char *) r + data);
 
     len = 0;
 
-    for (i = 0; i < n; i++) {
-
-        if (h[i]->hash == 0) {
+    for (th = h; th; th = th->next) {
+
+        if (th->hash == 0) {
             continue;
         }
 
-        len += h[i]->value.len + 2;
+        len += th->value.len + 2;
     }
 
     if (len == 0) {
@@ -879,9 +874,9 @@ ngx_http_variable_headers_internal(ngx_h
     v->no_cacheable = 0;
     v->not_found = 0;
 
-    if (n == 1) {
-        v->len = (*h)->value.len;
-        v->data = (*h)->value.data;
+    if (h->next == NULL) {
+        v->len = h->value.len;
+        v->data = h->value.data;
 
         return NGX_OK;
     }
@@ -894,17 +889,15 @@ ngx_http_variable_headers_internal(ngx_h
     v->len = len;
     v->data = p;
 
-    end = p + len;
-
-    for (i = 0; /* void */ ; i++) {
-
-        if (h[i]->hash == 0) {
+    for (th = h; th; th = th->next) {
+
+        if (th->hash == 0) {
             continue;
         }
 
-        p = ngx_copy(p, h[i]->value.data, h[i]->value.len);
-
-        if (p == end) {
+        p = ngx_copy(p, th->value.data, th->value.len);
+
+        if (th->next == NULL) {
             break;
         }
 
@@ -1102,8 +1095,8 @@ ngx_http_variable_cookie(ngx_http_reques
     s.len = name->len - (sizeof("cookie_") - 1);
     s.data = name->data + sizeof("cookie_") - 1;
 
-    if (ngx_http_parse_multi_header_lines(&r->headers_in.cookies, &s, &cookie)
-        == NGX_DECLINED)
+    if (ngx_http_parse_multi_header_lines(r, r->headers_in.cookie, &s, &cookie)
+        == NULL)
     {
         v->not_found = 1;
         return NGX_OK;
diff -r 08b3ea81ff5f -r ef6a3a99a81a src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c	Mon May 30 21:25:32 2022 +0300
+++ b/src/http/v2/ngx_http_v2_filter_module.c	Mon May 30 21:25:33 2022 +0300
@@ -674,14 +674,14 @@ ngx_http_v2_header_filter(ngx_http_reque
 static ngx_int_t
 ngx_http_v2_push_resources(ngx_http_request_t *r)
 {
-    u_char                     *start, *end, *last;
-    ngx_int_t                   rc;
-    ngx_str_t                   path;
-    ngx_uint_t                  i, push;
-    ngx_table_elt_t           **h;
-    ngx_http_v2_loc_conf_t     *h2lcf;
-    ngx_http_complex_value_t   *pushes;
-    ngx_str_t                   binary[NGX_HTTP_V2_PUSH_HEADERS];
+    u_char                    *start, *end, *last;
+    ngx_int_t                  rc;
+    ngx_str_t                  path;
+    ngx_uint_t                 i, push;
+    ngx_table_elt_t           *h;
+    ngx_http_v2_loc_conf_t    *h2lcf;
+    ngx_http_complex_value_t  *pushes;
+    ngx_str_t                  binary[NGX_HTTP_V2_PUSH_HEADERS];
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                    "http2 push resources");
@@ -725,15 +725,13 @@ ngx_http_v2_push_resources(ngx_http_requ
         return NGX_OK;
     }


From pluknet at nginx.com  Mon May 30 22:32:28 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:28 +0000
Subject: [nginx] All non-unique input headers are now linked lists.
Message-ID: <hg.c263f9ffa1fd.1653949948.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/c263f9ffa1fd
branches:  
changeset: 8025:c263f9ffa1fd
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:35 2022 +0300
description:
All non-unique input headers are now linked lists.

The ngx_http_process_multi_header_lines() function is removed, as it is
exactly equivalent to ngx_http_process_header_line().  Similarly,
ngx_http_variable_header() is used instead of ngx_http_variable_headers().

diffstat:

 src/http/ngx_http_request.c   |  38 ++++++++------------------------------
 src/http/ngx_http_variables.c |  33 ++++-----------------------------
 2 files changed, 12 insertions(+), 59 deletions(-)

diffs (158 lines):

diff -r ef6a3a99a81a -r c263f9ffa1fd src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c	Mon May 30 21:25:33 2022 +0300
+++ b/src/http/ngx_http_request.c	Mon May 30 21:25:35 2022 +0300
@@ -22,8 +22,6 @@ static ngx_int_t ngx_http_process_header
     ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t ngx_http_process_unique_header_line(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset);
-static ngx_int_t ngx_http_process_multi_header_lines(ngx_http_request_t *r,
-    ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t ngx_http_process_host(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t ngx_http_process_connection(ngx_http_request_t *r,
@@ -164,7 +162,7 @@ ngx_http_header_t  ngx_http_headers_in[]
 #if (NGX_HTTP_X_FORWARDED_FOR)
     { ngx_string("X-Forwarded-For"),
                  offsetof(ngx_http_headers_in_t, x_forwarded_for),
-                 ngx_http_process_multi_header_lines },
+                 ngx_http_process_header_line },
 #endif
 
 #if (NGX_HTTP_REALIP)
@@ -197,7 +195,7 @@ ngx_http_header_t  ngx_http_headers_in[]
 #endif
 
     { ngx_string("Cookie"), offsetof(ngx_http_headers_in_t, cookie),
-                 ngx_http_process_multi_header_lines },
+                 ngx_http_process_header_line },
 
     { ngx_null_string, 0, NULL }
 };
@@ -1742,10 +1740,10 @@ ngx_http_process_header_line(ngx_http_re
 
     ph = (ngx_table_elt_t **) ((char *) &r->headers_in + offset);
 
-    if (*ph == NULL) {
-        *ph = h;
-        h->next = NULL;
-    }
+    while (*ph) { ph = &(*ph)->next; }
+
+    *ph = h;
+    h->next = NULL;
 
     return NGX_OK;
 }
@@ -1851,13 +1849,10 @@ ngx_http_process_user_agent(ngx_http_req
 {
     u_char  *user_agent, *msie;
 
-    if (r->headers_in.user_agent) {
-        return NGX_OK;
+    if (ngx_http_process_header_line(r, h, offset) != NGX_OK) {
+        return NGX_ERROR;
     }
 
-    r->headers_in.user_agent = h;
-    h->next = NULL;
-
     /* check some widespread browsers while the header is in CPU cache */
 
     user_agent = h->value.data;
@@ -1919,23 +1914,6 @@ ngx_http_process_user_agent(ngx_http_req
 }
 
 
-static ngx_int_t
-ngx_http_process_multi_header_lines(ngx_http_request_t *r, ngx_table_elt_t *h,
-    ngx_uint_t offset)
-{
-    ngx_table_elt_t  **ph;
-
-    ph = (ngx_table_elt_t **) ((char *) &r->headers_in + offset);
-
-    while (*ph) { ph = &(*ph)->next; }
-
-    *ph = h;
-    h->next = NULL;
-
-    return NGX_OK;
-}
-
-
 ngx_int_t
 ngx_http_process_request_header(ngx_http_request_t *r)
 {
diff -r ef6a3a99a81a -r c263f9ffa1fd src/http/ngx_http_variables.c
--- a/src/http/ngx_http_variables.c	Mon May 30 21:25:33 2022 +0300
+++ b/src/http/ngx_http_variables.c	Mon May 30 21:25:35 2022 +0300
@@ -27,8 +27,6 @@ static ngx_int_t ngx_http_variable_heade
 
 static ngx_int_t ngx_http_variable_cookies(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
-static ngx_int_t ngx_http_variable_headers(ngx_http_request_t *r,
-    ngx_http_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_http_variable_headers_internal(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data, u_char sep);
 
@@ -178,7 +176,7 @@ static ngx_http_variable_t  ngx_http_cor
 #endif
 
 #if (NGX_HTTP_X_FORWARDED_FOR)
-    { ngx_string("http_x_forwarded_for"), NULL, ngx_http_variable_headers,
+    { ngx_string("http_x_forwarded_for"), NULL, ngx_http_variable_header,
       offsetof(ngx_http_request_t, headers_in.x_forwarded_for), 0, 0 },
 #endif
 
@@ -327,10 +325,10 @@ static ngx_http_variable_t  ngx_http_cor
     { ngx_string("sent_http_transfer_encoding"), NULL,
       ngx_http_variable_sent_transfer_encoding, 0, 0, 0 },
 
-    { ngx_string("sent_http_cache_control"), NULL, ngx_http_variable_headers,
+    { ngx_string("sent_http_cache_control"), NULL, ngx_http_variable_header,
       offsetof(ngx_http_request_t, headers_out.cache_control), 0, 0 },
 
-    { ngx_string("sent_http_link"), NULL, ngx_http_variable_headers,
+    { ngx_string("sent_http_link"), NULL, ngx_http_variable_header,
       offsetof(ngx_http_request_t, headers_out.link), 0, 0 },
 
     { ngx_string("limit_rate"), ngx_http_variable_set_limit_rate,
@@ -807,22 +805,7 @@ static ngx_int_t
 ngx_http_variable_header(ngx_http_request_t *r, ngx_http_variable_value_t *v,
     uintptr_t data)
 {
-    ngx_table_elt_t  *h;
-
-    h = *(ngx_table_elt_t **) ((char *) r + data);
-
-    if (h) {
-        v->len = h->value.len;
-        v->valid = 1;
-        v->no_cacheable = 0;
-        v->not_found = 0;
-        v->data = h->value.data;
-
-    } else {
-        v->not_found = 1;
-    }
-
-    return NGX_OK;
+    return ngx_http_variable_headers_internal(r, v, data, ',');
 }
 
 
@@ -835,14 +818,6 @@ ngx_http_variable_cookies(ngx_http_reque
 
 
 static ngx_int_t
-ngx_http_variable_headers(ngx_http_request_t *r,
-    ngx_http_variable_value_t *v, uintptr_t data)
-{
-    return ngx_http_variable_headers_internal(r, v, data, ',');
-}
-
-
-static ngx_int_t
 ngx_http_variable_headers_internal(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data, u_char sep)
 {


From pluknet at nginx.com  Mon May 30 22:32:31 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:31 +0000
Subject: [nginx] Perl: all known input headers are handled identically.
Message-ID: <hg.8b3860b52bb3.1653949951.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/8b3860b52bb3
branches:  
changeset: 8026:8b3860b52bb3
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:36 2022 +0300
description:
Perl: all known input headers are handled identically.

As all known input headers are now linked lists, these are now handled
identically.  In particular, this makes it possible to access properly
combined values of headers not specifically handled previously, such
as "Via" or "Connection".

diffstat:

 src/http/modules/perl/nginx.xs |  22 ++--------------------
 1 files changed, 2 insertions(+), 20 deletions(-)

diffs (34 lines):

diff -r c263f9ffa1fd -r 8b3860b52bb3 src/http/modules/perl/nginx.xs
--- a/src/http/modules/perl/nginx.xs	Mon May 30 21:25:35 2022 +0300
+++ b/src/http/modules/perl/nginx.xs	Mon May 30 21:25:36 2022 +0300
@@ -304,28 +304,10 @@ header_in(r, key)
 
         if (hh->offset == offsetof(ngx_http_headers_in_t, cookie)) {
             sep = ';';
-            goto multi;
-        }
-#if (NGX_HTTP_X_FORWARDED_FOR)
-        if (hh->offset == offsetof(ngx_http_headers_in_t, x_forwarded_for)) {
+
+        } else {
             sep = ',';
-            goto multi;
         }
-#endif
-
-        ph = (ngx_table_elt_t **) ((char *) &r->headers_in + hh->offset);
-
-        if (*ph) {
-            ngx_http_perl_set_targ((*ph)->value.data, (*ph)->value.len);
-
-            goto done;
-        }
-
-        XSRETURN_UNDEF;
-
-    multi:
-
-        /* Cookie, X-Forwarded-For */
 
         ph = (ngx_table_elt_t **) ((char *) &r->headers_in + hh->offset);
 


From pluknet at nginx.com  Mon May 30 22:32:34 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:34 +0000
Subject: [nginx] Perl: combining unknown headers during $r->header_in() lookup.
Message-ID: <hg.ca78312db071.1653949954.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/ca78312db071
branches:  
changeset: 8027:ca78312db071
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:38 2022 +0300
description:
Perl: combining unknown headers during $r->header_in() lookup.

diffstat:

 src/http/modules/perl/nginx.xs |  84 ++++++++++++++++++++++--------------------
 1 files changed, 44 insertions(+), 40 deletions(-)

diffs (116 lines):

diff -r 8b3860b52bb3 -r ca78312db071 src/http/modules/perl/nginx.xs
--- a/src/http/modules/perl/nginx.xs	Mon May 30 21:25:36 2022 +0300
+++ b/src/http/modules/perl/nginx.xs	Mon May 30 21:25:38 2022 +0300
@@ -272,7 +272,7 @@ header_in(r, key)
     ngx_uint_t                  i, n, hash;
     ngx_array_t                *a;
     ngx_list_part_t            *part;
-    ngx_table_elt_t            *h, **ph;
+    ngx_table_elt_t            *h, *header, **ph;
     ngx_http_header_t          *hh;
     ngx_http_core_main_conf_t  *cmcf;
 
@@ -311,47 +311,14 @@ header_in(r, key)
 
         ph = (ngx_table_elt_t **) ((char *) &r->headers_in + hh->offset);
 
-        if (*ph == NULL) {
-            XSRETURN_UNDEF;
-        }
-
-        if ((*ph)->next == NULL) {
-            ngx_http_perl_set_targ((*ph)->value.data, (*ph)->value.len);
-
-            goto done;
-        }
-
-        size = - (ssize_t) (sizeof("; ") - 1);
-
-        for (h = *ph; h; h = h->next) {
-            size += h->value.len + sizeof("; ") - 1;
-        }
-
-        value = ngx_pnalloc(r->pool, size);
-        if (value == NULL) {
-            ctx->error = 1;
-            croak("ngx_pnalloc() failed");
-        }
-
-        p = value;
-
-        for (h = *ph; h; h = h->next) {
-            p = ngx_copy(p, h->value.data, h->value.len);
-
-            if (h->next == NULL) {
-                break;
-            }
-
-            *p++ = sep; *p++ = ' ';
-        }
-
-        ngx_http_perl_set_targ(value, size);
-
-        goto done;
+        goto found;
     }
 
     /* iterate over all headers */
 
+    sep = ',';
+    ph = &header;
+
     part = &r->headers_in.headers.part;
     h = part->elts;
 
@@ -373,12 +340,49 @@ header_in(r, key)
             continue;
         }
 
-        ngx_http_perl_set_targ(h[i].value.data, h[i].value.len);
+        *ph = &h[i];
+        ph = &h[i].next;
+    }
+
+    *ph = NULL;
+    ph = &header;
 
+    found:
+
+    if (*ph == NULL) {
+        XSRETURN_UNDEF;
+    }
+
+    if ((*ph)->next == NULL) {
+        ngx_http_perl_set_targ((*ph)->value.data, (*ph)->value.len);
         goto done;
     }
 
-    XSRETURN_UNDEF;
+    size = - (ssize_t) (sizeof("; ") - 1);
+
+    for (h = *ph; h; h = h->next) {
+        size += h->value.len + sizeof("; ") - 1;
+    }
+
+    value = ngx_pnalloc(r->pool, size);
+    if (value == NULL) {
+        ctx->error = 1;
+        croak("ngx_pnalloc() failed");
+    }
+
+    p = value;
+
+    for (h = *ph; h; h = h->next) {
+        p = ngx_copy(p, h->value.data, h->value.len);
+
+        if (h->next == NULL) {
+            break;
+        }
+
+        *p++ = sep; *p++ = ' ';
+    }
+
+    ngx_http_perl_set_targ(value, size);
 
     done:
 


From pluknet at nginx.com  Mon May 30 22:32:37 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:37 +0000
Subject: [nginx] Upstream: style.
Message-ID: <hg.27d12017f300.1653949957.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/27d12017f300
branches:  
changeset: 8028:27d12017f300
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:40 2022 +0300
description:
Upstream: style.

diffstat:

 src/http/ngx_http_upstream.c |  24 +++++++++++++++++-------
 1 files changed, 17 insertions(+), 7 deletions(-)

diffs (65 lines):

diff -r ca78312db071 -r 27d12017f300 src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Mon May 30 21:25:38 2022 +0300
+++ b/src/http/ngx_http_upstream.c	Mon May 30 21:25:40 2022 +0300
@@ -4691,8 +4691,8 @@ static ngx_int_t
 ngx_http_upstream_process_cache_control(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset)
 {
-    ngx_table_elt_t     **ph;
-    ngx_http_upstream_t  *u;
+    ngx_table_elt_t      **ph;
+    ngx_http_upstream_t   *u;
 
     u = r->upstream;
     ph = &u->headers_in.cache_control;
@@ -4980,7 +4980,11 @@ static ngx_int_t
 ngx_http_upstream_process_charset(ngx_http_request_t *r, ngx_table_elt_t *h,
     ngx_uint_t offset)
 {
-    if (r->upstream->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_XA_CHARSET) {
+    ngx_http_upstream_t  *u;
+
+    u = r->upstream;
+
+    if (u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_XA_CHARSET) {
         return NGX_OK;
     }
 
@@ -4994,13 +4998,16 @@ static ngx_int_t
 ngx_http_upstream_process_connection(ngx_http_request_t *r, ngx_table_elt_t *h,
     ngx_uint_t offset)
 {
-    r->upstream->headers_in.connection = h;
+    ngx_http_upstream_t  *u;
+
+    u = r->upstream;
+    u->headers_in.connection = h;
 
     if (ngx_strlcasestrn(h->value.data, h->value.data + h->value.len,
                          (u_char *) "close", 5 - 1)
         != NULL)
     {
-        r->upstream->headers_in.connection_close = 1;
+        u->headers_in.connection_close = 1;
     }
 
     return NGX_OK;
@@ -5011,13 +5018,16 @@ static ngx_int_t
 ngx_http_upstream_process_transfer_encoding(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset)
 {
-    r->upstream->headers_in.transfer_encoding = h;
+    ngx_http_upstream_t  *u;
+
+    u = r->upstream;
+    u->headers_in.transfer_encoding = h;
 
     if (ngx_strlcasestrn(h->value.data, h->value.data + h->value.len,
                          (u_char *) "chunked", 7 - 1)
         != NULL)
     {
-        r->upstream->headers_in.chunked = 1;
+        u->headers_in.chunked = 1;
     }
 
     return NGX_OK;


From pluknet at nginx.com  Mon May 30 22:32:39 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:39 +0000
Subject: [nginx] Upstream: simplified Content-Encoding handling.
Message-ID: <hg.e0027c8438b0.1653949959.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/e0027c8438b0
branches:  
changeset: 8029:e0027c8438b0
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:42 2022 +0300
description:
Upstream: simplified Content-Encoding handling.

Since introduction of offset handling in ngx_http_upstream_copy_header_line()
in revision 573:58475592100c, the ngx_http_upstream_copy_content_encoding()
function is no longer needed, as its behaviour is exactly equivalent to
ngx_http_upstream_copy_header_line() with appropriate offset.  As such,
the ngx_http_upstream_copy_content_encoding() function was removed.

Further, the u->headers_in.content_encoding field is not used anywhere,
so it was removed as well.

Further, Content-Encoding handling no longer depends on NGX_HTTP_GZIP,
as it can be used even without any gzip handling compiled in (for example,
in the charset filter).

diffstat:

 src/http/ngx_http_upstream.c |  36 +++---------------------------------
 src/http/ngx_http_upstream.h |   4 ----
 2 files changed, 3 insertions(+), 37 deletions(-)

diffs (75 lines):

diff -r 27d12017f300 -r e0027c8438b0 src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Mon May 30 21:25:40 2022 +0300
+++ b/src/http/ngx_http_upstream.c	Mon May 30 21:25:42 2022 +0300
@@ -147,11 +147,6 @@ static ngx_int_t ngx_http_upstream_rewri
 static ngx_int_t ngx_http_upstream_copy_allow_ranges(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset);
 
-#if (NGX_HTTP_GZIP)
-static ngx_int_t ngx_http_upstream_copy_content_encoding(ngx_http_request_t *r,
-    ngx_table_elt_t *h, ngx_uint_t offset);
-#endif
-
 static ngx_int_t ngx_http_upstream_add_variables(ngx_conf_t *cf);
 static ngx_int_t ngx_http_upstream_addr_variable(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
@@ -316,12 +311,10 @@ static ngx_http_upstream_header_t  ngx_h
                  ngx_http_upstream_process_transfer_encoding, 0,
                  ngx_http_upstream_ignore_header_line, 0, 0 },
 
-#if (NGX_HTTP_GZIP)
     { ngx_string("Content-Encoding"),
-                 ngx_http_upstream_process_header_line,
-                 offsetof(ngx_http_upstream_headers_in_t, content_encoding),
-                 ngx_http_upstream_copy_content_encoding, 0, 0 },
-#endif
+                 ngx_http_upstream_ignore_header_line, 0,
+                 ngx_http_upstream_copy_header_line,
+                 offsetof(ngx_http_headers_out_t, content_encoding), 0 },
 
     { ngx_null_string, NULL, 0, NULL, 0, 0 }
 };
@@ -5349,29 +5342,6 @@ ngx_http_upstream_copy_allow_ranges(ngx_
 }
 
 
-#if (NGX_HTTP_GZIP)
-
-static ngx_int_t
-ngx_http_upstream_copy_content_encoding(ngx_http_request_t *r,
-    ngx_table_elt_t *h, ngx_uint_t offset)
-{
-    ngx_table_elt_t  *ho;
-
-    ho = ngx_list_push(&r->headers_out.headers);
-    if (ho == NULL) {
-        return NGX_ERROR;
-    }
-
-    *ho = *h;
-
-    r->headers_out.content_encoding = ho;
-
-    return NGX_OK;
-}
-
-#endif
-
-
 static ngx_int_t
 ngx_http_upstream_add_variables(ngx_conf_t *cf)
 {
diff -r 27d12017f300 -r e0027c8438b0 src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h	Mon May 30 21:25:40 2022 +0300
+++ b/src/http/ngx_http_upstream.h	Mon May 30 21:25:42 2022 +0300
@@ -285,10 +285,6 @@ typedef struct {
     ngx_table_elt_t                 *transfer_encoding;
     ngx_table_elt_t                 *vary;
 
-#if (NGX_HTTP_GZIP)
-    ngx_table_elt_t                 *content_encoding;
-#endif
-
     ngx_table_elt_t                 *cache_control;
     ngx_table_elt_t                 *set_cookie;
 


From pluknet at nginx.com  Mon May 30 22:32:42 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:42 +0000
Subject: [nginx] Upstream: simplified Accept-Ranges handling.
Message-ID: <hg.cdc2724858ca.1653949962.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/cdc2724858ca
branches:  
changeset: 8030:cdc2724858ca
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:43 2022 +0300
description:
Upstream: simplified Accept-Ranges handling.

The u->headers_in.accept_ranges field is not used anywhere and hence removed.

diffstat:

 src/http/ngx_http_upstream.c |  3 +--
 src/http/ngx_http_upstream.h |  1 -
 2 files changed, 1 insertions(+), 3 deletions(-)

diffs (24 lines):

diff -r e0027c8438b0 -r cdc2724858ca src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Mon May 30 21:25:42 2022 +0300
+++ b/src/http/ngx_http_upstream.c	Mon May 30 21:25:43 2022 +0300
@@ -259,8 +259,7 @@ static ngx_http_upstream_header_t  ngx_h
                  offsetof(ngx_http_headers_out_t, expires), 1 },
 
     { ngx_string("Accept-Ranges"),
-                 ngx_http_upstream_process_header_line,
-                 offsetof(ngx_http_upstream_headers_in_t, accept_ranges),
+                 ngx_http_upstream_ignore_header_line, 0,
                  ngx_http_upstream_copy_allow_ranges,
                  offsetof(ngx_http_headers_out_t, accept_ranges), 1 },
 
diff -r e0027c8438b0 -r cdc2724858ca src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h	Mon May 30 21:25:42 2022 +0300
+++ b/src/http/ngx_http_upstream.h	Mon May 30 21:25:43 2022 +0300
@@ -280,7 +280,6 @@ typedef struct {
 
     ngx_table_elt_t                 *last_modified;
     ngx_table_elt_t                 *location;
-    ngx_table_elt_t                 *accept_ranges;
     ngx_table_elt_t                 *www_authenticate;
     ngx_table_elt_t                 *transfer_encoding;
     ngx_table_elt_t                 *vary;


From pluknet at nginx.com  Mon May 30 22:32:45 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:45 +0000
Subject: [nginx] All known output headers can be linked lists now.
Message-ID: <hg.d26db4f82d7d.1653949965.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/d26db4f82d7d
branches:  
changeset: 8031:d26db4f82d7d
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:45 2022 +0300
description:
All known output headers can be linked lists now.

The h->next pointer properly provided as NULL in all cases where known
output headers are added.

Note that there are 3rd party modules which might not do this, and it
might be risky to rely on this for arbitrary headers.

diffstat:

 src/http/modules/ngx_http_auth_basic_module.c     |  1 +
 src/http/modules/ngx_http_auth_request_module.c   |  1 +
 src/http/modules/ngx_http_dav_module.c            |  1 +
 src/http/modules/ngx_http_gzip_filter_module.c    |  1 +
 src/http/modules/ngx_http_gzip_static_module.c    |  1 +
 src/http/modules/ngx_http_headers_filter_module.c |  2 ++
 src/http/modules/ngx_http_memcached_module.c      |  1 +
 src/http/modules/ngx_http_range_filter_module.c   |  3 +++
 src/http/modules/ngx_http_static_module.c         |  1 +
 src/http/modules/perl/nginx.xs                    |  1 +
 src/http/ngx_http_core_module.c                   |  3 +++
 src/http/ngx_http_script.c                        |  1 +
 src/http/ngx_http_special_response.c              |  1 +
 src/http/ngx_http_upstream.c                      |  7 +++++++
 14 files changed, 25 insertions(+), 0 deletions(-)

diffs (242 lines):

diff -r cdc2724858ca -r d26db4f82d7d src/http/modules/ngx_http_auth_basic_module.c
--- a/src/http/modules/ngx_http_auth_basic_module.c	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/modules/ngx_http_auth_basic_module.c	Mon May 30 21:25:45 2022 +0300
@@ -339,6 +339,7 @@ ngx_http_auth_basic_set_realm(ngx_http_r
     *p = '"';
 
     r->headers_out.www_authenticate->hash = 1;
+    r->headers_out.www_authenticate->next = NULL;
     ngx_str_set(&r->headers_out.www_authenticate->key, "WWW-Authenticate");
     r->headers_out.www_authenticate->value.data = basic;
     r->headers_out.www_authenticate->value.len = len;
diff -r cdc2724858ca -r d26db4f82d7d src/http/modules/ngx_http_auth_request_module.c
--- a/src/http/modules/ngx_http_auth_request_module.c	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/modules/ngx_http_auth_request_module.c	Mon May 30 21:25:45 2022 +0300
@@ -154,6 +154,7 @@ ngx_http_auth_request_handler(ngx_http_r
                 }
 
                 *ho = *h;
+                ho->next = NULL;
 
                 r->headers_out.www_authenticate = ho;
             }
diff -r cdc2724858ca -r d26db4f82d7d src/http/modules/ngx_http_dav_module.c
--- a/src/http/modules/ngx_http_dav_module.c	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/modules/ngx_http_dav_module.c	Mon May 30 21:25:45 2022 +0300
@@ -1082,6 +1082,7 @@ ngx_http_dav_location(ngx_http_request_t
     }
 
     r->headers_out.location->hash = 1;
+    r->headers_out.location->next = NULL;
     ngx_str_set(&r->headers_out.location->key, "Location");
 
     escape = 2 * ngx_escape_uri(NULL, r->uri.data, r->uri.len, NGX_ESCAPE_URI);
diff -r cdc2724858ca -r d26db4f82d7d src/http/modules/ngx_http_gzip_filter_module.c
--- a/src/http/modules/ngx_http_gzip_filter_module.c	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/modules/ngx_http_gzip_filter_module.c	Mon May 30 21:25:45 2022 +0300
@@ -280,6 +280,7 @@ ngx_http_gzip_header_filter(ngx_http_req
     }
 
     h->hash = 1;
+    h->next = NULL;
     ngx_str_set(&h->key, "Content-Encoding");
     ngx_str_set(&h->value, "gzip");
     r->headers_out.content_encoding = h;
diff -r cdc2724858ca -r d26db4f82d7d src/http/modules/ngx_http_gzip_static_module.c
--- a/src/http/modules/ngx_http_gzip_static_module.c	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/modules/ngx_http_gzip_static_module.c	Mon May 30 21:25:45 2022 +0300
@@ -242,6 +242,7 @@ ngx_http_gzip_static_handler(ngx_http_re
     }
 
     h->hash = 1;
+    h->next = NULL;
     ngx_str_set(&h->key, "Content-Encoding");
     ngx_str_set(&h->value, "gzip");
     r->headers_out.content_encoding = h;
diff -r cdc2724858ca -r d26db4f82d7d src/http/modules/ngx_http_headers_filter_module.c
--- a/src/http/modules/ngx_http_headers_filter_module.c	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/modules/ngx_http_headers_filter_module.c	Mon May 30 21:25:45 2022 +0300
@@ -362,6 +362,7 @@ ngx_http_set_expires(ngx_http_request_t 
         }
 
         r->headers_out.expires = e;
+        e->next = NULL;
 
         e->hash = 1;
         ngx_str_set(&e->key, "Expires");
@@ -621,6 +622,7 @@ ngx_http_set_response_header(ngx_http_re
         }
 
         *old = h;
+        h->next = NULL;
     }
 
     h->hash = 1;
diff -r cdc2724858ca -r d26db4f82d7d src/http/modules/ngx_http_memcached_module.c
--- a/src/http/modules/ngx_http_memcached_module.c	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/modules/ngx_http_memcached_module.c	Mon May 30 21:25:45 2022 +0300
@@ -401,6 +401,7 @@ found:
             }
 
             h->hash = 1;
+            h->next = NULL;
             ngx_str_set(&h->key, "Content-Encoding");
             ngx_str_set(&h->value, "gzip");
             r->headers_out.content_encoding = h;
diff -r cdc2724858ca -r d26db4f82d7d src/http/modules/ngx_http_range_filter_module.c
--- a/src/http/modules/ngx_http_range_filter_module.c	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/modules/ngx_http_range_filter_module.c	Mon May 30 21:25:45 2022 +0300
@@ -258,6 +258,7 @@ next_filter:
     }
 
     r->headers_out.accept_ranges->hash = 1;
+    r->headers_out.accept_ranges->next = NULL;
     ngx_str_set(&r->headers_out.accept_ranges->key, "Accept-Ranges");
     ngx_str_set(&r->headers_out.accept_ranges->value, "bytes");
 
@@ -427,6 +428,7 @@ ngx_http_range_singlepart_header(ngx_htt
     r->headers_out.content_range = content_range;
 
     content_range->hash = 1;
+    content_range->next = NULL;
     ngx_str_set(&content_range->key, "Content-Range");
 
     content_range->value.data = ngx_pnalloc(r->pool,
@@ -599,6 +601,7 @@ ngx_http_range_not_satisfiable(ngx_http_
     r->headers_out.content_range = content_range;
 
     content_range->hash = 1;
+    content_range->next = NULL;
     ngx_str_set(&content_range->key, "Content-Range");
 
     content_range->value.data = ngx_pnalloc(r->pool,
diff -r cdc2724858ca -r d26db4f82d7d src/http/modules/ngx_http_static_module.c
--- a/src/http/modules/ngx_http_static_module.c	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/modules/ngx_http_static_module.c	Mon May 30 21:25:45 2022 +0300
@@ -195,6 +195,7 @@ ngx_http_static_handler(ngx_http_request
         }
 
         r->headers_out.location->hash = 1;
+        r->headers_out.location->next = NULL;
         ngx_str_set(&r->headers_out.location->key, "Location");
         r->headers_out.location->value.len = len;
         r->headers_out.location->value.data = location;
diff -r cdc2724858ca -r d26db4f82d7d src/http/modules/perl/nginx.xs
--- a/src/http/modules/perl/nginx.xs	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/modules/perl/nginx.xs	Mon May 30 21:25:45 2022 +0300
@@ -573,6 +573,7 @@ header_out(r, key, value)
     }
 
     header->hash = 1;
+    header->next = NULL;
 
     if (ngx_http_perl_sv2str(aTHX_ r, &header->key, key) != NGX_OK) {
         header->hash = 0;
diff -r cdc2724858ca -r d26db4f82d7d src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/ngx_http_core_module.c	Mon May 30 21:25:45 2022 +0300
@@ -1007,6 +1007,7 @@ ngx_http_core_find_config_phase(ngx_http
         }
 
         r->headers_out.location->hash = 1;
+        r->headers_out.location->next = NULL;
         ngx_str_set(&r->headers_out.location->key, "Location");
 
         if (r->args.len == 0) {
@@ -1687,6 +1688,7 @@ ngx_http_set_etag(ngx_http_request_t *r)
     }
 
     etag->hash = 1;
+    etag->next = NULL;
     ngx_str_set(&etag->key, "ETag");
 
     etag->value.data = ngx_pnalloc(r->pool, NGX_OFF_T_LEN + NGX_TIME_T_LEN + 3);
@@ -1781,6 +1783,7 @@ ngx_http_send_response(ngx_http_request_
         }
 
         r->headers_out.location->hash = 1;
+        r->headers_out.location->next = NULL;
         ngx_str_set(&r->headers_out.location->key, "Location");
         r->headers_out.location->value = val;
 
diff -r cdc2724858ca -r d26db4f82d7d src/http/ngx_http_script.c
--- a/src/http/ngx_http_script.c	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/ngx_http_script.c	Mon May 30 21:25:45 2022 +0300
@@ -1243,6 +1243,7 @@ ngx_http_script_regex_end_code(ngx_http_
         }
 
         r->headers_out.location->hash = 1;
+        r->headers_out.location->next = NULL;
         ngx_str_set(&r->headers_out.location->key, "Location");
         r->headers_out.location->value = e->buf;
 
diff -r cdc2724858ca -r d26db4f82d7d src/http/ngx_http_special_response.c
--- a/src/http/ngx_http_special_response.c	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/ngx_http_special_response.c	Mon May 30 21:25:45 2022 +0300
@@ -649,6 +649,7 @@ ngx_http_send_error_page(ngx_http_reques
     }
 
     location->hash = 1;
+    location->next = NULL;
     ngx_str_set(&location->key, "Location");
     location->value = uri;
 
diff -r cdc2724858ca -r d26db4f82d7d src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Mon May 30 21:25:43 2022 +0300
+++ b/src/http/ngx_http_upstream.c	Mon May 30 21:25:45 2022 +0300
@@ -2681,6 +2681,7 @@ ngx_http_upstream_intercept_errors(ngx_h
                 }
 
                 *h = *u->headers_in.www_authenticate;
+                h->next = NULL;
 
                 r->headers_out.www_authenticate = h;
             }
@@ -5075,6 +5076,7 @@ ngx_http_upstream_copy_header_line(ngx_h
     if (offset) {
         ph = (ngx_table_elt_t **) ((char *) &r->headers_out + offset);
         *ph = ho;
+        ho->next = NULL;
     }
 
     return NGX_OK;
@@ -5169,6 +5171,7 @@ ngx_http_upstream_copy_last_modified(ngx
     }
 
     *ho = *h;
+    ho->next = NULL;
 
     r->headers_out.last_modified = ho;
     r->headers_out.last_modified_time =
@@ -5191,6 +5194,7 @@ ngx_http_upstream_rewrite_location(ngx_h
     }
 
     *ho = *h;
+    ho->next = NULL;
 
     if (r->upstream->rewrite_redirect) {
         rc = r->upstream->rewrite_redirect(r, ho, 0);
@@ -5236,6 +5240,7 @@ ngx_http_upstream_rewrite_refresh(ngx_ht
     }
 
     *ho = *h;
+    ho->next = NULL;
 
     if (r->upstream->rewrite_redirect) {
 
@@ -5281,6 +5286,7 @@ ngx_http_upstream_rewrite_set_cookie(ngx
     }
 
     *ho = *h;
+    ho->next = NULL;
 
     if (r->upstream->rewrite_cookie) {
         rc = r->upstream->rewrite_cookie(r, ho);
@@ -5334,6 +5340,7 @@ ngx_http_upstream_copy_allow_ranges(ngx_
     }
 
     *ho = *h;
+    ho->next = NULL;
 
     r->headers_out.accept_ranges = ho;
 


From pluknet at nginx.com  Mon May 30 22:32:48 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:48 +0000
Subject: [nginx] Upstream: all known headers in u->headers_in are linked lists
 now.
Message-ID: <hg.2025aae94739.1653949968.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/2025aae94739
branches:  
changeset: 8032:2025aae94739
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:46 2022 +0300
description:
Upstream: all known headers in u->headers_in are linked lists now.

diffstat:

 src/http/modules/ngx_http_proxy_module.c |  2 ++
 src/http/ngx_http_upstream.c             |  9 +++++++++
 2 files changed, 11 insertions(+), 0 deletions(-)

diffs (94 lines):

diff -r d26db4f82d7d -r 2025aae94739 src/http/modules/ngx_http_proxy_module.c
--- a/src/http/modules/ngx_http_proxy_module.c	Mon May 30 21:25:45 2022 +0300
+++ b/src/http/modules/ngx_http_proxy_module.c	Mon May 30 21:25:46 2022 +0300
@@ -1965,6 +1965,7 @@ ngx_http_proxy_process_header(ngx_http_r
                 ngx_str_set(&h->key, "Server");
                 ngx_str_null(&h->value);
                 h->lowcase_key = (u_char *) "server";
+                h->next = NULL;
             }
 
             if (r->upstream->headers_in.date == NULL) {
@@ -1978,6 +1979,7 @@ ngx_http_proxy_process_header(ngx_http_r
                 ngx_str_set(&h->key, "Date");
                 ngx_str_null(&h->value);
                 h->lowcase_key = (u_char *) "date";
+                h->next = NULL;
             }
 
             /* clear content length if response is chunked */
diff -r d26db4f82d7d -r 2025aae94739 src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Mon May 30 21:25:45 2022 +0300
+++ b/src/http/ngx_http_upstream.c	Mon May 30 21:25:46 2022 +0300
@@ -4610,6 +4610,7 @@ ngx_http_upstream_process_header_line(ng
 
     if (*ph == NULL) {
         *ph = h;
+        h->next = NULL;
     }
 
     return NGX_OK;
@@ -4632,6 +4633,7 @@ ngx_http_upstream_process_content_length
 
     u = r->upstream;
 
+    h->next = NULL;
     u->headers_in.content_length = h;
     u->headers_in.content_length_n = ngx_atoof(h->value.data, h->value.len);
 
@@ -4647,6 +4649,7 @@ ngx_http_upstream_process_last_modified(
 
     u = r->upstream;
 
+    h->next = NULL;
     u->headers_in.last_modified = h;
     u->headers_in.last_modified_time = ngx_parse_http_time(h->value.data,
                                                            h->value.len);
@@ -4816,6 +4819,7 @@ ngx_http_upstream_process_expires(ngx_ht
 
     u = r->upstream;
     u->headers_in.expires = h;
+    h->next = NULL;
 
 #if (NGX_HTTP_CACHE)
     {
@@ -4856,6 +4860,7 @@ ngx_http_upstream_process_accel_expires(
 
     u = r->upstream;
     u->headers_in.x_accel_expires = h;
+    h->next = NULL;
 
 #if (NGX_HTTP_CACHE)
     {
@@ -4915,6 +4920,7 @@ ngx_http_upstream_process_limit_rate(ngx
 
     u = r->upstream;
     u->headers_in.x_accel_limit_rate = h;
+    h->next = NULL;
 
     if (u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_XA_LIMIT_RATE) {
         return NGX_OK;
@@ -4995,6 +5001,7 @@ ngx_http_upstream_process_connection(ngx
 
     u = r->upstream;
     u->headers_in.connection = h;
+    h->next = NULL;
 
     if (ngx_strlcasestrn(h->value.data, h->value.data + h->value.len,
                          (u_char *) "close", 5 - 1)
@@ -5015,6 +5022,7 @@ ngx_http_upstream_process_transfer_encod
 
     u = r->upstream;
     u->headers_in.transfer_encoding = h;
+    h->next = NULL;
 
     if (ngx_strlcasestrn(h->value.data, h->value.data + h->value.len,
                          (u_char *) "chunked", 7 - 1)
@@ -5035,6 +5043,7 @@ ngx_http_upstream_process_vary(ngx_http_
 
     u = r->upstream;
     u->headers_in.vary = h;
+    h->next = NULL;
 
 #if (NGX_HTTP_CACHE)
 


From pluknet at nginx.com  Mon May 30 22:32:51 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:51 +0000
Subject: [nginx] Upstream: header handlers can now return parsing errors.
Message-ID: <hg.2bf7792c262e.1653949971.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/2bf7792c262e
branches:  
changeset: 8033:2bf7792c262e
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:48 2022 +0300
description:
Upstream: header handlers can now return parsing errors.

With this change, duplicate Content-Length and Transfer-Encoding headers
are now rejected.  Further, responses with invalid Content-Length or
Transfer-Encoding headers are now rejected, as well as responses with both
Content-Length and Transfer-Encoding.

diffstat:

 src/http/modules/ngx_http_fastcgi_module.c |   8 +++-
 src/http/modules/ngx_http_grpc_module.c    |   8 +++-
 src/http/modules/ngx_http_proxy_module.c   |   8 +++-
 src/http/modules/ngx_http_scgi_module.c    |   8 +++-
 src/http/modules/ngx_http_uwsgi_module.c   |   8 +++-
 src/http/ngx_http_upstream.c               |  53 ++++++++++++++++++++++++++++-
 6 files changed, 80 insertions(+), 13 deletions(-)

diffs (169 lines):

diff -r 2025aae94739 -r 2bf7792c262e src/http/modules/ngx_http_fastcgi_module.c
--- a/src/http/modules/ngx_http_fastcgi_module.c	Mon May 30 21:25:46 2022 +0300
+++ b/src/http/modules/ngx_http_fastcgi_module.c	Mon May 30 21:25:48 2022 +0300
@@ -2007,8 +2007,12 @@ ngx_http_fastcgi_process_header(ngx_http
                 hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
                                    h->lowcase_key, h->key.len);
 
-                if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
-                    return NGX_ERROR;
+                if (hh) {
+                    rc = hh->handler(r, h, hh->offset);
+
+                    if (rc != NGX_OK) {
+                        return rc;
+                    }
                 }
 
                 ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
diff -r 2025aae94739 -r 2bf7792c262e src/http/modules/ngx_http_grpc_module.c
--- a/src/http/modules/ngx_http_grpc_module.c	Mon May 30 21:25:46 2022 +0300
+++ b/src/http/modules/ngx_http_grpc_module.c	Mon May 30 21:25:48 2022 +0300
@@ -1891,8 +1891,12 @@ ngx_http_grpc_process_header(ngx_http_re
                 hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
                                    h->lowcase_key, h->key.len);
 
-                if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
-                    return NGX_ERROR;
+                if (hh) {
+                    rc = hh->handler(r, h, hh->offset);
+
+                    if (rc != NGX_OK) {
+                        return rc;
+                    }
                 }
 
                 continue;
diff -r 2025aae94739 -r 2bf7792c262e src/http/modules/ngx_http_proxy_module.c
--- a/src/http/modules/ngx_http_proxy_module.c	Mon May 30 21:25:46 2022 +0300
+++ b/src/http/modules/ngx_http_proxy_module.c	Mon May 30 21:25:48 2022 +0300
@@ -1930,8 +1930,12 @@ ngx_http_proxy_process_header(ngx_http_r
             hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
                                h->lowcase_key, h->key.len);
 
-            if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
-                return NGX_ERROR;
+            if (hh) {
+                rc = hh->handler(r, h, hh->offset);
+
+                if (rc != NGX_OK) {
+                    return rc;
+                }
             }
 
             ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
diff -r 2025aae94739 -r 2bf7792c262e src/http/modules/ngx_http_scgi_module.c
--- a/src/http/modules/ngx_http_scgi_module.c	Mon May 30 21:25:46 2022 +0300
+++ b/src/http/modules/ngx_http_scgi_module.c	Mon May 30 21:25:48 2022 +0300
@@ -1114,8 +1114,12 @@ ngx_http_scgi_process_header(ngx_http_re
             hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
                                h->lowcase_key, h->key.len);
 
-            if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
-                return NGX_ERROR;
+            if (hh) {
+                rc = hh->handler(r, h, hh->offset);
+
+                if (rc != NGX_OK) {
+                    return rc;
+                }
             }
 
             ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
diff -r 2025aae94739 -r 2bf7792c262e src/http/modules/ngx_http_uwsgi_module.c
--- a/src/http/modules/ngx_http_uwsgi_module.c	Mon May 30 21:25:46 2022 +0300
+++ b/src/http/modules/ngx_http_uwsgi_module.c	Mon May 30 21:25:48 2022 +0300
@@ -1340,8 +1340,12 @@ ngx_http_uwsgi_process_header(ngx_http_r
             hh = ngx_hash_find(&umcf->headers_in_hash, h->hash,
                                h->lowcase_key, h->key.len);
 
-            if (hh && hh->handler(r, h, hh->offset) != NGX_OK) {
-                return NGX_ERROR;
+            if (hh) {
+                rc = hh->handler(r, h, hh->offset);
+
+                if (rc != NGX_OK) {
+                    return rc;
+                }
             }
 
             ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
diff -r 2025aae94739 -r 2bf7792c262e src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Mon May 30 21:25:46 2022 +0300
+++ b/src/http/ngx_http_upstream.c	Mon May 30 21:25:48 2022 +0300
@@ -4633,10 +4633,34 @@ ngx_http_upstream_process_content_length
 
     u = r->upstream;
 
+    if (u->headers_in.content_length) {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "upstream sent duplicate header line: \"%V: %V\", "
+                      "previous value: \"%V: %V\"",
+                      &h->key, &h->value,
+                      &u->headers_in.content_length->key,
+                      &u->headers_in.content_length->value);
+        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
+    }
+
+    if (u->headers_in.transfer_encoding) {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "upstream sent \"Content-Length\" and "
+                      "\"Transfer-Encoding\" headers at the same time");
+        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
+    }
+
     h->next = NULL;
     u->headers_in.content_length = h;
     u->headers_in.content_length_n = ngx_atoof(h->value.data, h->value.len);
 
+    if (u->headers_in.content_length_n == NGX_ERROR) {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "upstream sent invalid \"Content-Length\" header: "
+                      "\"%V: %V\"", &h->key, &h->value);
+        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
+    }
+
     return NGX_OK;
 }
 
@@ -5021,14 +5045,37 @@ ngx_http_upstream_process_transfer_encod
     ngx_http_upstream_t  *u;
 
     u = r->upstream;
+
+    if (u->headers_in.transfer_encoding) {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "upstream sent duplicate header line: \"%V: %V\", "
+                      "previous value: \"%V: %V\"",
+                      &h->key, &h->value,
+                      &u->headers_in.transfer_encoding->key,
+                      &u->headers_in.transfer_encoding->value);
+        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
+    }
+
+    if (u->headers_in.content_length) {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "upstream sent \"Content-Length\" and "
+                      "\"Transfer-Encoding\" headers at the same time");
+        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
+    }
+
     u->headers_in.transfer_encoding = h;
     h->next = NULL;
 
-    if (ngx_strlcasestrn(h->value.data, h->value.data + h->value.len,
-                         (u_char *) "chunked", 7 - 1)
-        != NULL)
+    if (h->value.len == 7
+        && ngx_strncasecmp(h->value.data, (u_char *) "chunked", 7) == 0)
     {
         u->headers_in.chunked = 1;
+
+    } else {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "upstream sent unknown \"Transfer-Encoding\": \"%V\"",
+                      &h->value);
+        return NGX_HTTP_UPSTREAM_INVALID_HEADER;
     }
 
     return NGX_OK;


From pluknet at nginx.com  Mon May 30 22:32:54 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:54 +0000
Subject: [nginx] Upstream: duplicate headers ignored or properly linked.
Message-ID: <hg.413dbda22f7d.1653949974.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/413dbda22f7d
branches:  
changeset: 8034:413dbda22f7d
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:49 2022 +0300
description:
Upstream: duplicate headers ignored or properly linked.

Most of the known duplicate upstream response headers are now ignored
with a warning.

If syntax permits multiple headers, these are now properly linked to
the lists, notably Vary and WWW-Authenticate.  This makes it possible
to further handle such lists where it makes sense.

diffstat:

 src/http/ngx_http_upstream.c |  114 +++++++++++++++++++++++++++++++++++++++---
 src/http/ngx_http_upstream.h |    1 +
 2 files changed, 105 insertions(+), 10 deletions(-)

diffs (216 lines):

diff -r 2bf7792c262e -r 413dbda22f7d src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Mon May 30 21:25:48 2022 +0300
+++ b/src/http/ngx_http_upstream.c	Mon May 30 21:25:49 2022 +0300
@@ -101,6 +101,9 @@ static void ngx_http_upstream_finalize_r
 
 static ngx_int_t ngx_http_upstream_process_header_line(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset);
+static ngx_int_t
+    ngx_http_upstream_process_multi_header_lines(ngx_http_request_t *r,
+    ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t ngx_http_upstream_process_content_length(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset);
 static ngx_int_t ngx_http_upstream_process_last_modified(ngx_http_request_t *r,
@@ -226,7 +229,7 @@ static ngx_http_upstream_header_t  ngx_h
                  offsetof(ngx_http_headers_out_t, server), 0 },
 
     { ngx_string("WWW-Authenticate"),
-                 ngx_http_upstream_process_header_line,
+                 ngx_http_upstream_process_multi_header_lines,
                  offsetof(ngx_http_upstream_headers_in_t, www_authenticate),
                  ngx_http_upstream_copy_header_line, 0, 0 },
 
@@ -236,7 +239,8 @@ static ngx_http_upstream_header_t  ngx_h
                  ngx_http_upstream_rewrite_location, 0, 0 },
 
     { ngx_string("Refresh"),
-                 ngx_http_upstream_ignore_header_line, 0,
+                 ngx_http_upstream_process_header_line,
+                 offsetof(ngx_http_upstream_headers_in_t, refresh),
                  ngx_http_upstream_rewrite_refresh, 0, 0 },
 
     { ngx_string("Set-Cookie"),
@@ -2804,6 +2808,10 @@ ngx_http_upstream_process_headers(ngx_ht
                 i = 0;
             }
 
+            if (h[i].hash == 0) {
+                continue;
+            }
+
             hh = ngx_hash_find(&umcf->headers_in_hash, h[i].hash,
                                h[i].lowcase_key, h[i].key.len);
 
@@ -2857,6 +2865,10 @@ ngx_http_upstream_process_headers(ngx_ht
             i = 0;
         }
 
+        if (h[i].hash == 0) {
+            continue;
+        }
+
         if (ngx_hash_find(&u->conf->hide_headers_hash, h[i].hash,
                           h[i].lowcase_key, h[i].key.len))
         {
@@ -4608,10 +4620,35 @@ ngx_http_upstream_process_header_line(ng
 
     ph = (ngx_table_elt_t **) ((char *) &r->upstream->headers_in + offset);
 
-    if (*ph == NULL) {
-        *ph = h;
-        h->next = NULL;
-    }
+    if (*ph) {
+        ngx_log_error(NGX_LOG_WARN, r->connection->log, 0,
+                      "upstream sent duplicate header line: \"%V: %V\", "
+                      "previous value: \"%V: %V\", ignored",
+                      &h->key, &h->value,
+                      &(*ph)->key, &(*ph)->value);
+        h->hash = 0;
+        return NGX_OK;
+    }
+
+    *ph = h;
+    h->next = NULL;
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_http_upstream_process_multi_header_lines(ngx_http_request_t *r,
+    ngx_table_elt_t *h, ngx_uint_t offset)
+{
+    ngx_table_elt_t  **ph;
+
+    ph = (ngx_table_elt_t **) ((char *) &r->upstream->headers_in + offset);
+
+    while (*ph) { ph = &(*ph)->next; }
+
+    *ph = h;
+    h->next = NULL;
 
     return NGX_OK;
 }
@@ -4673,6 +4710,17 @@ ngx_http_upstream_process_last_modified(
 
     u = r->upstream;
 
+    if (u->headers_in.last_modified) {
+        ngx_log_error(NGX_LOG_WARN, r->connection->log, 0,
+                      "upstream sent duplicate header line: \"%V: %V\", "
+                      "previous value: \"%V: %V\", ignored",
+                      &h->key, &h->value,
+                      &u->headers_in.last_modified->key,
+                      &u->headers_in.last_modified->value);
+        h->hash = 0;
+        return NGX_OK;
+    }
+
     h->next = NULL;
     u->headers_in.last_modified = h;
     u->headers_in.last_modified_time = ngx_parse_http_time(h->value.data,
@@ -4842,6 +4890,18 @@ ngx_http_upstream_process_expires(ngx_ht
     ngx_http_upstream_t  *u;
 
     u = r->upstream;
+
+    if (u->headers_in.expires) {
+        ngx_log_error(NGX_LOG_WARN, r->connection->log, 0,
+                      "upstream sent duplicate header line: \"%V: %V\", "
+                      "previous value: \"%V: %V\", ignored",
+                      &h->key, &h->value,
+                      &u->headers_in.expires->key,
+                      &u->headers_in.expires->value);
+        h->hash = 0;
+        return NGX_OK;
+    }
+
     u->headers_in.expires = h;
     h->next = NULL;
 
@@ -4883,6 +4943,18 @@ ngx_http_upstream_process_accel_expires(
     ngx_http_upstream_t  *u;
 
     u = r->upstream;
+
+    if (u->headers_in.x_accel_expires) {
+        ngx_log_error(NGX_LOG_WARN, r->connection->log, 0,
+                      "upstream sent duplicate header line: \"%V: %V\", "
+                      "previous value: \"%V: %V\", ignored",
+                      &h->key, &h->value,
+                      &u->headers_in.x_accel_expires->key,
+                      &u->headers_in.x_accel_expires->value);
+        h->hash = 0;
+        return NGX_OK;
+    }
+
     u->headers_in.x_accel_expires = h;
     h->next = NULL;
 
@@ -4943,6 +5015,18 @@ ngx_http_upstream_process_limit_rate(ngx
     ngx_http_upstream_t  *u;
 
     u = r->upstream;
+
+    if (u->headers_in.x_accel_limit_rate) {
+        ngx_log_error(NGX_LOG_WARN, r->connection->log, 0,
+                      "upstream sent duplicate header line: \"%V: %V\", "
+                      "previous value: \"%V: %V\", ignored",
+                      &h->key, &h->value,
+                      &u->headers_in.x_accel_limit_rate->key,
+                      &u->headers_in.x_accel_limit_rate->value);
+        h->hash = 0;
+        return NGX_OK;
+    }
+
     u->headers_in.x_accel_limit_rate = h;
     h->next = NULL;
 
@@ -5021,10 +5105,15 @@ static ngx_int_t
 ngx_http_upstream_process_connection(ngx_http_request_t *r, ngx_table_elt_t *h,
     ngx_uint_t offset)
 {
-    ngx_http_upstream_t  *u;
+    ngx_table_elt_t      **ph;
+    ngx_http_upstream_t   *u;
 
     u = r->upstream;
-    u->headers_in.connection = h;
+    ph = &u->headers_in.connection;
+
+    while (*ph) { ph = &(*ph)->next; }
+
+    *ph = h;
     h->next = NULL;
 
     if (ngx_strlcasestrn(h->value.data, h->value.data + h->value.len,
@@ -5086,10 +5175,15 @@ static ngx_int_t
 ngx_http_upstream_process_vary(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset)
 {
-    ngx_http_upstream_t  *u;
+    ngx_table_elt_t      **ph;
+    ngx_http_upstream_t   *u;
 
     u = r->upstream;
-    u->headers_in.vary = h;
+    ph = &u->headers_in.vary;
+
+    while (*ph) { ph = &(*ph)->next; }
+
+    *ph = h;
     h->next = NULL;
 
 #if (NGX_HTTP_CACHE)
diff -r 2bf7792c262e -r 413dbda22f7d src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h	Mon May 30 21:25:48 2022 +0300
+++ b/src/http/ngx_http_upstream.h	Mon May 30 21:25:49 2022 +0300
@@ -280,6 +280,7 @@ typedef struct {
 
     ngx_table_elt_t                 *last_modified;
     ngx_table_elt_t                 *location;
+    ngx_table_elt_t                 *refresh;
     ngx_table_elt_t                 *www_authenticate;
     ngx_table_elt_t                 *transfer_encoding;
     ngx_table_elt_t                 *vary;


From pluknet at nginx.com  Mon May 30 22:32:57 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:57 +0000
Subject: [nginx] Upstream: handling of multiple Vary headers (ticket #1423).
Message-ID: <hg.cd73509f21e2.1653949977.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/cd73509f21e2
branches:  
changeset: 8035:cd73509f21e2
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:51 2022 +0300
description:
Upstream: handling of multiple Vary headers (ticket #1423).

Previously, only the last header value was used when caching.

diffstat:

 src/http/ngx_http_upstream.c |  48 +++++++++++++++++++++++++++++++++++++++----
 1 files changed, 43 insertions(+), 5 deletions(-)

diffs (71 lines):

diff -r 413dbda22f7d -r cd73509f21e2 src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Mon May 30 21:25:49 2022 +0300
+++ b/src/http/ngx_http_upstream.c	Mon May 30 21:25:51 2022 +0300
@@ -5175,6 +5175,9 @@ static ngx_int_t
 ngx_http_upstream_process_vary(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset)
 {
+    u_char                *p;
+    size_t                 len;
+    ngx_str_t              vary;
     ngx_table_elt_t      **ph;
     ngx_http_upstream_t   *u;
 
@@ -5192,17 +5195,52 @@ ngx_http_upstream_process_vary(ngx_http_
         return NGX_OK;
     }
 
-    if (r->cache == NULL) {
+    if (r->cache == NULL || !u->cacheable) {
+        return NGX_OK;
+    }
+
+    if (h->value.len == 1 && h->value.data[0] == '*') {
+        u->cacheable = 0;
         return NGX_OK;
     }
 
-    if (h->value.len > NGX_HTTP_CACHE_VARY_LEN
-        || (h->value.len == 1 && h->value.data[0] == '*'))
-    {
+    if (u->headers_in.vary->next) {
+
+        len = 0;
+
+        for (h = u->headers_in.vary; h; h = h->next) {
+            len += h->value.len + 2;
+        }
+
+        len -= 2;
+
+        p = ngx_pnalloc(r->pool, len);
+        if (p == NULL) {
+            return NGX_ERROR;
+        }
+
+        vary.len = len;
+        vary.data = p;
+
+        for (h = u->headers_in.vary; h; h = h->next) {
+            p = ngx_copy(p, h->value.data, h->value.len);
+
+            if (h->next == NULL) {
+                break;
+            }
+
+            *p++ = ','; *p++ = ' ';
+        }
+
+    } else {
+        vary = h->value;
+    }
+
+    if (vary.len > NGX_HTTP_CACHE_VARY_LEN) {
         u->cacheable = 0;
     }
 
-    r->cache->vary = h->value;
+    r->cache->vary = vary;
 
 #endif
 


From pluknet at nginx.com  Mon May 30 22:32:59 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:32:59 +0000
Subject: [nginx] Upstream: multiple WWW-Authenticate headers (ticket #485).
Message-ID: <hg.f739c8142fb2.1653949979.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/f739c8142fb2
branches:  
changeset: 8036:f739c8142fb2
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:53 2022 +0300
description:
Upstream: multiple WWW-Authenticate headers (ticket #485).

When using proxy_intercept_errors and an error page for error 401
(Unauthorized), multiple WWW-Authenticate headers from the upstream server
response are now properly copied to the response.

diffstat:

 src/http/ngx_http_upstream.c |  30 +++++++++++++++++++-----------
 1 files changed, 19 insertions(+), 11 deletions(-)

diffs (49 lines):

diff -r cd73509f21e2 -r f739c8142fb2 src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Mon May 30 21:25:51 2022 +0300
+++ b/src/http/ngx_http_upstream.c	Mon May 30 21:25:53 2022 +0300
@@ -2647,7 +2647,7 @@ ngx_http_upstream_intercept_errors(ngx_h
 {
     ngx_int_t                  status;
     ngx_uint_t                 i;
-    ngx_table_elt_t           *h;
+    ngx_table_elt_t           *h, *ho, **ph;
     ngx_http_err_page_t       *err_page;
     ngx_http_core_loc_conf_t  *clcf;
 
@@ -2676,18 +2676,26 @@ ngx_http_upstream_intercept_errors(ngx_h
             if (status == NGX_HTTP_UNAUTHORIZED
                 && u->headers_in.www_authenticate)
             {
-                h = ngx_list_push(&r->headers_out.headers);
-
-                if (h == NULL) {
-                    ngx_http_upstream_finalize_request(r, u,
+                h = u->headers_in.www_authenticate;
+                ph = &r->headers_out.www_authenticate;
+
+                while (h) {
+                    ho = ngx_list_push(&r->headers_out.headers);
+
+                    if (ho == NULL) {
+                        ngx_http_upstream_finalize_request(r, u,
                                                NGX_HTTP_INTERNAL_SERVER_ERROR);
-                    return NGX_OK;
+                        return NGX_OK;
+                    }
+
+                    *ho = *h;
+                    ho->next = NULL;
+
+                    *ph = ho;
+                    ph = &ho->next;
+
+                    h = h->next;
                 }
-
-                *h = *u->headers_in.www_authenticate;
-                h->next = NULL;
-
-                r->headers_out.www_authenticate = h;
             }
 
 #if (NGX_HTTP_CACHE)


From pluknet at nginx.com  Mon May 30 22:33:02 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:33:02 +0000
Subject: [nginx] Auth request: multiple WWW-Authenticate headers (ticket #485).
Message-ID: <hg.8272c823a7d0.1653949982.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/8272c823a7d0
branches:  
changeset: 8037:8272c823a7d0
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:54 2022 +0300
description:
Auth request: multiple WWW-Authenticate headers (ticket #485).

When using auth_request with an upstream server which returns 401
(Unauthorized), multiple WWW-Authenticate headers from the upstream server
response are now properly copied to the response.

diffstat:

 src/http/modules/ngx_http_auth_request_module.c |  11 ++++++++---
 1 files changed, 8 insertions(+), 3 deletions(-)

diffs (35 lines):

diff -r f739c8142fb2 -r 8272c823a7d0 src/http/modules/ngx_http_auth_request_module.c
--- a/src/http/modules/ngx_http_auth_request_module.c	Mon May 30 21:25:53 2022 +0300
+++ b/src/http/modules/ngx_http_auth_request_module.c	Mon May 30 21:25:54 2022 +0300
@@ -101,7 +101,7 @@ ngx_module_t  ngx_http_auth_request_modu
 static ngx_int_t
 ngx_http_auth_request_handler(ngx_http_request_t *r)
 {
-    ngx_table_elt_t               *h, *ho;
+    ngx_table_elt_t               *h, *ho, **ph;
     ngx_http_request_t            *sr;
     ngx_http_post_subrequest_t    *ps;
     ngx_http_auth_request_ctx_t   *ctx;
@@ -147,7 +147,9 @@ ngx_http_auth_request_handler(ngx_http_r
                 h = sr->upstream->headers_in.www_authenticate;
             }
 
-            if (h) {
+            ph = &r->headers_out.www_authenticate;
+
+            while (h) {
                 ho = ngx_list_push(&r->headers_out.headers);
                 if (ho == NULL) {
                     return NGX_ERROR;
@@ -156,7 +158,10 @@ ngx_http_auth_request_handler(ngx_http_r
                 *ho = *h;
                 ho->next = NULL;
 
-                r->headers_out.www_authenticate = ho;
+                *ph = ho;
+                ph = &ho->next;
+
+                h = h->next;
             }
 
             return ctx->status;


From pluknet at nginx.com  Mon May 30 22:33:05 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:33:05 +0000
Subject: [nginx] Multiple WWW-Authenticate headers with "satisfy any;".
Message-ID: <hg.711737177b77.1653949985.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/711737177b77
branches:  
changeset: 8038:711737177b77
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:56 2022 +0300
description:
Multiple WWW-Authenticate headers with "satisfy any;".

If a module adds multiple WWW-Authenticate headers (ticket #485) to the
response, linked in r->headers_out.www_authenticate, all headers are now
cleared if another module later allows access.

This change is a nop for standard modules, since the only access module which
can add multiple WWW-Authenticate headers is the auth request module, and
it is checked after other standard access modules.  Though this might
affect some third party access modules.

Note that if a 3rd party module adds a single WWW-Authenticate header
and not yet modified to set the header's next pointer to NULL, attempt to
clear such a header with this change will result in a segmentation fault.

diffstat:

 src/http/ngx_http_core_module.c |  5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diffs (22 lines):

diff -r 8272c823a7d0 -r 711737177b77 src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c	Mon May 30 21:25:54 2022 +0300
+++ b/src/http/ngx_http_core_module.c	Mon May 30 21:25:56 2022 +0300
@@ -1088,6 +1088,7 @@ ngx_int_t
 ngx_http_core_access_phase(ngx_http_request_t *r, ngx_http_phase_handler_t *ph)
 {
     ngx_int_t                  rc;
+    ngx_table_elt_t           *h;
     ngx_http_core_loc_conf_t  *clcf;
 
     if (r != r->main) {
@@ -1122,8 +1123,8 @@ ngx_http_core_access_phase(ngx_http_requ
         if (rc == NGX_OK) {
             r->access_code = 0;
 
-            if (r->headers_out.www_authenticate) {
-                r->headers_out.www_authenticate->hash = 0;
+            for (h = r->headers_out.www_authenticate; h; h = h->next) {
+                h->hash = 0;
             }
 
             r->phase_handler = ph->next;


From pluknet at nginx.com  Mon May 30 22:33:08 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:33:08 +0000
Subject: [nginx] Headers filter: improved memory allocation error handling.
Message-ID: <hg.e64a1f32065b.1653949988.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/e64a1f32065b
branches:  
changeset: 8039:e64a1f32065b
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon May 30 21:25:57 2022 +0300
description:
Headers filter: improved memory allocation error handling.

diffstat:

 src/http/modules/ngx_http_headers_filter_module.c |  4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diffs (28 lines):

diff -r 711737177b77 -r e64a1f32065b src/http/modules/ngx_http_headers_filter_module.c
--- a/src/http/modules/ngx_http_headers_filter_module.c	Mon May 30 21:25:56 2022 +0300
+++ b/src/http/modules/ngx_http_headers_filter_module.c	Mon May 30 21:25:57 2022 +0300
@@ -377,6 +377,7 @@ ngx_http_set_expires(ngx_http_request_t 
 
         cc = ngx_list_push(&r->headers_out.headers);
         if (cc == NULL) {
+            e->hash = 0;
             return NGX_ERROR;
         }
 
@@ -410,6 +411,8 @@ ngx_http_set_expires(ngx_http_request_t 
 
     e->value.data = ngx_pnalloc(r->pool, len);
     if (e->value.data == NULL) {
+        e->hash = 0;
+        cc->hash = 0;
         return NGX_ERROR;
     }
 
@@ -447,6 +450,7 @@ ngx_http_set_expires(ngx_http_request_t 
     cc->value.data = ngx_pnalloc(r->pool,
                                  sizeof("max-age=") + NGX_TIME_T_LEN + 1);
     if (cc->value.data == NULL) {
+        cc->hash = 0;
         return NGX_ERROR;
     }
 


From pluknet at nginx.com  Mon May 30 22:33:11 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 30 May 2022 22:33:11 +0000
Subject: [nginx] Upstream: fixed build without http cache (broken by
 cd73509f21e2).
Message-ID: <hg.e0cfab501dd1.1653949991.6026610855610030274@dev.nginx>

details:   https://hg.nginx.org/nginx/rev/e0cfab501dd1
branches:  
changeset: 8040:e0cfab501dd1
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue May 31 00:14:11 2022 +0300
description:
Upstream: fixed build without http cache (broken by cd73509f21e2).

diffstat:

 src/http/ngx_http_upstream.c |  9 +++++----
 1 files changed, 5 insertions(+), 4 deletions(-)

diffs (33 lines):

diff -r e64a1f32065b -r e0cfab501dd1 src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Mon May 30 21:25:57 2022 +0300
+++ b/src/http/ngx_http_upstream.c	Tue May 31 00:14:11 2022 +0300
@@ -5183,9 +5183,6 @@ static ngx_int_t
 ngx_http_upstream_process_vary(ngx_http_request_t *r,
     ngx_table_elt_t *h, ngx_uint_t offset)
 {
-    u_char                *p;
-    size_t                 len;
-    ngx_str_t              vary;
     ngx_table_elt_t      **ph;
     ngx_http_upstream_t   *u;
 
@@ -5198,6 +5195,10 @@ ngx_http_upstream_process_vary(ngx_http_
     h->next = NULL;
 
 #if (NGX_HTTP_CACHE)
+    {
+    u_char     *p;
+    size_t      len;
+    ngx_str_t   vary;
 
     if (u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_VARY) {
         return NGX_OK;
@@ -5249,7 +5250,7 @@ ngx_http_upstream_process_vary(ngx_http_
     }
 
     r->cache->vary = vary;
-
+    }
 #endif
 
     return NGX_OK;


From pluknet at nginx.com  Mon May 30 22:54:28 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Tue, 31 May 2022 02:54:28 +0400
Subject: [PATCH] Upstream: handling of certificates specified as an empty
 string
In-Reply-To: <MW4PR01MB6499737D56ED6863B0A9D3E5B1D89@MW4PR01MB6499.prod.exchangelabs.com>
References: <3bb1adbb74dfcd372f73.1653422757@enoparse.local>
 <MW4PR01MB6499737D56ED6863B0A9D3E5B1D89@MW4PR01MB6499.prod.exchangelabs.com>
Message-ID: <30373D2B-9DFD-413F-A143-1EC0A44AFCE3@nginx.com>


> On 27 May 2022, at 08:26, Pavel Pautov via nginx-devel <nginx-devel at nginx.org> wrote:
> 
> Hi,
> 
> Shall we restore original behavior for stream proxy as well?
> 

Fixed, tnx.  The relevant part:

diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c
--- a/src/stream/ngx_stream_proxy_module.c
+++ b/src/stream/ngx_stream_proxy_module.c
@@ -2244,7 +2244,7 @@ ngx_stream_proxy_set_ssl(ngx_conf_t *cf,
                 return NGX_ERROR;
             }
 
-        } else {
+        } else if (pscf->ssl_certificate->value.len) {
             if (ngx_ssl_certificate(cf, pscf->ssl,
                                     &pscf->ssl_certificate->value,
                                     &pscf->ssl_certificate_key->value,

-- 
Sergey Kandaurov


From arut at nginx.com  Tue May 31 06:32:26 2022
From: arut at nginx.com (Roman Arutyunyan)
Date: Tue, 31 May 2022 10:32:26 +0400
Subject: [PATCH 1 of 4] QUIC: fixed-length buffers for secrets
In-Reply-To: <YhS5n2MwHR361isD@vlpc>
References: <patchbomb.1645441830@vl.krasnogorsk.ru>
 <1a0a12bef7f00b5422d4.1645441831@vl.krasnogorsk.ru>
 <20220221145142.pic3fkrhd6dkm5xa@MacBook-Air-Sergey.local>
 <YhS5n2MwHR361isD@vlpc>
Message-ID: <20220531063226.bppptpart52bawy6@N00W24XTQX>

On Tue, Feb 22, 2022 at 01:23:27PM +0300, Vladimir Homutov wrote:
> On Mon, Feb 21, 2022 at 05:51:42PM +0300, Sergey Kandaurov wrote:
> > On Mon, Feb 21, 2022 at 02:10:31PM +0300, Vladimir Homutov wrote:
> > > Patch subject is complete summary.
> > >
> > >
> > >  src/event/quic/ngx_event_quic_protection.c |  202 +++++++++++++++-------------
> > >  1 files changed, 105 insertions(+), 97 deletions(-)
> > >
> > >
> >
> > > # HG changeset patch
> > > # User Vladimir Homutov <vl at nginx.com>
> > > # Date 1645440604 -10800
> > > #      Mon Feb 21 13:50:04 2022 +0300
> > > # Branch quic
> > > # Node ID 1a0a12bef7f00b5422d449b2d4642fff39e0a47e
> > > # Parent  55b38514729b8f848709b31295e72d6886a7a433
> > > QUIC: fixed-length buffers for secrets.
> > >
> > > diff --git a/src/event/quic/ngx_event_quic_protection.c b/src/event/quic/ngx_event_quic_protection.c
> > > --- a/src/event/quic/ngx_event_quic_protection.c
> > > +++ b/src/event/quic/ngx_event_quic_protection.c
> > > @@ -17,6 +17,8 @@
> > >
> > >  #define NGX_QUIC_AES_128_KEY_LEN      16
> > >
> > > +#define NGX_QUIC_KEY_LEN              32
> > > +
> > >  #define NGX_AES_128_GCM_SHA256        0x1301
> > >  #define NGX_AES_256_GCM_SHA384        0x1302
> > >  #define NGX_CHACHA20_POLY1305_SHA256  0x1303
> > > @@ -30,6 +32,27 @@
> > >
> > >
> > >  typedef struct {
> > > +    size_t                    len;
> > > +    u_char                    data[SHA256_DIGEST_LENGTH];
> > > +} ngx_quic_okm_t;
> > > +
> > > +typedef struct {
> > > +    size_t                    len;
> > > +    u_char                    data[NGX_QUIC_KEY_LEN];
> > > +} ngx_quic_key_t;
> > > +
> > > +typedef struct {
> > > +    size_t                    len;
> > > +    u_char                    data[NGX_QUIC_KEY_LEN];
> > > +} ngx_quic_hp_t;
> > > +
> > > +typedef struct {
> > > +    size_t                    len;
> > > +    u_char                    data[NGX_QUIC_IV_LEN];
> > > +} ngx_quic_iv_t;
> >
> > Style: two empty lines between struct declarations.
> 
> thanks, fixed this
> 
> >
> > > +
> > > +
> > > +typedef struct {
> > >      const ngx_quic_cipher_t  *c;
> > >      const EVP_CIPHER         *hp;
> > >      const EVP_MD             *d;
> > > @@ -37,10 +60,10 @@ typedef struct {
> > >
> > >
> > >  typedef struct ngx_quic_secret_s {
> > > -    ngx_str_t                 secret;
> > > -    ngx_str_t                 key;
> > > -    ngx_str_t                 iv;
> > > -    ngx_str_t                 hp;
> > > +    ngx_quic_okm_t            secret;
> > > +    ngx_quic_key_t            key;
> > > +    ngx_quic_iv_t             iv;
> > > +    ngx_quic_hp_t             hp;
> > >  } ngx_quic_secret_t;
> > >
> > >
> > > @@ -57,6 +80,29 @@ struct ngx_quic_keys_s {
> > >  };
> > >
> > >
> > > +typedef struct {
> > > +    size_t                    out_len;
> > > +    u_char                   *out;
> > > +
> > > +    size_t                    prk_len;
> > > +    const uint8_t            *prk;
> > > +
> > > +    size_t                    label_len;
> > > +    const u_char             *label;
> > > +
> > > +    size_t                    info_len;
> > > +    uint8_t                   info[20];
> > > +} ngx_quic_hkdf_t;
> > > +
> > > +#define ngx_quic_hkdf_set(label, out, prk)                                    \
> > > +    {                                                                         \
> > > +        (out)->len, (out)->data,                                              \
> > > +        (prk)->len, (prk)->data,                                              \
> > > +        (sizeof(label) - 1), (u_char *)(label),                               \
> > > +        0, { 0 }                                                              \
> > > +    }
> > > +
> > > +
> > >  static ngx_int_t ngx_hkdf_expand(u_char *out_key, size_t out_len,
> > >      const EVP_MD *digest, const u_char *prk, size_t prk_len,
> > >      const u_char *info, size_t info_len);
> > > @@ -78,8 +124,8 @@ static ngx_int_t ngx_quic_tls_seal(const
> > >      ngx_str_t *ad, ngx_log_t *log);
> > >  static ngx_int_t ngx_quic_tls_hp(ngx_log_t *log, const EVP_CIPHER *cipher,
> > >      ngx_quic_secret_t *s, u_char *out, u_char *in);
> > > -static ngx_int_t ngx_quic_hkdf_expand(ngx_pool_t *pool, const EVP_MD *digest,
> > > -    ngx_str_t *out, ngx_str_t *label, const uint8_t *prk, size_t prk_len);
> > > +static ngx_int_t ngx_quic_hkdf_expand(ngx_quic_hkdf_t *hkdf,
> > > +    const EVP_MD *digest, ngx_pool_t *pool);
> > >
> > >  static ngx_int_t ngx_quic_create_packet(ngx_quic_header_t *pkt,
> > >      ngx_str_t *res);
> > > @@ -204,28 +250,20 @@ ngx_quic_keys_set_initial_secret(ngx_poo
> > >      client->iv.len = NGX_QUIC_IV_LEN;
> > >      server->iv.len = NGX_QUIC_IV_LEN;
> > >
> > > -    struct {
> > > -        ngx_str_t   label;
> > > -        ngx_str_t  *key;
> > > -        ngx_str_t  *prk;
> > > -    } seq[] = {
> > > +    ngx_quic_hkdf_t seq[] = {
> > >          /* labels per RFC 9001, 5.1. Packet Protection Keys */
> > > -        { ngx_string("tls13 client in"), &client->secret, &iss },
> > > -        { ngx_string("tls13 quic key"),  &client->key,    &client->secret },
> > > -        { ngx_string("tls13 quic iv"),   &client->iv,     &client->secret },
> > > -        { ngx_string("tls13 quic hp"),   &client->hp,     &client->secret },
> > > -        { ngx_string("tls13 server in"), &server->secret, &iss },
> > > -        { ngx_string("tls13 quic key"),  &server->key,    &server->secret },
> > > -        { ngx_string("tls13 quic iv"),   &server->iv,     &server->secret },
> > > -        { ngx_string("tls13 quic hp"),   &server->hp,     &server->secret },
> > > +        ngx_quic_hkdf_set("tls13 client in", &client->secret, &iss),
> > > +        ngx_quic_hkdf_set("tls13 quic key",  &client->key,    &client->secret),
> > > +        ngx_quic_hkdf_set("tls13 quic iv",   &client->iv,     &client->secret),
> > > +        ngx_quic_hkdf_set("tls13 quic hp",   &client->hp,     &client->secret),
> > > +        ngx_quic_hkdf_set("tls13 server in", &server->secret, &iss),
> > > +        ngx_quic_hkdf_set("tls13 quic key",  &server->key,    &server->secret),
> > > +        ngx_quic_hkdf_set("tls13 quic iv",   &server->iv,     &server->secret),
> > > +        ngx_quic_hkdf_set("tls13 quic hp",   &server->hp,     &server->secret),
> > >      };
> > >
> > >      for (i = 0; i < (sizeof(seq) / sizeof(seq[0])); i++) {
> > > -
> > > -        if (ngx_quic_hkdf_expand(pool, digest, seq[i].key, &seq[i].label,
> > > -                                 seq[i].prk->data, seq[i].prk->len)
> > > -            != NGX_OK)
> > > -        {
> > > +        if (ngx_quic_hkdf_expand(&seq[i], digest, pool) != NGX_OK) {
> > >              return NGX_ERROR;
> > >          }
> > >      }
> > > @@ -235,40 +273,39 @@ ngx_quic_keys_set_initial_secret(ngx_poo
> > >
> > >
> > >  static ngx_int_t
> > > -ngx_quic_hkdf_expand(ngx_pool_t *pool, const EVP_MD *digest, ngx_str_t *out,
> > > -    ngx_str_t *label, const uint8_t *prk, size_t prk_len)
> > > +ngx_quic_hkdf_expand(ngx_quic_hkdf_t *h, const EVP_MD *digest, ngx_pool_t *pool)
> > >  {
> > > -    size_t    info_len;
> > >      uint8_t  *p;
> > > -    uint8_t   info[20];
> > >
> > > -    if (out->data == NULL) {
> > > -        out->data = ngx_pnalloc(pool, out->len);
> > > -        if (out->data == NULL) {
> > > +    if (h->out == NULL) {
> > > +        h->out = ngx_pnalloc(pool, h->out_len);
> > > +        if (h->out == NULL) {
> > >              return NGX_ERROR;
> > >          }
> > >      }
> > >
> > > -    info_len = 2 + 1 + label->len + 1;
> > > +    h->info_len = 2 + 1 + h->label_len + 1;
> > >
> > > -    info[0] = 0;
> > > -    info[1] = out->len;
> > > -    info[2] = label->len;
> > > -    p = ngx_cpymem(&info[3], label->data, label->len);
> > > +    h->info[0] = 0;
> > > +    h->info[1] = h->out_len;
> > > +    h->info[2] = h->label_len;
> >
> > Why?
> > info/info_len aren't used/useful outside of ngx_quic_hkdf_expand(),
> > they are barely one-time local storages to produce traffic secrets.
> 
> sorry, this is a leftover. Indeed, there is no meaning in keeping this.
> I had some ideas how to initialize it on declaration and merge 'info'
> with 'label', but the result was too clumsy and unsafe to use.
> 
> removed info from structure and restored local variable..
> 
> >
> > > +
> > > +    p = ngx_cpymem(&h->info[3], h->label, h->label_len);
> > >      *p = '\0';
> > >
> > > -    if (ngx_hkdf_expand(out->data, out->len, digest,
> > > -                        prk, prk_len, info, info_len)
> > > +    if (ngx_hkdf_expand(h->out, h->out_len, digest,
> > > +                        h->prk, h->prk_len, h->info, h->info_len)
> > >          != NGX_OK)
> > >      {
> > >          ngx_ssl_error(NGX_LOG_INFO, pool->log, 0,
> > > -                      "ngx_hkdf_expand(%V) failed", label);
> > > +                      "ngx_hkdf_expand(%*s) failed", h->label_len, h->label);
> > >          return NGX_ERROR;
> > >      }
> > >
> > >  #ifdef NGX_QUIC_DEBUG_CRYPTO
> > > -    ngx_log_debug3(NGX_LOG_DEBUG_EVENT, pool->log, 0,
> > > -                   "quic expand %V key len:%uz %xV", label, out->len, out);
> > > +    ngx_log_debug5(NGX_LOG_DEBUG_EVENT, pool->log, 0,
> > > +                   "quic expand \"%*s\" key len:%uz %*xs",
> > > +                   h->label_len, h->label, h->out_len, h->out_len, h->out);
> > >  #endif
> > >
> > >      return NGX_OK;
> > > @@ -652,6 +689,7 @@ ngx_quic_keys_set_encryption_secret(ngx_
> > >      const SSL_CIPHER *cipher, const uint8_t *secret, size_t secret_len)
> > >  {
> > >      ngx_int_t            key_len;
> > > +    ngx_str_t            secret_str;
> > >      ngx_uint_t           i;
> > >      ngx_quic_secret_t   *peer_secret;
> > >      ngx_quic_ciphers_t   ciphers;
> > > @@ -668,8 +706,9 @@ ngx_quic_keys_set_encryption_secret(ngx_
> > >          return NGX_ERROR;
> > >      }
> > >
> > > -    peer_secret->secret.data = ngx_pnalloc(pool, secret_len);
> > > -    if (peer_secret->secret.data == NULL) {
> > > +    if (sizeof(peer_secret->secret.data) < secret_len) {
> > > +        ngx_log_error(NGX_LOG_ERR, pool->log, 0,
> > > +                      "unexpected secret len: %uz", secret_len);
> > >          return NGX_ERROR;
> > >      }
> >
> > This won't work with cipher suite hash algorithms used to produce
> > HKDF Hash.length (read: secret length) above SHA256_DIGEST_LENGTH,
> > such as TLS_AES_256_GCM_SHA384 or any future TLSv1.3 cipher suites
> > with SHA384 or above.  The same for hardcoding NGX_QUIC_KEY_LEN.
> 
> yes, exactly. currently boringssl just defines max size 48 to keep
> SHA384, and I think we can safely follow it. If anything will appear
> in future, we can always adjust size.
> 
> I've renamed NGX_QUIC_KEY_LEN to NGX_QUIC_MAX_MD_SIZE and renamed
> structure holding buffer to ngx_quic_md_t, and use it for secret,
> key and hp, as it makes sens to use them uniformly.
> 
> > The error, if ever leave it there, deserves rasing logging level
> > to "alert" as clearly a programmatic error.
> >
> agreed. This place is the worst in patch - we get size from the library
> and hope that our bufer is big enough, otherwise fail with a runtime error.
> 
> On the other side, it is pretty clear that size here is just digest
> length, and we can expect reasonable numbers here.
> 

> # HG changeset patch
> # User Vladimir Homutov <vl at nginx.com>
> # Date 1645524401 -10800
> #      Tue Feb 22 13:06:41 2022 +0300
> # Branch quic
> # Node ID bb1717365759760bc8175b8d8084819a6ec35c26
> # Parent  55b38514729b8f848709b31295e72d6886a7a433
> QUIC: fixed-length buffers for secrets.
> 
> diff --git a/src/event/quic/ngx_event_quic_protection.c b/src/event/quic/ngx_event_quic_protection.c
> --- a/src/event/quic/ngx_event_quic_protection.c
> +++ b/src/event/quic/ngx_event_quic_protection.c
> @@ -17,6 +17,9 @@
>  
>  #define NGX_QUIC_AES_128_KEY_LEN      16
>  
> +/* largest hash used in TLS is SHA-384 */
> +#define NGX_QUIC_MAX_MD_SIZE          48
> +
>  #define NGX_AES_128_GCM_SHA256        0x1301
>  #define NGX_AES_256_GCM_SHA384        0x1302
>  #define NGX_CHACHA20_POLY1305_SHA256  0x1303
> @@ -30,6 +33,18 @@
>  
>  
>  typedef struct {
> +    size_t                    len;
> +    u_char                    data[NGX_QUIC_MAX_MD_SIZE];
> +} ngx_quic_md_t;
> +
> +
> +typedef struct {
> +    size_t                    len;
> +    u_char                    data[NGX_QUIC_IV_LEN];
> +} ngx_quic_iv_t;
> +
> +
> +typedef struct {
>      const ngx_quic_cipher_t  *c;
>      const EVP_CIPHER         *hp;
>      const EVP_MD             *d;
> @@ -37,10 +52,10 @@ typedef struct {
>  
>  
>  typedef struct ngx_quic_secret_s {
> -    ngx_str_t                 secret;
> -    ngx_str_t                 key;
> -    ngx_str_t                 iv;
> -    ngx_str_t                 hp;
> +    ngx_quic_md_t             secret;
> +    ngx_quic_md_t             key;
> +    ngx_quic_iv_t             iv;
> +    ngx_quic_md_t             hp;
>  } ngx_quic_secret_t;
>  
>  
> @@ -57,6 +72,25 @@ struct ngx_quic_keys_s {
>  };
>  
>  
> +typedef struct {
> +    size_t                    out_len;
> +    u_char                   *out;
> +
> +    size_t                    prk_len;
> +    const uint8_t            *prk;
> +
> +    size_t                    label_len;
> +    const u_char             *label;
> +} ngx_quic_hkdf_t;
> +
> +#define ngx_quic_hkdf_set(label, out, prk)                                    \
> +    {                                                                         \
> +        (out)->len, (out)->data,                                              \
> +        (prk)->len, (prk)->data,                                              \
> +        (sizeof(label) - 1), (u_char *)(label),                               \
> +    }
> +
> +
>  static ngx_int_t ngx_hkdf_expand(u_char *out_key, size_t out_len,
>      const EVP_MD *digest, const u_char *prk, size_t prk_len,
>      const u_char *info, size_t info_len);
> @@ -78,8 +112,8 @@ static ngx_int_t ngx_quic_tls_seal(const
>      ngx_str_t *ad, ngx_log_t *log);
>  static ngx_int_t ngx_quic_tls_hp(ngx_log_t *log, const EVP_CIPHER *cipher,
>      ngx_quic_secret_t *s, u_char *out, u_char *in);
> -static ngx_int_t ngx_quic_hkdf_expand(ngx_pool_t *pool, const EVP_MD *digest,
> -    ngx_str_t *out, ngx_str_t *label, const uint8_t *prk, size_t prk_len);
> +static ngx_int_t ngx_quic_hkdf_expand(ngx_quic_hkdf_t *hkdf,
> +    const EVP_MD *digest, ngx_pool_t *pool);
>  
>  static ngx_int_t ngx_quic_create_packet(ngx_quic_header_t *pkt,
>      ngx_str_t *res);
> @@ -204,28 +238,20 @@ ngx_quic_keys_set_initial_secret(ngx_poo
>      client->iv.len = NGX_QUIC_IV_LEN;
>      server->iv.len = NGX_QUIC_IV_LEN;
>  
> -    struct {
> -        ngx_str_t   label;
> -        ngx_str_t  *key;
> -        ngx_str_t  *prk;
> -    } seq[] = {
> +    ngx_quic_hkdf_t seq[] = {
>          /* labels per RFC 9001, 5.1. Packet Protection Keys */
> -        { ngx_string("tls13 client in"), &client->secret, &iss },
> -        { ngx_string("tls13 quic key"),  &client->key,    &client->secret },
> -        { ngx_string("tls13 quic iv"),   &client->iv,     &client->secret },
> -        { ngx_string("tls13 quic hp"),   &client->hp,     &client->secret },
> -        { ngx_string("tls13 server in"), &server->secret, &iss },
> -        { ngx_string("tls13 quic key"),  &server->key,    &server->secret },
> -        { ngx_string("tls13 quic iv"),   &server->iv,     &server->secret },
> -        { ngx_string("tls13 quic hp"),   &server->hp,     &server->secret },
> +        ngx_quic_hkdf_set("tls13 client in", &client->secret, &iss),
> +        ngx_quic_hkdf_set("tls13 quic key",  &client->key,    &client->secret),
> +        ngx_quic_hkdf_set("tls13 quic iv",   &client->iv,     &client->secret),
> +        ngx_quic_hkdf_set("tls13 quic hp",   &client->hp,     &client->secret),
> +        ngx_quic_hkdf_set("tls13 server in", &server->secret, &iss),
> +        ngx_quic_hkdf_set("tls13 quic key",  &server->key,    &server->secret),
> +        ngx_quic_hkdf_set("tls13 quic iv",   &server->iv,     &server->secret),
> +        ngx_quic_hkdf_set("tls13 quic hp",   &server->hp,     &server->secret),
>      };
>  
>      for (i = 0; i < (sizeof(seq) / sizeof(seq[0])); i++) {
> -
> -        if (ngx_quic_hkdf_expand(pool, digest, seq[i].key, &seq[i].label,
> -                                 seq[i].prk->data, seq[i].prk->len)
> -            != NGX_OK)
> -        {
> +        if (ngx_quic_hkdf_expand(&seq[i], digest, pool) != NGX_OK) {
>              return NGX_ERROR;
>          }
>      }
> @@ -235,40 +261,41 @@ ngx_quic_keys_set_initial_secret(ngx_poo
>  
>  
>  static ngx_int_t
> -ngx_quic_hkdf_expand(ngx_pool_t *pool, const EVP_MD *digest, ngx_str_t *out,
> -    ngx_str_t *label, const uint8_t *prk, size_t prk_len)
> +ngx_quic_hkdf_expand(ngx_quic_hkdf_t *h, const EVP_MD *digest, ngx_pool_t *pool)
>  {
>      size_t    info_len;
>      uint8_t  *p;
>      uint8_t   info[20];
>  
> -    if (out->data == NULL) {
> -        out->data = ngx_pnalloc(pool, out->len);
> -        if (out->data == NULL) {
> +    if (h->out == NULL) {
> +        h->out = ngx_pnalloc(pool, h->out_len);
> +        if (h->out == NULL) {
>              return NGX_ERROR;
>          }
>      }
>  
> -    info_len = 2 + 1 + label->len + 1;
> +    info_len = 2 + 1 + h->label_len + 1;
>  
>      info[0] = 0;
> -    info[1] = out->len;
> -    info[2] = label->len;
> -    p = ngx_cpymem(&info[3], label->data, label->len);
> +    info[1] = h->out_len;
> +    info[2] = h->label_len;
> +
> +    p = ngx_cpymem(&info[3], h->label, h->label_len);
>      *p = '\0';
>  
> -    if (ngx_hkdf_expand(out->data, out->len, digest,
> -                        prk, prk_len, info, info_len)
> +    if (ngx_hkdf_expand(h->out, h->out_len, digest,
> +                        h->prk, h->prk_len, info, info_len)
>          != NGX_OK)
>      {
>          ngx_ssl_error(NGX_LOG_INFO, pool->log, 0,
> -                      "ngx_hkdf_expand(%V) failed", label);
> +                      "ngx_hkdf_expand(%*s) failed", h->label_len, h->label);
>          return NGX_ERROR;
>      }
>  
>  #ifdef NGX_QUIC_DEBUG_CRYPTO
> -    ngx_log_debug3(NGX_LOG_DEBUG_EVENT, pool->log, 0,
> -                   "quic expand %V key len:%uz %xV", label, out->len, out);
> +    ngx_log_debug5(NGX_LOG_DEBUG_EVENT, pool->log, 0,
> +                   "quic expand \"%*s\" key len:%uz %*xs",
> +                   h->label_len, h->label, h->out_len, h->out_len, h->out);
>  #endif
>  
>      return NGX_OK;
> @@ -652,6 +679,7 @@ ngx_quic_keys_set_encryption_secret(ngx_
>      const SSL_CIPHER *cipher, const uint8_t *secret, size_t secret_len)
>  {
>      ngx_int_t            key_len;
> +    ngx_str_t            secret_str;
>      ngx_uint_t           i;
>      ngx_quic_secret_t   *peer_secret;
>      ngx_quic_ciphers_t   ciphers;
> @@ -668,8 +696,9 @@ ngx_quic_keys_set_encryption_secret(ngx_
>          return NGX_ERROR;
>      }
>  
> -    peer_secret->secret.data = ngx_pnalloc(pool, secret_len);
> -    if (peer_secret->secret.data == NULL) {
> +    if (sizeof(peer_secret->secret.data) < secret_len) {
> +        ngx_log_error(NGX_LOG_ALERT, pool->log, 0,
> +                      "unexpected secret len: %uz", secret_len);
>          return NGX_ERROR;
>      }
>  
> @@ -680,22 +709,17 @@ ngx_quic_keys_set_encryption_secret(ngx_
>      peer_secret->iv.len = NGX_QUIC_IV_LEN;
>      peer_secret->hp.len = key_len;
>  
> -    struct {
> -        ngx_str_t       label;
> -        ngx_str_t      *key;
> -        const uint8_t  *secret;
> -    } seq[] = {
> -        { ngx_string("tls13 quic key"), &peer_secret->key, secret },
> -        { ngx_string("tls13 quic iv"),  &peer_secret->iv,  secret },
> -        { ngx_string("tls13 quic hp"),  &peer_secret->hp,  secret },
> +    secret_str.len = secret_len;
> +    secret_str.data = (u_char *) secret;
> +
> +    ngx_quic_hkdf_t seq[] = {
> +        ngx_quic_hkdf_set("tls13 quic key", &peer_secret->key, &secret_str),
> +        ngx_quic_hkdf_set("tls13 quic iv", &peer_secret->iv, &secret_str),
> +        ngx_quic_hkdf_set("tls13 quic hp", &peer_secret->hp, &secret_str),
>      };
>  
>      for (i = 0; i < (sizeof(seq) / sizeof(seq[0])); i++) {
> -
> -        if (ngx_quic_hkdf_expand(pool, ciphers.d, seq[i].key, &seq[i].label,
> -                                 seq[i].secret, secret_len)
> -            != NGX_OK)
> -        {
> +        if (ngx_quic_hkdf_expand(&seq[i], ciphers.d, pool) != NGX_OK) {
>              return NGX_ERROR;
>          }
>      }
> @@ -769,49 +793,23 @@ ngx_quic_keys_update(ngx_connection_t *c
>      next->server.iv.len = NGX_QUIC_IV_LEN;
>      next->server.hp = current->server.hp;
>  
> -    struct {
> -        ngx_str_t   label;
> -        ngx_str_t  *key;
> -        ngx_str_t  *secret;
> -    } seq[] = {
> -        {
> -            ngx_string("tls13 quic ku"),
> -            &next->client.secret,
> -            &current->client.secret,
> -        },
> -        {
> -            ngx_string("tls13 quic key"),
> -            &next->client.key,
> -            &next->client.secret,
> -        },
> -        {
> -            ngx_string("tls13 quic iv"),
> -            &next->client.iv,
> -            &next->client.secret,
> -        },
> -        {
> -            ngx_string("tls13 quic ku"),
> -            &next->server.secret,
> -            &current->server.secret,
> -        },
> -        {
> -            ngx_string("tls13 quic key"),
> -            &next->server.key,
> -            &next->server.secret,
> -        },
> -        {
> -            ngx_string("tls13 quic iv"),
> -            &next->server.iv,
> -            &next->server.secret,
> -        },
> +    ngx_quic_hkdf_t seq[] = {
> +        ngx_quic_hkdf_set("tls13 quic ku",
> +                           &next->client.secret, &current->client.secret),

A space is missing here ^^

> +        ngx_quic_hkdf_set("tls13 quic key",
> +                          &next->client.key, &next->client.secret),
> +        ngx_quic_hkdf_set("tls13 quic iv",
> +                          &next->client.iv, &next->client.secret),
> +        ngx_quic_hkdf_set("tls13 quic ku",
> +                          &next->server.secret, &current->server.secret),
> +        ngx_quic_hkdf_set("tls13 quic key",
> +                          &next->server.key, &next->server.secret),
> +        ngx_quic_hkdf_set("tls13 quic iv",
> +                          &next->server.iv, &next->server.secret),
>      };
>  
>      for (i = 0; i < (sizeof(seq) / sizeof(seq[0])); i++) {
> -
> -        if (ngx_quic_hkdf_expand(c->pool, ciphers.d, seq[i].key, &seq[i].label,
> -                                 seq[i].secret->data, seq[i].secret->len)
> -            != NGX_OK)
> -        {
> +        if (ngx_quic_hkdf_expand(&seq[i], ciphers.d, c->pool) != NGX_OK) {
>              return NGX_ERROR;
>          }
>      }
> @@ -909,7 +907,7 @@ ngx_quic_create_retry_packet(ngx_quic_he
>      }
>  
>      secret.key.len = sizeof(key);
> -    secret.key.data = key;
> +    ngx_memcpy(secret.key.data, key, sizeof(key));
>      secret.iv.len = NGX_QUIC_IV_LEN;
>  
>      if (ngx_quic_tls_seal(ciphers.c, &secret, &itag, nonce, &in, &ad, pkt->log)

Overall, the patch looks good to me.

This version of the patch affects the following patches in the series.
I will resend the whole series.


From arut at nginx.com  Tue May 31 06:35:30 2022
From: arut at nginx.com (Roman Arutyunyan)
Date: Tue, 31 May 2022 10:35:30 +0400
Subject: [PATCH 2 of 4] QUIC: avoided pool usage in ngx_quic_protection.c
In-Reply-To: <950a45270e862b02f43e.1645441832@vl.krasnogorsk.ru>
References: <patchbomb.1645441830@vl.krasnogorsk.ru>
 <950a45270e862b02f43e.1645441832@vl.krasnogorsk.ru>
Message-ID: <20220531063530.mgvsrijulxwoh46x@N00W24XTQX>

On Mon, Feb 21, 2022 at 02:10:32PM +0300, Vladimir Homutov wrote:
> Patch subject is complete summary.
> 
> 
>  src/event/quic/ngx_event_quic.c            |   2 +-
>  src/event/quic/ngx_event_quic_output.c     |   2 +-
>  src/event/quic/ngx_event_quic_protection.c |  37 ++++++++++++-----------------
>  src/event/quic/ngx_event_quic_protection.h |   6 ++--
>  src/event/quic/ngx_event_quic_ssl.c        |   8 +++---
>  5 files changed, 24 insertions(+), 31 deletions(-)
> 
> 

Looks good to me


From arut at nginx.com  Tue May 31 06:42:11 2022
From: arut at nginx.com (Roman Arutyunyan)
Date: Tue, 31 May 2022 10:42:11 +0400
Subject: [PATCH 3 of 4] QUIC: removed ngx_quic_keys_new()
In-Reply-To: <dfc2fc335990e05da1a6.1645441833@vl.krasnogorsk.ru>
References: <patchbomb.1645441830@vl.krasnogorsk.ru>
 <dfc2fc335990e05da1a6.1645441833@vl.krasnogorsk.ru>
Message-ID: <20220531064211.e5di6tjfqmza7ds7@N00W24XTQX>

On Mon, Feb 21, 2022 at 02:10:33PM +0300, Vladimir Homutov wrote:
> The ngx_quic_keys_t structure is now exposed.
> This allows to use it in contexts where no pool/connection is available,
> i.e. early packet processing.
> 
> 
>  src/event/quic/ngx_event_quic.c            |   2 +-
>  src/event/quic/ngx_event_quic_output.c     |   8 ++--
>  src/event/quic/ngx_event_quic_protection.c |  53 ------------------------------
>  src/event/quic/ngx_event_quic_protection.h |  48 ++++++++++++++++++++++++++-
>  4 files changed, 52 insertions(+), 59 deletions(-)

This one is heavily affected by the change in the first patch.
Overall, I like the patch.


From arut at nginx.com  Tue May 31 06:55:35 2022
From: arut at nginx.com (Roman Arutyunyan)
Date: Tue, 31 May 2022 10:55:35 +0400
Subject: [PATCH 4 of 4] QUIC: avoided pool usage in token calculation
In-Reply-To: <b3fb81ecc3431c4dbf9e.1645441834@vl.krasnogorsk.ru>
References: <patchbomb.1645441830@vl.krasnogorsk.ru>
 <b3fb81ecc3431c4dbf9e.1645441834@vl.krasnogorsk.ru>
Message-ID: <20220531065535.ni2djnn3vhurvet2@N00W24XTQX>

On Mon, Feb 21, 2022 at 02:10:34PM +0300, Vladimir Homutov wrote:
> Patch subject is complete summary.
> 
> 
>  src/event/quic/ngx_event_quic_output.c    |  11 +++++++++--
>  src/event/quic/ngx_event_quic_tokens.c    |  22 ++++------------------
>  src/event/quic/ngx_event_quic_tokens.h    |  14 +++++++++++++-
>  src/event/quic/ngx_event_quic_transport.h |   1 +
>  4 files changed, 27 insertions(+), 21 deletions(-)
> 
> 

> # HG changeset patch
> # User Vladimir Homutov <vl at nginx.com>
> # Date 1645440587 -10800
> #      Mon Feb 21 13:49:47 2022 +0300
> # Branch quic
> # Node ID b3fb81ecc3431c4dbf9e849d72d13a84fe02703b
> # Parent  dfc2fc335990e05da1a6f087ca75721cbf8c8891
> QUIC: avoided pool usage in token calculation.
> 
> diff --git a/src/event/quic/ngx_event_quic_output.c b/src/event/quic/ngx_event_quic_output.c
> --- a/src/event/quic/ngx_event_quic_output.c
> +++ b/src/event/quic/ngx_event_quic_output.c
> @@ -1009,10 +1009,13 @@ ngx_quic_send_retry(ngx_connection_t *c,
>  
>      u_char             buf[NGX_QUIC_RETRY_BUFFER_SIZE];
>      u_char             dcid[NGX_QUIC_SERVER_CID_LEN];
> +    u_char             tbuf[NGX_QUIC_TOKEN_BUF_SIZE];
>  
>      expires = ngx_time() + NGX_QUIC_RETRY_TOKEN_LIFETIME;
>  
> -    if (ngx_quic_new_token(c, c->sockaddr, c->socklen, conf->av_token_key,
> +    token.data = tbuf;

This part looks fragile to me.  I suggest that we assign the available length
to token.len:

    token.len = NGX_QUIC_TOKEN_BUF_SIZE;

Then inside ngx_quic_new_token() verify if length is enough.

> +
> +    if (ngx_quic_new_token(c->log, c->sockaddr, c->socklen, conf->av_token_key,
>                             &token, &inpkt->dcid, expires, 1)
>          != NGX_OK)
>      {
> @@ -1075,11 +1078,15 @@ ngx_quic_send_new_token(ngx_connection_t
>      ngx_quic_frame_t       *frame;
>      ngx_quic_connection_t  *qc;
>  
> +    u_char                  tbuf[NGX_QUIC_TOKEN_BUF_SIZE];
> +
>      qc = ngx_quic_get_connection(c);
>  
>      expires = ngx_time() + NGX_QUIC_NEW_TOKEN_LIFETIME;
>  
> -    if (ngx_quic_new_token(c, path->sockaddr, path->socklen,
> +    token.data = tbuf;

Same here.

> +    if (ngx_quic_new_token(c->log, path->sockaddr, path->socklen,
>                             qc->conf->av_token_key, &token, NULL, expires, 0)
>          != NGX_OK)
>      {
> diff --git a/src/event/quic/ngx_event_quic_tokens.c b/src/event/quic/ngx_event_quic_tokens.c
> --- a/src/event/quic/ngx_event_quic_tokens.c
> +++ b/src/event/quic/ngx_event_quic_tokens.c
> @@ -11,14 +11,6 @@
>  #include <ngx_event_quic_connection.h>
>  
>  
> -#define NGX_QUIC_MAX_TOKEN_SIZE              64
> -    /* SHA-1(addr)=20 + sizeof(time_t) + retry(1) + odcid.len(1) + odcid */
> -
> -/* RFC 3602, 2.1 and 2.4 for AES-CBC block size and IV length */
> -#define NGX_QUIC_AES_256_CBC_IV_LEN          16
> -#define NGX_QUIC_AES_256_CBC_BLOCK_SIZE      16
> -
> -
>  static void ngx_quic_address_hash(struct sockaddr *sockaddr, socklen_t socklen,
>      ngx_uint_t no_port, u_char buf[20]);
>  
> @@ -48,7 +40,7 @@ ngx_quic_new_sr_token(ngx_connection_t *
>  
>  
>  ngx_int_t
> -ngx_quic_new_token(ngx_connection_t *c, struct sockaddr *sockaddr,
> +ngx_quic_new_token(ngx_log_t *log, struct sockaddr *sockaddr,
>      socklen_t socklen, u_char *key, ngx_str_t *token, ngx_str_t *odcid,
>      time_t exp, ngx_uint_t is_retry)
>  {
> @@ -81,10 +73,6 @@ ngx_quic_new_token(ngx_connection_t *c, 
>      iv_len = NGX_QUIC_AES_256_CBC_IV_LEN;
>  
>      token->len = iv_len + len + NGX_QUIC_AES_256_CBC_BLOCK_SIZE;

Since there's no allocation anymore, this assignment makes no sense.
As I suggested earlier, what could make sense is verifying token->data buffer
is large enough for the token:

    if ((size_t) (iv_len + len + NGX_QUIC_AES_256_CBC_BLOCK_SIZE) > token->len)
    {
        ngx_log_error(NGX_LOG_ALERT, log, 0, "quic token buffer is too small");
        return NGX_ERROR;
    }

> -    token->data = ngx_pnalloc(c->pool, token->len);
> -    if (token->data == NULL) {
> -        return NGX_ERROR;
> -    }
>  
>      ctx = EVP_CIPHER_CTX_new();
>      if (ctx == NULL) {
> @@ -119,7 +107,7 @@ ngx_quic_new_token(ngx_connection_t *c, 
>      EVP_CIPHER_CTX_free(ctx);
>  
>  #ifdef NGX_QUIC_DEBUG_PACKETS
> -    ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0,
> +    ngx_log_debug2(NGX_LOG_DEBUG_EVENT, log, 0,
>                     "quic new token len:%uz %xV", token->len, token);
>  #endif
>  
> @@ -268,10 +256,8 @@ ngx_quic_validate_token(ngx_connection_t
>  
>      if (odcid.len) {
>          pkt->odcid.len = odcid.len;
> -        pkt->odcid.data = ngx_pstrdup(c->pool, &odcid);
> -        if (pkt->odcid.data == NULL) {
> -            return NGX_ERROR;
> -        }
> +        pkt->odcid.data = pkt->odcid_data;
> +        ngx_memcpy(pkt->odcid.data, odcid.data, odcid.len);
>  
>      } else {
>          pkt->odcid = pkt->dcid;
> diff --git a/src/event/quic/ngx_event_quic_tokens.h b/src/event/quic/ngx_event_quic_tokens.h
> --- a/src/event/quic/ngx_event_quic_tokens.h
> +++ b/src/event/quic/ngx_event_quic_tokens.h
> @@ -12,9 +12,21 @@
>  #include <ngx_core.h>
>  
>  
> +#define NGX_QUIC_MAX_TOKEN_SIZE              64
> +    /* SHA-1(addr)=20 + sizeof(time_t) + retry(1) + odcid.len(1) + odcid */
> +
> +/* RFC 3602, 2.1 and 2.4 for AES-CBC block size and IV length */
> +#define NGX_QUIC_AES_256_CBC_IV_LEN          16
> +#define NGX_QUIC_AES_256_CBC_BLOCK_SIZE      16
> +
> +#define NGX_QUIC_TOKEN_BUF_SIZE             (NGX_QUIC_AES_256_CBC_IV_LEN      \
> +                                            + NGX_QUIC_MAX_TOKEN_SIZE         \
> +                                            + NGX_QUIC_AES_256_CBC_BLOCK_SIZE)
> +
> +
>  ngx_int_t ngx_quic_new_sr_token(ngx_connection_t *c, ngx_str_t *cid,
>      u_char *secret, u_char *token);
> -ngx_int_t ngx_quic_new_token(ngx_connection_t *c, struct sockaddr *sockaddr,
> +ngx_int_t ngx_quic_new_token(ngx_log_t *log, struct sockaddr *sockaddr,
>      socklen_t socklen, u_char *key, ngx_str_t *token, ngx_str_t *odcid,
>      time_t expires, ngx_uint_t is_retry);
>  ngx_int_t ngx_quic_validate_token(ngx_connection_t *c,
> diff --git a/src/event/quic/ngx_event_quic_transport.h b/src/event/quic/ngx_event_quic_transport.h
> --- a/src/event/quic/ngx_event_quic_transport.h
> +++ b/src/event/quic/ngx_event_quic_transport.h
> @@ -322,6 +322,7 @@ typedef struct {
>  
>      /* cleartext fields */
>      ngx_str_t                                   odcid; /* retry packet tag */
> +    u_char                                      odcid_data[NGX_QUIC_MAX_CID_LEN];
>      ngx_str_t                                   dcid;
>      ngx_str_t                                   scid;
>      uint64_t                                    pn;

I will include the proposed changes in the new series.


From arut at nginx.com  Tue May 31 07:06:30 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 31 May 2022 11:06:30 +0400
Subject: [PATCH 0 of 4] avoid pool allocations
In-Reply-To: < <patchbomb.1645441830@vl.krasnogorsk.ru>
References: < <patchbomb.1645441830@vl.krasnogorsk.ru>
Message-ID: <patchbomb.1653980790@arut-laptop>

Updated version of the series.


From arut at nginx.com  Tue May 31 07:06:31 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 31 May 2022 11:06:31 +0400
Subject: [PATCH 1 of 4] QUIC: fixed-length buffers for secrets
In-Reply-To: <patchbomb.1653980790@arut-laptop>
References: <patchbomb.1653980790@arut-laptop>
Message-ID: <a881ff28070262f38105.1653980791@arut-laptop>

# HG changeset patch
# User Vladimir Homutov <vl at nginx.com>
# Date 1645524401 -10800
#      Tue Feb 22 13:06:41 2022 +0300
# Branch quic
# Node ID a881ff28070262f3810517d5d3cb4ff67a4b7121
# Parent  5b1011b5702b5c5db2ba3d392a4da25596183cc2
QUIC: fixed-length buffers for secrets.

diff --git a/src/event/quic/ngx_event_quic_protection.c b/src/event/quic/ngx_event_quic_protection.c
--- a/src/event/quic/ngx_event_quic_protection.c
+++ b/src/event/quic/ngx_event_quic_protection.c
@@ -17,6 +17,9 @@
 
 #define NGX_QUIC_AES_128_KEY_LEN      16
 
+/* largest hash used in TLS is SHA-384 */
+#define NGX_QUIC_MAX_MD_SIZE          48
+
 #define NGX_AES_128_GCM_SHA256        0x1301
 #define NGX_AES_256_GCM_SHA384        0x1302
 #define NGX_CHACHA20_POLY1305_SHA256  0x1303
@@ -30,6 +33,18 @@
 
 
 typedef struct {
+    size_t                    len;
+    u_char                    data[NGX_QUIC_MAX_MD_SIZE];
+} ngx_quic_md_t;
+
+
+typedef struct {
+    size_t                    len;
+    u_char                    data[NGX_QUIC_IV_LEN];
+} ngx_quic_iv_t;
+
+
+typedef struct {
     const ngx_quic_cipher_t  *c;
     const EVP_CIPHER         *hp;
     const EVP_MD             *d;
@@ -37,10 +52,10 @@ typedef struct {
 
 
 typedef struct ngx_quic_secret_s {
-    ngx_str_t                 secret;
-    ngx_str_t                 key;
-    ngx_str_t                 iv;
-    ngx_str_t                 hp;
+    ngx_quic_md_t             secret;
+    ngx_quic_md_t             key;
+    ngx_quic_iv_t             iv;
+    ngx_quic_md_t             hp;
 } ngx_quic_secret_t;
 
 
@@ -57,6 +72,25 @@ struct ngx_quic_keys_s {
 };
 
 
+typedef struct {
+    size_t                    out_len;
+    u_char                   *out;
+
+    size_t                    prk_len;
+    const uint8_t            *prk;
+
+    size_t                    label_len;
+    const u_char             *label;
+} ngx_quic_hkdf_t;
+
+#define ngx_quic_hkdf_set(label, out, prk)                                    \
+    {                                                                         \
+        (out)->len, (out)->data,                                              \
+        (prk)->len, (prk)->data,                                              \
+        (sizeof(label) - 1), (u_char *)(label),                               \
+    }
+
+
 static ngx_int_t ngx_hkdf_expand(u_char *out_key, size_t out_len,
     const EVP_MD *digest, const u_char *prk, size_t prk_len,
     const u_char *info, size_t info_len);
@@ -78,8 +112,8 @@ static ngx_int_t ngx_quic_tls_seal(const
     ngx_str_t *ad, ngx_log_t *log);
 static ngx_int_t ngx_quic_tls_hp(ngx_log_t *log, const EVP_CIPHER *cipher,
     ngx_quic_secret_t *s, u_char *out, u_char *in);
-static ngx_int_t ngx_quic_hkdf_expand(ngx_pool_t *pool, const EVP_MD *digest,
-    ngx_str_t *out, ngx_str_t *label, const uint8_t *prk, size_t prk_len);
+static ngx_int_t ngx_quic_hkdf_expand(ngx_quic_hkdf_t *hkdf,
+    const EVP_MD *digest, ngx_pool_t *pool);
 
 static ngx_int_t ngx_quic_create_packet(ngx_quic_header_t *pkt,
     ngx_str_t *res);
@@ -204,28 +238,20 @@ ngx_quic_keys_set_initial_secret(ngx_poo
     client->iv.len = NGX_QUIC_IV_LEN;
     server->iv.len = NGX_QUIC_IV_LEN;
 
-    struct {
-        ngx_str_t   label;
-        ngx_str_t  *key;
-        ngx_str_t  *prk;
-    } seq[] = {
+    ngx_quic_hkdf_t seq[] = {
         /* labels per RFC 9001, 5.1. Packet Protection Keys */
-        { ngx_string("tls13 client in"), &client->secret, &iss },
-        { ngx_string("tls13 quic key"),  &client->key,    &client->secret },
-        { ngx_string("tls13 quic iv"),   &client->iv,     &client->secret },
-        { ngx_string("tls13 quic hp"),   &client->hp,     &client->secret },
-        { ngx_string("tls13 server in"), &server->secret, &iss },
-        { ngx_string("tls13 quic key"),  &server->key,    &server->secret },
-        { ngx_string("tls13 quic iv"),   &server->iv,     &server->secret },
-        { ngx_string("tls13 quic hp"),   &server->hp,     &server->secret },
+        ngx_quic_hkdf_set("tls13 client in", &client->secret, &iss),
+        ngx_quic_hkdf_set("tls13 quic key",  &client->key,    &client->secret),
+        ngx_quic_hkdf_set("tls13 quic iv",   &client->iv,     &client->secret),
+        ngx_quic_hkdf_set("tls13 quic hp",   &client->hp,     &client->secret),
+        ngx_quic_hkdf_set("tls13 server in", &server->secret, &iss),
+        ngx_quic_hkdf_set("tls13 quic key",  &server->key,    &server->secret),
+        ngx_quic_hkdf_set("tls13 quic iv",   &server->iv,     &server->secret),
+        ngx_quic_hkdf_set("tls13 quic hp",   &server->hp,     &server->secret),
     };
 
     for (i = 0; i < (sizeof(seq) / sizeof(seq[0])); i++) {
-
-        if (ngx_quic_hkdf_expand(pool, digest, seq[i].key, &seq[i].label,
-                                 seq[i].prk->data, seq[i].prk->len)
-            != NGX_OK)
-        {
+        if (ngx_quic_hkdf_expand(&seq[i], digest, pool) != NGX_OK) {
             return NGX_ERROR;
         }
     }
@@ -235,40 +261,41 @@ ngx_quic_keys_set_initial_secret(ngx_poo
 
 
 static ngx_int_t
-ngx_quic_hkdf_expand(ngx_pool_t *pool, const EVP_MD *digest, ngx_str_t *out,
-    ngx_str_t *label, const uint8_t *prk, size_t prk_len)
+ngx_quic_hkdf_expand(ngx_quic_hkdf_t *h, const EVP_MD *digest, ngx_pool_t *pool)
 {
     size_t    info_len;
     uint8_t  *p;
     uint8_t   info[20];
 
-    if (out->data == NULL) {
-        out->data = ngx_pnalloc(pool, out->len);
-        if (out->data == NULL) {
+    if (h->out == NULL) {
+        h->out = ngx_pnalloc(pool, h->out_len);
+        if (h->out == NULL) {
             return NGX_ERROR;
         }
     }
 
-    info_len = 2 + 1 + label->len + 1;
+    info_len = 2 + 1 + h->label_len + 1;
 
     info[0] = 0;
-    info[1] = out->len;
-    info[2] = label->len;
-    p = ngx_cpymem(&info[3], label->data, label->len);
+    info[1] = h->out_len;
+    info[2] = h->label_len;
+
+    p = ngx_cpymem(&info[3], h->label, h->label_len);
     *p = '\0';
 
-    if (ngx_hkdf_expand(out->data, out->len, digest,
-                        prk, prk_len, info, info_len)
+    if (ngx_hkdf_expand(h->out, h->out_len, digest,
+                        h->prk, h->prk_len, info, info_len)
         != NGX_OK)
     {
         ngx_ssl_error(NGX_LOG_INFO, pool->log, 0,
-                      "ngx_hkdf_expand(%V) failed", label);
+                      "ngx_hkdf_expand(%*s) failed", h->label_len, h->label);
         return NGX_ERROR;
     }
 
 #ifdef NGX_QUIC_DEBUG_CRYPTO
-    ngx_log_debug3(NGX_LOG_DEBUG_EVENT, pool->log, 0,
-                   "quic expand %V key len:%uz %xV", label, out->len, out);
+    ngx_log_debug5(NGX_LOG_DEBUG_EVENT, pool->log, 0,
+                   "quic expand \"%*s\" key len:%uz %*xs",
+                   h->label_len, h->label, h->out_len, h->out_len, h->out);
 #endif
 
     return NGX_OK;
@@ -652,6 +679,7 @@ ngx_quic_keys_set_encryption_secret(ngx_
     const SSL_CIPHER *cipher, const uint8_t *secret, size_t secret_len)
 {
     ngx_int_t            key_len;
+    ngx_str_t            secret_str;
     ngx_uint_t           i;
     ngx_quic_secret_t   *peer_secret;
     ngx_quic_ciphers_t   ciphers;
@@ -668,8 +696,9 @@ ngx_quic_keys_set_encryption_secret(ngx_
         return NGX_ERROR;
     }
 
-    peer_secret->secret.data = ngx_pnalloc(pool, secret_len);
-    if (peer_secret->secret.data == NULL) {
+    if (sizeof(peer_secret->secret.data) < secret_len) {
+        ngx_log_error(NGX_LOG_ALERT, pool->log, 0,
+                      "unexpected secret len: %uz", secret_len);
         return NGX_ERROR;
     }
 
@@ -680,22 +709,17 @@ ngx_quic_keys_set_encryption_secret(ngx_
     peer_secret->iv.len = NGX_QUIC_IV_LEN;
     peer_secret->hp.len = key_len;
 
-    struct {
-        ngx_str_t       label;
-        ngx_str_t      *key;
-        const uint8_t  *secret;
-    } seq[] = {
-        { ngx_string("tls13 quic key"), &peer_secret->key, secret },
-        { ngx_string("tls13 quic iv"),  &peer_secret->iv,  secret },
-        { ngx_string("tls13 quic hp"),  &peer_secret->hp,  secret },
+    secret_str.len = secret_len;
+    secret_str.data = (u_char *) secret;
+
+    ngx_quic_hkdf_t seq[] = {
+        ngx_quic_hkdf_set("tls13 quic key", &peer_secret->key, &secret_str),
+        ngx_quic_hkdf_set("tls13 quic iv", &peer_secret->iv, &secret_str),
+        ngx_quic_hkdf_set("tls13 quic hp", &peer_secret->hp, &secret_str),
     };
 
     for (i = 0; i < (sizeof(seq) / sizeof(seq[0])); i++) {
-
-        if (ngx_quic_hkdf_expand(pool, ciphers.d, seq[i].key, &seq[i].label,
-                                 seq[i].secret, secret_len)
-            != NGX_OK)
-        {
+        if (ngx_quic_hkdf_expand(&seq[i], ciphers.d, pool) != NGX_OK) {
             return NGX_ERROR;
         }
     }
@@ -769,49 +793,23 @@ ngx_quic_keys_update(ngx_connection_t *c
     next->server.iv.len = NGX_QUIC_IV_LEN;
     next->server.hp = current->server.hp;
 
-    struct {
-        ngx_str_t   label;
-        ngx_str_t  *key;
-        ngx_str_t  *secret;
-    } seq[] = {
-        {
-            ngx_string("tls13 quic ku"),
-            &next->client.secret,
-            &current->client.secret,
-        },
-        {
-            ngx_string("tls13 quic key"),
-            &next->client.key,
-            &next->client.secret,
-        },
-        {
-            ngx_string("tls13 quic iv"),
-            &next->client.iv,
-            &next->client.secret,
-        },
-        {
-            ngx_string("tls13 quic ku"),
-            &next->server.secret,
-            &current->server.secret,
-        },
-        {
-            ngx_string("tls13 quic key"),
-            &next->server.key,
-            &next->server.secret,
-        },
-        {
-            ngx_string("tls13 quic iv"),
-            &next->server.iv,
-            &next->server.secret,
-        },
+    ngx_quic_hkdf_t seq[] = {
+        ngx_quic_hkdf_set("tls13 quic ku",
+                          &next->client.secret, &current->client.secret),
+        ngx_quic_hkdf_set("tls13 quic key",
+                          &next->client.key, &next->client.secret),
+        ngx_quic_hkdf_set("tls13 quic iv",
+                          &next->client.iv, &next->client.secret),
+        ngx_quic_hkdf_set("tls13 quic ku",
+                          &next->server.secret, &current->server.secret),
+        ngx_quic_hkdf_set("tls13 quic key",
+                          &next->server.key, &next->server.secret),
+        ngx_quic_hkdf_set("tls13 quic iv",
+                          &next->server.iv, &next->server.secret),
     };
 
     for (i = 0; i < (sizeof(seq) / sizeof(seq[0])); i++) {
-
-        if (ngx_quic_hkdf_expand(c->pool, ciphers.d, seq[i].key, &seq[i].label,
-                                 seq[i].secret->data, seq[i].secret->len)
-            != NGX_OK)
-        {
+        if (ngx_quic_hkdf_expand(&seq[i], ciphers.d, c->pool) != NGX_OK) {
             return NGX_ERROR;
         }
     }
@@ -909,7 +907,7 @@ ngx_quic_create_retry_packet(ngx_quic_he
     }
 
     secret.key.len = sizeof(key);
-    secret.key.data = key;
+    ngx_memcpy(secret.key.data, key, sizeof(key));
     secret.iv.len = NGX_QUIC_IV_LEN;
 
     if (ngx_quic_tls_seal(ciphers.c, &secret, &itag, nonce, &in, &ad, pkt->log)


From arut at nginx.com  Tue May 31 07:06:32 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 31 May 2022 11:06:32 +0400
Subject: [PATCH 2 of 4] QUIC: avoided pool usage in ngx_quic_protection.c
In-Reply-To: <patchbomb.1653980790@arut-laptop>
References: <patchbomb.1653980790@arut-laptop>
Message-ID: <41f47332273e03501572.1653980792@arut-laptop>

# HG changeset patch
# User Vladimir Homutov <vl at nginx.com>
# Date 1653644250 -14400
#      Fri May 27 13:37:30 2022 +0400
# Branch quic
# Node ID 41f47332273e0350157258cc40dd0ede4ee86c69
# Parent  a881ff28070262f3810517d5d3cb4ff67a4b7121
QUIC: avoided pool usage in ngx_quic_protection.c.

diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c
--- a/src/event/quic/ngx_event_quic.c
+++ b/src/event/quic/ngx_event_quic.c
@@ -325,7 +325,7 @@ ngx_quic_new_connection(ngx_connection_t
         }
     }
 
-    if (ngx_quic_keys_set_initial_secret(c->pool, qc->keys, &pkt->dcid)
+    if (ngx_quic_keys_set_initial_secret(qc->keys, &pkt->dcid, c->log)
         != NGX_OK)
     {
         return NULL;
diff --git a/src/event/quic/ngx_event_quic_output.c b/src/event/quic/ngx_event_quic_output.c
--- a/src/event/quic/ngx_event_quic_output.c
+++ b/src/event/quic/ngx_event_quic_output.c
@@ -961,7 +961,7 @@ ngx_quic_send_early_cc(ngx_connection_t 
         return NGX_ERROR;
     }
 
-    if (ngx_quic_keys_set_initial_secret(c->pool, pkt.keys, &inpkt->dcid)
+    if (ngx_quic_keys_set_initial_secret(pkt.keys, &inpkt->dcid, c->log)
         != NGX_OK)
     {
         return NGX_ERROR;
diff --git a/src/event/quic/ngx_event_quic_protection.c b/src/event/quic/ngx_event_quic_protection.c
--- a/src/event/quic/ngx_event_quic_protection.c
+++ b/src/event/quic/ngx_event_quic_protection.c
@@ -113,7 +113,7 @@ static ngx_int_t ngx_quic_tls_seal(const
 static ngx_int_t ngx_quic_tls_hp(ngx_log_t *log, const EVP_CIPHER *cipher,
     ngx_quic_secret_t *s, u_char *out, u_char *in);
 static ngx_int_t ngx_quic_hkdf_expand(ngx_quic_hkdf_t *hkdf,
-    const EVP_MD *digest, ngx_pool_t *pool);
+    const EVP_MD *digest, ngx_log_t *log);
 
 static ngx_int_t ngx_quic_create_packet(ngx_quic_header_t *pkt,
     ngx_str_t *res);
@@ -179,8 +179,8 @@ ngx_quic_ciphers(ngx_uint_t id, ngx_quic
 
 
 ngx_int_t
-ngx_quic_keys_set_initial_secret(ngx_pool_t *pool, ngx_quic_keys_t *keys,
-    ngx_str_t *secret)
+ngx_quic_keys_set_initial_secret(ngx_quic_keys_t *keys, ngx_str_t *secret,
+    ngx_log_t *log)
 {
     size_t              is_len;
     uint8_t             is[SHA256_DIGEST_LENGTH];
@@ -217,12 +217,12 @@ ngx_quic_keys_set_initial_secret(ngx_poo
         .len = is_len
     };
 
-    ngx_log_debug0(NGX_LOG_DEBUG_EVENT, pool->log, 0,
+    ngx_log_debug0(NGX_LOG_DEBUG_EVENT, log, 0,
                    "quic ngx_quic_set_initial_secret");
 #ifdef NGX_QUIC_DEBUG_CRYPTO
-    ngx_log_debug3(NGX_LOG_DEBUG_EVENT, pool->log, 0,
+    ngx_log_debug3(NGX_LOG_DEBUG_EVENT, log, 0,
                    "quic salt len:%uz %*xs", sizeof(salt), sizeof(salt), salt);
-    ngx_log_debug3(NGX_LOG_DEBUG_EVENT, pool->log, 0,
+    ngx_log_debug3(NGX_LOG_DEBUG_EVENT, log, 0,
                    "quic initial secret len:%uz %*xs", is_len, is_len, is);
 #endif
 
@@ -251,7 +251,7 @@ ngx_quic_keys_set_initial_secret(ngx_poo
     };
 
     for (i = 0; i < (sizeof(seq) / sizeof(seq[0])); i++) {
-        if (ngx_quic_hkdf_expand(&seq[i], digest, pool) != NGX_OK) {
+        if (ngx_quic_hkdf_expand(&seq[i], digest, log) != NGX_OK) {
             return NGX_ERROR;
         }
     }
@@ -261,19 +261,12 @@ ngx_quic_keys_set_initial_secret(ngx_poo
 
 
 static ngx_int_t
-ngx_quic_hkdf_expand(ngx_quic_hkdf_t *h, const EVP_MD *digest, ngx_pool_t *pool)
+ngx_quic_hkdf_expand(ngx_quic_hkdf_t *h, const EVP_MD *digest, ngx_log_t *log)
 {
     size_t    info_len;
     uint8_t  *p;
     uint8_t   info[20];
 
-    if (h->out == NULL) {
-        h->out = ngx_pnalloc(pool, h->out_len);
-        if (h->out == NULL) {
-            return NGX_ERROR;
-        }
-    }
-
     info_len = 2 + 1 + h->label_len + 1;
 
     info[0] = 0;
@@ -287,13 +280,13 @@ ngx_quic_hkdf_expand(ngx_quic_hkdf_t *h,
                         h->prk, h->prk_len, info, info_len)
         != NGX_OK)
     {
-        ngx_ssl_error(NGX_LOG_INFO, pool->log, 0,
+        ngx_ssl_error(NGX_LOG_INFO, log, 0,
                       "ngx_hkdf_expand(%*s) failed", h->label_len, h->label);
         return NGX_ERROR;
     }
 
 #ifdef NGX_QUIC_DEBUG_CRYPTO
-    ngx_log_debug5(NGX_LOG_DEBUG_EVENT, pool->log, 0,
+    ngx_log_debug5(NGX_LOG_DEBUG_EVENT, log, 0,
                    "quic expand \"%*s\" key len:%uz %*xs",
                    h->label_len, h->label, h->out_len, h->out_len, h->out);
 #endif
@@ -674,7 +667,7 @@ failed:
 
 
 ngx_int_t
-ngx_quic_keys_set_encryption_secret(ngx_pool_t *pool, ngx_uint_t is_write,
+ngx_quic_keys_set_encryption_secret(ngx_log_t *log, ngx_uint_t is_write,
     ngx_quic_keys_t *keys, enum ssl_encryption_level_t level,
     const SSL_CIPHER *cipher, const uint8_t *secret, size_t secret_len)
 {
@@ -692,12 +685,12 @@ ngx_quic_keys_set_encryption_secret(ngx_
     key_len = ngx_quic_ciphers(keys->cipher, &ciphers, level);
 
     if (key_len == NGX_ERROR) {
-        ngx_ssl_error(NGX_LOG_INFO, pool->log, 0, "unexpected cipher");
+        ngx_ssl_error(NGX_LOG_INFO, log, 0, "unexpected cipher");
         return NGX_ERROR;
     }
 
     if (sizeof(peer_secret->secret.data) < secret_len) {
-        ngx_log_error(NGX_LOG_ALERT, pool->log, 0,
+        ngx_log_error(NGX_LOG_ALERT, log, 0,
                       "unexpected secret len: %uz", secret_len);
         return NGX_ERROR;
     }
@@ -719,7 +712,7 @@ ngx_quic_keys_set_encryption_secret(ngx_
     };
 
     for (i = 0; i < (sizeof(seq) / sizeof(seq[0])); i++) {
-        if (ngx_quic_hkdf_expand(&seq[i], ciphers.d, pool) != NGX_OK) {
+        if (ngx_quic_hkdf_expand(&seq[i], ciphers.d, log) != NGX_OK) {
             return NGX_ERROR;
         }
     }
@@ -809,7 +802,7 @@ ngx_quic_keys_update(ngx_connection_t *c
     };
 
     for (i = 0; i < (sizeof(seq) / sizeof(seq[0])); i++) {
-        if (ngx_quic_hkdf_expand(&seq[i], ciphers.d, c->pool) != NGX_OK) {
+        if (ngx_quic_hkdf_expand(&seq[i], ciphers.d, c->log) != NGX_OK) {
             return NGX_ERROR;
         }
     }
diff --git a/src/event/quic/ngx_event_quic_protection.h b/src/event/quic/ngx_event_quic_protection.h
--- a/src/event/quic/ngx_event_quic_protection.h
+++ b/src/event/quic/ngx_event_quic_protection.h
@@ -18,9 +18,9 @@
 
 
 ngx_quic_keys_t *ngx_quic_keys_new(ngx_pool_t *pool);
-ngx_int_t ngx_quic_keys_set_initial_secret(ngx_pool_t *pool,
-    ngx_quic_keys_t *keys, ngx_str_t *secret);
-ngx_int_t ngx_quic_keys_set_encryption_secret(ngx_pool_t *pool,
+ngx_int_t ngx_quic_keys_set_initial_secret(ngx_quic_keys_t *keys,
+    ngx_str_t *secret, ngx_log_t *log);
+ngx_int_t ngx_quic_keys_set_encryption_secret(ngx_log_t *log,
     ngx_uint_t is_write, ngx_quic_keys_t *keys,
     enum ssl_encryption_level_t level, const SSL_CIPHER *cipher,
     const uint8_t *secret, size_t secret_len);
diff --git a/src/event/quic/ngx_event_quic_ssl.c b/src/event/quic/ngx_event_quic_ssl.c
--- a/src/event/quic/ngx_event_quic_ssl.c
+++ b/src/event/quic/ngx_event_quic_ssl.c
@@ -73,7 +73,7 @@ ngx_quic_set_read_secret(ngx_ssl_conn_t 
                    secret_len, rsecret);
 #endif
 
-    if (ngx_quic_keys_set_encryption_secret(c->pool, 0, qc->keys, level,
+    if (ngx_quic_keys_set_encryption_secret(c->log, 0, qc->keys, level,
                                             cipher, rsecret, secret_len)
         != NGX_OK)
     {
@@ -109,7 +109,7 @@ ngx_quic_set_write_secret(ngx_ssl_conn_t
                    secret_len, wsecret);
 #endif
 
-    if (ngx_quic_keys_set_encryption_secret(c->pool, 1, qc->keys, level,
+    if (ngx_quic_keys_set_encryption_secret(c->log, 1, qc->keys, level,
                                             cipher, wsecret, secret_len)
         != NGX_OK)
     {
@@ -143,7 +143,7 @@ ngx_quic_set_encryption_secrets(ngx_ssl_
 
     cipher = SSL_get_current_cipher(ssl_conn);
 
-    if (ngx_quic_keys_set_encryption_secret(c->pool, 0, qc->keys, level,
+    if (ngx_quic_keys_set_encryption_secret(c->log, 0, qc->keys, level,
                                             cipher, rsecret, secret_len)
         != NGX_OK)
     {
@@ -164,7 +164,7 @@ ngx_quic_set_encryption_secrets(ngx_ssl_
                    secret_len, wsecret);
 #endif
 
-    if (ngx_quic_keys_set_encryption_secret(c->pool, 1, qc->keys, level,
+    if (ngx_quic_keys_set_encryption_secret(c->log, 1, qc->keys, level,
                                             cipher, wsecret, secret_len)
         != NGX_OK)
     {


From arut at nginx.com  Tue May 31 07:06:33 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 31 May 2022 11:06:33 +0400
Subject: [PATCH 3 of 4] QUIC: removed ngx_quic_keys_new()
In-Reply-To: <patchbomb.1653980790@arut-laptop>
References: <patchbomb.1653980790@arut-laptop>
Message-ID: <7929cae8d65fd1f41d07.1653980793@arut-laptop>

# HG changeset patch
# User Vladimir Homutov <vl at nginx.com>
# Date 1653652352 -14400
#      Fri May 27 15:52:32 2022 +0400
# Branch quic
# Node ID 7929cae8d65fd1f41d07365cae93970b29f2d03d
# Parent  41f47332273e0350157258cc40dd0ede4ee86c69
QUIC: removed ngx_quic_keys_new().

The ngx_quic_keys_t structure is now exposed.
This allows to use it in contexts where no pool/connection is available,
i.e. early packet processing.

diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c
--- a/src/event/quic/ngx_event_quic.c
+++ b/src/event/quic/ngx_event_quic.c
@@ -238,7 +238,7 @@ ngx_quic_new_connection(ngx_connection_t
         return NULL;
     }
 
-    qc->keys = ngx_quic_keys_new(c->pool);
+    qc->keys = ngx_pcalloc(c->pool, sizeof(ngx_quic_keys_t));
     if (qc->keys == NULL) {
         return NULL;
     }
diff --git a/src/event/quic/ngx_event_quic_output.c b/src/event/quic/ngx_event_quic_output.c
--- a/src/event/quic/ngx_event_quic_output.c
+++ b/src/event/quic/ngx_event_quic_output.c
@@ -928,6 +928,7 @@ ngx_quic_send_early_cc(ngx_connection_t 
 {
     ssize_t            len;
     ngx_str_t          res;
+    ngx_quic_keys_t    keys;
     ngx_quic_frame_t   frame;
     ngx_quic_header_t  pkt;
 
@@ -956,10 +957,9 @@ ngx_quic_send_early_cc(ngx_connection_t 
         return NGX_ERROR;
     }
 
-    pkt.keys = ngx_quic_keys_new(c->pool);
-    if (pkt.keys == NULL) {
-        return NGX_ERROR;
-    }
+    ngx_memzero(&keys, sizeof(ngx_quic_keys_t));
+
+    pkt.keys = &keys;
 
     if (ngx_quic_keys_set_initial_secret(pkt.keys, &inpkt->dcid, c->log)
         != NGX_OK)
diff --git a/src/event/quic/ngx_event_quic_protection.c b/src/event/quic/ngx_event_quic_protection.c
--- a/src/event/quic/ngx_event_quic_protection.c
+++ b/src/event/quic/ngx_event_quic_protection.c
@@ -10,16 +10,11 @@
 #include <ngx_event_quic_connection.h>
 
 
-/* RFC 5116, 5.1 and RFC 8439, 2.3 for all supported ciphers */
-#define NGX_QUIC_IV_LEN               12
 /* RFC 9001, 5.4.1.  Header Protection Application: 5-byte mask */
 #define NGX_QUIC_HP_LEN               5
 
 #define NGX_QUIC_AES_128_KEY_LEN      16
 
-/* largest hash used in TLS is SHA-384 */
-#define NGX_QUIC_MAX_MD_SIZE          48
-
 #define NGX_AES_128_GCM_SHA256        0x1301
 #define NGX_AES_256_GCM_SHA384        0x1302
 #define NGX_CHACHA20_POLY1305_SHA256  0x1303
@@ -33,45 +28,12 @@
 
 
 typedef struct {
-    size_t                    len;
-    u_char                    data[NGX_QUIC_MAX_MD_SIZE];
-} ngx_quic_md_t;
-
-
-typedef struct {
-    size_t                    len;
-    u_char                    data[NGX_QUIC_IV_LEN];
-} ngx_quic_iv_t;
-
-
-typedef struct {
     const ngx_quic_cipher_t  *c;
     const EVP_CIPHER         *hp;
     const EVP_MD             *d;
 } ngx_quic_ciphers_t;
 
 
-typedef struct ngx_quic_secret_s {
-    ngx_quic_md_t             secret;
-    ngx_quic_md_t             key;
-    ngx_quic_iv_t             iv;
-    ngx_quic_md_t             hp;
-} ngx_quic_secret_t;
-
-
-typedef struct {
-    ngx_quic_secret_t         client;
-    ngx_quic_secret_t         server;
-} ngx_quic_secrets_t;
-
-
-struct ngx_quic_keys_s {
-    ngx_quic_secrets_t        secrets[NGX_QUIC_ENCRYPTION_LAST];
-    ngx_quic_secrets_t        next_key;
-    ngx_uint_t                cipher;
-};
-
-
 typedef struct {
     size_t                    out_len;
     u_char                   *out;
@@ -721,13 +683,6 @@ ngx_quic_keys_set_encryption_secret(ngx_
 }
 
 
-ngx_quic_keys_t *
-ngx_quic_keys_new(ngx_pool_t *pool)
-{
-    return ngx_pcalloc(pool, sizeof(ngx_quic_keys_t));
-}
-
-
 ngx_uint_t
 ngx_quic_keys_available(ngx_quic_keys_t *keys,
     enum ssl_encryption_level_t level)
diff --git a/src/event/quic/ngx_event_quic_protection.h b/src/event/quic/ngx_event_quic_protection.h
--- a/src/event/quic/ngx_event_quic_protection.h
+++ b/src/event/quic/ngx_event_quic_protection.h
@@ -16,8 +16,46 @@
 
 #define NGX_QUIC_ENCRYPTION_LAST  ((ssl_encryption_application) + 1)
 
+/* RFC 5116, 5.1 and RFC 8439, 2.3 for all supported ciphers */
+#define NGX_QUIC_IV_LEN               12
 
-ngx_quic_keys_t *ngx_quic_keys_new(ngx_pool_t *pool);
+/* largest hash used in TLS is SHA-384 */
+#define NGX_QUIC_MAX_MD_SIZE          48
+
+
+typedef struct {
+    size_t                    len;
+    u_char                    data[NGX_QUIC_MAX_MD_SIZE];
+} ngx_quic_md_t;
+
+
+typedef struct {
+    size_t                    len;
+    u_char                    data[NGX_QUIC_IV_LEN];
+} ngx_quic_iv_t;
+
+
+typedef struct ngx_quic_secret_s {
+    ngx_quic_md_t             secret;
+    ngx_quic_md_t             key;
+    ngx_quic_iv_t             iv;
+    ngx_quic_md_t             hp;
+} ngx_quic_secret_t;
+
+
+typedef struct {
+    ngx_quic_secret_t         client;
+    ngx_quic_secret_t         server;
+} ngx_quic_secrets_t;
+
+
+struct ngx_quic_keys_s {
+    ngx_quic_secrets_t        secrets[NGX_QUIC_ENCRYPTION_LAST];
+    ngx_quic_secrets_t        next_key;
+    ngx_uint_t                cipher;
+};
+
+
 ngx_int_t ngx_quic_keys_set_initial_secret(ngx_quic_keys_t *keys,
     ngx_str_t *secret, ngx_log_t *log);
 ngx_int_t ngx_quic_keys_set_encryption_secret(ngx_log_t *log,


From arut at nginx.com  Tue May 31 07:06:34 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 31 May 2022 11:06:34 +0400
Subject: [PATCH 4 of 4] QUIC: avoided pool usage in token calculation
In-Reply-To: <patchbomb.1653980790@arut-laptop>
References: <patchbomb.1653980790@arut-laptop>
Message-ID: <a231f30606317a61efa3.1653980794@arut-laptop>

# HG changeset patch
# User Vladimir Homutov <vl at nginx.com>
# Date 1653980722 -14400
#      Tue May 31 11:05:22 2022 +0400
# Branch quic
# Node ID a231f30606317a61efa3005fda9b43eb91d148d9
# Parent  7929cae8d65fd1f41d07365cae93970b29f2d03d
QUIC: avoided pool usage in token calculation.

diff --git a/src/event/quic/ngx_event_quic_output.c b/src/event/quic/ngx_event_quic_output.c
--- a/src/event/quic/ngx_event_quic_output.c
+++ b/src/event/quic/ngx_event_quic_output.c
@@ -1009,10 +1009,14 @@ ngx_quic_send_retry(ngx_connection_t *c,
 
     u_char             buf[NGX_QUIC_RETRY_BUFFER_SIZE];
     u_char             dcid[NGX_QUIC_SERVER_CID_LEN];
+    u_char             tbuf[NGX_QUIC_TOKEN_BUF_SIZE];
 
     expires = ngx_time() + NGX_QUIC_RETRY_TOKEN_LIFETIME;
 
-    if (ngx_quic_new_token(c, c->sockaddr, c->socklen, conf->av_token_key,
+    token.data = tbuf;
+    token.len = NGX_QUIC_TOKEN_BUF_SIZE;
+
+    if (ngx_quic_new_token(c->log, c->sockaddr, c->socklen, conf->av_token_key,
                            &token, &inpkt->dcid, expires, 1)
         != NGX_OK)
     {
@@ -1075,11 +1079,16 @@ ngx_quic_send_new_token(ngx_connection_t
     ngx_quic_frame_t       *frame;
     ngx_quic_connection_t  *qc;
 
+    u_char                  tbuf[NGX_QUIC_TOKEN_BUF_SIZE];
+
     qc = ngx_quic_get_connection(c);
 
     expires = ngx_time() + NGX_QUIC_NEW_TOKEN_LIFETIME;
 
-    if (ngx_quic_new_token(c, path->sockaddr, path->socklen,
+    token.data = tbuf;
+    token.len = NGX_QUIC_TOKEN_BUF_SIZE;
+
+    if (ngx_quic_new_token(c->log, path->sockaddr, path->socklen,
                            qc->conf->av_token_key, &token, NULL, expires, 0)
         != NGX_OK)
     {
diff --git a/src/event/quic/ngx_event_quic_tokens.c b/src/event/quic/ngx_event_quic_tokens.c
--- a/src/event/quic/ngx_event_quic_tokens.c
+++ b/src/event/quic/ngx_event_quic_tokens.c
@@ -11,14 +11,6 @@
 #include <ngx_event_quic_connection.h>
 
 
-#define NGX_QUIC_MAX_TOKEN_SIZE              64
-    /* SHA-1(addr)=20 + sizeof(time_t) + retry(1) + odcid.len(1) + odcid */
-
-/* RFC 3602, 2.1 and 2.4 for AES-CBC block size and IV length */
-#define NGX_QUIC_AES_256_CBC_IV_LEN          16
-#define NGX_QUIC_AES_256_CBC_BLOCK_SIZE      16
-
-
 static void ngx_quic_address_hash(struct sockaddr *sockaddr, socklen_t socklen,
     ngx_uint_t no_port, u_char buf[20]);
 
@@ -48,7 +40,7 @@ ngx_quic_new_sr_token(ngx_connection_t *
 
 
 ngx_int_t
-ngx_quic_new_token(ngx_connection_t *c, struct sockaddr *sockaddr,
+ngx_quic_new_token(ngx_log_t *log, struct sockaddr *sockaddr,
     socklen_t socklen, u_char *key, ngx_str_t *token, ngx_str_t *odcid,
     time_t exp, ngx_uint_t is_retry)
 {
@@ -80,9 +72,9 @@ ngx_quic_new_token(ngx_connection_t *c, 
     cipher = EVP_aes_256_cbc();
     iv_len = NGX_QUIC_AES_256_CBC_IV_LEN;
 
-    token->len = iv_len + len + NGX_QUIC_AES_256_CBC_BLOCK_SIZE;
-    token->data = ngx_pnalloc(c->pool, token->len);
-    if (token->data == NULL) {
+    if ((size_t) (iv_len + len + NGX_QUIC_AES_256_CBC_BLOCK_SIZE) > token->len)
+    {
+        ngx_log_error(NGX_LOG_ALERT, log, 0, "quic token buffer is too small");
         return NGX_ERROR;
     }
 
@@ -119,7 +111,7 @@ ngx_quic_new_token(ngx_connection_t *c, 
     EVP_CIPHER_CTX_free(ctx);
 
 #ifdef NGX_QUIC_DEBUG_PACKETS
-    ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0,
+    ngx_log_debug2(NGX_LOG_DEBUG_EVENT, log, 0,
                    "quic new token len:%uz %xV", token->len, token);
 #endif
 
@@ -268,10 +260,8 @@ ngx_quic_validate_token(ngx_connection_t
 
     if (odcid.len) {
         pkt->odcid.len = odcid.len;
-        pkt->odcid.data = ngx_pstrdup(c->pool, &odcid);
-        if (pkt->odcid.data == NULL) {
-            return NGX_ERROR;
-        }
+        pkt->odcid.data = pkt->odcid_buf;
+        ngx_memcpy(pkt->odcid.data, odcid.data, odcid.len);
 
     } else {
         pkt->odcid = pkt->dcid;
diff --git a/src/event/quic/ngx_event_quic_tokens.h b/src/event/quic/ngx_event_quic_tokens.h
--- a/src/event/quic/ngx_event_quic_tokens.h
+++ b/src/event/quic/ngx_event_quic_tokens.h
@@ -12,9 +12,21 @@
 #include <ngx_core.h>
 
 
+#define NGX_QUIC_MAX_TOKEN_SIZE              64
+    /* SHA-1(addr)=20 + sizeof(time_t) + retry(1) + odcid.len(1) + odcid */
+
+/* RFC 3602, 2.1 and 2.4 for AES-CBC block size and IV length */
+#define NGX_QUIC_AES_256_CBC_IV_LEN          16
+#define NGX_QUIC_AES_256_CBC_BLOCK_SIZE      16
+
+#define NGX_QUIC_TOKEN_BUF_SIZE             (NGX_QUIC_AES_256_CBC_IV_LEN      \
+                                             + NGX_QUIC_MAX_TOKEN_SIZE        \
+                                             + NGX_QUIC_AES_256_CBC_BLOCK_SIZE)
+
+
 ngx_int_t ngx_quic_new_sr_token(ngx_connection_t *c, ngx_str_t *cid,
     u_char *secret, u_char *token);
-ngx_int_t ngx_quic_new_token(ngx_connection_t *c, struct sockaddr *sockaddr,
+ngx_int_t ngx_quic_new_token(ngx_log_t *log, struct sockaddr *sockaddr,
     socklen_t socklen, u_char *key, ngx_str_t *token, ngx_str_t *odcid,
     time_t expires, ngx_uint_t is_retry);
 ngx_int_t ngx_quic_validate_token(ngx_connection_t *c,
diff --git a/src/event/quic/ngx_event_quic_transport.h b/src/event/quic/ngx_event_quic_transport.h
--- a/src/event/quic/ngx_event_quic_transport.h
+++ b/src/event/quic/ngx_event_quic_transport.h
@@ -322,6 +322,7 @@ typedef struct {
 
     /* cleartext fields */
     ngx_str_t                                   odcid; /* retry packet tag */
+    u_char                                      odcid_buf[NGX_QUIC_MAX_CID_LEN];
     ngx_str_t                                   dcid;
     ngx_str_t                                   scid;
     uint64_t                                    pn;


From xeioex at nginx.com  Tue May 31 16:32:56 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 31 May 2022 16:32:56 +0000
Subject: [njs] Version bump.
Message-ID: <hg.4e4bfbc09cb6.1654014776.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/4e4bfbc09cb6
branches:  
changeset: 1866:4e4bfbc09cb6
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue May 31 08:53:18 2022 -0700
description:
Version bump.

diffstat:

 src/njs.h |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff -r 640ed49b2dce -r 4e4bfbc09cb6 src/njs.h
--- a/src/njs.h	Tue May 24 09:10:52 2022 -0700
+++ b/src/njs.h	Tue May 31 08:53:18 2022 -0700
@@ -11,8 +11,8 @@
 
 #include <njs_auto_config.h>
 
-#define NJS_VERSION                 "0.7.4"
-#define NJS_VERSION_NUMBER          0x000704
+#define NJS_VERSION                 "0.7.5"
+#define NJS_VERSION_NUMBER          0x000705
 
 
 #include <unistd.h>                 /* STDOUT_FILENO, STDERR_FILENO */


From xeioex at nginx.com  Tue May 31 16:32:58 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 31 May 2022 16:32:58 +0000
Subject: [njs] HTTP: improved memory allocation error handling.
Message-ID: <hg.6ccb060f328b.1654014778.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/6ccb060f328b
branches:  
changeset: 1867:6ccb060f328b
user:      Sergey Kandaurov <pluknet at nginx.com>
date:      Thu May 26 15:28:06 2022 +0400
description:
HTTP: improved memory allocation error handling.

diffstat:

 nginx/ngx_http_js_module.c |  5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diffs (43 lines):

diff -r 4e4bfbc09cb6 -r 6ccb060f328b nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c	Tue May 31 08:53:18 2022 -0700
+++ b/nginx/ngx_http_js_module.c	Thu May 26 15:28:06 2022 +0400
@@ -1563,6 +1563,7 @@ ngx_http_js_header_out_special(njs_vm_t 
 
         p = ngx_pnalloc(r->pool, v->length);
         if (p == NULL) {
+            h->hash = 0;
             return NJS_ERROR;
         }
 
@@ -1575,6 +1576,7 @@ ngx_http_js_header_out_special(njs_vm_t 
     if (h != NULL) {
         p = ngx_pnalloc(r->pool, s.length);
         if (p == NULL) {
+            h->hash = 0;
             return NJS_ERROR;
         }
 
@@ -1803,6 +1805,7 @@ ngx_http_js_header_generic(njs_vm_t *vm,
 
         p = ngx_pnalloc(r->pool, name->length);
         if (p == NULL) {
+            h->hash = 0;
             return NJS_ERROR;
         }
 
@@ -1813,6 +1816,7 @@ ngx_http_js_header_generic(njs_vm_t *vm,
 
         p = ngx_pnalloc(r->pool, s.length);
         if (p == NULL) {
+            h->hash = 0;
             return NJS_ERROR;
         }
 
@@ -2858,6 +2862,7 @@ ngx_http_js_ext_variables(njs_vm_t *vm, 
 
     vv->data = ngx_pnalloc(r->pool, s.length);
     if (vv->data == NULL) {
+        vv->valid = 0;
         njs_vm_error(vm, "internal error");
         return NJS_ERROR;
     }


From xeioex at nginx.com  Tue May 31 16:33:00 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 31 May 2022 16:33:00 +0000
Subject: [njs] HTTP: adapting to changes in nginx header structures.
Message-ID: <hg.5b7676ec600d.1654014780.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/5b7676ec600d
branches:  
changeset: 1868:5b7676ec600d
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue May 31 09:26:47 2022 -0700
description:
HTTP: adapting to changes in nginx header structures.

diffstat:

 nginx/ngx_http_js_module.c |  617 +++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 617 insertions(+), 0 deletions(-)

diffs (748 lines):

diff -r 6ccb060f328b -r 5b7676ec600d nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c	Thu May 26 15:28:06 2022 +0400
+++ b/nginx/ngx_http_js_module.c	Tue May 31 09:26:47 2022 -0700
@@ -12,6 +12,11 @@
 #include "ngx_js.h"
 
 
+#define NJS_HEADER_SEMICOLON   0x1
+#define NJS_HEADER_SINGLE      0x2
+#define NJS_HEADER_ARRAY       0x4
+
+
 typedef struct {
     njs_vm_t              *vm;
     ngx_array_t           *imports;
@@ -84,10 +89,17 @@ typedef struct {
 
 typedef struct {
     njs_str_t              name;
+#if defined(nginx_version) && (nginx_version >= 1023000)
+    unsigned               flags;
+    njs_int_t            (*handler)(njs_vm_t *vm, ngx_http_request_t *r,
+                                    unsigned flags, njs_str_t *name,
+                                    njs_value_t *setval, njs_value_t *retval);
+#else
     njs_int_t            (*handler)(njs_vm_t *vm, ngx_http_request_t *r,
                                     ngx_list_t *headers, njs_str_t *name,
                                     njs_value_t *setval, njs_value_t *retval);
 
+#endif
 }  ngx_http_js_header_t;
 
 
@@ -107,14 +119,17 @@ static void ngx_http_js_cleanup_vm(void 
 
 static njs_int_t ngx_http_js_ext_keys_header(njs_vm_t *vm, njs_value_t *value,
     njs_value_t *keys, ngx_list_t *headers);
+#if defined(nginx_version) && (nginx_version < 1023000)
 static ngx_table_elt_t *ngx_http_js_get_header(ngx_list_part_t *part,
     u_char *data, size_t len);
+#endif
 static njs_int_t ngx_http_js_ext_raw_header(njs_vm_t *vm,
     njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval,
     njs_value_t *retval);
 static njs_int_t ngx_http_js_ext_header_out(njs_vm_t *vm,
     njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval,
     njs_value_t *retval);
+#if defined(nginx_version) && (nginx_version < 1023000)
 static njs_int_t ngx_http_js_header_single(njs_vm_t *vm,
     ngx_http_request_t *r, ngx_list_t *headers, njs_str_t *name,
     njs_value_t *setval, njs_value_t *retval);
@@ -136,6 +151,7 @@ static njs_int_t ngx_http_js_content_enc
 static njs_int_t ngx_http_js_content_type(njs_vm_t *vm, ngx_http_request_t *r,
     ngx_list_t *headers, njs_str_t *name, njs_value_t *setval,
     njs_value_t *retval);
+#endif
 static njs_int_t ngx_http_js_ext_keys_header_out(njs_vm_t *vm,
     njs_value_t *value, njs_value_t *keys);
 static njs_int_t ngx_http_js_ext_status(njs_vm_t *vm, njs_object_prop_t *prop,
@@ -169,6 +185,7 @@ static njs_int_t ngx_http_js_ext_get_req
 static njs_int_t ngx_http_js_ext_header_in(njs_vm_t *vm,
     njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval,
     njs_value_t *retval);
+#if defined(nginx_version) && (nginx_version < 1023000)
 static njs_int_t ngx_http_js_header_cookie(njs_vm_t *vm,
     ngx_http_request_t *r, ngx_list_t *headers, njs_str_t *name,
     njs_value_t *setval, njs_value_t *retval);
@@ -179,6 +196,7 @@ static njs_int_t ngx_http_js_header_x_fo
 #endif
 static njs_int_t ngx_http_js_header_in_array(njs_vm_t *vm,
     ngx_http_request_t *r, ngx_array_t *array, u_char sep, njs_value_t *retval);
+#endif
 static njs_int_t ngx_http_js_ext_keys_header_in(njs_vm_t *vm,
     njs_value_t *value, njs_value_t *keys);
 static njs_int_t ngx_http_js_ext_get_arg(njs_vm_t *vm,
@@ -203,6 +221,28 @@ static njs_int_t ngx_http_js_ext_get_res
     njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval,
     njs_value_t *retval);
 
+#if defined(nginx_version) && (nginx_version >= 1023000)
+static njs_int_t ngx_http_js_header_in(njs_vm_t *vm, ngx_http_request_t *r,
+    unsigned flags, njs_str_t *name, njs_value_t *retval);
+static njs_int_t ngx_http_js_header_out(njs_vm_t *vm, ngx_http_request_t *r,
+    unsigned flags, njs_str_t *name, njs_value_t *setval, njs_value_t *retval);
+static njs_int_t ngx_http_js_content_encoding(njs_vm_t *vm,
+    ngx_http_request_t *r, unsigned flags, njs_str_t *name,
+    njs_value_t *setval, njs_value_t *retval);
+static njs_int_t ngx_http_js_content_length(njs_vm_t *vm, ngx_http_request_t *r,
+    unsigned flags, njs_str_t *name, njs_value_t *setval,
+    njs_value_t *retval);
+static njs_int_t ngx_http_js_content_type(njs_vm_t *vm, ngx_http_request_t *r,
+    unsigned flags, njs_str_t *name, njs_value_t *setval,
+    njs_value_t *retval);
+static njs_int_t ngx_http_js_header_out_special(njs_vm_t *vm,
+    ngx_http_request_t *r, njs_str_t *v, njs_value_t *setval,
+    njs_value_t *retval, ngx_table_elt_t **hh);
+static njs_int_t ngx_http_js_header_generic(njs_vm_t *vm,
+    ngx_http_request_t *r, ngx_list_t *headers, ngx_table_elt_t **ph,
+    unsigned flags, njs_str_t *name, njs_value_t *retval);
+#endif
+
 static njs_host_event_t ngx_http_js_set_timer(njs_external_ptr_t external,
     uint64_t delay, njs_vm_event_t vm_event);
 static void ngx_http_js_clear_timer(njs_external_ptr_t external,
@@ -1319,6 +1359,7 @@ ngx_http_js_ext_keys_header(njs_vm_t *vm
 }
 
 
+#if defined(nginx_version) && (nginx_version < 1023000)
 static ngx_table_elt_t *
 ngx_http_js_get_header(ngx_list_part_t *part, u_char *data, size_t len)
 {
@@ -1352,6 +1393,7 @@ ngx_http_js_get_header(ngx_list_part_t *
 
     return NULL;
 }
+#endif
 
 
 static njs_int_t
@@ -1446,6 +1488,7 @@ ngx_http_js_ext_header_out(njs_vm_t *vm,
     ngx_http_js_header_t  *h;
 
     static ngx_http_js_header_t headers_out[] = {
+#if defined(nginx_version) && (nginx_version < 1023000)
         { njs_str("Age"), ngx_http_js_header_single },
         { njs_str("Content-Type"), ngx_http_js_content_type },
         { njs_str("Content-Length"), ngx_http_js_content_length },
@@ -1457,6 +1500,19 @@ ngx_http_js_ext_header_out(njs_vm_t *vm,
         { njs_str("Set-Cookie"), ngx_http_js_header_array },
         { njs_str("Retry-After"), ngx_http_js_header_single },
         { njs_str(""), ngx_http_js_header_generic },
+#else
+        { njs_str("Age"), NJS_HEADER_SINGLE, ngx_http_js_header_out },
+        { njs_str("Content-Encoding"), 0, ngx_http_js_content_encoding },
+        { njs_str("Content-Length"), 0, ngx_http_js_content_length },
+        { njs_str("Content-Type"), 0, ngx_http_js_content_type },
+        { njs_str("Etag"), NJS_HEADER_SINGLE, ngx_http_js_header_out },
+        { njs_str("Expires"), NJS_HEADER_SINGLE, ngx_http_js_header_out },
+        { njs_str("Last-Modified"), NJS_HEADER_SINGLE, ngx_http_js_header_out },
+        { njs_str("Location"), NJS_HEADER_SINGLE, ngx_http_js_header_out },
+        { njs_str("Set-Cookie"), NJS_HEADER_ARRAY, ngx_http_js_header_out },
+        { njs_str("Retry-After"), NJS_HEADER_SINGLE, ngx_http_js_header_out },
+        { njs_str(""), 0, ngx_http_js_header_out },
+#endif
     };
 
     r = njs_vm_external(vm, ngx_http_js_request_proto_id, value);
@@ -1485,10 +1541,15 @@ ngx_http_js_ext_header_out(njs_vm_t *vm,
         }
     }
 
+#if defined(nginx_version) && (nginx_version < 1023000)
     return h->handler(vm, r, &r->headers_out.headers, &name, setval, retval);
+#else
+    return h->handler(vm, r, h->flags, &name, setval, retval);
+#endif
 }
 
 
+#if defined(nginx_version) && (nginx_version < 1023000)
 static njs_int_t
 ngx_http_js_header_single(njs_vm_t *vm, ngx_http_request_t *r,
     ngx_list_t *headers, njs_str_t *name, njs_value_t *setval,
@@ -1940,6 +2001,7 @@ ngx_http_js_content_type(njs_vm_t *vm, n
 
     return NJS_OK;
 }
+#endif
 
 
 static njs_int_t
@@ -2545,6 +2607,7 @@ done:
 }
 
 
+#if defined(nginx_version) && (nginx_version < 1023000)
 static njs_int_t
 ngx_http_js_ext_header_in(njs_vm_t *vm, njs_object_prop_t *prop,
     njs_value_t *value, njs_value_t *setval, njs_value_t *retval)
@@ -2672,6 +2735,59 @@ ngx_http_js_header_in_array(njs_vm_t *vm
 
     return NJS_OK;
 }
+#else
+static njs_int_t
+ngx_http_js_ext_header_in(njs_vm_t *vm, njs_object_prop_t *prop,
+    njs_value_t *value, njs_value_t *unused, njs_value_t *retval)
+{
+    unsigned             flags;
+    njs_int_t            rc;
+    njs_str_t            name, *h;
+    ngx_http_request_t  *r;
+
+    static njs_str_t single_headers_in[] = {
+        njs_str("Content-Type"),
+        njs_str("ETag"),
+        njs_str("From"),
+        njs_str("Max-Forwards"),
+        njs_str("Referer"),
+        njs_str("Proxy-Authorization"),
+        njs_str("User-Agent"),
+        njs_str(""),
+    };
+
+    r = njs_vm_external(vm, ngx_http_js_request_proto_id, value);
+    if (r == NULL) {
+        if (retval != NULL) {
+            njs_value_undefined_set(retval);
+        }
+
+        return NJS_DECLINED;
+    }
+
+    rc = njs_vm_prop_name(vm, prop, &name);
+    if (rc != NJS_OK) {
+        if (retval != NULL) {
+            njs_value_undefined_set(retval);
+        }
+
+        return NJS_DECLINED;
+    }
+
+    flags = 0;
+
+    for (h = single_headers_in; h->length > 0; h++) {
+        if (h->length == name.length
+            && ngx_strncasecmp(h->start, name.start, name.length) == 0)
+        {
+            flags |= NJS_HEADER_SINGLE;
+            break;
+        }
+    }
+
+    return ngx_http_js_header_in(vm, r, flags, &name, retval);
+}
+#endif
 
 
 static njs_int_t
@@ -3392,6 +3508,507 @@ ngx_http_js_ext_get_response_body(njs_vm
 }
 
 
+#if defined(nginx_version) && (nginx_version >= 1023000)
+static njs_int_t
+ngx_http_js_header_in(njs_vm_t *vm, ngx_http_request_t *r, unsigned flags,
+    njs_str_t *name, njs_value_t *retval)
+{
+    u_char                      *lowcase_key;
+    ngx_uint_t                   hash;
+    ngx_table_elt_t            **ph;
+    ngx_http_header_t           *hh;
+    ngx_http_core_main_conf_t   *cmcf;
+
+    if (retval == NULL) {
+        return NJS_OK;
+    }
+
+    /* look up hashed headers */
+
+    lowcase_key = ngx_pnalloc(r->pool, name->length);
+    if (lowcase_key == NULL) {
+        njs_vm_memory_error(vm);
+        return NJS_ERROR;
+    }
+
+    hash = ngx_hash_strlow(lowcase_key, name->start, name->length);
+
+    cmcf = ngx_http_get_module_main_conf(r, ngx_http_core_module);
+
+    hh = ngx_hash_find(&cmcf->headers_in_hash, hash, lowcase_key,
+                       name->length);
+
+    ph = NULL;
+
+    if (hh) {
+        if (hh->offset == offsetof(ngx_http_headers_in_t, cookie)) {
+            flags |= NJS_HEADER_SEMICOLON;
+        }
+
+        ph = (ngx_table_elt_t **) ((char *) &r->headers_in + hh->offset);
+    }
+
+    return ngx_http_js_header_generic(vm, r, &r->headers_in.headers, ph, flags,
+                                      name, retval);
+}
+
+
+static njs_int_t
+ngx_http_js_header_out(njs_vm_t *vm, ngx_http_request_t *r, unsigned flags,
+    njs_str_t *name, njs_value_t *setval, njs_value_t *retval)
+{
+    u_char              *p;
+    int64_t              length;
+    njs_value_t         *array;
+    njs_int_t            rc;
+    njs_str_t            s;
+    ngx_uint_t           i;
+    ngx_list_part_t     *part;
+    ngx_table_elt_t     *header, *h, **ph;
+    njs_opaque_value_t   lvalue;
+
+    if (retval != NULL && setval == NULL) {
+        return ngx_http_js_header_generic(vm, r, &r->headers_out.headers, NULL,
+                                          flags, name, retval);
+
+    }
+
+    part = &r->headers_out.headers.part;
+    header = part->elts;
+
+    for (i = 0; /* void */ ; i++) {
+
+        if (i >= part->nelts) {
+            if (part->next == NULL) {
+                break;
+            }
+
+            part = part->next;
+            header = part->elts;
+            i = 0;
+        }
+
+        h = &header[i];
+
+        if (h->hash == 0
+            || h->key.len != name->length
+            || ngx_strncasecmp(h->key.data, name->start, name->length) != 0)
+        {
+            continue;
+        }
+
+        h->hash = 0;
+        h->next = NULL;
+    }
+
+    if (retval == NULL) {
+        return NJS_OK;
+    }
+
+    if (njs_value_is_array(setval)) {
+        array = setval;
+
+        rc = njs_vm_array_length(vm, array, &length);
+        if (rc != NJS_OK) {
+            return NJS_ERROR;
+        }
+
+        if (length == 0) {
+            return NJS_OK;
+        }
+
+    } else {
+        array = NULL;
+        length = 1;
+    }
+
+    ph = &header;
+
+    for (i = 0; i < (ngx_uint_t) length; i++) {
+        if (array != NULL) {
+            setval = njs_vm_array_prop(vm, array, i, &lvalue);
+        }
+
+        if (ngx_js_string(vm, setval, &s) != NGX_OK) {
+            return NJS_ERROR;
+        }
+
+        if (s.length == 0) {
+            continue;
+        }
+
+        h = ngx_list_push(&r->headers_out.headers);
+        if (h == NULL) {
+            njs_vm_memory_error(vm);
+            return NJS_ERROR;
+        }
+
+        p = ngx_pnalloc(r->pool, name->length);
+        if (p == NULL) {
+            h->hash = 0;
+            njs_vm_memory_error(vm);
+            return NJS_ERROR;
+        }
+
+        ngx_memcpy(p, name->start, name->length);
+
+        h->key.data = p;
+        h->key.len = name->length;
+
+        p = ngx_pnalloc(r->pool, s.length);
+        if (p == NULL) {
+            h->hash = 0;
+            njs_vm_memory_error(vm);
+            return NJS_ERROR;
+        }
+
+        ngx_memcpy(p, s.start, s.length);
+
+        h->value.data = p;
+        h->value.len = s.length;
+        h->hash = 1;
+
+        *ph = h;
+        ph = &h->next;
+    }
+
+    *ph = NULL;
+
+    return NJS_OK;
+}
+
+
+static njs_int_t
+ngx_http_js_content_encoding(njs_vm_t *vm, ngx_http_request_t *r,
+    unsigned flags, njs_str_t *v, njs_value_t *setval, njs_value_t *retval)
+{
+    njs_int_t         rc;
+    ngx_table_elt_t  *h;
+
+    rc = ngx_http_js_header_out_special(vm, r, v, setval, retval, &h);
+    if (rc != NJS_OK) {
+        return NJS_ERROR;
+    }
+
+    if (setval != NULL || retval == NULL) {
+        r->headers_out.content_encoding = h;
+    }
+
+    return NJS_OK;
+}
+
+
+static njs_int_t
+ngx_http_js_content_length(njs_vm_t *vm, ngx_http_request_t *r,
+    unsigned flags, njs_str_t *v, njs_value_t *setval, njs_value_t *retval)
+{
+    u_char           *p, *start;
+    njs_int_t         rc;
+    ngx_int_t         n;
+    ngx_table_elt_t  *h;
+    u_char            content_len[NGX_OFF_T_LEN];
+
+    if (retval != NULL && setval == NULL) {
+        if (r->headers_out.content_length == NULL
+            && r->headers_out.content_length_n >= 0)
+        {
+            p = ngx_sprintf(content_len, "%O", r->headers_out.content_length_n);
+
+            start = njs_vm_value_string_alloc(vm, retval, p - content_len);
+            if (start == NULL) {
+                return NJS_ERROR;
+            }
+
+            ngx_memcpy(start, content_len, p - content_len);
+
+            return NJS_OK;
+        }
+    }
+
+    rc = ngx_http_js_header_out_special(vm, r, v, setval, retval, &h);
+    if (rc != NJS_OK) {
+        return NJS_ERROR;
+    }
+
+    if (setval != NULL || retval == NULL) {
+        if (h != NULL) {
+            n = ngx_atoi(h->value.data, h->value.len);
+            if (n == NGX_ERROR) {
+                h->hash = 0;
+                njs_vm_error(vm, "failed converting argument "
+                             "to positive integer");
+                return NJS_ERROR;
+            }
+
+            r->headers_out.content_length = h;
+            r->headers_out.content_length_n = n;
+
+        } else {
+            ngx_http_clear_content_length(r);
+        }
+    }
+
+    return NJS_OK;
+}
+
+
+static njs_int_t
+ngx_http_js_content_type(njs_vm_t *vm, ngx_http_request_t *r,
+    unsigned flags, njs_str_t *v, njs_value_t *setval, njs_value_t *retval)
+{
+    int64_t              length;
+    njs_int_t            rc;
+    njs_str_t            s;
+    ngx_str_t           *hdr;
+    njs_opaque_value_t   lvalue;
+
+    if (retval != NULL && setval == NULL) {
+        hdr = &r->headers_out.content_type;
+        return njs_vm_value_string_set(vm, retval, hdr->data, hdr->len);
+    }
+
+    if (setval != NULL && njs_value_is_array(setval)) {
+        rc = njs_vm_array_length(vm, setval, &length);
+        if (rc != NJS_OK) {
+            return NJS_ERROR;
+        }
+
+        setval = njs_vm_array_prop(vm, setval, length - 1, &lvalue);
+    }
+
+    if (ngx_js_string(vm, setval, &s) != NGX_OK) {
+        return NJS_ERROR;
+    }
+
+    r->headers_out.content_type.len = s.length;
+    r->headers_out.content_type_len = r->headers_out.content_type.len;
+    r->headers_out.content_type.data = s.start;
+    r->headers_out.content_type_lowcase = NULL;
+
+    return NJS_OK;
+}
+
+
+static njs_int_t
+ngx_http_js_header_out_special(njs_vm_t *vm, ngx_http_request_t *r,
+    njs_str_t *v, njs_value_t *setval, njs_value_t *retval,
+    ngx_table_elt_t **hh)
+{
+    u_char              *p;
+    int64_t              length;
+    njs_int_t            rc;
+    njs_str_t            s;
+    ngx_uint_t           i;
+    ngx_list_t          *headers;
+    ngx_list_part_t     *part;
+    ngx_table_elt_t     *header, *h;
+    njs_opaque_value_t   lvalue;
+
+    headers = &r->headers_out.headers;
+
+    if (retval != NULL && setval == NULL) {
+        return ngx_http_js_header_out(vm, r, NJS_HEADER_SINGLE, v, setval,
+                                      retval);
+    }
+
+    if (setval != NULL && njs_value_is_array(setval)) {
+        rc = njs_vm_array_length(vm, setval, &length);
+        if (rc != NJS_OK) {
+            return NJS_ERROR;
+        }
+
+        setval = njs_vm_array_prop(vm, setval, length - 1, &lvalue);
+    }
+
+    if (ngx_js_string(vm, setval, &s) != NGX_OK) {
+        return NJS_ERROR;
+    }
+
+    h = NULL;
+    part = &headers->part;
+    header = part->elts;
+
+    for (i = 0; /* void */ ; i++) {
+
+        if (i >= part->nelts) {
+            if (part->next == NULL) {
+                break;
+            }
+
+            part = part->next;
+            header = part->elts;
+            i = 0;
+        }
+
+        h = &header[i];
+
+        if (h->hash == 0) {
+            continue;
+        }
+
+        if (h->key.len == v->length
+            && ngx_strncasecmp(h->key.data, v->start, v->length) == 0)
+        {
+            break;
+        }
+    }
+
+    if (h != NULL && s.length == 0) {
+        h->hash = 0;
+        h = NULL;
+    }
+
+    if (h == NULL && s.length != 0) {
+        h = ngx_list_push(headers);
+        if (h == NULL) {
+            njs_vm_memory_error(vm);
+            return NJS_ERROR;
+        }
+
+        p = ngx_pnalloc(r->pool, v->length);
+        if (p == NULL) {
+            h->hash = 0;
+            njs_vm_memory_error(vm);
+            return NJS_ERROR;
+        }
+
+        ngx_memcpy(p, v->start, v->length);
+
+        h->key.data = p;
+        h->key.len = v->length;
+    }
+
+    if (h != NULL) {
+        p = ngx_pnalloc(r->pool, s.length);
+        if (p == NULL) {
+            h->hash = 0;
+            njs_vm_memory_error(vm);
+            return NJS_ERROR;
+        }
+
+        ngx_memcpy(p, s.start, s.length);
+
+        h->value.data = p;
+        h->value.len = s.length;
+        h->hash = 1;
+    }
+
+    if (hh != NULL) {
+        *hh = h;
+    }
+
+    return NJS_OK;
+}
+
+
+static njs_int_t
+ngx_http_js_header_generic(njs_vm_t *vm, ngx_http_request_t *r,
+    ngx_list_t *headers, ngx_table_elt_t **ph, unsigned flags, njs_str_t *name,
+    njs_value_t *retval)
+{
+    u_char           *p, sep;
+    ssize_t           size;
+    njs_int_t         rc;
+    ngx_uint_t        i;
+    njs_value_t      *value;
+    ngx_list_part_t  *part;
+    ngx_table_elt_t  *header, *h;
+
+    if (ph == NULL) {
+        /* iterate over all headers */
+
+        ph = &header;
+        part = &headers->part;
+        h = part->elts;
+
+        for (i = 0; /* void */ ; i++) {
+
+            if (i >= part->nelts) {
+                if (part->next == NULL) {
+                    break;
+                }
+
+                part = part->next;
+                h = part->elts;
+                i = 0;
+            }
+
+            if (h[i].hash == 0
+                || name->length != h[i].key.len
+                || ngx_strncasecmp(name->start, h[i].key.data, name->length)
+                   != 0)
+            {
+                continue;
+            }
+
+            *ph = &h[i];
+            ph = &h[i].next;
+        }
+
+        *ph = NULL;
+        ph = &header;
+    }
+
+    if (*ph == NULL) {
+        njs_value_undefined_set(retval);
+        return NJS_DECLINED;
+    }
+
+    if (flags & NJS_HEADER_ARRAY) {
+        rc = njs_vm_array_alloc(vm, retval, 4);
+        if (rc != NJS_OK) {
+            return NJS_ERROR;
+        }
+
+        for (h = *ph; h; h = h->next) {
+            value = njs_vm_array_push(vm, retval);
+            if (value == NULL) {
+                return NJS_ERROR;
+            }
+
+            rc = njs_vm_value_string_set(vm, value, h->value.data,
+                                         h->value.len);
+            if (rc != NJS_OK) {
+                return NJS_ERROR;
+            }
+        }
+
+        return NJS_OK;
+    }
+
+    if ((*ph)->next == NULL || flags & NJS_HEADER_SINGLE) {
+        return njs_vm_value_string_set(vm, retval, (*ph)->value.data,
+                                       (*ph)->value.len);
+    }
+
+    size = - (ssize_t) njs_length("; ");
+
+    for (h = *ph; h; h = h->next) {
+        size += h->value.len + njs_length("; ");
+    }
+
+    p = njs_vm_value_string_alloc(vm, retval, size);
+    if (p == NULL) {
+        return NJS_ERROR;
+    }
+
+    sep = flags & NJS_HEADER_SEMICOLON ? ';' : ',';
+
+    for (h = *ph; h; h = h->next) {
+        p = ngx_copy(p, h->value.data, h->value.len);
+
+        if (h->next == NULL) {
+            break;
+        }
+
+        *p++ = sep; *p++ = ' ';
+    }
+
+    return NJS_OK;
+}
+#endif
+
+
 static njs_host_event_t
 ngx_http_js_set_timer(njs_external_ptr_t external, uint64_t delay,
     njs_vm_event_t vm_event)


From P.Pautov at F5.com  Tue May 31 21:24:45 2022
From: P.Pautov at F5.com (Pavel Pautov)
Date: Tue, 31 May 2022 21:24:45 +0000
Subject: [PATCH] Upstream: handling of certificates specified as an empty
 string
In-Reply-To: <30373D2B-9DFD-413F-A143-1EC0A44AFCE3@nginx.com>
References: <3bb1adbb74dfcd372f73.1653422757@enoparse.local>
 <MW4PR01MB6499737D56ED6863B0A9D3E5B1D89@MW4PR01MB6499.prod.exchangelabs.com>
 <30373D2B-9DFD-413F-A143-1EC0A44AFCE3@nginx.com>
Message-ID:  <SJ0PR01MB6510CED2B1C9E924679C159FB1DC9@SJ0PR01MB6510.prod.exchangelabs.com>

LGTM.

> -----Original Message-----
> From: Sergey Kandaurov <pluknet at nginx.com>
> Sent: Monday, May 30, 2022 15:54
> To: nginx-devel at nginx.org
> Cc: Pavel Pautov <P.Pautov at F5.com>
> Subject: Re: [PATCH] Upstream: handling of certificates specified as an empty
> string
> 
> EXTERNAL MAIL: pluknet at nginx.com
> 
> > On 27 May 2022, at 08:26, Pavel Pautov via nginx-devel <nginx-
> devel at nginx.org> wrote:
> >
> > Hi,
> >
> > Shall we restore original behavior for stream proxy as well?
> >
> 
> Fixed, tnx.  The relevant part:
> 
> diff --git a/src/stream/ngx_stream_proxy_module.c
> b/src/stream/ngx_stream_proxy_module.c
> --- a/src/stream/ngx_stream_proxy_module.c
> +++ b/src/stream/ngx_stream_proxy_module.c
> @@ -2244,7 +2244,7 @@ ngx_stream_proxy_set_ssl(ngx_conf_t *cf,
>                  return NGX_ERROR;
>              }
> 
> -        } else {
> +        } else if (pscf->ssl_certificate->value.len) {
>              if (ngx_ssl_certificate(cf, pscf->ssl,
>                                      &pscf->ssl_certificate->value,
>                                      &pscf->ssl_certificate_key->value,
> 
> --
> Sergey Kandaurov
> 


From pluknet at nginx.com  Tue May 31 23:11:45 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Wed, 1 Jun 2022 03:11:45 +0400
Subject: [PATCH 00 of 10] multiple headers tests
In-Reply-To: <patchbomb.1650494261@vm-bsd.mdounin.ru>
References: <patchbomb.1650493120@vm-bsd.mdounin.ru>
 <patchbomb.1650494261@vm-bsd.mdounin.ru>
Message-ID: <D550DF2C-BA2E-437B-8FDC-2701DFEBA678@nginx.com>


> On 21 Apr 2022, at 02:37, Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> Hello!
> 
> Tests for the multiple headers handling patch series.
> 

What do you think about extending the tests
for ngx_http_link_multi_headers() ?
Searching for previous headers is a novel routine
so it might make sense to improve its coverage.

Something like this:

# HG changeset patch
# User Sergey Kandaurov <pluknet at nginx.com>
# Date 1654038532 -14400
#      Wed Jun 01 03:08:52 2022 +0400
# Node ID 3c2a768ef7e3eff6c943e0fa4dd10c9ec487c973
# Parent  83ec649296126965883e0d65c92ecd99f5397dc7
Tests: added fastcgi tests for catched bugs in combining headers.

This improves coverage in a search for previous headers.

diff --git a/fastcgi_header_params.t b/fastcgi_header_params.t
--- a/fastcgi_header_params.t
+++ b/fastcgi_header_params.t
@@ -25,7 +25,7 @@ eval { require FCGI; };
 plan(skip_all => 'FCGI not installed') if $@;
 plan(skip_all => 'win32') if $^O eq 'MSWin32';
 
-my $t = Test::Nginx->new()->has(qw/http fastcgi/)->plan(4)
+my $t = Test::Nginx->new()->has(qw/http fastcgi/)->plan(5)
 	->write_file_expand('nginx.conf', <<'EOF');
 
 %%TEST_GLOBALS%%
@@ -58,6 +58,8 @@ EOF
 
 like(http_get_headers('/'), qr/SEE-THIS/,
 	'fastcgi request with many ignored headers');
+like(http_get_headers('/', diff => 1), qr/SEE-THIS/,
+	'fastcgi request with many different headers');
 
 TODO: {
 local $TODO = 'not yet' unless $t->has_version('1.23.0');
@@ -94,7 +96,7 @@ like($r, qr/X-Foo: foo, bar, bazz/,
 
 sub http_get_headers {
 	my ($url, %extra) = @_;
-	return http(<<EOF, %extra);
+	my $r = <<EOF;
 GET $url HTTP/1.0
 Host: localhost
 X-Blah: ignored header
@@ -116,8 +118,12 @@ X-Blah: ignored header
 X-Blah: ignored header
 X-Blah: ignored header
 X-Blah: ignored header
+X-Blah: ignored header
 
 EOF
+	my $n = 0;
+	$r =~ s/(?<=Blah)(?=:)/$n++/ge if $extra{diff};
+	return http($r);
 }
 
 ###############################################################################

-- 
Sergey Kandaurov