From wangjiahao at openresty.com Wed Jun 1 04:26:41 2022 From: wangjiahao at openresty.com (Jiahao Wang) Date: Wed, 1 Jun 2022 12:26:41 +0800 Subject: [ANN] Test::Nginx 0.30 is released Message-ID: Hi there, I am happy to announce the new 0.30 release of Test::Nginx: https://openresty.org/en/ann-test-nginx-030.html This version has many new features and fixes several bugs since 0.29, refer to the above link for details. This Perl module provides a test scaffold for automated testing in Nginx C module or OpenResty-based Lua library development and regression testing. This class inherits from Test::Base, thus bringing all its declarative power to the Nginx C module testing practices. All of our OpenResty projects are using this test scaffold for automated regression testing. Enjoy! Best regards, Jiahao -------------- next part -------------- An HTML attachment was scrubbed... URL: From xeioex at nginx.com Wed Jun 1 04:53:52 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 01 Jun 2022 04:53:52 +0000 Subject: [njs] Fixed typo in debug code introduced in 3e754413e907 (0.7.4). Message-ID: details: https://hg.nginx.org/njs/rev/1347c84742c0 branches: changeset: 1869:1347c84742c0 user: Dmitry Volyntsev date: Tue May 31 21:48:19 2022 -0700 description: Fixed typo in debug code introduced in 3e754413e907 (0.7.4). diffstat: src/njs_vmcode.h | 15 ++++++++------- 1 files changed, 8 insertions(+), 7 deletions(-) diffs (26 lines): diff -r 5b7676ec600d -r 1347c84742c0 src/njs_vmcode.h --- a/src/njs_vmcode.h Tue May 31 09:26:47 2022 -0700 +++ b/src/njs_vmcode.h Tue May 31 21:48:19 2022 -0700 @@ -452,14 +452,15 @@ njs_object_t *njs_function_new_object(nj #ifdef NJS_OPCODE_DEBUG #define njs_vmcode_debug(vm, pc, prefix) { \ - do { \ - njs_vm_code_t *code; \ + do { \ + njs_vm_code_t *code; \ + \ + code = njs_lookup_code(vm, pc); \ \ - code = njs_lookup_code(vm, pc); \ - \ - njs_printf("%s %V\n", prefix, \ - (code != NULL) ? &code->name : &njs_entry_unknown); \ - } while (0) + njs_printf("%s %V\n", prefix, \ + (code != NULL) ? &code->name : &njs_entry_unknown); \ + } while (0); \ + } #else #define njs_vmcode_debug(vm, pc, prefix) #endif From xeioex at nginx.com Wed Jun 1 04:53:54 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 01 Jun 2022 04:53:54 +0000 Subject: [njs] Improved OPCODE debug. Message-ID: details: https://hg.nginx.org/njs/rev/bd520efc0df1 branches: changeset: 1870:bd520efc0df1 user: Dmitry Volyntsev date: Tue May 31 21:48:26 2022 -0700 description: Improved OPCODE debug. diffstat: src/njs_vmcode.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 1347c84742c0 -r bd520efc0df1 src/njs_vmcode.c --- a/src/njs_vmcode.c Tue May 31 21:48:19 2022 -0700 +++ b/src/njs_vmcode.c Tue May 31 21:48:26 2022 -0700 @@ -801,6 +801,8 @@ next: goto error; } + njs_vmcode_debug(vm, pc, "RESUME"); + ret = sizeof(njs_vmcode_function_call_t); break; From xeioex at nginx.com Wed Jun 1 04:53:56 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 01 Jun 2022 04:53:56 +0000 Subject: [njs] Fixed catching of the exception thrown from an awaited function. Message-ID: details: https://hg.nginx.org/njs/rev/2774fed7b864 branches: changeset: 1871:2774fed7b864 user: Dmitry Volyntsev date: Tue May 31 21:48:46 2022 -0700 description: Fixed catching of the exception thrown from an awaited function. This closes #500 issue on Github. diffstat: src/njs_async.c | 6 +++--- test/js/async_try_catch_call.t.js | 29 +++++++++++++++++++++++++++++ test/js/async_try_catch_expression.t.js | 28 ++++++++++++++++++++++++++++ 3 files changed, 60 insertions(+), 3 deletions(-) diffs (95 lines): diff -r bd520efc0df1 -r 2774fed7b864 src/njs_async.c --- a/src/njs_async.c Tue May 31 21:48:26 2022 -0700 +++ b/src/njs_async.c Tue May 31 21:48:46 2022 -0700 @@ -67,9 +67,6 @@ njs_await_fulfilled(njs_vm_t *vm, njs_va ctx = vm->top_frame->function->context; value = njs_arg(args, nargs, 1); - if (njs_is_error(value)) { - goto failed; - } async_frame = ctx->await; async = &async_frame->native; @@ -143,6 +140,7 @@ njs_await_rejected(njs_vm_t *vm, njs_val value = njs_arg(args, nargs, 1); if (ctx->await->native.pc == ctx->pc) { + /* No catch block was set before await. */ (void) njs_function_call(vm, njs_function(&ctx->capability->reject), &njs_value_undefined, value, 1, &vm->retval); @@ -151,6 +149,8 @@ njs_await_rejected(njs_vm_t *vm, njs_val return NJS_ERROR; } + /* ctx->await->native.pc points to a catch block here. */ + ctx->pc = ctx->await->native.pc; return njs_await_fulfilled(vm, args, nargs, unused); diff -r bd520efc0df1 -r 2774fed7b864 test/js/async_try_catch_call.t.js --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/js/async_try_catch_call.t.js Tue May 31 21:48:46 2022 -0700 @@ -0,0 +1,29 @@ +/*--- +includes: [compareArray.js] +flags: [async] +---*/ + +let stages = []; +const fn = async () => { throw new Error('Oops') }; + +async function af() { + try { + await fn(); + + $DONOTEVALUATE(); + } + catch (v) { + stages.push(`catch:${v}`); + } + finally { + stages.push('finally'); + } + + return "end"; +}; + +af().then(v => { + stages.push(v); + assert.compareArray(stages, ['catch:Error: Oops', 'finally', 'end']); +}) +.then($DONE, $DONE) diff -r bd520efc0df1 -r 2774fed7b864 test/js/async_try_catch_expression.t.js --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/js/async_try_catch_expression.t.js Tue May 31 21:48:46 2022 -0700 @@ -0,0 +1,28 @@ +/*--- +includes: [compareArray.js] +flags: [async] +---*/ + +let stages = []; + +async function af() { + try { + await ({}).a.a(); + + $DONOTEVALUATE(); + } + catch (v) { + stages.push('catch'); + } + finally { + stages.push('finally'); + } + + return "end"; +}; + +af().then(v => { + stages.push(v); + assert.compareArray(stages, ['catch', 'finally', 'end']); +}) +.then($DONE, $DONE) From mdounin at mdounin.ru Wed Jun 1 14:41:27 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 1 Jun 2022 17:41:27 +0300 Subject: [PATCH] Stream: don't flush empty buffers created for read errors. In-Reply-To: References: Message-ID: Hello! On Tue, May 24, 2022 at 04:37:11PM +0000, Aleksei Bavshin via nginx-devel wrote: > In all honesty, the main motivation for the patch is to address > a regression in UDP healthchecks caused by the behavior > described below. It's already covered by tests, but the > corresponding cases were disabled by default. > > I considered > cl->buf->flush = !src->read->error; > or even > cl->buf->flush = !src->read->eof; but neither of those are > exactly equivalent to the code below (with the latter having > significantly different behavior), and IMO it would look > less obvious. > > # HG changeset patch > # User Aleksei Bavshin > # Date 1653330584 25200 > # Mon May 23 11:29:44 2022 -0700 > # Branch se > # Node ID 5a98e9cb437f7719afa2bde62de68e174fd8e03e > # Parent 8902674cc7fe759cada415c10340f31ae4a90fba > Stream: don't flush empty buffers created for read errors. > > When we generate the last_buf buffer for an UDP upstream recv error, it does > not contain any data from the wire. ngx_stream_write_filter attempts to forward > it anyways, which is incorrect (e.g., UDP upstream ECONNREFUSED will be > translated to an empty packet). > > Reproduction: > > stream { > upstream unreachable { > server 127.0.0.1:8880; > } > server { > listen 127.0.0.1:8998 udp; > proxy_pass unreachable; > } > } > > 1 0.000000000 127.0.0.1 → 127.0.0.1 UDP 47 45588 → 8998 Len=5 > 2 0.000166300 127.0.0.1 → 127.0.0.1 UDP 47 51149 → 8880 Len=5 > 3 0.000172600 127.0.0.1 → 127.0.0.1 ICMP 75 Destination unreachable (Port > unreachable) > 4 0.000202400 127.0.0.1 → 127.0.0.1 UDP 42 8998 → 45588 Len=0 > > Fixes d127837c714f. > > diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c > --- a/src/stream/ngx_stream_proxy_module.c > +++ b/src/stream/ngx_stream_proxy_module.c > @@ -1676,7 +1676,7 @@ ngx_stream_proxy_process(ngx_stream_sess > ssize_t n; > ngx_buf_t *b; > ngx_int_t rc; > - ngx_uint_t flags, *packets; > + ngx_uint_t flags, flush, *packets; > ngx_msec_t delay; > ngx_chain_t *cl, **ll, **out, **busy; > ngx_connection_t *c, *pc, *src, *dst; > @@ -1803,9 +1803,12 @@ ngx_stream_proxy_process(ngx_stream_sess > break; > } > > + flush = 1; > + > if (n == NGX_ERROR) { > src->read->eof = 1; > n = 0; > + flush = 0; > } > > if (n >= 0) { > @@ -1846,7 +1849,7 @@ ngx_stream_proxy_process(ngx_stream_sess > > cl->buf->temporary = (n ? 1 : 0); > cl->buf->last_buf = src->read->eof; > - cl->buf->flush = 1; > + cl->buf->flush = flush; > > (*packets)++; > *received += n; I tend to think this patch is wrong, for at least two reasons: 1. It introduces a zero-sized non-special buffer in the chain. This is generally wrong, explicitly rejected in many code paths (see "zero size buf in ..." alerts), and might easily result in infinite loops in common processing patterns unless explicitly handled. 2. Semantically, in UDP proxying a sequence of one or more buffer ending with a flush buffer corresponds to an UDP packet. That is, the empty buffer added by the code is a start of the packet - and it can be handled as such by intermediate filters. Consider, for example, a filter which prepends each packet with "foo:" - with this change, a packet with "foo:" will still be sent in case of an error. I think a better solution would be to avoid adding buffers in case of errors at all. -- Maxim Dounin http://mdounin.ru/ From A.Bavshin at F5.com Wed Jun 1 15:04:24 2022 From: A.Bavshin at F5.com (Aleksei Bavshin) Date: Wed, 1 Jun 2022 15:04:24 +0000 Subject: [PATCH] Stream: don't flush empty buffers created for read errors. In-Reply-To: References: Message-ID: > -----Original Message----- > From: Maxim Dounin > Sent: Wednesday, June 1, 2022 7:41 AM > To: Aleksei Bavshin via nginx-devel > Subject: Re: [PATCH] Stream: don't flush empty buffers created for read errors. > > EXTERNAL MAIL: nginx-devel-bounces at nginx.org > > Hello! > > On Tue, May 24, 2022 at 04:37:11PM +0000, Aleksei Bavshin via nginx-devel > wrote: > > > In all honesty, the main motivation for the patch is to address > > a regression in UDP healthchecks caused by the behavior > > described below. It's already covered by tests, but the > > corresponding cases were disabled by default. > > > > I considered > > cl->buf->flush = !src->read->error; > > or even > > cl->buf->flush = !src->read->eof; but neither of those are > > exactly equivalent to the code below (with the latter having > > significantly different behavior), and IMO it would look > > less obvious. > > > > # HG changeset patch > > # User Aleksei Bavshin > > # Date 1653330584 25200 > > # Mon May 23 11:29:44 2022 -0700 > > # Branch se > > # Node ID 5a98e9cb437f7719afa2bde62de68e174fd8e03e > > # Parent 8902674cc7fe759cada415c10340f31ae4a90fba > > Stream: don't flush empty buffers created for read errors. > > > > When we generate the last_buf buffer for an UDP upstream recv error, it > does > > not contain any data from the wire. ngx_stream_write_filter attempts to > forward > > it anyways, which is incorrect (e.g., UDP upstream ECONNREFUSED will be > > translated to an empty packet). > > > > Reproduction: > > > > stream { > > upstream unreachable { > > server 127.0.0.1:8880; > > } > > server { > > listen 127.0.0.1:8998 udp; > > proxy_pass unreachable; > > } > > } > > > > 1 0.000000000 127.0.0.1 → 127.0.0.1 UDP 47 45588 → 8998 Len=5 > > 2 0.000166300 127.0.0.1 → 127.0.0.1 UDP 47 51149 → 8880 Len=5 > > 3 0.000172600 127.0.0.1 → 127.0.0.1 ICMP 75 Destination unreachable > (Port > > unreachable) > > 4 0.000202400 127.0.0.1 → 127.0.0.1 UDP 42 8998 → 45588 Len=0 > > > > Fixes d127837c714f. > > > > diff --git a/src/stream/ngx_stream_proxy_module.c > b/src/stream/ngx_stream_proxy_module.c > > --- a/src/stream/ngx_stream_proxy_module.c > > +++ b/src/stream/ngx_stream_proxy_module.c > > @@ -1676,7 +1676,7 @@ ngx_stream_proxy_process(ngx_stream_sess > > ssize_t n; > > ngx_buf_t *b; > > ngx_int_t rc; > > - ngx_uint_t flags, *packets; > > + ngx_uint_t flags, flush, *packets; > > ngx_msec_t delay; > > ngx_chain_t *cl, **ll, **out, **busy; > > ngx_connection_t *c, *pc, *src, *dst; > > @@ -1803,9 +1803,12 @@ ngx_stream_proxy_process(ngx_stream_sess > > break; > > } > > > > + flush = 1; > > + > > if (n == NGX_ERROR) { > > src->read->eof = 1; > > n = 0; > > + flush = 0; > > } > > > > if (n >= 0) { > > @@ -1846,7 +1849,7 @@ ngx_stream_proxy_process(ngx_stream_sess > > > > cl->buf->temporary = (n ? 1 : 0); > > cl->buf->last_buf = src->read->eof; > > - cl->buf->flush = 1; > > + cl->buf->flush = flush; > > > > (*packets)++; > > *received += n; > > I tend to think this patch is wrong, for at least two reasons: > > 1. It introduces a zero-sized non-special buffer in the chain. > This is generally wrong, explicitly rejected in many code paths > (see "zero size buf in ..." alerts), and might easily result in > infinite loops in common processing patterns unless explicitly > handled. I think you misunderstood what the patch does. The buffer is guaranteed to be special; it will have last_buf set just because it signals an unrecoverable connection error. The problem is that if *both* last_buf and flush are set, the buffer becomes special enough to be sent to the wire even if there's no actual buffered data (no packet to terminate). An alternative approach would be to ignore the flush bit for a buffer with last_buf set: @@ -234,8 +234,8 @@ ngx_stream_write_filter(ngx_stream_sessi if (size == 0 && !(c->buffered & NGX_LOWLEVEL_BUFFERED) - && !(last && c->need_last_buf) - && !(flush && c->need_flush_buf)) + && !(last ? c->need_last_buf + : (flush && c->need_flush_buf))) { if (last || flush || sync) { for (cl = *out; cl; /* void */) { but that doesn't look as safe as this patch to me. > 2. Semantically, in UDP proxying a sequence of one or more buffer > ending with a flush buffer corresponds to an UDP packet. That is, > the empty buffer added by the code is a start of the packet - and > it can be handled as such by intermediate filters. Consider, for > example, a filter which prepends each packet with "foo:" - with > this change, a packet with "foo:" will still be sent in case of an > error. > > I think a better solution would be to avoid adding buffers in case > of errors at all. See above: as we're closing the connection in case of the error, the buffer automatically gets last_buf and becomes 'special'. If I understand Nginx flow control correctly, special buffer with last_buf set is expected to appear before closing the connection. From mdounin at mdounin.ru Thu Jun 2 00:28:06 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 2 Jun 2022 03:28:06 +0300 Subject: [PATCH] Stream: don't flush empty buffers created for read errors. In-Reply-To: References: Message-ID: Hello! On Wed, Jun 01, 2022 at 03:04:24PM +0000, Aleksei Bavshin via nginx-devel wrote: > > On Tue, May 24, 2022 at 04:37:11PM +0000, Aleksei Bavshin via nginx-devel > > wrote: > > > > > In all honesty, the main motivation for the patch is to address > > > a regression in UDP healthchecks caused by the behavior > > > described below. It's already covered by tests, but the > > > corresponding cases were disabled by default. > > > > > > I considered > > > cl->buf->flush = !src->read->error; > > > or even > > > cl->buf->flush = !src->read->eof; but neither of those are > > > exactly equivalent to the code below (with the latter having > > > significantly different behavior), and IMO it would look > > > less obvious. > > > > > > # HG changeset patch > > > # User Aleksei Bavshin > > > # Date 1653330584 25200 > > > # Mon May 23 11:29:44 2022 -0700 > > > # Branch se > > > # Node ID 5a98e9cb437f7719afa2bde62de68e174fd8e03e > > > # Parent 8902674cc7fe759cada415c10340f31ae4a90fba > > > Stream: don't flush empty buffers created for read errors. > > > > > > When we generate the last_buf buffer for an UDP upstream recv error, it > > does > > > not contain any data from the wire. ngx_stream_write_filter attempts to > > forward > > > it anyways, which is incorrect (e.g., UDP upstream ECONNREFUSED will be > > > translated to an empty packet). > > > > > > Reproduction: > > > > > > stream { > > > upstream unreachable { > > > server 127.0.0.1:8880; > > > } > > > server { > > > listen 127.0.0.1:8998 udp; > > > proxy_pass unreachable; > > > } > > > } > > > > > > 1 0.000000000 127.0.0.1 → 127.0.0.1 UDP 47 45588 → 8998 Len=5 > > > 2 0.000166300 127.0.0.1 → 127.0.0.1 UDP 47 51149 → 8880 Len=5 > > > 3 0.000172600 127.0.0.1 → 127.0.0.1 ICMP 75 Destination unreachable > > (Port > > > unreachable) > > > 4 0.000202400 127.0.0.1 → 127.0.0.1 UDP 42 8998 → 45588 Len=0 > > > > > > Fixes d127837c714f. > > > > > > diff --git a/src/stream/ngx_stream_proxy_module.c > > b/src/stream/ngx_stream_proxy_module.c > > > --- a/src/stream/ngx_stream_proxy_module.c > > > +++ b/src/stream/ngx_stream_proxy_module.c > > > @@ -1676,7 +1676,7 @@ ngx_stream_proxy_process(ngx_stream_sess > > > ssize_t n; > > > ngx_buf_t *b; > > > ngx_int_t rc; > > > - ngx_uint_t flags, *packets; > > > + ngx_uint_t flags, flush, *packets; > > > ngx_msec_t delay; > > > ngx_chain_t *cl, **ll, **out, **busy; > > > ngx_connection_t *c, *pc, *src, *dst; > > > @@ -1803,9 +1803,12 @@ ngx_stream_proxy_process(ngx_stream_sess > > > break; > > > } > > > > > > + flush = 1; > > > + > > > if (n == NGX_ERROR) { > > > src->read->eof = 1; > > > n = 0; > > > + flush = 0; > > > } > > > > > > if (n >= 0) { > > > @@ -1846,7 +1849,7 @@ ngx_stream_proxy_process(ngx_stream_sess > > > > > > cl->buf->temporary = (n ? 1 : 0); > > > cl->buf->last_buf = src->read->eof; > > > - cl->buf->flush = 1; > > > + cl->buf->flush = flush; > > > > > > (*packets)++; > > > *received += n; > > > > I tend to think this patch is wrong, for at least two reasons: > > > > 1. It introduces a zero-sized non-special buffer in the chain. > > This is generally wrong, explicitly rejected in many code paths > > (see "zero size buf in ..." alerts), and might easily result in > > infinite loops in common processing patterns unless explicitly > > handled. > > I think you misunderstood what the patch does. The buffer is > guaranteed to be special; it will have last_buf set just because > it signals an unrecoverable connection error. > The problem is that if *both* last_buf and flush are set, the > buffer becomes special enough to be sent to the wire even if > there's no actual buffered data (no packet to terminate). Ah, sorry, missed that the buffer will also had the last_buf set. This is probably wrong though, as last_buf implies correct end of data, and we've got an error instead. For example, in HTTP code last_buf is not set when the upstream response is abnormally terminated, and, as a result, no final chunk is sent if the response is using chunked transfer encoding, making it possible for the client to detect incomplete responses. This is mostly unrelated to the patch though, and should be addressed separately. > An alternative approach would be to ignore the flush bit for a buffer with last_buf set: > @@ -234,8 +234,8 @@ ngx_stream_write_filter(ngx_stream_sessi > > if (size == 0 > && !(c->buffered & NGX_LOWLEVEL_BUFFERED) > - && !(last && c->need_last_buf) > - && !(flush && c->need_flush_buf)) > + && !(last ? c->need_last_buf > + : (flush && c->need_flush_buf))) > { > if (last || flush || sync) { > for (cl = *out; cl; /* void */) { > but that doesn't look as safe as this patch to me. I don't think it's the way to go. Semantically, flush and last_buf are mutually exclusive: last_buf implies flushing of all remaining data, and there should be no flush set if last_buf is set. The "cl->buf->flush = !src->read->eof" approach might be actually better and more obvious: it simply ensures that last_buf and flush are not set at the same time. I don't immediately see any cases where it can break things, and this should fix the original problem with empty UDP packets being sent. > > 2. Semantically, in UDP proxying a sequence of one or more buffer > > ending with a flush buffer corresponds to an UDP packet. That is, > > the empty buffer added by the code is a start of the packet - and > > it can be handled as such by intermediate filters. Consider, for > > example, a filter which prepends each packet with "foo:" - with > > this change, a packet with "foo:" will still be sent in case of an > > error. > > > > I think a better solution would be to avoid adding buffers in case > > of errors at all. > > See above: as we're closing the connection in case of the error, > the buffer automatically gets last_buf and becomes 'special'. > If I understand Nginx flow control correctly, special buffer > with last_buf set is expected to appear before closing the > connection. That's not exactly the case, see above. -- Maxim Dounin http://mdounin.ru/ From A.Bavshin at F5.com Thu Jun 2 03:20:54 2022 From: A.Bavshin at F5.com (Aleksei Bavshin) Date: Thu, 2 Jun 2022 03:20:54 +0000 Subject: [PATCH] Stream: don't flush empty buffers created for read errors. In-Reply-To: References: Message-ID: > I don't think it's the way to go. Semantically, flush and > last_buf are mutually exclusive: last_buf implies flushing of all > remaining data, and there should be no flush set if last_buf is > set. > > The "cl->buf->flush = !src->read->eof" approach might be actually > better and more obvious: it simply ensures that last_buf and flush > are not set at the same time. I don't immediately see any cases > where it can break things, and this should fix the original > problem with empty UDP packets being sent. Thanks for confirming that, I wasn't sure it is the right approach exactly because I had doubts regarding the mutual exclusivity of the flags. Updated patch below; tests are passing on both branches and all systems we have in CI. # HG changeset patch # User Aleksei Bavshin # Date 1653330584 25200 # Mon May 23 11:29:44 2022 -0700 # Node ID 40926829be83986a3a5a5f941d2014000a0acd2e # Parent e0cfab501dd11fdd23ad492419692269d3a01fc7 Stream: don't flush empty buffers created for read errors. When we generate the last_buf buffer for an UDP upstream recv error, it does not contain any data from the wire. ngx_stream_write_filter attempts to forward it anyways, which is incorrect (e.g., UDP upstream ECONNREFUSED will be translated to an empty packet). This happens because we mark the buffer as both 'flush' and 'last_buf', and ngx_stream_write_filter has special handling for flush with certain types of connections (see d127837c714f, 32b0ba4855a6). The flags are meant to be mutually exclusive, so the fix is to ensure that flush and last_buf are not set at the same time. Reproduction: stream { upstream unreachable { server 127.0.0.1:8880; } server { listen 127.0.0.1:8998 udp; proxy_pass unreachable; } } 1 0.000000000 127.0.0.1 → 127.0.0.1 UDP 47 45588 → 8998 Len=5 2 0.000166300 127.0.0.1 → 127.0.0.1 UDP 47 51149 → 8880 Len=5 3 0.000172600 127.0.0.1 → 127.0.0.1 ICMP 75 Destination unreachable (Port unreachable) 4 0.000202400 127.0.0.1 → 127.0.0.1 UDP 42 8998 → 45588 Len=0 Fixes d127837c714f. diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c --- a/src/stream/ngx_stream_proxy_module.c +++ b/src/stream/ngx_stream_proxy_module.c @@ -1735,7 +1735,7 @@ ngx_stream_proxy_process(ngx_stream_sess cl->buf->temporary = (n ? 1 : 0); cl->buf->last_buf = src->read->eof; - cl->buf->flush = 1; + cl->buf->flush = !src->read->eof; (*packets)++; *received += n; From A.Bavshin at F5.com Thu Jun 2 03:49:50 2022 From: A.Bavshin at F5.com (Aleksei Bavshin) Date: Thu, 2 Jun 2022 03:49:50 +0000 Subject: [PATCH] Resolver: make TCP write timer event cancelable. In-Reply-To: References: Message-ID: Updated patch, with maybe too verbose description. For the record, I did check a few similar modules I'm aware of (nginx-upstream-dynamic-servers, ngx_upstream_jdomain) and haven't found any affected code. So, the effect on third-party modules is hypothetical. # HG changeset patch # User Aleksei Bavshin # Date 1654139843 25200 # Wed Jun 01 20:17:23 2022 -0700 # Node ID 488f7a926ba54303c930b0b1e19c2b27e0af5079 # Parent 40926829be83986a3a5a5f941d2014000a0acd2e Resolver: make TCP write timer event cancelable. Similar to 70e65bf8dfd7, the change is made to ensure that the ability to cancel resolver tasks is fully controlled by the caller. As mentioned in the referenced commit, it is safe to make this timer cancelable because resolve tasks can have their own timeouts that are not cancelable. The scenario where this may become a problem is a periodic background resolve task (not tied to a specific request or a client connection), which receives a response with short TTL, large enough to warrant fallback to a TCP query. With each event loop wakeup, we either have a previously set write timer instance or schedule a new one. The non-cancelable write timer can delay or block graceful shutdown of a worker even if the ngx_resolver_ctx_t->cancelable flag is set by the API user, and there are no other tasks or connections. We use the resolver API in this way to maintain the list of upstream server addresses specified with the 'resolve' parameter, and there could be third-party modules implementing similar logic. diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c +++ b/src/core/ngx_resolver.c @@ -1389,6 +1389,7 @@ ngx_resolver_send_tcp_query(ngx_resolver rec->tcp->data = rec; rec->tcp->write->handler = ngx_resolver_tcp_write; + rec->tcp->write->cancelable = 1; rec->tcp->read->handler = ngx_resolver_tcp_read; rec->tcp->read->resolver = 1; From xeioex at nginx.com Thu Jun 2 04:55:48 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 02 Jun 2022 04:55:48 +0000 Subject: [njs] Fixed function value initialization. Message-ID: details: https://hg.nginx.org/njs/rev/ddec2f61c6d4 branches: changeset: 1872:ddec2f61c6d4 user: Dmitry Volyntsev date: Wed Jun 01 21:54:32 2022 -0700 description: Fixed function value initialization. This closes #470 issue on Github. diffstat: src/njs_vmcode.c | 6 +++--- src/test/njs_unit_test.c | 6 ++++-- 2 files changed, 7 insertions(+), 5 deletions(-) diffs (34 lines): diff -r 2774fed7b864 -r ddec2f61c6d4 src/njs_vmcode.c --- a/src/njs_vmcode.c Tue May 31 21:48:46 2022 -0700 +++ b/src/njs_vmcode.c Wed Jun 01 21:54:32 2022 -0700 @@ -1259,9 +1259,9 @@ njs_vmcode_function_copy(njs_vm_t *vm, n njs_value_t *retval; njs_function_t *function; - retval = njs_scope_valid_value(vm, retidx); - - if (njs_is_undefined(retval)) { + retval = njs_scope_value(vm, retidx); + + if (!njs_is_valid(retval)) { *retval = *value; function = njs_function_value_copy(vm, retval); diff -r 2774fed7b864 -r ddec2f61c6d4 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Tue May 31 21:48:46 2022 -0700 +++ b/src/test/njs_unit_test.c Wed Jun 01 21:54:32 2022 -0700 @@ -10200,10 +10200,12 @@ static njs_unit_test_t njs_test[] = "c()()()"), njs_str("1") }, -#if 0 { njs_str("function f() {}; f += 1; f"), njs_str("[object Function]1") }, -#endif + + { njs_str("function f() { function g() { g = undefined; }; g(); g(); };" + "f()"), + njs_str("TypeError: undefined is not a function") }, { njs_str("function f() {}; function g() { return f }; g()"), njs_str("[object Function]") }, From arut at nginx.com Thu Jun 2 13:41:38 2022 From: arut at nginx.com (Roman Arutyunyan) Date: Thu, 2 Jun 2022 17:41:38 +0400 Subject: [PATCH 1 of 2] QUIC: reusable and idle modes for main connection In-Reply-To: <47D0EFBF-31D0-4C69-AC0F-271D92516717@nginx.com> References: <67ae4b649f2e38a44b24.1652857034@arut-laptop> <47D0EFBF-31D0-4C69-AC0F-271D92516717@nginx.com> Message-ID: <20220602134138.yqxwlncdy7ra5irt@N00W24XTQX> On Mon, May 30, 2022 at 05:54:34PM +0400, Sergey Kandaurov wrote: > > > On 18 May 2022, at 10:57, Roman Arutyunyan wrote: > > > > # HG changeset patch > > # User Roman Arutyunyan > > # Date 1652852691 -14400 > > # Wed May 18 09:44:51 2022 +0400 > > # Branch quic > > # Node ID 67ae4b649f2e38a44b245b7a842cf396c8250f02 > > # Parent c2f5d79cde64457f1fa7344c56a5248a677a7e46 > > QUIC: reusable and idle modes for main connection. > > > > The modes are controlled by application layer. For HTTP/3, they are enabled > > when there are no active request streams. > > > > diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c > > --- a/src/event/quic/ngx_event_quic.c > > +++ b/src/event/quic/ngx_event_quic.c > > @@ -414,8 +414,8 @@ ngx_quic_input_handler(ngx_event_t *rev) > > } > > > > if (c->close) { > > - qc->error_reason = "graceful shutdown"; > > - ngx_quic_close_connection(c, NGX_OK); > > + qc->error = NGX_QUIC_ERR_NO_ERROR; > > + ngx_quic_close_connection(c, NGX_ERROR); > > return; > > } > > > > diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c > > --- a/src/event/quic/ngx_event_quic_streams.c > > +++ b/src/event/quic/ngx_event_quic_streams.c > > @@ -161,6 +161,7 @@ ngx_quic_close_streams(ngx_connection_t > > ngx_pool_t *pool; > > ngx_queue_t *q; > > ngx_rbtree_t *tree; > > + ngx_connection_t *sc; > > ngx_rbtree_node_t *node; > > ngx_quic_stream_t *qs; > > > > @@ -185,24 +186,41 @@ ngx_quic_close_streams(ngx_connection_t > > return NGX_OK; > > } > > > > -#if (NGX_DEBUG) > > - ns = 0; > > -#endif > > - > > node = ngx_rbtree_min(tree->root, tree->sentinel); > > > > while (node) { > > qs = (ngx_quic_stream_t *) node; > > node = ngx_rbtree_next(tree, node); > > + sc = qs->connection; > > + > > + if (sc == NULL) { > > + ngx_quic_close_stream(qs); > > + continue; > > + } > > + > > + if (c->close || sc->idle || sc->reusable) { > > + sc->close = 1; > > + sc->read->handler(sc->read); > > + } > > + } > > + > > + if (tree->root == tree->sentinel) { > > + return NGX_OK; > > + } > > + > > +#if (NGX_DEBUG) > > + ns = 0; > > +#endif > > + > > + for (node = ngx_rbtree_min(tree->root, tree->sentinel); > > + node; > > + node = ngx_rbtree_next(tree, node)) > > + { > > + qs = (ngx_quic_stream_t *) node; > > > > qs->recv_state = NGX_QUIC_STREAM_RECV_RESET_RECVD; > > qs->send_state = NGX_QUIC_STREAM_SEND_RESET_SENT; > > > > - if (qs->connection == NULL) { > > - ngx_quic_close_stream(qs); > > - continue; > > - } > > - > > ngx_quic_set_event(qs->connection->read); > > ngx_quic_set_event(qs->connection->write); > > > > @@ -587,6 +605,7 @@ ngx_quic_create_stream(ngx_connection_t > > { > > ngx_log_t *log; > > ngx_pool_t *pool; > > + ngx_uint_t reusable; > > ngx_queue_t *q; > > ngx_connection_t *sc; > > ngx_quic_stream_t *qs; > > @@ -639,7 +658,13 @@ ngx_quic_create_stream(ngx_connection_t > > *log = *c->log; > > pool->log = log; > > > > + reusable = c->reusable; > > + ngx_reusable_connection(c, 0); > > + > > sc = ngx_get_connection(c->fd, log); > > + > > + ngx_reusable_connection(c, reusable); > > + > > jftr, this pollutes debug log a bit, but generally looks ok. > > > if (sc == NULL) { > > ngx_destroy_pool(pool); > > ngx_queue_insert_tail(&qc->streams.free, &qs->queue); > > diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c > > --- a/src/http/v3/ngx_http_v3_request.c > > +++ b/src/http/v3/ngx_http_v3_request.c > > @@ -210,6 +210,11 @@ ngx_http_v3_init_request_stream(ngx_conn > > > > h3c = ngx_http_v3_get_session(c); > > > > + if (ngx_terminate || ngx_exiting) { > > + h3c->goaway = 1; > > + (void) ngx_http_v3_send_goaway(c, c->quic->id); > > + } > > + > > This results in ever increasing GOAWAY identifiers if multiple > new client streams received after graceful shutdown initiated. > This is forbidden per rfc9114, 5.2. Connection Shutdown. > > I wonder if it should move after the below condition. We cannot move it below since we need the below condition to work in case of (ngx_terminating || ngx_exiting) too. I suggest that we add !h3c->goaway to the new condition. > > if (h3c->goaway) { > > c->close = 1; > > ngx_http_close_connection(c); > > @@ -258,7 +263,7 @@ ngx_http_v3_wait_request_handler(ngx_eve > > size_t size; > > ssize_t n; > > ngx_buf_t *b; > > - ngx_connection_t *c; > > + ngx_connection_t *c, *pc; > > ngx_pool_cleanup_t *cln; > > ngx_http_request_t *r; > > ngx_http_connection_t *hc; > > @@ -385,6 +390,10 @@ ngx_http_v3_wait_request_handler(ngx_eve > > h3c = ngx_http_v3_get_session(c); > > h3c->nrequests++; > > > > + pc = c->quic->parent; > > + pc->idle = 0; > > + ngx_reusable_connection(pc, 0); > > + > > if (h3c->keepalive.timer_set) { > > ngx_del_timer(&h3c->keepalive); > > } > > @@ -430,7 +439,7 @@ ngx_http_v3_cleanup_request(void *data) > > { > > ngx_http_request_t *r = data; > > > > - ngx_connection_t *c; > > + ngx_connection_t *c, *pc; > > ngx_http_v3_session_t *h3c; > > ngx_http_core_loc_conf_t *clcf; > > > > @@ -443,6 +452,16 @@ ngx_http_v3_cleanup_request(void *data) > > h3c = ngx_http_v3_get_session(c); > > > > if (--h3c->nrequests == 0) { > > + pc = c->quic->parent; > > + > > + if (ngx_terminate || ngx_exiting) { > > + ngx_quic_finalize_connection(pc, NGX_HTTP_V3_ERR_NO_ERROR, NULL); > > + return; > > + } > > + > > + pc->idle = 1; > > + ngx_reusable_connection(pc, 1); > > + > > clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module); > > ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout); > > } > > diff --git a/src/http/v3/ngx_http_v3_uni.c b/src/http/v3/ngx_http_v3_uni.c > > --- a/src/http/v3/ngx_http_v3_uni.c > > +++ b/src/http/v3/ngx_http_v3_uni.c > > @@ -182,6 +182,11 @@ ngx_http_v3_uni_read_handler(ngx_event_t > > > > ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 read handler"); > > > > + if (c->close) { > > + ngx_http_v3_close_uni_stream(c); > > + return; > > + } > > + > > ngx_memzero(&b, sizeof(ngx_buf_t)); > > > > while (rev->ready) { > > @@ -262,6 +267,11 @@ ngx_http_v3_uni_dummy_read_handler(ngx_e > > > > ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 dummy read handler"); > > > > + if (c->close) { > > + ngx_http_v3_close_uni_stream(c); > > + return; > > + } > > + > > if (rev->ready) { > > if (c->recv(c, &ch, 1) != 0) { > > ngx_http_v3_finalize_connection(c, NGX_HTTP_V3_ERR_NO_ERROR, NULL); > > > > -- > Sergey Kandaurov > > _______________________________________________ > nginx-devel mailing list -- nginx-devel at nginx.org > To unsubscribe send an email to nginx-devel-leave at nginx.org From arut at nginx.com Thu Jun 2 13:45:09 2022 From: arut at nginx.com (Roman Arutyunyan) Date: Thu, 2 Jun 2022 17:45:09 +0400 Subject: [PATCH 2 of 2] QUIC: init_streams() callback In-Reply-To: <22737FC4-820A-4DDD-870F-1C841999D822@nginx.com> References: <22737FC4-820A-4DDD-870F-1C841999D822@nginx.com> Message-ID: <20220602134509.axtskzyypeiyjyk3@N00W24XTQX> On Tue, May 31, 2022 at 01:53:08AM +0400, Sergey Kandaurov wrote: > > > On 18 May 2022, at 10:57, Roman Arutyunyan wrote: > > > > # HG changeset patch > > # User Roman Arutyunyan > > # Date 1652856132 -14400 > > # Wed May 18 10:42:12 2022 +0400 > > # Branch quic > > # Node ID a0f2d69f1fe43dfc718262235bf04d7b05f1fd68 > > # Parent 67ae4b649f2e38a44b245b7a842cf396c8250f02 > > QUIC: init_streams() callback. > > > > It's called after handshake completion to initialize application-level data > > prior to creating streams. > > See below for 0-RTT case. > > > > > HTTP/3 callback implementation switches main QUIC connection to idle and > > reusable modes and sets keepalive timer. > > > > diff --git a/src/event/quic/ngx_event_quic.h b/src/event/quic/ngx_event_quic.h > > --- a/src/event/quic/ngx_event_quic.h > > +++ b/src/event/quic/ngx_event_quic.h > > @@ -28,6 +28,9 @@ > > #define NGX_QUIC_STREAM_UNIDIRECTIONAL 0x02 > > > > > > +typedef ngx_int_t (*ngx_quic_init_streams_pt)(ngx_connection_t *c); > > + > > + > > typedef enum { > > NGX_QUIC_STREAM_SEND_READY = 0, > > NGX_QUIC_STREAM_SEND_SEND, > > @@ -74,6 +77,8 @@ typedef struct { > > ngx_int_t stream_reject_code_uni; > > ngx_int_t stream_reject_code_bidi; > > > > + ngx_quic_init_streams_pt init_streams; > > + > > u_char av_token_key[NGX_QUIC_AV_KEY_LEN]; > > u_char sr_token_key[NGX_QUIC_SR_KEY_LEN]; > > } ngx_quic_conf_t; > > diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c > > --- a/src/event/quic/ngx_event_quic_streams.c > > +++ b/src/event/quic/ngx_event_quic_streams.c > > @@ -21,6 +21,7 @@ static ngx_quic_stream_t *ngx_quic_get_s > > static ngx_int_t ngx_quic_reject_stream(ngx_connection_t *c, uint64_t id); > > static void ngx_quic_init_stream_handler(ngx_event_t *ev); > > static void ngx_quic_init_streams_handler(ngx_connection_t *c); > > +static ngx_int_t ngx_quic_do_init_streams(ngx_connection_t *c); > > static ngx_quic_stream_t *ngx_quic_create_stream(ngx_connection_t *c, > > uint64_t id); > > static void ngx_quic_empty_handler(ngx_event_t *ev); > > @@ -571,15 +572,22 @@ ngx_quic_init_streams(ngx_connection_t * > > return NGX_OK; > > } > > > > - ngx_quic_init_streams_handler(c); > > - > > - return NGX_OK; > > + return ngx_quic_do_init_streams(c); > > } > > > > > > static void > > ngx_quic_init_streams_handler(ngx_connection_t *c) > > { > > + if (ngx_quic_do_init_streams(c) != NGX_OK) { > > + ngx_quic_close_connection(c, NGX_ERROR); > > + } > > +} > > + > > + > > +static ngx_int_t > > +ngx_quic_do_init_streams(ngx_connection_t *c) > > +{ > > ngx_queue_t *q; > > ngx_quic_stream_t *qs; > > ngx_quic_connection_t *qc; > > @@ -588,6 +596,12 @@ ngx_quic_init_streams_handler(ngx_connec > > > > qc = ngx_quic_get_connection(c); > > > > + if (qc->conf->init_streams) { > > + if (qc->conf->init_streams(c) != NGX_OK) { > > + return NGX_ERROR; > > + } > > + } > > + > > for (q = ngx_queue_head(&qc->streams.uninitialized); > > q != ngx_queue_sentinel(&qc->streams.uninitialized); > > q = ngx_queue_next(q)) > > @@ -597,6 +611,8 @@ ngx_quic_init_streams_handler(ngx_connec > > } > > > > qc->streams.initialized = 1; > > + > > + return NGX_OK; > > } > > > > > > diff --git a/src/http/v3/ngx_http_v3.c b/src/http/v3/ngx_http_v3.c > > --- a/src/http/v3/ngx_http_v3.c > > +++ b/src/http/v3/ngx_http_v3.c > > @@ -17,21 +17,15 @@ static void ngx_http_v3_cleanup_session( > > ngx_int_t > > ngx_http_v3_init_session(ngx_connection_t *c) > > { > > - ngx_connection_t *pc; > > ngx_pool_cleanup_t *cln; > > ngx_http_connection_t *hc; > > ngx_http_v3_session_t *h3c; > > > > - pc = c->quic->parent; > > - hc = pc->data; > > - > > - if (hc->v3_session) { > > - return NGX_OK; > > - } > > + hc = c->data; > > > > ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 init session"); > > > > - h3c = ngx_pcalloc(pc->pool, sizeof(ngx_http_v3_session_t)); > > + h3c = ngx_pcalloc(c->pool, sizeof(ngx_http_v3_session_t)); > > if (h3c == NULL) { > > goto failed; > > } > > @@ -42,16 +36,16 @@ ngx_http_v3_init_session(ngx_connection_ > > ngx_queue_init(&h3c->blocked); > > ngx_queue_init(&h3c->pushing); > > > > - h3c->keepalive.log = pc->log; > > - h3c->keepalive.data = pc; > > + h3c->keepalive.log = c->log; > > + h3c->keepalive.data = c; > > h3c->keepalive.handler = ngx_http_v3_keepalive_handler; > > h3c->keepalive.cancelable = 1; > > > > - h3c->table.send_insert_count.log = pc->log; > > - h3c->table.send_insert_count.data = pc; > > + h3c->table.send_insert_count.log = c->log; > > + h3c->table.send_insert_count.data = c; > > h3c->table.send_insert_count.handler = ngx_http_v3_inc_insert_count_handler; > > > > - cln = ngx_pool_cleanup_add(pc->pool, 0); > > + cln = ngx_pool_cleanup_add(c->pool, 0); > > if (cln == NULL) { > > goto failed; > > } > > diff --git a/src/http/v3/ngx_http_v3.h b/src/http/v3/ngx_http_v3.h > > --- a/src/http/v3/ngx_http_v3.h > > +++ b/src/http/v3/ngx_http_v3.h > > @@ -153,6 +153,7 @@ struct ngx_http_v3_session_s { > > > > void ngx_http_v3_init(ngx_connection_t *c); > > void ngx_http_v3_reset_connection(ngx_connection_t *c); > > +ngx_int_t ngx_http_v3_init_streams(ngx_connection_t *c); > > ngx_int_t ngx_http_v3_init_session(ngx_connection_t *c); > > ngx_int_t ngx_http_v3_check_flood(ngx_connection_t *c); > > > > diff --git a/src/http/v3/ngx_http_v3_module.c b/src/http/v3/ngx_http_v3_module.c > > --- a/src/http/v3/ngx_http_v3_module.c > > +++ b/src/http/v3/ngx_http_v3_module.c > > @@ -249,6 +249,8 @@ ngx_http_v3_create_srv_conf(ngx_conf_t * > > h3scf->quic.stream_reject_code_bidi = NGX_HTTP_V3_ERR_REQUEST_REJECTED; > > h3scf->quic.active_connection_id_limit = NGX_CONF_UNSET_UINT; > > > > + h3scf->quic.init_streams = ngx_http_v3_init_streams; > > + > > return h3scf; > > } > > > > diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c > > --- a/src/http/v3/ngx_http_v3_request.c > > +++ b/src/http/v3/ngx_http_v3_request.c > > @@ -93,11 +93,6 @@ ngx_http_v3_init(ngx_connection_t *c) > > } > > #endif > > > > - if (ngx_http_v3_init_session(c) != NGX_OK) { > > - ngx_http_close_connection(c); > > - return; > > - } > > - > > if (c->quic->id & NGX_QUIC_STREAM_UNIDIRECTIONAL) { > > ngx_http_v3_init_uni_stream(c); > > > > @@ -107,6 +102,43 @@ ngx_http_v3_init(ngx_connection_t *c) > > } > > > > > > +ngx_int_t > > +ngx_http_v3_init_streams(ngx_connection_t *c) > > +{ > > + ngx_http_v3_session_t *h3c; > > + ngx_http_connection_t *hc; > > + ngx_http_v3_srv_conf_t *h3scf; > > + ngx_http_core_loc_conf_t *clcf; > > + > > + if (ngx_terminate || ngx_exiting) { > > + return NGX_ERROR; > > + } > > + > > + hc = c->data; > > + > > + h3scf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_v3_module); > > + > > +#if (NGX_HTTP_V3_HQ) > > + if (h3scf->hq) { > > + return NGX_OK; > > + } > > +#endif > > Clang complains about unused variables: > > diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c > --- a/src/http/v3/ngx_http_v3_request.c > +++ b/src/http/v3/ngx_http_v3_request.c > @@ -106,22 +106,26 @@ ngx_int_t > ngx_http_v3_init_streams(ngx_connection_t *c) > { > ngx_http_v3_session_t *h3c; > +#if (NGX_HTTP_V3_HQ) > ngx_http_connection_t *hc; > ngx_http_v3_srv_conf_t *h3scf; > +#endif > ngx_http_core_loc_conf_t *clcf; > > if (ngx_terminate || ngx_exiting) { > return NGX_ERROR; > } > > +#if (NGX_HTTP_V3_HQ) > + > hc = c->data; > > h3scf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_v3_module); > > -#if (NGX_HTTP_V3_HQ) > if (h3scf->hq) { > return NGX_OK; > } > + > #endif Thanks, fixed. > if (ngx_http_v3_init_session(c) != NGX_OK) { > > > + > > + if (ngx_http_v3_init_session(c) != NGX_OK) { > > + return NGX_ERROR; > > + } > > If 0-RTT data is accepted, ngx_http_v3_init_session() is called too early, > before SSL_do_handshake() is used to apply client transport parameters. > In particular that means inability to open a server unidirectional stream. Discussed this. The issue manifests itself with QuicTLS because of callback order. With BoringSSL is ok. I suggest another patch that relocates ngx_quic_init_streams() call from where we receive early keys to where we successfully decrypt the first early data packet. This relocation should fix the issue for all libraries. > > + > > + c->idle = 1; > > + ngx_reusable_connection(c, 1); > > + > > + h3c = ngx_http_v3_get_session(c); > > + clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module); > > + ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout); > > + > > + return NGX_OK; > > +} > > + > > + > > #if (NGX_HTTP_V3_HQ) > > > > static void > > > > -- > Sergey Kandaurov > > _______________________________________________ > nginx-devel mailing list -- nginx-devel at nginx.org > To unsubscribe send an email to nginx-devel-leave at nginx.org From arut at nginx.com Thu Jun 2 13:52:44 2022 From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=) Date: Thu, 02 Jun 2022 17:52:44 +0400 Subject: [PATCH 0 of 3] QUIC connection reuse In-Reply-To: References: Message-ID: Fixed a few issues + added a new patch relocating streams initialization. From arut at nginx.com Thu Jun 2 13:52:45 2022 From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=) Date: Thu, 02 Jun 2022 17:52:45 +0400 Subject: [PATCH 1 of 3] QUIC: reusable and idle modes for main connection In-Reply-To: References: Message-ID: <617ec472690620cc75c4.1654177965@arut-laptop> # HG changeset patch # User Roman Arutyunyan # Date 1652852691 -14400 # Wed May 18 09:44:51 2022 +0400 # Branch quic # Node ID 617ec472690620cc75c473f97555138a4bc7d38b # Parent a231f30606317a61efa3005fda9b43eb91d148d9 QUIC: reusable and idle modes for main connection. The modes are controlled by application layer. For HTTP/3, they are enabled when there are no active request streams. diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c --- a/src/event/quic/ngx_event_quic.c +++ b/src/event/quic/ngx_event_quic.c @@ -414,8 +414,8 @@ ngx_quic_input_handler(ngx_event_t *rev) } if (c->close) { - qc->error_reason = "graceful shutdown"; - ngx_quic_close_connection(c, NGX_OK); + qc->error = NGX_QUIC_ERR_NO_ERROR; + ngx_quic_close_connection(c, NGX_ERROR); return; } diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c --- a/src/event/quic/ngx_event_quic_streams.c +++ b/src/event/quic/ngx_event_quic_streams.c @@ -161,6 +161,7 @@ ngx_quic_close_streams(ngx_connection_t ngx_pool_t *pool; ngx_queue_t *q; ngx_rbtree_t *tree; + ngx_connection_t *sc; ngx_rbtree_node_t *node; ngx_quic_stream_t *qs; @@ -185,24 +186,41 @@ ngx_quic_close_streams(ngx_connection_t return NGX_OK; } -#if (NGX_DEBUG) - ns = 0; -#endif - node = ngx_rbtree_min(tree->root, tree->sentinel); while (node) { qs = (ngx_quic_stream_t *) node; node = ngx_rbtree_next(tree, node); + sc = qs->connection; + + if (sc == NULL) { + ngx_quic_close_stream(qs); + continue; + } + + if (c->close || sc->idle || sc->reusable) { + sc->close = 1; + sc->read->handler(sc->read); + } + } + + if (tree->root == tree->sentinel) { + return NGX_OK; + } + +#if (NGX_DEBUG) + ns = 0; +#endif + + for (node = ngx_rbtree_min(tree->root, tree->sentinel); + node; + node = ngx_rbtree_next(tree, node)) + { + qs = (ngx_quic_stream_t *) node; qs->recv_state = NGX_QUIC_STREAM_RECV_RESET_RECVD; qs->send_state = NGX_QUIC_STREAM_SEND_RESET_SENT; - if (qs->connection == NULL) { - ngx_quic_close_stream(qs); - continue; - } - ngx_quic_set_event(qs->connection->read); ngx_quic_set_event(qs->connection->write); @@ -587,6 +605,7 @@ ngx_quic_create_stream(ngx_connection_t { ngx_log_t *log; ngx_pool_t *pool; + ngx_uint_t reusable; ngx_queue_t *q; ngx_connection_t *sc; ngx_quic_stream_t *qs; @@ -639,7 +658,13 @@ ngx_quic_create_stream(ngx_connection_t *log = *c->log; pool->log = log; + reusable = c->reusable; + ngx_reusable_connection(c, 0); + sc = ngx_get_connection(c->fd, log); + + ngx_reusable_connection(c, reusable); + if (sc == NULL) { ngx_destroy_pool(pool); ngx_queue_insert_tail(&qc->streams.free, &qs->queue); diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c --- a/src/http/v3/ngx_http_v3_request.c +++ b/src/http/v3/ngx_http_v3_request.c @@ -210,6 +210,11 @@ ngx_http_v3_init_request_stream(ngx_conn h3c = ngx_http_v3_get_session(c); + if (!h3c->goaway && (ngx_terminate || ngx_exiting)) { + h3c->goaway = 1; + (void) ngx_http_v3_send_goaway(c, c->quic->id); + } + if (h3c->goaway) { c->close = 1; ngx_http_close_connection(c); @@ -258,7 +263,7 @@ ngx_http_v3_wait_request_handler(ngx_eve size_t size; ssize_t n; ngx_buf_t *b; - ngx_connection_t *c; + ngx_connection_t *c, *pc; ngx_pool_cleanup_t *cln; ngx_http_request_t *r; ngx_http_connection_t *hc; @@ -385,6 +390,10 @@ ngx_http_v3_wait_request_handler(ngx_eve h3c = ngx_http_v3_get_session(c); h3c->nrequests++; + pc = c->quic->parent; + pc->idle = 0; + ngx_reusable_connection(pc, 0); + if (h3c->keepalive.timer_set) { ngx_del_timer(&h3c->keepalive); } @@ -430,7 +439,7 @@ ngx_http_v3_cleanup_request(void *data) { ngx_http_request_t *r = data; - ngx_connection_t *c; + ngx_connection_t *c, *pc; ngx_http_v3_session_t *h3c; ngx_http_core_loc_conf_t *clcf; @@ -443,6 +452,16 @@ ngx_http_v3_cleanup_request(void *data) h3c = ngx_http_v3_get_session(c); if (--h3c->nrequests == 0) { + pc = c->quic->parent; + + if (ngx_terminate || ngx_exiting) { + ngx_quic_finalize_connection(pc, NGX_HTTP_V3_ERR_NO_ERROR, NULL); + return; + } + + pc->idle = 1; + ngx_reusable_connection(pc, 1); + clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module); ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout); } diff --git a/src/http/v3/ngx_http_v3_uni.c b/src/http/v3/ngx_http_v3_uni.c --- a/src/http/v3/ngx_http_v3_uni.c +++ b/src/http/v3/ngx_http_v3_uni.c @@ -182,6 +182,11 @@ ngx_http_v3_uni_read_handler(ngx_event_t ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 read handler"); + if (c->close) { + ngx_http_v3_close_uni_stream(c); + return; + } + ngx_memzero(&b, sizeof(ngx_buf_t)); while (rev->ready) { @@ -262,6 +267,11 @@ ngx_http_v3_uni_dummy_read_handler(ngx_e ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 dummy read handler"); + if (c->close) { + ngx_http_v3_close_uni_stream(c); + return; + } + if (rev->ready) { if (c->recv(c, &ch, 1) != 0) { ngx_http_v3_finalize_connection(c, NGX_HTTP_V3_ERR_NO_ERROR, NULL); From arut at nginx.com Thu Jun 2 13:52:46 2022 From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=) Date: Thu, 02 Jun 2022 17:52:46 +0400 Subject: [PATCH 2 of 3] QUIC: relocated early streams initialization In-Reply-To: References: Message-ID: # HG changeset patch # User Roman Arutyunyan # Date 1654177819 -14400 # Thu Jun 02 17:50:19 2022 +0400 # Branch quic # Node ID d3857a6df35c7a1d063e1a5776b6392afde16db2 # Parent 617ec472690620cc75c473f97555138a4bc7d38b QUIC: relocated early streams initialization. Previously, early streams initialization took place right after receiving early data keys. Now it's done after successful decryption of the first early data packet. The changes allows to perform operations in the streams initializer function, which require client transport parameters. An example of such operation is creating a server stream. diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c --- a/src/event/quic/ngx_event_quic.c +++ b/src/event/quic/ngx_event_quic.c @@ -1032,6 +1032,12 @@ ngx_quic_handle_payload(ngx_connection_t return ngx_quic_send_cc(c); } + if (pkt->level == ssl_encryption_early_data) { + if (ngx_quic_init_streams(c) != NGX_OK) { + return NGX_ERROR; + } + } + pkt->received = ngx_current_msec; c->log->action = "handling payload"; diff --git a/src/event/quic/ngx_event_quic_ssl.c b/src/event/quic/ngx_event_quic_ssl.c --- a/src/event/quic/ngx_event_quic_ssl.c +++ b/src/event/quic/ngx_event_quic_ssl.c @@ -80,12 +80,6 @@ ngx_quic_set_read_secret(ngx_ssl_conn_t return 0; } - if (level == ssl_encryption_early_data) { - if (ngx_quic_init_streams(c) != NGX_OK) { - return 0; - } - } - return 1; } @@ -150,14 +144,6 @@ ngx_quic_set_encryption_secrets(ngx_ssl_ return 0; } - if (level == ssl_encryption_early_data) { - if (ngx_quic_init_streams(c) != NGX_OK) { - return 0; - } - - return 1; - } - #ifdef NGX_QUIC_DEBUG_CRYPTO ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0, "quic write secret len:%uz %*xs", secret_len, From arut at nginx.com Thu Jun 2 13:52:47 2022 From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=) Date: Thu, 02 Jun 2022 17:52:47 +0400 Subject: [PATCH 3 of 3] QUIC: init_streams() callback In-Reply-To: References: Message-ID: <79f9c421058b43144aef.1654177967@arut-laptop> # HG changeset patch # User Roman Arutyunyan # Date 1653981374 -14400 # Tue May 31 11:16:14 2022 +0400 # Branch quic # Node ID 79f9c421058b43144aefb077532e4a8d4f22c983 # Parent d3857a6df35c7a1d063e1a5776b6392afde16db2 QUIC: init_streams() callback. It's called after handshake completion to initialize application-level data prior to creating streams. HTTP/3 callback implementation switches main QUIC connection to idle and reusable modes and sets keepalive timer. diff --git a/src/event/quic/ngx_event_quic.h b/src/event/quic/ngx_event_quic.h --- a/src/event/quic/ngx_event_quic.h +++ b/src/event/quic/ngx_event_quic.h @@ -28,6 +28,9 @@ #define NGX_QUIC_STREAM_UNIDIRECTIONAL 0x02 +typedef ngx_int_t (*ngx_quic_init_streams_pt)(ngx_connection_t *c); + + typedef enum { NGX_QUIC_STREAM_SEND_READY = 0, NGX_QUIC_STREAM_SEND_SEND, @@ -74,6 +77,8 @@ typedef struct { ngx_int_t stream_reject_code_uni; ngx_int_t stream_reject_code_bidi; + ngx_quic_init_streams_pt init_streams; + u_char av_token_key[NGX_QUIC_AV_KEY_LEN]; u_char sr_token_key[NGX_QUIC_SR_KEY_LEN]; } ngx_quic_conf_t; diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c --- a/src/event/quic/ngx_event_quic_streams.c +++ b/src/event/quic/ngx_event_quic_streams.c @@ -21,6 +21,7 @@ static ngx_quic_stream_t *ngx_quic_get_s static ngx_int_t ngx_quic_reject_stream(ngx_connection_t *c, uint64_t id); static void ngx_quic_init_stream_handler(ngx_event_t *ev); static void ngx_quic_init_streams_handler(ngx_connection_t *c); +static ngx_int_t ngx_quic_do_init_streams(ngx_connection_t *c); static ngx_quic_stream_t *ngx_quic_create_stream(ngx_connection_t *c, uint64_t id); static void ngx_quic_empty_handler(ngx_event_t *ev); @@ -571,15 +572,22 @@ ngx_quic_init_streams(ngx_connection_t * return NGX_OK; } - ngx_quic_init_streams_handler(c); - - return NGX_OK; + return ngx_quic_do_init_streams(c); } static void ngx_quic_init_streams_handler(ngx_connection_t *c) { + if (ngx_quic_do_init_streams(c) != NGX_OK) { + ngx_quic_close_connection(c, NGX_ERROR); + } +} + + +static ngx_int_t +ngx_quic_do_init_streams(ngx_connection_t *c) +{ ngx_queue_t *q; ngx_quic_stream_t *qs; ngx_quic_connection_t *qc; @@ -588,6 +596,12 @@ ngx_quic_init_streams_handler(ngx_connec qc = ngx_quic_get_connection(c); + if (qc->conf->init_streams) { + if (qc->conf->init_streams(c) != NGX_OK) { + return NGX_ERROR; + } + } + for (q = ngx_queue_head(&qc->streams.uninitialized); q != ngx_queue_sentinel(&qc->streams.uninitialized); q = ngx_queue_next(q)) @@ -597,6 +611,8 @@ ngx_quic_init_streams_handler(ngx_connec } qc->streams.initialized = 1; + + return NGX_OK; } diff --git a/src/http/v3/ngx_http_v3.c b/src/http/v3/ngx_http_v3.c --- a/src/http/v3/ngx_http_v3.c +++ b/src/http/v3/ngx_http_v3.c @@ -17,21 +17,15 @@ static void ngx_http_v3_cleanup_session( ngx_int_t ngx_http_v3_init_session(ngx_connection_t *c) { - ngx_connection_t *pc; ngx_pool_cleanup_t *cln; ngx_http_connection_t *hc; ngx_http_v3_session_t *h3c; - pc = c->quic->parent; - hc = pc->data; - - if (hc->v3_session) { - return NGX_OK; - } + hc = c->data; ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 init session"); - h3c = ngx_pcalloc(pc->pool, sizeof(ngx_http_v3_session_t)); + h3c = ngx_pcalloc(c->pool, sizeof(ngx_http_v3_session_t)); if (h3c == NULL) { goto failed; } @@ -42,16 +36,16 @@ ngx_http_v3_init_session(ngx_connection_ ngx_queue_init(&h3c->blocked); ngx_queue_init(&h3c->pushing); - h3c->keepalive.log = pc->log; - h3c->keepalive.data = pc; + h3c->keepalive.log = c->log; + h3c->keepalive.data = c; h3c->keepalive.handler = ngx_http_v3_keepalive_handler; h3c->keepalive.cancelable = 1; - h3c->table.send_insert_count.log = pc->log; - h3c->table.send_insert_count.data = pc; + h3c->table.send_insert_count.log = c->log; + h3c->table.send_insert_count.data = c; h3c->table.send_insert_count.handler = ngx_http_v3_inc_insert_count_handler; - cln = ngx_pool_cleanup_add(pc->pool, 0); + cln = ngx_pool_cleanup_add(c->pool, 0); if (cln == NULL) { goto failed; } diff --git a/src/http/v3/ngx_http_v3.h b/src/http/v3/ngx_http_v3.h --- a/src/http/v3/ngx_http_v3.h +++ b/src/http/v3/ngx_http_v3.h @@ -153,6 +153,7 @@ struct ngx_http_v3_session_s { void ngx_http_v3_init(ngx_connection_t *c); void ngx_http_v3_reset_connection(ngx_connection_t *c); +ngx_int_t ngx_http_v3_init_streams(ngx_connection_t *c); ngx_int_t ngx_http_v3_init_session(ngx_connection_t *c); ngx_int_t ngx_http_v3_check_flood(ngx_connection_t *c); diff --git a/src/http/v3/ngx_http_v3_module.c b/src/http/v3/ngx_http_v3_module.c --- a/src/http/v3/ngx_http_v3_module.c +++ b/src/http/v3/ngx_http_v3_module.c @@ -249,6 +249,8 @@ ngx_http_v3_create_srv_conf(ngx_conf_t * h3scf->quic.stream_reject_code_bidi = NGX_HTTP_V3_ERR_REQUEST_REJECTED; h3scf->quic.active_connection_id_limit = NGX_CONF_UNSET_UINT; + h3scf->quic.init_streams = ngx_http_v3_init_streams; + return h3scf; } diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c --- a/src/http/v3/ngx_http_v3_request.c +++ b/src/http/v3/ngx_http_v3_request.c @@ -93,11 +93,6 @@ ngx_http_v3_init(ngx_connection_t *c) } #endif - if (ngx_http_v3_init_session(c) != NGX_OK) { - ngx_http_close_connection(c); - return; - } - if (c->quic->id & NGX_QUIC_STREAM_UNIDIRECTIONAL) { ngx_http_v3_init_uni_stream(c); @@ -107,6 +102,47 @@ ngx_http_v3_init(ngx_connection_t *c) } +ngx_int_t +ngx_http_v3_init_streams(ngx_connection_t *c) +{ + ngx_http_v3_session_t *h3c; +#if (NGX_HTTP_V3_HQ) + ngx_http_connection_t *hc; + ngx_http_v3_srv_conf_t *h3scf; +#endif + ngx_http_core_loc_conf_t *clcf; + + if (ngx_terminate || ngx_exiting) { + return NGX_ERROR; + } + +#if (NGX_HTTP_V3_HQ) + + hc = c->data; + + h3scf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_v3_module); + + if (h3scf->hq) { + return NGX_OK; + } + +#endif + + if (ngx_http_v3_init_session(c) != NGX_OK) { + return NGX_ERROR; + } + + c->idle = 1; + ngx_reusable_connection(c, 1); + + h3c = ngx_http_v3_get_session(c); + clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module); + ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout); + + return NGX_OK; +} + + #if (NGX_HTTP_V3_HQ) static void From xeioex at nginx.com Thu Jun 2 23:33:29 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 02 Jun 2022 23:33:29 +0000 Subject: [njs] Fixed njs_vmcode_interpreter() when await fails. Message-ID: details: https://hg.nginx.org/njs/rev/e99d83c24517 branches: changeset: 1873:e99d83c24517 user: Dmitry Volyntsev date: Thu Jun 02 16:32:38 2022 -0700 description: Fixed njs_vmcode_interpreter() when await fails. Previously, while interpreting a user function, njs_vmcode_interpreter() might return prematurely when an error happens in await instruction. This is not correct because the current frame has to be unwound (or exception caught) first. The fix is to exit through only 5 appropriate exit points to ensure proper unwinding. The patch correctly fixes issue reported in 07ef6c1f04f1 (0.7.3). This closes #506 issue on Github. diffstat: src/njs_vmcode.c | 9 ++++++++- test/js/async_exception_in_await.t.js | 22 ++++++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletions(-) diffs (54 lines): diff -r ddec2f61c6d4 -r e99d83c24517 src/njs_vmcode.c --- a/src/njs_vmcode.c Wed Jun 01 21:54:32 2022 -0700 +++ b/src/njs_vmcode.c Thu Jun 02 16:32:38 2022 -0700 @@ -856,9 +856,15 @@ next: case NJS_VMCODE_AWAIT: await = (njs_vmcode_await_t *) pc; + ret = njs_vmcode_await(vm, await, promise_cap, async_ctx); + njs_vmcode_debug(vm, pc, "EXIT AWAIT"); - return njs_vmcode_await(vm, await, promise_cap, async_ctx); + if (njs_slow_path(ret == NJS_ERROR)) { + goto error; + } + + return ret; case NJS_VMCODE_TRY_START: ret = njs_vmcode_try_start(vm, value1, value2, pc); @@ -1923,6 +1929,7 @@ njs_vmcode_await(njs_vm_t *vm, njs_vmcod value = njs_scope_valid_value(vm, await->retval); if (njs_slow_path(value == NULL)) { + njs_internal_error(vm, "await->retval is invalid"); return NJS_ERROR; } diff -r ddec2f61c6d4 -r e99d83c24517 test/js/async_exception_in_await.t.js --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/js/async_exception_in_await.t.js Thu Jun 02 16:32:38 2022 -0700 @@ -0,0 +1,22 @@ +/*--- +includes: [] +flags: [] +---*/ + +var p = new Promise(() => 0); +Object.defineProperty(p, "constructor", {get: () => ({}).a.a}); + +async function g() { + try { + await p; + } catch (e) { + } +} + +function f() { + g(); + + return 42; +} + +assert.sameValue(f(), 42); From mdounin at mdounin.ru Fri Jun 3 21:13:05 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 4 Jun 2022 00:13:05 +0300 Subject: [PATCH] Stream: don't flush empty buffers created for read errors. In-Reply-To: References: Message-ID: Hello! On Thu, Jun 02, 2022 at 03:20:54AM +0000, Aleksei Bavshin via nginx-devel wrote: > > I don't think it's the way to go. Semantically, flush and > > last_buf are mutually exclusive: last_buf implies flushing of all > > remaining data, and there should be no flush set if last_buf is > > set. > > > > The "cl->buf->flush = !src->read->eof" approach might be actually > > better and more obvious: it simply ensures that last_buf and flush > > are not set at the same time. I don't immediately see any cases > > where it can break things, and this should fix the original > > problem with empty UDP packets being sent. > > Thanks for confirming that, I wasn't sure it is the right approach exactly because I had doubts regarding the mutual exclusivity of the flags. > Updated patch below; tests are passing on both branches and all systems we have in CI. > > # HG changeset patch > # User Aleksei Bavshin > # Date 1653330584 25200 > # Mon May 23 11:29:44 2022 -0700 > # Node ID 40926829be83986a3a5a5f941d2014000a0acd2e > # Parent e0cfab501dd11fdd23ad492419692269d3a01fc7 > Stream: don't flush empty buffers created for read errors. > > When we generate the last_buf buffer for an UDP upstream recv error, it does > not contain any data from the wire. ngx_stream_write_filter attempts to forward > it anyways, which is incorrect (e.g., UDP upstream ECONNREFUSED will be > translated to an empty packet). > > This happens because we mark the buffer as both 'flush' and 'last_buf', and > ngx_stream_write_filter has special handling for flush with certain types of > connections (see d127837c714f, 32b0ba4855a6). The flags are meant to be > mutually exclusive, so the fix is to ensure that flush and last_buf are not set > at the same time. > > Reproduction: > > stream { > upstream unreachable { > server 127.0.0.1:8880; > } > server { > listen 127.0.0.1:8998 udp; > proxy_pass unreachable; > } > } > > 1 0.000000000 127.0.0.1 → 127.0.0.1 UDP 47 45588 → 8998 Len=5 > 2 0.000166300 127.0.0.1 → 127.0.0.1 UDP 47 51149 → 8880 Len=5 > 3 0.000172600 127.0.0.1 → 127.0.0.1 ICMP 75 Destination unreachable (Port > unreachable) > 4 0.000202400 127.0.0.1 → 127.0.0.1 UDP 42 8998 → 45588 Len=0 > > Fixes d127837c714f. > > diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c > --- a/src/stream/ngx_stream_proxy_module.c > +++ b/src/stream/ngx_stream_proxy_module.c > @@ -1735,7 +1735,7 @@ ngx_stream_proxy_process(ngx_stream_sess > > cl->buf->temporary = (n ? 1 : 0); > cl->buf->last_buf = src->read->eof; > - cl->buf->flush = 1; > + cl->buf->flush = !src->read->eof; > > (*packets)++; > *received += n; Looks good to me. Roman, please take another look. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Fri Jun 3 23:24:40 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 4 Jun 2022 02:24:40 +0300 Subject: [PATCH 00 of 10] multiple headers tests In-Reply-To: References: Message-ID: Hello! On Wed, Jun 01, 2022 at 03:11:45AM +0400, Sergey Kandaurov wrote: > > > On 21 Apr 2022, at 02:37, Maxim Dounin wrote: > > > > Hello! > > > > Tests for the multiple headers handling patch series. > > > > What do you think about extending the tests > for ngx_http_link_multi_headers() ? > Searching for previous headers is a novel routine > so it might make sense to improve its coverage. > > Something like this: > > # HG changeset patch > # User Sergey Kandaurov > # Date 1654038532 -14400 > # Wed Jun 01 03:08:52 2022 +0400 > # Node ID 3c2a768ef7e3eff6c943e0fa4dd10c9ec487c973 > # Parent 83ec649296126965883e0d65c92ecd99f5397dc7 > Tests: added fastcgi tests for catched bugs in combining headers. > > This improves coverage in a search for previous headers. > > diff --git a/fastcgi_header_params.t b/fastcgi_header_params.t > --- a/fastcgi_header_params.t > +++ b/fastcgi_header_params.t > @@ -25,7 +25,7 @@ eval { require FCGI; }; > plan(skip_all => 'FCGI not installed') if $@; > plan(skip_all => 'win32') if $^O eq 'MSWin32'; > > -my $t = Test::Nginx->new()->has(qw/http fastcgi/)->plan(4) > +my $t = Test::Nginx->new()->has(qw/http fastcgi/)->plan(5) > ->write_file_expand('nginx.conf', <<'EOF'); > > %%TEST_GLOBALS%% > @@ -58,6 +58,8 @@ EOF > > like(http_get_headers('/'), qr/SEE-THIS/, > 'fastcgi request with many ignored headers'); > +like(http_get_headers('/', diff => 1), qr/SEE-THIS/, > + 'fastcgi request with many different headers'); > > TODO: { > local $TODO = 'not yet' unless $t->has_version('1.23.0'); > @@ -94,7 +96,7 @@ like($r, qr/X-Foo: foo, bar, bazz/, > > sub http_get_headers { > my ($url, %extra) = @_; > - return http(< + my $r = < GET $url HTTP/1.0 > Host: localhost > X-Blah: ignored header > @@ -116,8 +118,12 @@ X-Blah: ignored header > X-Blah: ignored header > X-Blah: ignored header > X-Blah: ignored header > +X-Blah: ignored header > > EOF > + my $n = 0; > + $r =~ s/(?<=Blah)(?=:)/$n++/ge if $extra{diff}; > + return http($r); > } I don't think the particular change is a good idea: it interferes with quite different test, designed to catch issues with hiding multiple headers due to not enough memory allocated for the ignored array (4015:e0a435f5f504). As a result, it would be hard to find out what is actually tested in both tests and how many headers are needed in test. If you think that we need a test with multiple different headers (presumably with multiple list parts, so more than 20 headers as per ngx_list_init() for r->headers_in.headers), it probably should be added separately. And it might be a good idea to actually test headers are being correctly passed/merged/ignored across list parts. Not sure we need it though. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Fri Jun 3 23:30:12 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 4 Jun 2022 02:30:12 +0300 Subject: [PATCH] Resolver: make TCP write timer event cancelable. In-Reply-To: References: Message-ID: Hello! On Thu, Jun 02, 2022 at 03:49:50AM +0000, Aleksei Bavshin via nginx-devel wrote: > Updated patch, with maybe too verbose description. > For the record, I did check a few similar modules I'm aware of (nginx-upstream-dynamic-servers, ngx_upstream_jdomain) and haven't found any affected code. So, the effect on third-party modules is hypothetical. > > # HG changeset patch > # User Aleksei Bavshin > # Date 1654139843 25200 > # Wed Jun 01 20:17:23 2022 -0700 > # Node ID 488f7a926ba54303c930b0b1e19c2b27e0af5079 > # Parent 40926829be83986a3a5a5f941d2014000a0acd2e > Resolver: make TCP write timer event cancelable. > > Similar to 70e65bf8dfd7, the change is made to ensure that the ability to > cancel resolver tasks is fully controlled by the caller. As mentioned in the > referenced commit, it is safe to make this timer cancelable because resolve > tasks can have their own timeouts that are not cancelable. > > The scenario where this may become a problem is a periodic background resolve > task (not tied to a specific request or a client connection), which receives a > response with short TTL, large enough to warrant fallback to a TCP query. > With each event loop wakeup, we either have a previously set write timer > instance or schedule a new one. The non-cancelable write timer can delay or > block graceful shutdown of a worker even if the ngx_resolver_ctx_t->cancelable > flag is set by the API user, and there are no other tasks or connections. > > We use the resolver API in this way to maintain the list of upstream server > addresses specified with the 'resolve' parameter, and there could be third-party > modules implementing similar logic. > > diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c > --- a/src/core/ngx_resolver.c > +++ b/src/core/ngx_resolver.c > @@ -1389,6 +1389,7 @@ ngx_resolver_send_tcp_query(ngx_resolver > > rec->tcp->data = rec; > rec->tcp->write->handler = ngx_resolver_tcp_write; > + rec->tcp->write->cancelable = 1; > rec->tcp->read->handler = ngx_resolver_tcp_read; > rec->tcp->read->resolver = 1; > > Looks good. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Sat Jun 4 00:09:11 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 4 Jun 2022 03:09:11 +0300 Subject: [PATCH] Upstream: handling of certificates specified as an empty string In-Reply-To: <3bb1adbb74dfcd372f73.1653422757@enoparse.local> References: <3bb1adbb74dfcd372f73.1653422757@enoparse.local> Message-ID: Hello! On Wed, May 25, 2022 at 12:05:57AM +0400, Sergey Kandaurov wrote: > # HG changeset patch > # User Sergey Kandaurov > # Date 1653422583 -14400 > # Wed May 25 00:03:03 2022 +0400 > # Node ID 3bb1adbb74dfcd372f7369530967cfb415900778 > # Parent 8a54733c9d1290e6dc2f86af18e8a976a6352e4f > Upstream: handling of certificates specified as an empty string. > > Now, if the directive is given an empty string, such configuration cancels > loading of certificates should they be inherited from the previous level. > This restores a previous behaviour, before variables support in certificates > was introduced (3ab8e1e2f0f7). > > diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c > --- a/src/http/modules/ngx_http_grpc_module.c > +++ b/src/http/modules/ngx_http_grpc_module.c > @@ -4921,7 +4921,7 @@ ngx_http_grpc_set_ssl(ngx_conf_t *cf, ng > return NGX_ERROR; > } > > - } else { > + } else if (glcf->upstream.ssl_certificate->value.len) { > if (ngx_ssl_certificate(cf, glcf->upstream.ssl, > &glcf->upstream.ssl_certificate->value, > &glcf->upstream.ssl_certificate_key->value, > diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c > --- a/src/http/modules/ngx_http_proxy_module.c > +++ b/src/http/modules/ngx_http_proxy_module.c > @@ -4970,7 +4970,7 @@ ngx_http_proxy_set_ssl(ngx_conf_t *cf, n > return NGX_ERROR; > } > > - } else { > + } else if (plcf->upstream.ssl_certificate->value.len) { > if (ngx_ssl_certificate(cf, plcf->upstream.ssl, > &plcf->upstream.ssl_certificate->value, > &plcf->upstream.ssl_certificate_key->value, > diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c > --- a/src/http/modules/ngx_http_uwsgi_module.c > +++ b/src/http/modules/ngx_http_uwsgi_module.c > @@ -2457,7 +2457,7 @@ ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, n > return NGX_ERROR; > } > > - } else { > + } else if (uwcf->upstream.ssl_certificate->value.len) { > if (ngx_ssl_certificate(cf, uwcf->upstream.ssl, > &uwcf->upstream.ssl_certificate->value, > &uwcf->upstream.ssl_certificate_key->value, It probably should in the outer if instead: diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c +++ b/src/http/modules/ngx_http_proxy_module.c @@ -4955,8 +4955,9 @@ ngx_http_proxy_set_ssl(ngx_conf_t *cf, n return NGX_ERROR; } - if (plcf->upstream.ssl_certificate) { - + if (plcf->upstream.ssl_certificate + && plcf->upstream.ssl_certificate->value.len) + { if (plcf->upstream.ssl_certificate_key == NULL) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "no \"proxy_ssl_certificate_key\" is defined " With the corresponding change to the upstream module: diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -1690,8 +1690,10 @@ ngx_http_upstream_ssl_init_connection(ng } } - if (u->conf->ssl_certificate && (u->conf->ssl_certificate->lengths - || u->conf->ssl_certificate_key->lengths)) + if (u->conf->ssl_certificate + && u->conf->ssl_certificate->value.len + && (u->conf->ssl_certificate->lengths + || u->conf->ssl_certificate_key->lengths)) { if (ngx_http_upstream_ssl_certificate(r, u, c) != NGX_OK) { ngx_http_upstream_finalize_request(r, u, This is more in line with the previous logic, and will avoid errors in configurations like the following: proxy_ssl_certificate foo.crt; location / { proxy_pass https://...; proxy_ssl_certificate ""; } location /foo { proxy_pass https://...; proxy_ssl_certificate_key foo.key; } (Which is rather strange, but not very different from the idea of clearing proxy_ssl_certificate.) -- Maxim Dounin http://mdounin.ru/ From xeioex at nginx.com Sat Jun 4 07:20:59 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Sat, 04 Jun 2022 07:20:59 +0000 Subject: [njs] Fixed typed-array ctor when source array is changed while iterating. Message-ID: details: https://hg.nginx.org/njs/rev/317fae74dc2f branches: changeset: 1874:317fae74dc2f user: Dmitry Volyntsev date: Fri Jun 03 23:24:08 2022 -0700 description: Fixed typed-array ctor when source array is changed while iterating. Previously, the function used optimization for ordinary arrays with no gaps (so called fast arrays). For a fast array code took elements directly from internal flat C array. The direct pointer may become invalid as side-effect of custom valueOf() method for an element. The fix is to eliminate the micro-optimization which uses direct pointers. The problem is similar to the 9578cc729205 (0.7.2) commit. This closes #523 issue on Github. diffstat: src/njs_typed_array.c | 26 +++----------------------- 1 files changed, 3 insertions(+), 23 deletions(-) diffs (57 lines): diff -r e99d83c24517 -r 317fae74dc2f src/njs_typed_array.c --- a/src/njs_typed_array.c Thu Jun 02 16:32:38 2022 -0700 +++ b/src/njs_typed_array.c Fri Jun 03 23:24:08 2022 -0700 @@ -34,7 +34,6 @@ njs_typed_array_alloc(njs_vm_t *vm, njs_ uint64_t size, offset; njs_int_t ret; njs_value_t *value, prop; - njs_array_t *src_array; njs_typed_array_t *array, *src_tarray; njs_array_buffer_t *buffer; @@ -43,7 +42,6 @@ njs_typed_array_alloc(njs_vm_t *vm, njs_ offset = 0; buffer = NULL; - src_array = NULL; src_tarray = NULL; element_size = njs_typed_array_element_size(type); @@ -115,15 +113,9 @@ njs_typed_array_alloc(njs_vm_t *vm, njs_ size = (uint64_t) njs_typed_array_length(src_tarray) * element_size; } else if (njs_is_object(value)) { - if (njs_is_fast_array(value)) { - src_array = njs_array(value); - length = src_array->length; - - } else { - ret = njs_object_length(vm, value, &length); - if (njs_slow_path(ret == NJS_ERROR)) { - return NULL; - } + ret = njs_object_length(vm, value, &length); + if (njs_slow_path(ret == NJS_ERROR)) { + return NULL; } size = length * element_size; @@ -166,18 +158,6 @@ njs_typed_array_alloc(njs_vm_t *vm, njs_ memcpy(&buffer->u.u8[0], &src_tarray->buffer->u.u8[0], size); } - } else if (src_array != NULL) { - for (i = 0; i < length; i++) { - ret = njs_value_to_number(vm, &src_array->start[i], &num); - if (njs_slow_path(ret == NJS_ERROR)) { - return NULL; - } - - if (ret == NJS_OK) { - njs_typed_array_prop_set(vm, array, i, num); - } - } - } else if (!njs_is_array_buffer(value) && njs_is_object(value)) { for (i = 0; i < length; i++) { ret = njs_value_property_i64(vm, value, i, &prop); From xeioex at nginx.com Sat Jun 4 07:21:01 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Sat, 04 Jun 2022 07:21:01 +0000 Subject: [njs] Improved OPCODE debug by tracing native functions. Message-ID: details: https://hg.nginx.org/njs/rev/5cad81ba52ed branches: changeset: 1875:5cad81ba52ed user: Dmitry Volyntsev date: Fri Jun 03 23:24:10 2022 -0700 description: Improved OPCODE debug by tracing native functions. diffstat: src/njs_function.c | 11 +++++++++++ 1 files changed, 11 insertions(+), 0 deletions(-) diffs (21 lines): diff -r 317fae74dc2f -r 5cad81ba52ed src/njs_function.c --- a/src/njs_function.c Fri Jun 03 23:24:08 2022 -0700 +++ b/src/njs_function.c Fri Jun 03 23:24:10 2022 -0700 @@ -711,6 +711,17 @@ njs_function_native_call(njs_vm_t *vm) native = vm->top_frame; function = native->function; +#ifdef NJS_OPCODE_DEBUG + njs_str_t name; + + ret = njs_builtin_match_native_function(vm, function, &name); + if (ret != NJS_OK) { + name = njs_entry_unknown; + } + + njs_printf("CALL NATIVE %V\n", &name); +#endif + if (njs_fast_path(function->bound == NULL)) { call = function->u.native; From xeioex at nginx.com Sat Jun 4 07:21:03 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Sat, 04 Jun 2022 07:21:03 +0000 Subject: [njs] Introduced njs_is_byte_or_ascii_string(). Message-ID: details: https://hg.nginx.org/njs/rev/dd4d6e7cd9ad branches: changeset: 1876:dd4d6e7cd9ad user: Dmitry Volyntsev date: Fri Jun 03 23:24:11 2022 -0700 description: Introduced njs_is_byte_or_ascii_string(). diffstat: src/njs_regexp.c | 2 +- src/njs_string.c | 17 ++++++----------- src/njs_string.h | 7 +++++++ src/njs_vmcode.c | 9 +++------ 4 files changed, 17 insertions(+), 18 deletions(-) diffs (114 lines): diff -r 5cad81ba52ed -r dd4d6e7cd9ad src/njs_regexp.c --- a/src/njs_regexp.c Fri Jun 03 23:24:10 2022 -0700 +++ b/src/njs_regexp.c Fri Jun 03 23:24:11 2022 -0700 @@ -1627,7 +1627,7 @@ njs_regexp_prototype_symbol_split(njs_vm utf8 = NJS_STRING_BYTE; - if (s.length != 0 && s.length != s.size) { + if (!njs_is_byte_or_ascii_string(&s)) { utf8 = NJS_STRING_UTF8; } diff -r 5cad81ba52ed -r dd4d6e7cd9ad src/njs_string.c --- a/src/njs_string.c Fri Jun 03 23:24:10 2022 -0700 +++ b/src/njs_string.c Fri Jun 03 23:24:11 2022 -0700 @@ -926,7 +926,7 @@ njs_string_prototype_concat(njs_vm_t *vm size += string.size; length += string.length; - if (string.length == 0 && string.size != 0) { + if (njs_is_byte_string(&string)) { mask = 0; } } @@ -2642,8 +2642,7 @@ njs_string_prototype_to_lower_case(njs_v (void) njs_string_prop(&string, njs_argument(args, 0)); - if (string.length == 0 || string.length == string.size) { - /* Byte or ASCII string. */ + if (njs_is_byte_or_ascii_string(&string)) { p = njs_string_alloc(vm, &vm->retval, string.size, string.length); if (njs_slow_path(p == NULL)) { @@ -2714,8 +2713,7 @@ njs_string_prototype_to_upper_case(njs_v (void) njs_string_prop(&string, njs_argument(args, 0)); - if (string.length == 0 || string.length == string.size) { - /* Byte or ASCII string. */ + if (njs_is_byte_or_ascii_string(&string)) { p = njs_string_alloc(vm, &vm->retval, string.size, string.length); if (njs_slow_path(p == NULL)) { @@ -2787,8 +2785,7 @@ njs_string_prototype_trim(njs_vm_t *vm, start = string.start; end = string.start + string.size; - if (string.length == 0 || string.length == string.size) { - /* Byte or ASCII string. */ + if (njs_is_byte_or_ascii_string(&string)) { if (mode & NJS_TRIM_START) { for ( ;; ) { @@ -4392,8 +4389,7 @@ njs_string_encode_uri(njs_vm_t *vm, njs_ src = string.start; end = src + string.size; - if (string.length == 0 || string.length == string.size) { - /* Byte or ASCII string. */ + if (njs_is_byte_or_ascii_string(&string)) { while (src < end) { byte = *src++; @@ -4450,8 +4446,7 @@ njs_string_encode_uri(njs_vm_t *vm, njs_ src = string.start; - if (string.length == 0 || string.length == string.size) { - /* Byte or ASCII string. */ + if (njs_is_byte_or_ascii_string(&string)) { (void) njs_string_encode(escape, string.size, src, dst); return NJS_OK; } diff -r 5cad81ba52ed -r dd4d6e7cd9ad src/njs_string.h --- a/src/njs_string.h Fri Jun 03 23:24:10 2022 -0700 +++ b/src/njs_string.h Fri Jun 03 23:24:11 2022 -0700 @@ -105,6 +105,13 @@ njs_is_byte_string(njs_string_prop_t *st } +njs_inline njs_bool_t +njs_is_byte_or_ascii_string(njs_string_prop_t *string) +{ + return (string->length == 0 || string->length == string->size); +} + + njs_inline uint32_t njs_string_calc_length(njs_utf8_t utf8, const u_char *start, size_t size) { diff -r 5cad81ba52ed -r dd4d6e7cd9ad src/njs_vmcode.c --- a/src/njs_vmcode.c Fri Jun 03 23:24:10 2022 -0700 +++ b/src/njs_vmcode.c Fri Jun 03 23:24:11 2022 -0700 @@ -1606,16 +1606,13 @@ njs_string_concat(njs_vm_t *vm, njs_valu (void) njs_string_prop(&string1, val1); (void) njs_string_prop(&string2, val2); + length = string1.length + string2.length; + /* * A result of concatenation of Byte and ASCII or UTF-8 strings * is a Byte string. */ - if ((string1.length != 0 || string1.size == 0) - && (string2.length != 0 || string2.size == 0)) - { - length = string1.length + string2.length; - - } else { + if (njs_is_byte_string(&string1) || njs_is_byte_string(&string2)) { length = 0; } From xeioex at nginx.com Sat Jun 4 07:21:05 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Sat, 04 Jun 2022 07:21:05 +0000 Subject: [njs] Fixed String.prototype.replace() with byte strings. Message-ID: details: https://hg.nginx.org/njs/rev/9c4858617097 branches: changeset: 1877:9c4858617097 user: Dmitry Volyntsev date: Fri Jun 03 23:24:12 2022 -0700 description: Fixed String.prototype.replace() with byte strings. This closes #522 issue on Github. diffstat: src/njs_string.c | 23 +++++++++++++++-------- src/test/njs_unit_test.c | 5 +++++ 2 files changed, 20 insertions(+), 8 deletions(-) diffs (69 lines): diff -r dd4d6e7cd9ad -r 9c4858617097 src/njs_string.c --- a/src/njs_string.c Fri Jun 03 23:24:11 2022 -0700 +++ b/src/njs_string.c Fri Jun 03 23:24:12 2022 -0700 @@ -3663,7 +3663,7 @@ njs_string_prototype_replace(njs_vm_t *v njs_index_t unused) { u_char *r; - size_t length, search_length, ret_length, size; + size_t length, size; int64_t pos; njs_int_t ret; njs_value_t *this, *search, *replace; @@ -3722,8 +3722,8 @@ njs_string_prototype_replace(njs_vm_t *v } } - length = njs_string_prop(&string, this); - search_length = njs_string_prop(&s, search); + (void) njs_string_prop(&string, this); + (void) njs_string_prop(&s, search); pos = njs_string_index_of(&string, &s, 0); if (pos < 0) { @@ -3757,7 +3757,7 @@ njs_string_prototype_replace(njs_vm_t *v } } - if (length == string.size) { + if (njs_is_byte_or_ascii_string(&string)) { p = string.start + pos; } else { @@ -3765,10 +3765,17 @@ njs_string_prototype_replace(njs_vm_t *v p = njs_string_offset(string.start, string.start + string.size, pos); } - ret_length = njs_string_prop(&ret_string, &retval); - - size = string.size + ret_string.size - s.size; - length += ret_length - search_length; + (void) njs_string_prop(&ret_string, &retval); + + size = string.size + ret_string.size - s.size; + length = string.length + ret_string.length - s.length; + + if (njs_is_byte_string(&string) + || njs_is_byte_string(&s) + || njs_is_byte_string(&ret_string)) + { + length = 0; + } r = njs_string_alloc(vm, &vm->retval, size, length); if (njs_slow_path(r == NULL)) { diff -r dd4d6e7cd9ad -r 9c4858617097 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Fri Jun 03 23:24:11 2022 -0700 +++ b/src/test/njs_unit_test.c Fri Jun 03 23:24:12 2022 -0700 @@ -8438,6 +8438,11 @@ static njs_unit_test_t njs_test[] = { njs_str("var r = 'αβγ'.replace('', 'X'); [r, r.length]"), njs_str("Xαβγ,4") }, + { njs_str("var s = 'αz'.toUTF8();" + "var r = s.replace('z', 'β');" + "r.length"), + njs_str("4") }, + { njs_str("'abc'.replace('b', (m, o, s) => `|${s}|${o}|${m}|`)"), njs_str("a|abc|1|b|c") }, From devarajan.dk at zohocorp.com Mon Jun 6 06:54:35 2022 From: devarajan.dk at zohocorp.com (Devarajan D) Date: Mon, 06 Jun 2022 12:24:35 +0530 Subject: Changeset: Dynamic Virtual Server Selection Based on Host's Cnamed domain Message-ID: <18137cb3e20.db44da5a19378.6352067189672045247@zohocorp.com> Dear Members, # HG changeset patch # User devarajan.dk # Date 1654428592 -19800 #      Sun Jun 05 16:59:52 2022 +0530 # Node ID 8d73e94b379091440ad8c836e2a052fb5edda160 # Parent  e0cfab501dd11fdd23ad492419692269d3a01fc7 Dynamic Virtual Server Selection Based on Host's Cnamed domain Use Case: Preventing nginx.conf reload on new customer domain addition. Many B2B Cloud Services provide option for rebranding customer domains through CNAME. eg. Customers - mail.schroeppel.me, mail.vey.de cnamed to ghs.googlehosted.com The hosting Service can be having multiple such services reverse proxied through same nginx instance. (eg. Virtual Servers - mail.ghost.com, docs.ghost.com, drive.ghost.com, etc. all virtual services listening on the same IP and port; consider 100s of such virtual services) While deployment, the cloud service has to add the customer domain name in 'server_name' directive of corresponding virtual server for the request to reach the right virtual server block. Also, as the customer has given control to the hosting cloud service, the service will also purchase certificate for the customer domain and offload through nginx. As and when the new rebranding customers starts to subscribe for the cloud service, the cloud service has to add the certificate name & key in ssl_certificate & ssl_certificate_key directives respectively and customer's domain name in server_name directive of corresponding virtual server. So the nginx.conf has to be reloaded every time a new customer starts using the service. As the frequency of new customers joining the service increases, the hosting service may suffer from frequent nginx reloads and may not scale well. (consider 100s of customers joining the service every hour) Thanks to the Dynamic Certificate Loading feature in nginx that uses $ssl_server_name variable to dynamically select the certificate on runtime and no need to reload conf after deploying certificate. But still there is a need to reload the conf due to the addition of customer's domain name in server_name directive. To prevent this reload, we can dynamically choose Virtual server based on the cname of the customer domains. Only the cnamed domain ie. mail.ghost.com will be written in the server_name directive of the virtual server. All customer domains cnamed to mail.ghost.com will be routed to right virtual server based on the cname. This cname check can be turned on/off by a http level directive - 'resolve_cname' (off by default) Note: 1. Similar to Dynamic Certificate Loading, the pros of dynamic Virtual Server selection come    at the cost of increase in latency during request processing time due to DNS lookup. 2. The cloud service can choose to deploy their customer domains in hybrid mode too.    (ie. old customers domains can be explicitly configured in server_name and    customer domains who newly subscribed for the cloud service in between two nginx.conf reloads    can be deployed in nginx dynamically) diff -r e0cfab501dd1 -r 8d73e94b3790 src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c   Tue May 31 00:14:11 2022 +0300 +++ b/src/http/ngx_http_core_module.c   Sun Jun 05 16:59:52 2022 +0530 @@ -252,6 +252,13 @@        offsetof(ngx_http_core_srv_conf_t, large_client_header_buffers),        NULL },   +    { ngx_string("resolve_cname"), +      NGX_HTTP_MAIN_CONF|NGX_CONF_FLAG, +      ngx_conf_set_flag_slot, +      NGX_HTTP_MAIN_CONF_OFFSET, +      offsetof(ngx_http_core_main_conf_t, resolve_cname), +      NULL }, +      { ngx_string("ignore_invalid_headers"),        NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_FLAG,        ngx_conf_set_flag_slot, @@ -3404,6 +3411,7 @@      cmcf->variables_hash_max_size = NGX_CONF_UNSET_UINT;      cmcf->variables_hash_bucket_size = NGX_CONF_UNSET_UINT;   +    cmcf->resolve_cname = NGX_CONF_UNSET;      return cmcf;  }   diff -r e0cfab501dd1 -r 8d73e94b3790 src/http/ngx_http_core_module.h --- a/src/http/ngx_http_core_module.h   Tue May 31 00:14:11 2022 +0300 +++ b/src/http/ngx_http_core_module.h   Sun Jun 05 16:59:52 2022 +0530 @@ -172,6 +172,8 @@        ngx_array_t               *ports;   +    ngx_flag_t                 resolve_cname; +      ngx_http_phase_t           phases[NGX_HTTP_LOG_PHASE + 1];  } ngx_http_core_main_conf_t;   diff -r e0cfab501dd1 -r 8d73e94b3790 src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c       Tue May 31 00:14:11 2022 +0300 +++ b/src/http/ngx_http_request.c       Sun Jun 05 16:59:52 2022 +0530 @@ -8,6 +8,7 @@  #include  #include  #include +#include      static void ngx_http_wait_request_handler(ngx_event_t *ev); @@ -36,6 +37,8 @@  static ngx_int_t ngx_http_find_virtual_server(ngx_connection_t *c,      ngx_http_virtual_names_t *virtual_names, ngx_str_t *host,      ngx_http_request_t *r, ngx_http_core_srv_conf_t **cscfp); +     +static void ngx_http_find_cname(ngx_str_t* host, ngx_str_t* cname, ngx_http_request_t *r);    static void ngx_http_request_handler(ngx_event_t *ev);  static void ngx_http_terminate_request(ngx_http_request_t *r, ngx_int_t rc); @@ -2269,6 +2272,27 @@      return NGX_OK;  }   +static void +ngx_http_find_cname(ngx_str_t* host, ngx_str_t* cname, ngx_http_request_t *r) +{ +    struct hostent *he; +    char **alias_list; + +    if ( (he = gethostbyname( (char*) host->data ) ) == NULL) { +        return; +    } + +    alias_list = (char **) he->h_aliases; + +    for(int i = 0; alias_list[i] != NULL; i++) { +        if (alias_list[i+1] == NULL) { +            cname->data = (u_char *) alias_list[i]; +            cname->len = ngx_strlen(cname->data); +        } +    } +    ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, +                   "Host - %s cnamed to %s", host->data, cname->data); +}    static ngx_int_t  ngx_http_find_virtual_server(ngx_connection_t *c, @@ -2290,6 +2314,24 @@          return NGX_OK;      }   +    ngx_http_core_main_conf_t *cmcf; + +    cmcf = ngx_http_get_module_main_conf(r, ngx_http_core_module); + +    if(cmcf->resolve_cname) { +        ngx_str_t cname = ngx_null_string; +        ngx_http_find_cname(host, &cname, r); +        if (cname.data != NULL) { +            cscf = ngx_hash_find_combined(&virtual_names->names, +                                      ngx_hash_key(cname.data, cname.len), +                                      cname.data, cname.len); +            if (cscf) { +                *cscfp = cscf; +                return NGX_OK; +            } +        } +    } +  #if (NGX_PCRE)        if (host->len && virtual_names->nregex) { Thanks & Regards, Devarajan D., Member Technical Staff, ZOHO Corporation Pvt. Ltd. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: cname.patch Type: application/octet-stream Size: 3594 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: sample_nginx.conf Type: application/octet-stream Size: 585 bytes Desc: not available URL: From pluknet at nginx.com Mon Jun 6 10:34:16 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 6 Jun 2022 14:34:16 +0400 Subject: [PATCH 1 of 3] QUIC: reusable and idle modes for main connection In-Reply-To: <617ec472690620cc75c4.1654177965@arut-laptop> References: <617ec472690620cc75c4.1654177965@arut-laptop> Message-ID: > On 2 Jun 2022, at 17:52, Roman Arutyunyan wrote: > > # HG changeset patch > # User Roman Arutyunyan > # Date 1652852691 -14400 > # Wed May 18 09:44:51 2022 +0400 > # Branch quic > # Node ID 617ec472690620cc75c473f97555138a4bc7d38b > # Parent a231f30606317a61efa3005fda9b43eb91d148d9 > QUIC: reusable and idle modes for main connection. > > The modes are controlled by application layer. For HTTP/3, they are enabled > when there are no active request streams. > > diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c > --- a/src/event/quic/ngx_event_quic.c > +++ b/src/event/quic/ngx_event_quic.c > @@ -414,8 +414,8 @@ ngx_quic_input_handler(ngx_event_t *rev) > } > > if (c->close) { > - qc->error_reason = "graceful shutdown"; > - ngx_quic_close_connection(c, NGX_OK); > + qc->error = NGX_QUIC_ERR_NO_ERROR; > + ngx_quic_close_connection(c, NGX_ERROR); Now a valid connection is closed with INTERNAL_ERROR on graceful shutdown. What's the real intention? > return; > } > > diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c > --- a/src/event/quic/ngx_event_quic_streams.c > +++ b/src/event/quic/ngx_event_quic_streams.c > @@ -161,6 +161,7 @@ ngx_quic_close_streams(ngx_connection_t > ngx_pool_t *pool; > ngx_queue_t *q; > ngx_rbtree_t *tree; > + ngx_connection_t *sc; > ngx_rbtree_node_t *node; > ngx_quic_stream_t *qs; > > @@ -185,24 +186,41 @@ ngx_quic_close_streams(ngx_connection_t > return NGX_OK; > } > > -#if (NGX_DEBUG) > - ns = 0; > -#endif > - > node = ngx_rbtree_min(tree->root, tree->sentinel); > > while (node) { > qs = (ngx_quic_stream_t *) node; > node = ngx_rbtree_next(tree, node); > + sc = qs->connection; > + > + if (sc == NULL) { > + ngx_quic_close_stream(qs); > + continue; > + } > + > + if (c->close || sc->idle || sc->reusable) { I'm not sure how the sc->idle test is a worthwhile as it seems never be set on HTTP/3 stream connections. Originally, c->idle was used to close HTTP/1.x keep-alive connections, and in HTTP/2 it is always set (see below). Testing sc->reusable here is a moot point and looks like an abuse of what when short on worker connections. For tracking active streams, HTTP/2 uses h2c->processing. If implemented same way in HTTP/3, sc->reusable becomes unnecessary as well (not sure if that's possible). > + sc->close = 1; > + sc->read->handler(sc->read); > + } > + } > + > + if (tree->root == tree->sentinel) { > + return NGX_OK; > + } > + > +#if (NGX_DEBUG) > + ns = 0; > +#endif > + > + for (node = ngx_rbtree_min(tree->root, tree->sentinel); > + node; > + node = ngx_rbtree_next(tree, node)) > + { > + qs = (ngx_quic_stream_t *) node; > > qs->recv_state = NGX_QUIC_STREAM_RECV_RESET_RECVD; > qs->send_state = NGX_QUIC_STREAM_SEND_RESET_SENT; > > - if (qs->connection == NULL) { > - ngx_quic_close_stream(qs); > - continue; > - } > - > ngx_quic_set_event(qs->connection->read); > ngx_quic_set_event(qs->connection->write); > > @@ -587,6 +605,7 @@ ngx_quic_create_stream(ngx_connection_t > { > ngx_log_t *log; > ngx_pool_t *pool; > + ngx_uint_t reusable; > ngx_queue_t *q; > ngx_connection_t *sc; > ngx_quic_stream_t *qs; > @@ -639,7 +658,13 @@ ngx_quic_create_stream(ngx_connection_t > *log = *c->log; > pool->log = log; > > + reusable = c->reusable; > + ngx_reusable_connection(c, 0); > + > sc = ngx_get_connection(c->fd, log); > + > + ngx_reusable_connection(c, reusable); > + > if (sc == NULL) { > ngx_destroy_pool(pool); > ngx_queue_insert_tail(&qc->streams.free, &qs->queue); > diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c > --- a/src/http/v3/ngx_http_v3_request.c > +++ b/src/http/v3/ngx_http_v3_request.c > @@ -210,6 +210,11 @@ ngx_http_v3_init_request_stream(ngx_conn > > h3c = ngx_http_v3_get_session(c); > > + if (!h3c->goaway && (ngx_terminate || ngx_exiting)) { > + h3c->goaway = 1; > + (void) ngx_http_v3_send_goaway(c, c->quic->id); > + } > + > if (h3c->goaway) { > c->close = 1; > ngx_http_close_connection(c); > @@ -258,7 +263,7 @@ ngx_http_v3_wait_request_handler(ngx_eve > size_t size; > ssize_t n; > ngx_buf_t *b; > - ngx_connection_t *c; > + ngx_connection_t *c, *pc; > ngx_pool_cleanup_t *cln; > ngx_http_request_t *r; > ngx_http_connection_t *hc; > @@ -385,6 +390,10 @@ ngx_http_v3_wait_request_handler(ngx_eve > h3c = ngx_http_v3_get_session(c); > h3c->nrequests++; > > + pc = c->quic->parent; > + pc->idle = 0; > + ngx_reusable_connection(pc, 0); > + > if (h3c->keepalive.timer_set) { > ngx_del_timer(&h3c->keepalive); > } > @@ -430,7 +439,7 @@ ngx_http_v3_cleanup_request(void *data) > { > ngx_http_request_t *r = data; > > - ngx_connection_t *c; > + ngx_connection_t *c, *pc; > ngx_http_v3_session_t *h3c; > ngx_http_core_loc_conf_t *clcf; > > @@ -443,6 +452,16 @@ ngx_http_v3_cleanup_request(void *data) > h3c = ngx_http_v3_get_session(c); > > if (--h3c->nrequests == 0) { > + pc = c->quic->parent; > + > + if (ngx_terminate || ngx_exiting) { > + ngx_quic_finalize_connection(pc, NGX_HTTP_V3_ERR_NO_ERROR, NULL); > + return; > + } > + > + pc->idle = 1; > + ngx_reusable_connection(pc, 1); > + Such way it doesn't take any action when there are requests streams. It could send GOAWAY proactively. By the way, with the patch, after graceful shutdown, it still allows processing of additional new streams and rearms idle timeout, so it may never end. HTTP/2 had set c->idle similarly before 5e95b9fb33b7. Now it always sets c->idle and performs accordingly to the state. Probably, HTTP/3 should be modified in somewhat the same direction. > clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module); > ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout); > } > diff --git a/src/http/v3/ngx_http_v3_uni.c b/src/http/v3/ngx_http_v3_uni.c > --- a/src/http/v3/ngx_http_v3_uni.c > +++ b/src/http/v3/ngx_http_v3_uni.c > @@ -182,6 +182,11 @@ ngx_http_v3_uni_read_handler(ngx_event_t > > ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 read handler"); > > + if (c->close) { > + ngx_http_v3_close_uni_stream(c); > + return; > + } > + > ngx_memzero(&b, sizeof(ngx_buf_t)); > > while (rev->ready) { > @@ -262,6 +267,11 @@ ngx_http_v3_uni_dummy_read_handler(ngx_e > > ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 dummy read handler"); > > + if (c->close) { > + ngx_http_v3_close_uni_stream(c); > + return; > + } > + > if (rev->ready) { > if (c->recv(c, &ch, 1) != 0) { > ngx_http_v3_finalize_connection(c, NGX_HTTP_V3_ERR_NO_ERROR, NULL); > -- Sergey Kandaurov From pluknet at nginx.com Mon Jun 6 10:43:40 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 6 Jun 2022 14:43:40 +0400 Subject: [PATCH 2 of 3] QUIC: relocated early streams initialization In-Reply-To: References: Message-ID: <2F9FF4A5-39A3-41C7-8B6B-E1C4D67F2F61@nginx.com> > > On 2 Jun 2022, at 17:52, Roman Arutyunyan wrote: > > # HG changeset patch > # User Roman Arutyunyan > # Date 1654177819 -14400 > # Thu Jun 02 17:50:19 2022 +0400 > # Branch quic > # Node ID d3857a6df35c7a1d063e1a5776b6392afde16db2 > # Parent 617ec472690620cc75c473f97555138a4bc7d38b > QUIC: relocated early streams initialization. > > Previously, early streams initialization took place right after receiving early > data keys. Now it's done after successful decryption of the first early data > packet. > > The changes allows to perform operations in the streams initializer function, > which require client transport parameters. An example of such operation is > creating a server stream. > > diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c > --- a/src/event/quic/ngx_event_quic.c > +++ b/src/event/quic/ngx_event_quic.c > @@ -1032,6 +1032,12 @@ ngx_quic_handle_payload(ngx_connection_t > return ngx_quic_send_cc(c); > } > > + if (pkt->level == ssl_encryption_early_data) { > + if (ngx_quic_init_streams(c) != NGX_OK) { > + return NGX_ERROR; > + } > + } > + > pkt->received = ngx_current_msec; > > c->log->action = "handling payload"; > diff --git a/src/event/quic/ngx_event_quic_ssl.c b/src/event/quic/ngx_event_quic_ssl.c > --- a/src/event/quic/ngx_event_quic_ssl.c > +++ b/src/event/quic/ngx_event_quic_ssl.c > @@ -80,12 +80,6 @@ ngx_quic_set_read_secret(ngx_ssl_conn_t > return 0; > } > > - if (level == ssl_encryption_early_data) { > - if (ngx_quic_init_streams(c) != NGX_OK) { > - return 0; > - } > - } > - > return 1; > } > > @@ -150,14 +144,6 @@ ngx_quic_set_encryption_secrets(ngx_ssl_ > return 0; > } > > - if (level == ssl_encryption_early_data) { > - if (ngx_quic_init_streams(c) != NGX_OK) { > - return 0; > - } > - > - return 1; > - } > - As discussed in private, it still needs surrounding condition. Otherwise, seems to be good. > #ifdef NGX_QUIC_DEBUG_CRYPTO > ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0, > "quic write secret len:%uz %*xs", secret_len, > -- Sergey Kandaurov -------------- next part -------------- An HTML attachment was scrubbed... URL: From pluknet at nginx.com Mon Jun 6 13:42:20 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 6 Jun 2022 17:42:20 +0400 Subject: [PATCH] Upstream: prioritise Cache-Control over Expires In-Reply-To: References: <4de897b7-e1b2-dbcd-7e20-f18ea91f8463@cdnnow.ru> <151ef936-f5d2-b681-bb54-6476c654a3ae@cdnnow.ru> <00218a02-f3e7-d1fe-04ba-1e61049f17be@cdnnow.ru> Message-ID: <20220606134220.afco4q4u6ujj2s3h@Y9MQ9X2QVV> On Sun, Apr 24, 2022 at 06:42:47PM +0300, Maxim Dounin wrote: > Hello! > > On Sun, Apr 24, 2022 at 07:55:17AM +0300, Maxim Dounin wrote: > > [...] > > > As far as I can tell, proper behaviour, assuming we parse cache > > control extensions independently of X-Accel-Expires, can be > > implemented by using just one flag. > > No, that's wrong, as with just one flag it wouldn't be possible to > correctly disable caching of responses with: > > Cache-Control: private > Cache-Control: max-age=10 > > So it needs at least two flags. Updated patch below, review and > testing appreciated. > > # HG changeset patch > # User Maxim Dounin > # Date 1650814681 -10800 > # Sun Apr 24 18:38:01 2022 +0300 > # Node ID 940ba4317a97c72d1ee6700cbf58a543fee04c7a > # Parent a736a7a613ea6e182ff86fbadcb98bb0f8891c0b > Upstream: fixed X-Accel-Expires/Cache-Control/Expires handling. > > Previously, if caching was disabled due to Expires in the past, nginx > failed to cache the response even if it was cacheable as per subsequently > parsed Cache-Control header (ticket #964). > > Similarly, if caching was disabled due to Expires in the past, > "Cache-Control: no-cache" or "Cache-Control: max-age=0", caching was not > used if it was cacheable as per subsequently parsed X-Accel-Expires header. > > Fix is to avoid disabling caching immediately after parsing Expires in > the past or Cache-Control, but rather set flags which are later checked by > ngx_http_upstream_process_headers() (and cleared by "Cache-Control: max-age" > and X-Accel-Expires). > > Additionally, now X-Accel-Expires does not prevent parsing of cache control > extensions, notably stale-while-revalidate and stale-if-error. This > ensures that order of the X-Accel-Expires and Cache-Control headers is not > important. > > Prodded by Vadim Fedorenko and Yugo Horie. > > diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c > --- a/src/http/ngx_http_upstream.c > +++ b/src/http/ngx_http_upstream.c > @@ -2697,6 +2697,10 @@ ngx_http_upstream_intercept_errors(ngx_h > > if (r->cache) { > > + if (u->headers_in.no_cache || u->headers_in.expired) { > + u->cacheable = 0; > + } > + > if (u->cacheable) { > time_t valid; > > @@ -2791,6 +2795,10 @@ ngx_http_upstream_process_headers(ngx_ht > > umcf = ngx_http_get_module_main_conf(r, ngx_http_upstream_module); > > + if (u->headers_in.no_cache || u->headers_in.expired) { > + u->cacheable = 0; > + } > + > if (u->headers_in.x_accel_redirect > && !(u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_XA_REDIRECT)) > { > @@ -4735,18 +4743,18 @@ ngx_http_upstream_process_cache_control( > return NGX_OK; > } > > - if (r->cache->valid_sec != 0 && u->headers_in.x_accel_expires != NULL) { > - return NGX_OK; > - } > - > start = h->value.data; > last = start + h->value.len; > > + if (r->cache->valid_sec != 0 && u->headers_in.x_accel_expires != NULL) { > + goto extensions; > + } > + > if (ngx_strlcasestrn(start, last, (u_char *) "no-cache", 8 - 1) != NULL > || ngx_strlcasestrn(start, last, (u_char *) "no-store", 8 - 1) != NULL > || ngx_strlcasestrn(start, last, (u_char *) "private", 7 - 1) != NULL) > { > - u->cacheable = 0; > + u->headers_in.no_cache = 1; > return NGX_OK; > } > > @@ -4776,12 +4784,15 @@ ngx_http_upstream_process_cache_control( > } > > if (n == 0) { > - u->cacheable = 0; > + u->headers_in.no_cache = 1; > return NGX_OK; > } > > r->cache->valid_sec = ngx_time() + n; > - } > + u->headers_in.expired = 0; > + } > + > +extensions: > > p = ngx_strlcasestrn(start, last, (u_char *) "stale-while-revalidate=", > 23 - 1); > @@ -4863,7 +4874,7 @@ ngx_http_upstream_process_expires(ngx_ht > expires = ngx_parse_http_time(h->value.data, h->value.len); > > if (expires == NGX_ERROR || expires < ngx_time()) { > - u->cacheable = 0; > + u->headers_in.expired = 1; > return NGX_OK; > } > > @@ -4914,6 +4925,8 @@ ngx_http_upstream_process_accel_expires( > > default: > r->cache->valid_sec = ngx_time() + n; > + u->headers_in.no_cache = 0; > + u->headers_in.expired = 0; > return NGX_OK; > } > } > @@ -4925,6 +4938,8 @@ ngx_http_upstream_process_accel_expires( > > if (n != NGX_ERROR) { > r->cache->valid_sec = n; > + u->headers_in.no_cache = 0; > + u->headers_in.expired = 0; Handling of "X-Accel-Redirect: @0" is still missed. Per the docs, it should set caching time to Epoch resulting in the expired response. Instead, such header is handled different, and it can be overridden with Cache-Control/Expires headers. Probably, it should be handled separately and disable caching, similar to "X-Accel-Redirect: 0". As the above note covers a corner case of X-Accel-Expires, I believe it's fine to commit as is. > } > } > #endif > diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h > --- a/src/http/ngx_http_upstream.h > +++ b/src/http/ngx_http_upstream.h > @@ -297,6 +297,8 @@ typedef struct { > > unsigned connection_close:1; > unsigned chunked:1; > + unsigned no_cache:1; > + unsigned expired:1; > } ngx_http_upstream_headers_in_t; > > > From pluknet at nginx.com Mon Jun 6 14:41:21 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 6 Jun 2022 18:41:21 +0400 Subject: [PATCH] Upstream: prioritise Cache-Control over Expires In-Reply-To: <20220606134220.afco4q4u6ujj2s3h@Y9MQ9X2QVV> References: <4de897b7-e1b2-dbcd-7e20-f18ea91f8463@cdnnow.ru> <151ef936-f5d2-b681-bb54-6476c654a3ae@cdnnow.ru> <00218a02-f3e7-d1fe-04ba-1e61049f17be@cdnnow.ru> <20220606134220.afco4q4u6ujj2s3h@Y9MQ9X2QVV> Message-ID: <1B25CB88-5640-45FD-A584-BF55F7C60445@nginx.com> > On 6 Jun 2022, at 17:42, Sergey Kandaurov wrote: > > On Sun, Apr 24, 2022 at 06:42:47PM +0300, Maxim Dounin wrote: >> Hello! >> >> On Sun, Apr 24, 2022 at 07:55:17AM +0300, Maxim Dounin wrote: >> >> [...] >> >>> As far as I can tell, proper behaviour, assuming we parse cache >>> control extensions independently of X-Accel-Expires, can be >>> implemented by using just one flag. >> >> No, that's wrong, as with just one flag it wouldn't be possible to >> correctly disable caching of responses with: >> >> Cache-Control: private >> Cache-Control: max-age=10 >> >> So it needs at least two flags. Updated patch below, review and >> testing appreciated. >> >> # HG changeset patch >> # User Maxim Dounin >> # Date 1650814681 -10800 >> # Sun Apr 24 18:38:01 2022 +0300 >> # Node ID 940ba4317a97c72d1ee6700cbf58a543fee04c7a >> # Parent a736a7a613ea6e182ff86fbadcb98bb0f8891c0b >> Upstream: fixed X-Accel-Expires/Cache-Control/Expires handling. >> >> Previously, if caching was disabled due to Expires in the past, nginx >> failed to cache the response even if it was cacheable as per subsequently >> parsed Cache-Control header (ticket #964). >> >> Similarly, if caching was disabled due to Expires in the past, >> "Cache-Control: no-cache" or "Cache-Control: max-age=0", caching was not >> used if it was cacheable as per subsequently parsed X-Accel-Expires header. >> >> Fix is to avoid disabling caching immediately after parsing Expires in >> the past or Cache-Control, but rather set flags which are later checked by >> ngx_http_upstream_process_headers() (and cleared by "Cache-Control: max-age" >> and X-Accel-Expires). >> >> Additionally, now X-Accel-Expires does not prevent parsing of cache control >> extensions, notably stale-while-revalidate and stale-if-error. This >> ensures that order of the X-Accel-Expires and Cache-Control headers is not >> important. >> >> Prodded by Vadim Fedorenko and Yugo Horie. >> >> diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c >> --- a/src/http/ngx_http_upstream.c >> +++ b/src/http/ngx_http_upstream.c >> @@ -2697,6 +2697,10 @@ ngx_http_upstream_intercept_errors(ngx_h >> >> if (r->cache) { >> >> + if (u->headers_in.no_cache || u->headers_in.expired) { >> + u->cacheable = 0; >> + } >> + >> if (u->cacheable) { >> time_t valid; >> >> @@ -2791,6 +2795,10 @@ ngx_http_upstream_process_headers(ngx_ht >> >> umcf = ngx_http_get_module_main_conf(r, ngx_http_upstream_module); >> >> + if (u->headers_in.no_cache || u->headers_in.expired) { >> + u->cacheable = 0; >> + } >> + >> if (u->headers_in.x_accel_redirect >> && !(u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_XA_REDIRECT)) >> { >> @@ -4735,18 +4743,18 @@ ngx_http_upstream_process_cache_control( >> return NGX_OK; >> } >> >> - if (r->cache->valid_sec != 0 && u->headers_in.x_accel_expires != NULL) { >> - return NGX_OK; >> - } >> - >> start = h->value.data; >> last = start + h->value.len; >> >> + if (r->cache->valid_sec != 0 && u->headers_in.x_accel_expires != NULL) { >> + goto extensions; >> + } >> + >> if (ngx_strlcasestrn(start, last, (u_char *) "no-cache", 8 - 1) != NULL >> || ngx_strlcasestrn(start, last, (u_char *) "no-store", 8 - 1) != NULL >> || ngx_strlcasestrn(start, last, (u_char *) "private", 7 - 1) != NULL) >> { >> - u->cacheable = 0; >> + u->headers_in.no_cache = 1; >> return NGX_OK; >> } >> >> @@ -4776,12 +4784,15 @@ ngx_http_upstream_process_cache_control( >> } >> >> if (n == 0) { >> - u->cacheable = 0; >> + u->headers_in.no_cache = 1; >> return NGX_OK; >> } >> >> r->cache->valid_sec = ngx_time() + n; >> - } >> + u->headers_in.expired = 0; >> + } >> + >> +extensions: >> >> p = ngx_strlcasestrn(start, last, (u_char *) "stale-while-revalidate=", >> 23 - 1); >> @@ -4863,7 +4874,7 @@ ngx_http_upstream_process_expires(ngx_ht >> expires = ngx_parse_http_time(h->value.data, h->value.len); >> >> if (expires == NGX_ERROR || expires < ngx_time()) { >> - u->cacheable = 0; >> + u->headers_in.expired = 1; >> return NGX_OK; >> } >> >> @@ -4914,6 +4925,8 @@ ngx_http_upstream_process_accel_expires( >> >> default: >> r->cache->valid_sec = ngx_time() + n; >> + u->headers_in.no_cache = 0; >> + u->headers_in.expired = 0; >> return NGX_OK; >> } >> } >> @@ -4925,6 +4938,8 @@ ngx_http_upstream_process_accel_expires( >> >> if (n != NGX_ERROR) { >> r->cache->valid_sec = n; >> + u->headers_in.no_cache = 0; >> + u->headers_in.expired = 0; > > Handling of "X-Accel-Redirect: @0" is still missed. Per the docs, > it should set caching time to Epoch resulting in the expired response. > Instead, such header is handled different, and it can be overridden > with Cache-Control/Expires headers. Probably, it should be handled > separately and disable caching, similar to "X-Accel-Redirect: 0". > Of course, this should be read as "X-Accel-Expires", bad copy'n'paste. > As the above note covers a corner case of X-Accel-Expires, > I believe it's fine to commit as is. > >> } >> } >> #endif >> diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h >> --- a/src/http/ngx_http_upstream.h >> +++ b/src/http/ngx_http_upstream.h >> @@ -297,6 +297,8 @@ typedef struct { >> >> unsigned connection_close:1; >> unsigned chunked:1; >> + unsigned no_cache:1; >> + unsigned expired:1; >> } ngx_http_upstream_headers_in_t; >> >> >> > _______________________________________________ > nginx-devel mailing list -- nginx-devel at nginx.org > To unsubscribe send an email to nginx-devel-leave at nginx.org -- Sergey Kandaurov From pluknet at nginx.com Mon Jun 6 15:42:53 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 6 Jun 2022 19:42:53 +0400 Subject: [PATCH] Upstream: handling of certificates specified as an empty string In-Reply-To: References: <3bb1adbb74dfcd372f73.1653422757@enoparse.local> Message-ID: <64FD3DCA-8448-4308-BD72-569B963B0D70@nginx.com> > On 4 Jun 2022, at 04:09, Maxim Dounin wrote: > > Hello! > > On Wed, May 25, 2022 at 12:05:57AM +0400, Sergey Kandaurov wrote: > >> # HG changeset patch >> # User Sergey Kandaurov >> # Date 1653422583 -14400 >> # Wed May 25 00:03:03 2022 +0400 >> # Node ID 3bb1adbb74dfcd372f7369530967cfb415900778 >> # Parent 8a54733c9d1290e6dc2f86af18e8a976a6352e4f >> Upstream: handling of certificates specified as an empty string. >> >> Now, if the directive is given an empty string, such configuration cancels >> loading of certificates should they be inherited from the previous level. >> This restores a previous behaviour, before variables support in certificates >> was introduced (3ab8e1e2f0f7). >> >> diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c >> --- a/src/http/modules/ngx_http_grpc_module.c >> +++ b/src/http/modules/ngx_http_grpc_module.c >> @@ -4921,7 +4921,7 @@ ngx_http_grpc_set_ssl(ngx_conf_t *cf, ng >> return NGX_ERROR; >> } >> >> - } else { >> + } else if (glcf->upstream.ssl_certificate->value.len) { >> if (ngx_ssl_certificate(cf, glcf->upstream.ssl, >> &glcf->upstream.ssl_certificate->value, >> &glcf->upstream.ssl_certificate_key->value, >> diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c >> --- a/src/http/modules/ngx_http_proxy_module.c >> +++ b/src/http/modules/ngx_http_proxy_module.c >> @@ -4970,7 +4970,7 @@ ngx_http_proxy_set_ssl(ngx_conf_t *cf, n >> return NGX_ERROR; >> } >> >> - } else { >> + } else if (plcf->upstream.ssl_certificate->value.len) { >> if (ngx_ssl_certificate(cf, plcf->upstream.ssl, >> &plcf->upstream.ssl_certificate->value, >> &plcf->upstream.ssl_certificate_key->value, >> diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c >> --- a/src/http/modules/ngx_http_uwsgi_module.c >> +++ b/src/http/modules/ngx_http_uwsgi_module.c >> @@ -2457,7 +2457,7 @@ ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, n >> return NGX_ERROR; >> } >> >> - } else { >> + } else if (uwcf->upstream.ssl_certificate->value.len) { >> if (ngx_ssl_certificate(cf, uwcf->upstream.ssl, >> &uwcf->upstream.ssl_certificate->value, >> &uwcf->upstream.ssl_certificate_key->value, > > It probably should in the outer if instead: > > diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c > --- a/src/http/modules/ngx_http_proxy_module.c > +++ b/src/http/modules/ngx_http_proxy_module.c > @@ -4955,8 +4955,9 @@ ngx_http_proxy_set_ssl(ngx_conf_t *cf, n > return NGX_ERROR; > } > > - if (plcf->upstream.ssl_certificate) { > - > + if (plcf->upstream.ssl_certificate > + && plcf->upstream.ssl_certificate->value.len) > + { > if (plcf->upstream.ssl_certificate_key == NULL) { > ngx_log_error(NGX_LOG_EMERG, cf->log, 0, > "no \"proxy_ssl_certificate_key\" is defined " > > With the corresponding change to the upstream module: > > diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c > --- a/src/http/ngx_http_upstream.c > +++ b/src/http/ngx_http_upstream.c > @@ -1690,8 +1690,10 @@ ngx_http_upstream_ssl_init_connection(ng > } > } > > - if (u->conf->ssl_certificate && (u->conf->ssl_certificate->lengths > - || u->conf->ssl_certificate_key->lengths)) > + if (u->conf->ssl_certificate > + && u->conf->ssl_certificate->value.len > + && (u->conf->ssl_certificate->lengths > + || u->conf->ssl_certificate_key->lengths)) > { > if (ngx_http_upstream_ssl_certificate(r, u, c) != NGX_OK) { > ngx_http_upstream_finalize_request(r, u, > > > This is more in line with the previous logic, and will avoid > errors in configurations like the following: > > proxy_ssl_certificate foo.crt; > > location / { > proxy_pass https://...; > proxy_ssl_certificate ""; > } > > location /foo { > proxy_pass https://...; > proxy_ssl_certificate_key foo.key; > } > > (Which is rather strange, but not very different from the idea of > clearing proxy_ssl_certificate.) > Agreed, thanks. Below is an updated patch with minor commit message adjustment: # HG changeset patch # User Sergey Kandaurov # Date 1654529998 -14400 # Mon Jun 06 19:39:58 2022 +0400 # Node ID 0e9638b52608a0e611dffaacad1fc8e54f3c156c # Parent e0cfab501dd11fdd23ad492419692269d3a01fc7 Upstream: handling of certificates specified as an empty string. Now, if the directive is given an empty string, such configuration cancels loading of certificates, in particular, if they would be otherwise inherited from the previous level. This restores previous behaviour, before variables support in certificates was introduced (3ab8e1e2f0f7). diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c --- a/src/http/modules/ngx_http_grpc_module.c +++ b/src/http/modules/ngx_http_grpc_module.c @@ -4906,8 +4906,9 @@ ngx_http_grpc_set_ssl(ngx_conf_t *cf, ng return NGX_ERROR; } - if (glcf->upstream.ssl_certificate) { - + if (glcf->upstream.ssl_certificate + && glcf->upstream.ssl_certificate->value.len) + { if (glcf->upstream.ssl_certificate_key == NULL) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "no \"grpc_ssl_certificate_key\" is defined " diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c +++ b/src/http/modules/ngx_http_proxy_module.c @@ -4955,8 +4955,9 @@ ngx_http_proxy_set_ssl(ngx_conf_t *cf, n return NGX_ERROR; } - if (plcf->upstream.ssl_certificate) { - + if (plcf->upstream.ssl_certificate + && plcf->upstream.ssl_certificate->value.len) + { if (plcf->upstream.ssl_certificate_key == NULL) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "no \"proxy_ssl_certificate_key\" is defined " diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c +++ b/src/http/modules/ngx_http_uwsgi_module.c @@ -2487,8 +2487,9 @@ ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, n return NGX_ERROR; } - if (uwcf->upstream.ssl_certificate) { - + if (uwcf->upstream.ssl_certificate + && uwcf->upstream.ssl_certificate->value.len) + { if (uwcf->upstream.ssl_certificate_key == NULL) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "no \"uwsgi_ssl_certificate_key\" is defined " diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -1690,8 +1690,10 @@ ngx_http_upstream_ssl_init_connection(ng } } - if (u->conf->ssl_certificate && (u->conf->ssl_certificate->lengths - || u->conf->ssl_certificate_key->lengths)) + if (u->conf->ssl_certificate + && u->conf->ssl_certificate->value.len + && (u->conf->ssl_certificate->lengths + || u->conf->ssl_certificate_key->lengths)) { if (ngx_http_upstream_ssl_certificate(r, u, c) != NGX_OK) { ngx_http_upstream_finalize_request(r, u, diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c --- a/src/stream/ngx_stream_proxy_module.c +++ b/src/stream/ngx_stream_proxy_module.c @@ -1069,8 +1069,10 @@ ngx_stream_proxy_ssl_init_connection(ngx } } - if (pscf->ssl_certificate && (pscf->ssl_certificate->lengths - || pscf->ssl_certificate_key->lengths)) + if (pscf->ssl_certificate + && pscf->ssl_certificate->value.len + && (pscf->ssl_certificate->lengths + || pscf->ssl_certificate_key->lengths)) { if (ngx_stream_proxy_ssl_certificate(s) != NGX_OK) { ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR); @@ -2225,8 +2227,9 @@ ngx_stream_proxy_set_ssl(ngx_conf_t *cf, return NGX_ERROR; } - if (pscf->ssl_certificate) { - + if (pscf->ssl_certificate + && pscf->ssl_certificate->value.len) + { if (pscf->ssl_certificate_key == NULL) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "no \"proxy_ssl_certificate_key\" is defined " -- Sergey Kandaurov From mdounin at mdounin.ru Mon Jun 6 17:38:33 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 6 Jun 2022 20:38:33 +0300 Subject: Changeset: Dynamic Virtual Server Selection Based on Host's Cnamed domain In-Reply-To: <18137cb3e20.db44da5a19378.6352067189672045247@zohocorp.com> References: <18137cb3e20.db44da5a19378.6352067189672045247@zohocorp.com> Message-ID: Hello! On Mon, Jun 06, 2022 at 12:24:35PM +0530, Devarajan D via nginx-devel wrote: > Dear Members, > > # HG changeset patch > # User devarajan.dk > # Date 1654428592 -19800 > #      Sun Jun 05 16:59:52 2022 +0530 > # Node ID 8d73e94b379091440ad8c836e2a052fb5edda160 > # Parent  e0cfab501dd11fdd23ad492419692269d3a01fc7 > Dynamic Virtual Server Selection Based on Host's Cnamed domain > > Use Case: > Preventing nginx.conf reload on new customer domain addition. > > Many B2B Cloud Services provide option for rebranding customer domains through CNAME. > eg. Customers - mail.schroeppel.me, mail.vey.de cnamed to ghs.googlehosted.com > The hosting Service can be having multiple such services reverse proxied through > same nginx instance. (eg. Virtual Servers - mail.ghost.com, docs.ghost.com, drive.ghost.com, etc. > all virtual services listening on the same IP and port; consider 100s of such virtual services) > > While deployment, the cloud service has to add the customer domain name in 'server_name' directive > of corresponding virtual server for the request to reach the right virtual server block. > Also, as the customer has given control to the hosting cloud service, the service will also > purchase certificate for the customer domain and offload through nginx. > > As and when the new rebranding customers starts to subscribe for the cloud service, the cloud service > has to add the certificate name & key in ssl_certificate & ssl_certificate_key directives respectively > and customer's domain name in server_name directive of corresponding virtual server. > So the nginx.conf has to be reloaded every time a new customer starts using the service. > > As the frequency of new customers joining the service increases, the hosting service > may suffer from frequent nginx reloads and may not scale well. > (consider 100s of customers joining the service every hour) > Thanks to the Dynamic Certificate Loading feature in nginx that uses $ssl_server_name variable > to dynamically select the certificate on runtime and no need to reload conf after deploying certificate. > But still there is a need to reload the conf due to the addition of customer's domain name in server_name directive. > > To prevent this reload, we can dynamically choose Virtual server based on the cname of the customer domains. > Only the cnamed domain ie. mail.ghost.com will be written in the server_name directive of the virtual server. > All customer domains cnamed to mail.ghost.com will be routed to right virtual server based on the cname. > This cname check can be turned on/off by a http level directive - 'resolve_cname' (off by default) > > Note: > 1. Similar to Dynamic Certificate Loading, the pros of dynamic Virtual Server selection come >    at the cost of increase in latency during request processing time due to DNS lookup. > 2. The cloud service can choose to deploy their customer domains in hybrid mode too. >    (ie. old customers domains can be explicitly configured in server_name and >    customer domains who newly subscribed for the cloud service in between two nginx.conf reloads >    can be deployed in nginx dynamically) [...] Thank you for the patch. It doesn't look like a good idea to me though, for multiple reasons. In no particular order: - It is generally a bad idea to do DNS lookups on per-request basis. Further, even if you do DNS lookups for some reason, it is generally a bad idea to do user-controlled DNS lookups. It can be easily used as an attack vector. - You should never ever use gethostbyname() at runtime in nginx (or any other event-based servers): it is blocking and a long-running DNS request will stop all the request in the worker processes. Combined with user-controlled name, this is an immediate DoS on the server. To make it possible to do DNS lookups when needed, nginx provides the resolver infrastructure (http://nginx.org/r/resolver). Further, it looks like what you are trying to do can be easily achieved with at least the following options: - The most simple solution to all issues involving configuration reloads is to do reloads periodically, applying all the changes accumulated at once. For example, reloading configurations once in several minutes might be an option - especially considering that typical DNS update time is more than 1 hour. - In this particular case, it is trivial to configure nginx to process all the names by using distinct IP-based virtual servers (if needed, with further additional restrictions applied via nginx configuration, such as checking if appropriate files exists, and/or at the backend level). If using just one IP address is absolutely a must, further distinction between services can be implemented within a single server{} block. Hope this helps. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Mon Jun 6 21:09:08 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 7 Jun 2022 00:09:08 +0300 Subject: [PATCH] Upstream: prioritise Cache-Control over Expires In-Reply-To: <20220606134220.afco4q4u6ujj2s3h@Y9MQ9X2QVV> References: <4de897b7-e1b2-dbcd-7e20-f18ea91f8463@cdnnow.ru> <151ef936-f5d2-b681-bb54-6476c654a3ae@cdnnow.ru> <00218a02-f3e7-d1fe-04ba-1e61049f17be@cdnnow.ru> <20220606134220.afco4q4u6ujj2s3h@Y9MQ9X2QVV> Message-ID: Hello! On Mon, Jun 06, 2022 at 05:42:20PM +0400, Sergey Kandaurov wrote: > On Sun, Apr 24, 2022 at 06:42:47PM +0300, Maxim Dounin wrote: > > Hello! > > > > On Sun, Apr 24, 2022 at 07:55:17AM +0300, Maxim Dounin wrote: > > > > [...] > > > > > As far as I can tell, proper behaviour, assuming we parse cache > > > control extensions independently of X-Accel-Expires, can be > > > implemented by using just one flag. > > > > No, that's wrong, as with just one flag it wouldn't be possible to > > correctly disable caching of responses with: > > > > Cache-Control: private > > Cache-Control: max-age=10 > > > > So it needs at least two flags. Updated patch below, review and > > testing appreciated. > > > > # HG changeset patch > > # User Maxim Dounin > > # Date 1650814681 -10800 > > # Sun Apr 24 18:38:01 2022 +0300 > > # Node ID 940ba4317a97c72d1ee6700cbf58a543fee04c7a > > # Parent a736a7a613ea6e182ff86fbadcb98bb0f8891c0b > > Upstream: fixed X-Accel-Expires/Cache-Control/Expires handling. > > > > Previously, if caching was disabled due to Expires in the past, nginx > > failed to cache the response even if it was cacheable as per subsequently > > parsed Cache-Control header (ticket #964). > > > > Similarly, if caching was disabled due to Expires in the past, > > "Cache-Control: no-cache" or "Cache-Control: max-age=0", caching was not > > used if it was cacheable as per subsequently parsed X-Accel-Expires header. > > > > Fix is to avoid disabling caching immediately after parsing Expires in > > the past or Cache-Control, but rather set flags which are later checked by > > ngx_http_upstream_process_headers() (and cleared by "Cache-Control: max-age" > > and X-Accel-Expires). > > > > Additionally, now X-Accel-Expires does not prevent parsing of cache control > > extensions, notably stale-while-revalidate and stale-if-error. This > > ensures that order of the X-Accel-Expires and Cache-Control headers is not > > important. > > > > Prodded by Vadim Fedorenko and Yugo Horie. > > > > diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c > > --- a/src/http/ngx_http_upstream.c > > +++ b/src/http/ngx_http_upstream.c > > @@ -2697,6 +2697,10 @@ ngx_http_upstream_intercept_errors(ngx_h > > > > if (r->cache) { > > > > + if (u->headers_in.no_cache || u->headers_in.expired) { > > + u->cacheable = 0; > > + } > > + > > if (u->cacheable) { > > time_t valid; > > > > @@ -2791,6 +2795,10 @@ ngx_http_upstream_process_headers(ngx_ht > > > > umcf = ngx_http_get_module_main_conf(r, ngx_http_upstream_module); > > > > + if (u->headers_in.no_cache || u->headers_in.expired) { > > + u->cacheable = 0; > > + } > > + > > if (u->headers_in.x_accel_redirect > > && !(u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_XA_REDIRECT)) > > { > > @@ -4735,18 +4743,18 @@ ngx_http_upstream_process_cache_control( > > return NGX_OK; > > } > > > > - if (r->cache->valid_sec != 0 && u->headers_in.x_accel_expires != NULL) { > > - return NGX_OK; > > - } > > - > > start = h->value.data; > > last = start + h->value.len; > > > > + if (r->cache->valid_sec != 0 && u->headers_in.x_accel_expires != NULL) { > > + goto extensions; > > + } > > + > > if (ngx_strlcasestrn(start, last, (u_char *) "no-cache", 8 - 1) != NULL > > || ngx_strlcasestrn(start, last, (u_char *) "no-store", 8 - 1) != NULL > > || ngx_strlcasestrn(start, last, (u_char *) "private", 7 - 1) != NULL) > > { > > - u->cacheable = 0; > > + u->headers_in.no_cache = 1; > > return NGX_OK; > > } > > > > @@ -4776,12 +4784,15 @@ ngx_http_upstream_process_cache_control( > > } > > > > if (n == 0) { > > - u->cacheable = 0; > > + u->headers_in.no_cache = 1; > > return NGX_OK; > > } > > > > r->cache->valid_sec = ngx_time() + n; > > - } > > + u->headers_in.expired = 0; > > + } > > + > > +extensions: > > > > p = ngx_strlcasestrn(start, last, (u_char *) "stale-while-revalidate=", > > 23 - 1); > > @@ -4863,7 +4874,7 @@ ngx_http_upstream_process_expires(ngx_ht > > expires = ngx_parse_http_time(h->value.data, h->value.len); > > > > if (expires == NGX_ERROR || expires < ngx_time()) { > > - u->cacheable = 0; > > + u->headers_in.expired = 1; > > return NGX_OK; > > } > > > > @@ -4914,6 +4925,8 @@ ngx_http_upstream_process_accel_expires( > > > > default: > > r->cache->valid_sec = ngx_time() + n; > > + u->headers_in.no_cache = 0; > > + u->headers_in.expired = 0; > > return NGX_OK; > > } > > } > > @@ -4925,6 +4938,8 @@ ngx_http_upstream_process_accel_expires( > > > > if (n != NGX_ERROR) { > > r->cache->valid_sec = n; > > + u->headers_in.no_cache = 0; > > + u->headers_in.expired = 0; > > Handling of "X-Accel-Redirect: @0" is still missed. Per the docs, > it should set caching time to Epoch resulting in the expired response. > Instead, such header is handled different, and it can be overridden > with Cache-Control/Expires headers. Probably, it should be handled > separately and disable caching, similar to "X-Accel-Redirect: 0". > > As the above note covers a corner case of X-Accel-Expires, > I believe it's fine to commit as is. This patch doesn't try to change handling of X-Accel-Expires in the past. And, honestly, the only documented way to _disable_ caching is to use 'X-Accel-Expires: 0'. The 'X-Accel-Expires: @0' only documented to set the time up to which the response may be cached to 0 seconds the Epoch. Expires and Cache-Control handling used to disable caching in similar situations, yet this behaviour was questioned more than once. X-Accel-Expires never disabled caching for absolute times in the past, and this is known to be used as a workaround to cache a pre-expires response[1][2]. Pushed to http://mdounin.ru/hg/nginx. [1] https://trac.nginx.org/nginx/ticket/1182 [2] https://mailman.nginx.org/pipermail/nginx-ru/2013-November/052614.html -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Mon Jun 6 22:03:14 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 7 Jun 2022 01:03:14 +0300 Subject: [PATCH] Upstream: handling of certificates specified as an empty string In-Reply-To: <64FD3DCA-8448-4308-BD72-569B963B0D70@nginx.com> References: <3bb1adbb74dfcd372f73.1653422757@enoparse.local> <64FD3DCA-8448-4308-BD72-569B963B0D70@nginx.com> Message-ID: Hello! On Mon, Jun 06, 2022 at 07:42:53PM +0400, Sergey Kandaurov wrote: > > On 4 Jun 2022, at 04:09, Maxim Dounin wrote: > > > > On Wed, May 25, 2022 at 12:05:57AM +0400, Sergey Kandaurov wrote: > > > >> # HG changeset patch > >> # User Sergey Kandaurov > >> # Date 1653422583 -14400 > >> # Wed May 25 00:03:03 2022 +0400 > >> # Node ID 3bb1adbb74dfcd372f7369530967cfb415900778 > >> # Parent 8a54733c9d1290e6dc2f86af18e8a976a6352e4f > >> Upstream: handling of certificates specified as an empty string. > >> > >> Now, if the directive is given an empty string, such configuration cancels > >> loading of certificates should they be inherited from the previous level. > >> This restores a previous behaviour, before variables support in certificates > >> was introduced (3ab8e1e2f0f7). > >> > >> diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c > >> --- a/src/http/modules/ngx_http_grpc_module.c > >> +++ b/src/http/modules/ngx_http_grpc_module.c > >> @@ -4921,7 +4921,7 @@ ngx_http_grpc_set_ssl(ngx_conf_t *cf, ng > >> return NGX_ERROR; > >> } > >> > >> - } else { > >> + } else if (glcf->upstream.ssl_certificate->value.len) { > >> if (ngx_ssl_certificate(cf, glcf->upstream.ssl, > >> &glcf->upstream.ssl_certificate->value, > >> &glcf->upstream.ssl_certificate_key->value, > >> diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c > >> --- a/src/http/modules/ngx_http_proxy_module.c > >> +++ b/src/http/modules/ngx_http_proxy_module.c > >> @@ -4970,7 +4970,7 @@ ngx_http_proxy_set_ssl(ngx_conf_t *cf, n > >> return NGX_ERROR; > >> } > >> > >> - } else { > >> + } else if (plcf->upstream.ssl_certificate->value.len) { > >> if (ngx_ssl_certificate(cf, plcf->upstream.ssl, > >> &plcf->upstream.ssl_certificate->value, > >> &plcf->upstream.ssl_certificate_key->value, > >> diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c > >> --- a/src/http/modules/ngx_http_uwsgi_module.c > >> +++ b/src/http/modules/ngx_http_uwsgi_module.c > >> @@ -2457,7 +2457,7 @@ ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, n > >> return NGX_ERROR; > >> } > >> > >> - } else { > >> + } else if (uwcf->upstream.ssl_certificate->value.len) { > >> if (ngx_ssl_certificate(cf, uwcf->upstream.ssl, > >> &uwcf->upstream.ssl_certificate->value, > >> &uwcf->upstream.ssl_certificate_key->value, > > > > It probably should in the outer if instead: > > > > diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c > > --- a/src/http/modules/ngx_http_proxy_module.c > > +++ b/src/http/modules/ngx_http_proxy_module.c > > @@ -4955,8 +4955,9 @@ ngx_http_proxy_set_ssl(ngx_conf_t *cf, n > > return NGX_ERROR; > > } > > > > - if (plcf->upstream.ssl_certificate) { > > - > > + if (plcf->upstream.ssl_certificate > > + && plcf->upstream.ssl_certificate->value.len) > > + { > > if (plcf->upstream.ssl_certificate_key == NULL) { > > ngx_log_error(NGX_LOG_EMERG, cf->log, 0, > > "no \"proxy_ssl_certificate_key\" is defined " > > > > With the corresponding change to the upstream module: > > > > diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c > > --- a/src/http/ngx_http_upstream.c > > +++ b/src/http/ngx_http_upstream.c > > @@ -1690,8 +1690,10 @@ ngx_http_upstream_ssl_init_connection(ng > > } > > } > > > > - if (u->conf->ssl_certificate && (u->conf->ssl_certificate->lengths > > - || u->conf->ssl_certificate_key->lengths)) > > + if (u->conf->ssl_certificate > > + && u->conf->ssl_certificate->value.len > > + && (u->conf->ssl_certificate->lengths > > + || u->conf->ssl_certificate_key->lengths)) > > { > > if (ngx_http_upstream_ssl_certificate(r, u, c) != NGX_OK) { > > ngx_http_upstream_finalize_request(r, u, > > > > > > This is more in line with the previous logic, and will avoid > > errors in configurations like the following: > > > > proxy_ssl_certificate foo.crt; > > > > location / { > > proxy_pass https://...; > > proxy_ssl_certificate ""; > > } > > > > location /foo { > > proxy_pass https://...; > > proxy_ssl_certificate_key foo.key; > > } > > > > (Which is rather strange, but not very different from the idea of > > clearing proxy_ssl_certificate.) > > > > Agreed, thanks. Below is an updated patch > with minor commit message adjustment: > > # HG changeset patch > # User Sergey Kandaurov > # Date 1654529998 -14400 > # Mon Jun 06 19:39:58 2022 +0400 > # Node ID 0e9638b52608a0e611dffaacad1fc8e54f3c156c > # Parent e0cfab501dd11fdd23ad492419692269d3a01fc7 > Upstream: handling of certificates specified as an empty string. > > Now, if the directive is given an empty string, such configuration cancels > loading of certificates, in particular, if they would be otherwise inherited > from the previous level. This restores previous behaviour, before variables > support in certificates was introduced (3ab8e1e2f0f7). > > diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c > --- a/src/http/modules/ngx_http_grpc_module.c > +++ b/src/http/modules/ngx_http_grpc_module.c > @@ -4906,8 +4906,9 @@ ngx_http_grpc_set_ssl(ngx_conf_t *cf, ng > return NGX_ERROR; > } > > - if (glcf->upstream.ssl_certificate) { > - > + if (glcf->upstream.ssl_certificate > + && glcf->upstream.ssl_certificate->value.len) > + { > if (glcf->upstream.ssl_certificate_key == NULL) { > ngx_log_error(NGX_LOG_EMERG, cf->log, 0, > "no \"grpc_ssl_certificate_key\" is defined " > diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c > --- a/src/http/modules/ngx_http_proxy_module.c > +++ b/src/http/modules/ngx_http_proxy_module.c > @@ -4955,8 +4955,9 @@ ngx_http_proxy_set_ssl(ngx_conf_t *cf, n > return NGX_ERROR; > } > > - if (plcf->upstream.ssl_certificate) { > - > + if (plcf->upstream.ssl_certificate > + && plcf->upstream.ssl_certificate->value.len) > + { > if (plcf->upstream.ssl_certificate_key == NULL) { > ngx_log_error(NGX_LOG_EMERG, cf->log, 0, > "no \"proxy_ssl_certificate_key\" is defined " > diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c > --- a/src/http/modules/ngx_http_uwsgi_module.c > +++ b/src/http/modules/ngx_http_uwsgi_module.c > @@ -2487,8 +2487,9 @@ ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, n > return NGX_ERROR; > } > > - if (uwcf->upstream.ssl_certificate) { > - > + if (uwcf->upstream.ssl_certificate > + && uwcf->upstream.ssl_certificate->value.len) > + { > if (uwcf->upstream.ssl_certificate_key == NULL) { > ngx_log_error(NGX_LOG_EMERG, cf->log, 0, > "no \"uwsgi_ssl_certificate_key\" is defined " > diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c > --- a/src/http/ngx_http_upstream.c > +++ b/src/http/ngx_http_upstream.c > @@ -1690,8 +1690,10 @@ ngx_http_upstream_ssl_init_connection(ng > } > } > > - if (u->conf->ssl_certificate && (u->conf->ssl_certificate->lengths > - || u->conf->ssl_certificate_key->lengths)) > + if (u->conf->ssl_certificate > + && u->conf->ssl_certificate->value.len > + && (u->conf->ssl_certificate->lengths > + || u->conf->ssl_certificate_key->lengths)) > { > if (ngx_http_upstream_ssl_certificate(r, u, c) != NGX_OK) { > ngx_http_upstream_finalize_request(r, u, > diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c > --- a/src/stream/ngx_stream_proxy_module.c > +++ b/src/stream/ngx_stream_proxy_module.c > @@ -1069,8 +1069,10 @@ ngx_stream_proxy_ssl_init_connection(ngx > } > } > > - if (pscf->ssl_certificate && (pscf->ssl_certificate->lengths > - || pscf->ssl_certificate_key->lengths)) > + if (pscf->ssl_certificate > + && pscf->ssl_certificate->value.len > + && (pscf->ssl_certificate->lengths > + || pscf->ssl_certificate_key->lengths)) > { > if (ngx_stream_proxy_ssl_certificate(s) != NGX_OK) { > ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR); > @@ -2225,8 +2227,9 @@ ngx_stream_proxy_set_ssl(ngx_conf_t *cf, > return NGX_ERROR; > } > > - if (pscf->ssl_certificate) { > - > + if (pscf->ssl_certificate > + && pscf->ssl_certificate->value.len) > + { > if (pscf->ssl_certificate_key == NULL) { > ngx_log_error(NGX_LOG_EMERG, cf->log, 0, > "no \"proxy_ssl_certificate_key\" is defined " Looks good. -- Maxim Dounin http://mdounin.ru/ From arut at nginx.com Tue Jun 7 07:01:52 2022 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 7 Jun 2022 11:01:52 +0400 Subject: [PATCH] Stream: don't flush empty buffers created for read errors. In-Reply-To: References: Message-ID: <20220607070152.vmx4rnnpmob6ujpb@N00W24XTQX> On Sat, Jun 04, 2022 at 12:13:05AM +0300, Maxim Dounin wrote: > Hello! > > On Thu, Jun 02, 2022 at 03:20:54AM +0000, Aleksei Bavshin via nginx-devel wrote: > > > > I don't think it's the way to go. Semantically, flush and > > > last_buf are mutually exclusive: last_buf implies flushing of all > > > remaining data, and there should be no flush set if last_buf is > > > set. > > > > > > The "cl->buf->flush = !src->read->eof" approach might be actually > > > better and more obvious: it simply ensures that last_buf and flush > > > are not set at the same time. I don't immediately see any cases > > > where it can break things, and this should fix the original > > > problem with empty UDP packets being sent. > > > > Thanks for confirming that, I wasn't sure it is the right approach exactly because I had doubts regarding the mutual exclusivity of the flags. > > Updated patch below; tests are passing on both branches and all systems we have in CI. > > > > # HG changeset patch > > # User Aleksei Bavshin > > # Date 1653330584 25200 > > # Mon May 23 11:29:44 2022 -0700 > > # Node ID 40926829be83986a3a5a5f941d2014000a0acd2e > > # Parent e0cfab501dd11fdd23ad492419692269d3a01fc7 > > Stream: don't flush empty buffers created for read errors. > > > > When we generate the last_buf buffer for an UDP upstream recv error, it does > > not contain any data from the wire. ngx_stream_write_filter attempts to forward > > it anyways, which is incorrect (e.g., UDP upstream ECONNREFUSED will be > > translated to an empty packet). > > > > This happens because we mark the buffer as both 'flush' and 'last_buf', and > > ngx_stream_write_filter has special handling for flush with certain types of > > connections (see d127837c714f, 32b0ba4855a6). The flags are meant to be > > mutually exclusive, so the fix is to ensure that flush and last_buf are not set > > at the same time. > > > > Reproduction: > > > > stream { > > upstream unreachable { > > server 127.0.0.1:8880; > > } > > server { > > listen 127.0.0.1:8998 udp; > > proxy_pass unreachable; > > } > > } > > > > 1 0.000000000 127.0.0.1 → 127.0.0.1 UDP 47 45588 → 8998 Len=5 > > 2 0.000166300 127.0.0.1 → 127.0.0.1 UDP 47 51149 → 8880 Len=5 > > 3 0.000172600 127.0.0.1 → 127.0.0.1 ICMP 75 Destination unreachable (Port > > unreachable) > > 4 0.000202400 127.0.0.1 → 127.0.0.1 UDP 42 8998 → 45588 Len=0 > > > > Fixes d127837c714f. > > > > diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c > > --- a/src/stream/ngx_stream_proxy_module.c > > +++ b/src/stream/ngx_stream_proxy_module.c > > @@ -1735,7 +1735,7 @@ ngx_stream_proxy_process(ngx_stream_sess > > > > cl->buf->temporary = (n ? 1 : 0); > > cl->buf->last_buf = src->read->eof; > > - cl->buf->flush = 1; > > + cl->buf->flush = !src->read->eof; > > > > (*packets)++; > > *received += n; > > Looks good to me. > > Roman, please take another look. Good to me too. This version is equivalent to the original patch for UDP. The only difference is a side-effect for TCP. Now flush is not set in the last buffer. > > -- > Maxim Dounin > http://mdounin.ru/ > _______________________________________________ > nginx-devel mailing list -- nginx-devel at nginx.org > To unsubscribe send an email to nginx-devel-leave at nginx.org From pluknet at nginx.com Tue Jun 7 09:05:43 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 07 Jun 2022 09:05:43 +0000 Subject: [nginx] Upstream: fixed X-Accel-Expires/Cache-Control/Expires handling. Message-ID: details: https://hg.nginx.org/nginx/rev/0784ab86ad08 branches: changeset: 8041:0784ab86ad08 user: Maxim Dounin date: Tue Jun 07 00:07:12 2022 +0300 description: Upstream: fixed X-Accel-Expires/Cache-Control/Expires handling. Previously, if caching was disabled due to Expires in the past, nginx failed to cache the response even if it was cacheable as per subsequently parsed Cache-Control header (ticket #964). Similarly, if caching was disabled due to Expires in the past, "Cache-Control: no-cache" or "Cache-Control: max-age=0", caching was not used if it was cacheable as per subsequently parsed X-Accel-Expires header. Fix is to avoid disabling caching immediately after parsing Expires in the past or Cache-Control, but rather set flags which are later checked by ngx_http_upstream_process_headers() (and cleared by "Cache-Control: max-age" and X-Accel-Expires). Additionally, now X-Accel-Expires does not prevent parsing of cache control extensions, notably stale-while-revalidate and stale-if-error. This ensures that order of the X-Accel-Expires and Cache-Control headers is not important. Prodded by Vadim Fedorenko and Yugo Horie. diffstat: src/http/ngx_http_upstream.c | 31 +++++++++++++++++++++++-------- src/http/ngx_http_upstream.h | 2 ++ 2 files changed, 25 insertions(+), 8 deletions(-) diffs (106 lines): diff -r e0cfab501dd1 -r 0784ab86ad08 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Tue May 31 00:14:11 2022 +0300 +++ b/src/http/ngx_http_upstream.c Tue Jun 07 00:07:12 2022 +0300 @@ -2702,6 +2702,10 @@ ngx_http_upstream_intercept_errors(ngx_h if (r->cache) { + if (u->headers_in.no_cache || u->headers_in.expired) { + u->cacheable = 0; + } + if (u->cacheable) { time_t valid; @@ -2796,6 +2800,10 @@ ngx_http_upstream_process_headers(ngx_ht umcf = ngx_http_get_module_main_conf(r, ngx_http_upstream_module); + if (u->headers_in.no_cache || u->headers_in.expired) { + u->cacheable = 0; + } + if (u->headers_in.x_accel_redirect && !(u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_XA_REDIRECT)) { @@ -4791,18 +4799,18 @@ ngx_http_upstream_process_cache_control( return NGX_OK; } - if (r->cache->valid_sec != 0 && u->headers_in.x_accel_expires != NULL) { - return NGX_OK; - } - start = h->value.data; last = start + h->value.len; + if (r->cache->valid_sec != 0 && u->headers_in.x_accel_expires != NULL) { + goto extensions; + } + if (ngx_strlcasestrn(start, last, (u_char *) "no-cache", 8 - 1) != NULL || ngx_strlcasestrn(start, last, (u_char *) "no-store", 8 - 1) != NULL || ngx_strlcasestrn(start, last, (u_char *) "private", 7 - 1) != NULL) { - u->cacheable = 0; + u->headers_in.no_cache = 1; return NGX_OK; } @@ -4832,12 +4840,15 @@ ngx_http_upstream_process_cache_control( } if (n == 0) { - u->cacheable = 0; + u->headers_in.no_cache = 1; return NGX_OK; } r->cache->valid_sec = ngx_time() + n; - } + u->headers_in.expired = 0; + } + +extensions: p = ngx_strlcasestrn(start, last, (u_char *) "stale-while-revalidate=", 23 - 1); @@ -4932,7 +4943,7 @@ ngx_http_upstream_process_expires(ngx_ht expires = ngx_parse_http_time(h->value.data, h->value.len); if (expires == NGX_ERROR || expires < ngx_time()) { - u->cacheable = 0; + u->headers_in.expired = 1; return NGX_OK; } @@ -4996,6 +5007,8 @@ ngx_http_upstream_process_accel_expires( default: r->cache->valid_sec = ngx_time() + n; + u->headers_in.no_cache = 0; + u->headers_in.expired = 0; return NGX_OK; } } @@ -5007,6 +5020,8 @@ ngx_http_upstream_process_accel_expires( if (n != NGX_ERROR) { r->cache->valid_sec = n; + u->headers_in.no_cache = 0; + u->headers_in.expired = 0; } } #endif diff -r e0cfab501dd1 -r 0784ab86ad08 src/http/ngx_http_upstream.h --- a/src/http/ngx_http_upstream.h Tue May 31 00:14:11 2022 +0300 +++ b/src/http/ngx_http_upstream.h Tue Jun 07 00:07:12 2022 +0300 @@ -293,6 +293,8 @@ typedef struct { unsigned connection_close:1; unsigned chunked:1; + unsigned no_cache:1; + unsigned expired:1; } ngx_http_upstream_headers_in_t; From arut at nginx.com Tue Jun 7 13:16:45 2022 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 7 Jun 2022 17:16:45 +0400 Subject: [PATCH] Mp4: fixed potential overflow in ngx_http_mp4_crop_stts_data() In-Reply-To: References: Message-ID: <20220607131645.lp3rd7bzprwfds4z@N00W24XTQX> On Tue, May 31, 2022 at 12:05:13AM +0300, Maxim Dounin wrote: > # HG changeset patch > # User Maxim Dounin > # Date 1653942033 -10800 > # Mon May 30 23:20:33 2022 +0300 > # Node ID d5363be9fa61e0327574b8aa1342c874efd027b0 > # Parent cd40709c91e245036e7f71a6c3e5190d9533be75 > Mp4: fixed potential overflow in ngx_http_mp4_crop_stts_data(). > > Both "count" and "duration" variables are 32-bit, so their product might > potentially overflow. It is used to reduce 64-bit start_time variable, > and with very large start_time this can result in incorrect seeking. > > Found by Coverity (CID 1499904). > > diff --git a/src/http/modules/ngx_http_mp4_module.c b/src/http/modules/ngx_http_mp4_module.c > --- a/src/http/modules/ngx_http_mp4_module.c > +++ b/src/http/modules/ngx_http_mp4_module.c > @@ -2331,7 +2331,7 @@ ngx_http_mp4_crop_stts_data(ngx_http_mp4 > } > > start_sample += count; > - start_time -= count * duration; > + start_time -= (uint64_t) count * duration; > entries--; > entry++; > } Looks fine From pluknet at nginx.com Tue Jun 7 15:58:36 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 7 Jun 2022 19:58:36 +0400 Subject: [PATCH] Upstream: prioritise Cache-Control over Expires In-Reply-To: References: <4de897b7-e1b2-dbcd-7e20-f18ea91f8463@cdnnow.ru> <151ef936-f5d2-b681-bb54-6476c654a3ae@cdnnow.ru> <00218a02-f3e7-d1fe-04ba-1e61049f17be@cdnnow.ru> <20220606134220.afco4q4u6ujj2s3h@Y9MQ9X2QVV> Message-ID: <6DC932A4-16AA-42D5-9D06-A3F599439908@nginx.com> > On 7 Jun 2022, at 01:09, Maxim Dounin wrote: > > Hello! > > On Mon, Jun 06, 2022 at 05:42:20PM +0400, Sergey Kandaurov wrote: > >> On Sun, Apr 24, 2022 at 06:42:47PM +0300, Maxim Dounin wrote: >>> Hello! >>> >>> On Sun, Apr 24, 2022 at 07:55:17AM +0300, Maxim Dounin wrote: >>> >>> [...] >>> >>>> As far as I can tell, proper behaviour, assuming we parse cache >>>> control extensions independently of X-Accel-Expires, can be >>>> implemented by using just one flag. >>> >>> No, that's wrong, as with just one flag it wouldn't be possible to >>> correctly disable caching of responses with: >>> >>> Cache-Control: private >>> Cache-Control: max-age=10 >>> >>> So it needs at least two flags. Updated patch below, review and >>> testing appreciated. >>> >>> # HG changeset patch >>> # User Maxim Dounin >>> # Date 1650814681 -10800 >>> # Sun Apr 24 18:38:01 2022 +0300 >>> # Node ID 940ba4317a97c72d1ee6700cbf58a543fee04c7a >>> # Parent a736a7a613ea6e182ff86fbadcb98bb0f8891c0b >>> Upstream: fixed X-Accel-Expires/Cache-Control/Expires handling. >>> [..] >> [..] >> As the above note covers a corner case of X-Accel-Expires, >> I believe it's fine to commit as is. > > This patch doesn't try to change handling of X-Accel-Expires in > the past. > > And, honestly, the only documented way to _disable_ caching is to > use 'X-Accel-Expires: 0'. The 'X-Accel-Expires: @0' only > documented to set the time up to which the response may be cached > to 0 seconds the Epoch. Expires and Cache-Control handling used > to disable caching in similar situations, yet this behaviour was > questioned more than once. X-Accel-Expires never disabled caching > for absolute times in the past, and this is known to be used as a > workaround to cache a pre-expires response[1][2]. > > Pushed to http://mdounin.ru/hg/nginx. > > [1] https://trac.nginx.org/nginx/ticket/1182 > [2] https://mailman.nginx.org/pipermail/nginx-ru/2013-November/052614.html > Tests: # HG changeset patch # User Sergey Kandaurov # Date 1654617432 -14400 # Tue Jun 07 19:57:12 2022 +0400 # Node ID ed5fe35807f6c4853be1eb8ac429524e04ac5d67 # Parent 4f238efded81e00c635c146df6ecb585fa0907ec Tests: added X-Accel-Expires tests. diff --git a/proxy_xae.t b/proxy_xae.t new file mode 100644 --- /dev/null +++ b/proxy_xae.t @@ -0,0 +1,160 @@ +#!/usr/bin/perl + +# (C) Sergey Kandaurov +# (C) Nginx, Inc. + +# Tests for proxy Expires / Cache-Control / X-Accel-Expires preference. + +############################################################################### + +use warnings; +use strict; + +use Test::More; + +BEGIN { use FindBin; chdir($FindBin::Bin); } + +use lib 'lib'; +use Test::Nginx; + +############################################################################### + +select STDERR; $| = 1; +select STDOUT; $| = 1; + +my $t = Test::Nginx->new()->has(qw/http proxy cache rewrite map/)->plan(19); + +$t->write_file_expand('nginx.conf', <<'EOF'); + +%%TEST_GLOBALS%% + +daemon off; + +events { +} + +http { + %%TEST_GLOBALS_HTTP%% + + proxy_cache_path %%TESTDIR%%/cache levels=1:2 + keys_zone=NAME:1m; + + map $arg_e $e { + epoch "Thu, 01 Jan 1970 00:00:01 GMT"; + max "Thu, 31 Dec 2037 23:55:55 GMT"; + } + + server { + listen 127.0.0.1:8080; + server_name localhost; + + location / { + proxy_pass http://127.0.0.1:8081; + proxy_cache NAME; + + proxy_cache_background_update on; + + add_header X-Cache-Status $upstream_cache_status; + } + + location /ignore { + proxy_pass http://127.0.0.1:8081/return-xae; + proxy_cache NAME; + + proxy_ignore_headers X-Accel-Expires; + + add_header X-Cache-Status $upstream_cache_status; + } + } + + server { + listen 127.0.0.1:8081; + server_name localhost; + + location / { + add_header Expires $e; + add_header Cache-Control $arg_c; + add_header X-Accel-Expires $arg_x; + add_header X-Accel-Expires $arg_xx; + + return 204; + } + + location /rev { + add_header X-Accel-Expires $arg_x; + add_header Cache-Control $arg_c; + add_header Expires $e; + + return 204; + } + } +} + +EOF + +$t->run(); + +############################################################################### + +# Cache-Control is preferred over Expires, and +# X-Accel-Expires is preferred over both Cache-Control and Expires, + +like(get('/?e=max'), qr/HIT/, 'Expires'); +like(get('/?c=max-age=60'), qr/HIT/, 'Cache-Control'); +like(get('/?x=60'), qr/HIT/, 'X-Accel-Expires'); +like(get('/?x=@2145902155'), qr/HIT/, 'X-Accel-Expires at'); + +like(get('/ignore?x=60&ign=1'), qr/MISS/, 'X-Accel-Expires ignored'); + +TODO: { +local $TODO = 'not yet' unless $t->has_version('1.23.0'); + +like(get('/?x=60&xx=0'), qr/HIT/, 'X-Accel-Expires duplicate'); + +} + +# handling of caching parameters in Expires / Cache-Control / X-Accel-Expires + +like(get('/?e=max&c=no-cache'), qr/MISS/, 'Expires on CC off'); +like(get('/?e=max&x=0'), qr/MISS/, 'Expires on X-Accel-Expires off'); +like(get('/?c=max-age=60&x=0'), qr/MISS/, 'CC on X-Accel-Expires off'); + +TODO: { +local $TODO = 'not yet' unless $t->has_version('1.23.0'); + +like(get('/?e=epoch&c=max-age=60'), qr/HIT/, 'Expires off CC on'); +like(get('/?e=epoch&x=60'), qr/HIT/, 'Expires off X-Accel-Expires on'); +like(get('/?c=no-cache&x=60'), qr/HIT/, 'CC off X-Accel-Expires on'); + +} + +# and in the reversed order: X-Accel-Expires / Cache-Control / Expires + +like(get('/rev/?e=max&c=no-cache'), qr/MISS/, 'CC off Expires on'); +like(get('/rev?e=max&x=0'), qr/MISS/, 'X-Accel-Expires off Expires on'); +like(get('/rev?c=max-age=60&x=0'), qr/MISS/, 'X-Accel-Expires off CC on'); + +like(get('/rev?e=epoch&c=max-age=60'), qr/HIT/, 'CC on Expires off'); +like(get('/rev?e=epoch&x=60'), qr/HIT/, 'X-Accel-Expires on Expires off'); +like(get('/rev?c=no-cache&x=60'), qr/HIT/, 'X-Accel-Expires on CC off'); + +# Cache-Control caching extensions preserved after X-Accel-Expires + +TODO: { +local $TODO = 'not yet' unless $t->has_version('1.23.0'); + +my $time = time() - 1; +like(get("/rev?c=max-age=60,stale-while-revalidate=60&x=\@$time"), + qr/STALE/, 'Cache-Control extensions after X-Accel-Expires'); + +} + +############################################################################### + +sub get { + my ($uri) = @_; + http_get($uri); + http_get($uri); +} + +############################################################################### -- Sergey Kandaurov From pluknet at nginx.com Tue Jun 7 16:54:17 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 07 Jun 2022 16:54:17 +0000 Subject: [nginx] Upstream: handling of certificates specified as an empty string. Message-ID: details: https://hg.nginx.org/nginx/rev/c7e25324be11 branches: changeset: 8042:c7e25324be11 user: Sergey Kandaurov date: Tue Jun 07 20:08:57 2022 +0400 description: Upstream: handling of certificates specified as an empty string. Now, if the directive is given an empty string, such configuration cancels loading of certificates, in particular, if they would be otherwise inherited from the previous level. This restores previous behaviour, before variables support in certificates was introduced (3ab8e1e2f0f7). diffstat: src/http/modules/ngx_http_grpc_module.c | 5 +++-- src/http/modules/ngx_http_proxy_module.c | 5 +++-- src/http/modules/ngx_http_uwsgi_module.c | 5 +++-- src/http/ngx_http_upstream.c | 6 ++++-- src/stream/ngx_stream_proxy_module.c | 11 +++++++---- 5 files changed, 20 insertions(+), 12 deletions(-) diffs (89 lines): diff -r 0784ab86ad08 -r c7e25324be11 src/http/modules/ngx_http_grpc_module.c --- a/src/http/modules/ngx_http_grpc_module.c Tue Jun 07 00:07:12 2022 +0300 +++ b/src/http/modules/ngx_http_grpc_module.c Tue Jun 07 20:08:57 2022 +0400 @@ -4906,8 +4906,9 @@ ngx_http_grpc_set_ssl(ngx_conf_t *cf, ng return NGX_ERROR; } - if (glcf->upstream.ssl_certificate) { - + if (glcf->upstream.ssl_certificate + && glcf->upstream.ssl_certificate->value.len) + { if (glcf->upstream.ssl_certificate_key == NULL) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "no \"grpc_ssl_certificate_key\" is defined " diff -r 0784ab86ad08 -r c7e25324be11 src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c Tue Jun 07 00:07:12 2022 +0300 +++ b/src/http/modules/ngx_http_proxy_module.c Tue Jun 07 20:08:57 2022 +0400 @@ -4955,8 +4955,9 @@ ngx_http_proxy_set_ssl(ngx_conf_t *cf, n return NGX_ERROR; } - if (plcf->upstream.ssl_certificate) { - + if (plcf->upstream.ssl_certificate + && plcf->upstream.ssl_certificate->value.len) + { if (plcf->upstream.ssl_certificate_key == NULL) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "no \"proxy_ssl_certificate_key\" is defined " diff -r 0784ab86ad08 -r c7e25324be11 src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c Tue Jun 07 00:07:12 2022 +0300 +++ b/src/http/modules/ngx_http_uwsgi_module.c Tue Jun 07 20:08:57 2022 +0400 @@ -2487,8 +2487,9 @@ ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, n return NGX_ERROR; } - if (uwcf->upstream.ssl_certificate) { - + if (uwcf->upstream.ssl_certificate + && uwcf->upstream.ssl_certificate->value.len) + { if (uwcf->upstream.ssl_certificate_key == NULL) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "no \"uwsgi_ssl_certificate_key\" is defined " diff -r 0784ab86ad08 -r c7e25324be11 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Tue Jun 07 00:07:12 2022 +0300 +++ b/src/http/ngx_http_upstream.c Tue Jun 07 20:08:57 2022 +0400 @@ -1690,8 +1690,10 @@ ngx_http_upstream_ssl_init_connection(ng } } - if (u->conf->ssl_certificate && (u->conf->ssl_certificate->lengths - || u->conf->ssl_certificate_key->lengths)) + if (u->conf->ssl_certificate + && u->conf->ssl_certificate->value.len + && (u->conf->ssl_certificate->lengths + || u->conf->ssl_certificate_key->lengths)) { if (ngx_http_upstream_ssl_certificate(r, u, c) != NGX_OK) { ngx_http_upstream_finalize_request(r, u, diff -r 0784ab86ad08 -r c7e25324be11 src/stream/ngx_stream_proxy_module.c --- a/src/stream/ngx_stream_proxy_module.c Tue Jun 07 00:07:12 2022 +0300 +++ b/src/stream/ngx_stream_proxy_module.c Tue Jun 07 20:08:57 2022 +0400 @@ -1069,8 +1069,10 @@ ngx_stream_proxy_ssl_init_connection(ngx } } - if (pscf->ssl_certificate && (pscf->ssl_certificate->lengths - || pscf->ssl_certificate_key->lengths)) + if (pscf->ssl_certificate + && pscf->ssl_certificate->value.len + && (pscf->ssl_certificate->lengths + || pscf->ssl_certificate_key->lengths)) { if (ngx_stream_proxy_ssl_certificate(s) != NGX_OK) { ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR); @@ -2225,8 +2227,9 @@ ngx_stream_proxy_set_ssl(ngx_conf_t *cf, return NGX_ERROR; } - if (pscf->ssl_certificate) { - + if (pscf->ssl_certificate + && pscf->ssl_certificate->value.len) + { if (pscf->ssl_certificate_key == NULL) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "no \"proxy_ssl_certificate_key\" is defined " From mdounin at mdounin.ru Tue Jun 7 19:00:53 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 7 Jun 2022 22:00:53 +0300 Subject: [PATCH] Mp4: fixed potential overflow in ngx_http_mp4_crop_stts_data() In-Reply-To: <20220607131645.lp3rd7bzprwfds4z@N00W24XTQX> References: <20220607131645.lp3rd7bzprwfds4z@N00W24XTQX> Message-ID: Hello! On Tue, Jun 07, 2022 at 05:16:45PM +0400, Roman Arutyunyan wrote: > On Tue, May 31, 2022 at 12:05:13AM +0300, Maxim Dounin wrote: > > # HG changeset patch > > # User Maxim Dounin > > # Date 1653942033 -10800 > > # Mon May 30 23:20:33 2022 +0300 > > # Node ID d5363be9fa61e0327574b8aa1342c874efd027b0 > > # Parent cd40709c91e245036e7f71a6c3e5190d9533be75 > > Mp4: fixed potential overflow in ngx_http_mp4_crop_stts_data(). > > > > Both "count" and "duration" variables are 32-bit, so their product might > > potentially overflow. It is used to reduce 64-bit start_time variable, > > and with very large start_time this can result in incorrect seeking. > > > > Found by Coverity (CID 1499904). > > > > diff --git a/src/http/modules/ngx_http_mp4_module.c b/src/http/modules/ngx_http_mp4_module.c > > --- a/src/http/modules/ngx_http_mp4_module.c > > +++ b/src/http/modules/ngx_http_mp4_module.c > > @@ -2331,7 +2331,7 @@ ngx_http_mp4_crop_stts_data(ngx_http_mp4 > > } > > > > start_sample += count; > > - start_time -= count * duration; > > + start_time -= (uint64_t) count * duration; > > entries--; > > entry++; > > } > > Looks fine Pushed to http://mdounin.ru/hg/nginx. -- Maxim Dounin http://mdounin.ru/ From pluknet at nginx.com Tue Jun 7 19:28:24 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 07 Jun 2022 19:28:24 +0000 Subject: [nginx] Mp4: fixed potential overflow in ngx_http_mp4_crop_stts_data(). Message-ID: details: https://hg.nginx.org/nginx/rev/1afd19dc7161 branches: changeset: 8043:1afd19dc7161 user: Maxim Dounin date: Tue Jun 07 21:58:52 2022 +0300 description: Mp4: fixed potential overflow in ngx_http_mp4_crop_stts_data(). Both "count" and "duration" variables are 32-bit, so their product might potentially overflow. It is used to reduce 64-bit start_time variable, and with very large start_time this can result in incorrect seeking. Found by Coverity (CID 1499904). diffstat: src/http/modules/ngx_http_mp4_module.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r c7e25324be11 -r 1afd19dc7161 src/http/modules/ngx_http_mp4_module.c --- a/src/http/modules/ngx_http_mp4_module.c Tue Jun 07 20:08:57 2022 +0400 +++ b/src/http/modules/ngx_http_mp4_module.c Tue Jun 07 21:58:52 2022 +0300 @@ -2331,7 +2331,7 @@ ngx_http_mp4_crop_stts_data(ngx_http_mp4 } start_sample += count; - start_time -= count * duration; + start_time -= (uint64_t) count * duration; entries--; entry++; } From mdounin at mdounin.ru Wed Jun 8 00:57:58 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 8 Jun 2022 03:57:58 +0300 Subject: [PATCH] Upstream: prioritise Cache-Control over Expires In-Reply-To: <6DC932A4-16AA-42D5-9D06-A3F599439908@nginx.com> References: <4de897b7-e1b2-dbcd-7e20-f18ea91f8463@cdnnow.ru> <151ef936-f5d2-b681-bb54-6476c654a3ae@cdnnow.ru> <00218a02-f3e7-d1fe-04ba-1e61049f17be@cdnnow.ru> <20220606134220.afco4q4u6ujj2s3h@Y9MQ9X2QVV> <6DC932A4-16AA-42D5-9D06-A3F599439908@nginx.com> Message-ID: Hello! On Tue, Jun 07, 2022 at 07:58:36PM +0400, Sergey Kandaurov wrote: > > On 7 Jun 2022, at 01:09, Maxim Dounin wrote: > > > > On Mon, Jun 06, 2022 at 05:42:20PM +0400, Sergey Kandaurov wrote: > > > >> On Sun, Apr 24, 2022 at 06:42:47PM +0300, Maxim Dounin wrote: > >>> Hello! > >>> > >>> On Sun, Apr 24, 2022 at 07:55:17AM +0300, Maxim Dounin wrote: > >>> > >>> [...] > >>> > >>>> As far as I can tell, proper behaviour, assuming we parse cache > >>>> control extensions independently of X-Accel-Expires, can be > >>>> implemented by using just one flag. > >>> > >>> No, that's wrong, as with just one flag it wouldn't be possible to > >>> correctly disable caching of responses with: > >>> > >>> Cache-Control: private > >>> Cache-Control: max-age=10 > >>> > >>> So it needs at least two flags. Updated patch below, review and > >>> testing appreciated. > >>> > >>> # HG changeset patch > >>> # User Maxim Dounin > >>> # Date 1650814681 -10800 > >>> # Sun Apr 24 18:38:01 2022 +0300 > >>> # Node ID 940ba4317a97c72d1ee6700cbf58a543fee04c7a > >>> # Parent a736a7a613ea6e182ff86fbadcb98bb0f8891c0b > >>> Upstream: fixed X-Accel-Expires/Cache-Control/Expires handling. > >>> [..] > >> [..] > >> As the above note covers a corner case of X-Accel-Expires, > >> I believe it's fine to commit as is. > > > > This patch doesn't try to change handling of X-Accel-Expires in > > the past. > > > > And, honestly, the only documented way to _disable_ caching is to > > use 'X-Accel-Expires: 0'. The 'X-Accel-Expires: @0' only > > documented to set the time up to which the response may be cached > > to 0 seconds the Epoch. Expires and Cache-Control handling used > > to disable caching in similar situations, yet this behaviour was > > questioned more than once. X-Accel-Expires never disabled caching > > for absolute times in the past, and this is known to be used as a > > workaround to cache a pre-expires response[1][2]. > > > > Pushed to http://mdounin.ru/hg/nginx. > > > > [1] https://trac.nginx.org/nginx/ticket/1182 > > [2] https://mailman.nginx.org/pipermail/nginx-ru/2013-November/052614.html > > > > Tests: > > > # HG changeset patch > # User Sergey Kandaurov > # Date 1654617432 -14400 > # Tue Jun 07 19:57:12 2022 +0400 > # Node ID ed5fe35807f6c4853be1eb8ac429524e04ac5d67 > # Parent 4f238efded81e00c635c146df6ecb585fa0907ec > Tests: added X-Accel-Expires tests. This probably needs to be adjusted. > > diff --git a/proxy_xae.t b/proxy_xae.t > new file mode 100644 > --- /dev/null > +++ b/proxy_xae.t Should be "proxy_cache_*.t", probably "proxy_cache_control.t" would be a good enough name. Related existing tests are in proxy_cache_valid.t, though a separate file is probably more readable given the number of tests. > @@ -0,0 +1,160 @@ > +#!/usr/bin/perl > + > +# (C) Sergey Kandaurov > +# (C) Nginx, Inc. > + > +# Tests for proxy Expires / Cache-Control / X-Accel-Expires preference. > + > +############################################################################### > + > +use warnings; > +use strict; > + > +use Test::More; > + > +BEGIN { use FindBin; chdir($FindBin::Bin); } > + > +use lib 'lib'; > +use Test::Nginx; > + > +############################################################################### > + > +select STDERR; $| = 1; > +select STDOUT; $| = 1; > + > +my $t = Test::Nginx->new()->has(qw/http proxy cache rewrite map/)->plan(19); > + > +$t->write_file_expand('nginx.conf', <<'EOF'); > + > +%%TEST_GLOBALS%% > + > +daemon off; > + > +events { > +} > + > +http { > + %%TEST_GLOBALS_HTTP%% > + > + proxy_cache_path %%TESTDIR%%/cache levels=1:2 > + keys_zone=NAME:1m; > + > + map $arg_e $e { > + epoch "Thu, 01 Jan 1970 00:00:01 GMT"; > + max "Thu, 31 Dec 2037 23:55:55 GMT"; > + } > + > + server { > + listen 127.0.0.1:8080; > + server_name localhost; > + > + location / { > + proxy_pass http://127.0.0.1:8081; > + proxy_cache NAME; > + > + proxy_cache_background_update on; > + > + add_header X-Cache-Status $upstream_cache_status; This probably can be moved to server level. > + } > + > + location /ignore { > + proxy_pass http://127.0.0.1:8081/return-xae; Any reasons for "/return-xae"? > + proxy_cache NAME; > + > + proxy_ignore_headers X-Accel-Expires; > + > + add_header X-Cache-Status $upstream_cache_status; > + } > + } > + > + server { > + listen 127.0.0.1:8081; > + server_name localhost; > + > + location / { > + add_header Expires $e; > + add_header Cache-Control $arg_c; > + add_header X-Accel-Expires $arg_x; > + add_header X-Accel-Expires $arg_xx; > + > + return 204; > + } > + > + location /rev { > + add_header X-Accel-Expires $arg_x; > + add_header Cache-Control $arg_c; > + add_header Expires $e; > + > + return 204; > + } > + } > +} > + > +EOF > + > +$t->run(); > + > +############################################################################### > + > +# Cache-Control is preferred over Expires, and > +# X-Accel-Expires is preferred over both Cache-Control and Expires, > + > +like(get('/?e=max'), qr/HIT/, 'Expires'); > +like(get('/?c=max-age=60'), qr/HIT/, 'Cache-Control'); > +like(get('/?x=60'), qr/HIT/, 'X-Accel-Expires'); > +like(get('/?x=@2145902155'), qr/HIT/, 'X-Accel-Expires at'); It would be great to have test names in lowercase unless there are specific reasons to write them differently. Header names are usually written in lowercase. > + > +like(get('/ignore?x=60&ign=1'), qr/MISS/, 'X-Accel-Expires ignored'); > + > +TODO: { > +local $TODO = 'not yet' unless $t->has_version('1.23.0'); > + > +like(get('/?x=60&xx=0'), qr/HIT/, 'X-Accel-Expires duplicate'); > + > +} > + It might be better to use explicitly returned headers in appropriately named locations, at least for some specific tests like the one with duplicate X-Accel-Expires, e.g.: location /duplicate-accel-expires { add_header X-Accel-Expires 60; add_header X-Accel-Expires 0; return 204; } Another test with multiple headers to consider, given that this specific case requires two flags instead of just one: location /cache-control-no-cache-multi { add_header Cache-Control no-cache; add_header Cache-Control max-age=60; return 204; } Probably along with the simple variant, location /cache-control-no-cache-one { add_header Cache-Control "no-cache, max-age=60"; return 204; } which is expected to be exactly equivalent. Also, it might be a good idea to make sure that various other factors to disable cache, such as Set-Cookie, are not ignored when a resource is returned with Expires in the past and valid Cache-Control max-age. There were multiple attempts to submit patches which failed to handle this properly. E.g.: location /set-cookie { add_header Set-Cookie foo; add_header Cache-Control max-age=60; return 204; } > +# handling of caching parameters in Expires / Cache-Control / X-Accel-Expires > + > +like(get('/?e=max&c=no-cache'), qr/MISS/, 'Expires on CC off'); I would really like to see Cache-Control non-abbreviated, even if it requires some line wrapping. > +like(get('/?e=max&x=0'), qr/MISS/, 'Expires on X-Accel-Expires off'); > +like(get('/?c=max-age=60&x=0'), qr/MISS/, 'CC on X-Accel-Expires off'); > + > +TODO: { > +local $TODO = 'not yet' unless $t->has_version('1.23.0'); > + > +like(get('/?e=epoch&c=max-age=60'), qr/HIT/, 'Expires off CC on'); > +like(get('/?e=epoch&x=60'), qr/HIT/, 'Expires off X-Accel-Expires on'); > +like(get('/?c=no-cache&x=60'), qr/HIT/, 'CC off X-Accel-Expires on'); > + > +} Any tests with all three headers? Overall, we need to test the following basic things: - Expires is ignored when Cache-Control is present. This implies that "Cache-Control: max-age=60" enables caching even if the response has Expires header in the past, regardless of the order, i.e.: location /cache-control-after-expires { add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; add_header Cache-Control max-age=60; return 204; } location /cache-control-before-expires { add_header Cache-Control max-age=60; add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; return 204; } location /cache-control-no-cache-after-expires { add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; add_header Cache-Control no-cache; return 204; } location /cache-control-no-cache-before-expires { add_header Cache-Control no-cache; add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; return 204; } - Expires and Cache-Control are ignored when X-Accel-Expires is present. Quite similar to the above, but with Expires / Cache-Control vs. X-Accel-Expires. location /x-accel-expires-after { add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; add_header Cache-Control no-cache; add_header X-Accel-Expires 60; return 204; } location /x-accel-expires-before { add_header X-Accel-Expires 60; add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; add_header Cache-Control no-cache; return 204; } location /x-accel-expires-0-after { add_header Cache-Control max-age=60; add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; add_header X-Accel-Expires 0; return 204; } location /x-accel-expires-0-before { add_header X-Accel-Expires 0; add_header Cache-Control max-age=60; add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; return 204; } > + > +# and in the reversed order: X-Accel-Expires / Cache-Control / Expires > + > +like(get('/rev/?e=max&c=no-cache'), qr/MISS/, 'CC off Expires on'); > +like(get('/rev?e=max&x=0'), qr/MISS/, 'X-Accel-Expires off Expires on'); > +like(get('/rev?c=max-age=60&x=0'), qr/MISS/, 'X-Accel-Expires off CC on'); > + > +like(get('/rev?e=epoch&c=max-age=60'), qr/HIT/, 'CC on Expires off'); > +like(get('/rev?e=epoch&x=60'), qr/HIT/, 'X-Accel-Expires on Expires off'); > +like(get('/rev?c=no-cache&x=60'), qr/HIT/, 'X-Accel-Expires on CC off'); > + > +# Cache-Control caching extensions preserved after X-Accel-Expires > + > +TODO: { > +local $TODO = 'not yet' unless $t->has_version('1.23.0'); > + > +my $time = time() - 1; > +like(get("/rev?c=max-age=60,stale-while-revalidate=60&x=\@$time"), > + qr/STALE/, 'Cache-Control extensions after X-Accel-Expires'); Something like: X-Accel-Expires: @1 Cache-Control: stale-while-revalidate=2145902155 shouldn't require any calculations (and can be placed in an explicit location). And the opposite order probably worth checking too. Not sure if "max-age=60" is at all needed here, though probably it can catch some potential bugs. [...] -- Maxim Dounin http://mdounin.ru/ From xeioex at nginx.com Wed Jun 8 17:13:10 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 08 Jun 2022 17:13:10 +0000 Subject: [njs] HTTP: fixed r.headersOut special getters when value is absent. Message-ID: details: https://hg.nginx.org/njs/rev/6ef77c24c924 branches: changeset: 1878:6ef77c24c924 user: Dmitry Volyntsev date: Mon Jun 06 23:27:11 2022 -0700 description: HTTP: fixed r.headersOut special getters when value is absent. Previously, when Content-Encoding or Content-Length header was absent, an exception was thrown erroneously. This closes #537 issue on Github. diffstat: nginx/ngx_http_js_module.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diffs (39 lines): diff -r 9c4858617097 -r 6ef77c24c924 nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Fri Jun 03 23:24:12 2022 -0700 +++ b/nginx/ngx_http_js_module.c Mon Jun 06 23:27:11 2022 -0700 @@ -1900,7 +1900,7 @@ ngx_http_js_content_encoding(njs_vm_t *v ngx_table_elt_t *h; rc = ngx_http_js_header_out_special(vm, r, v, setval, retval, &h); - if (rc != NJS_OK) { + if (rc == NJS_ERROR) { return NJS_ERROR; } @@ -1940,7 +1940,7 @@ ngx_http_js_content_length(njs_vm_t *vm, } rc = ngx_http_js_header_out_special(vm, r, v, setval, retval, &h); - if (rc != NJS_OK) { + if (rc == NJS_ERROR) { return NJS_ERROR; } @@ -3686,7 +3686,7 @@ ngx_http_js_content_encoding(njs_vm_t *v ngx_table_elt_t *h; rc = ngx_http_js_header_out_special(vm, r, v, setval, retval, &h); - if (rc != NJS_OK) { + if (rc == NJS_ERROR) { return NJS_ERROR; } @@ -3726,7 +3726,7 @@ ngx_http_js_content_length(njs_vm_t *vm, } rc = ngx_http_js_header_out_special(vm, r, v, setval, retval, &h); - if (rc != NJS_OK) { + if (rc == NJS_ERROR) { return NJS_ERROR; } From xeioex at nginx.com Wed Jun 8 17:13:12 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 08 Jun 2022 17:13:12 +0000 Subject: [njs] HTTP: returing undefined for Content-Type when the header is absent. Message-ID: details: https://hg.nginx.org/njs/rev/cf267407eabe branches: changeset: 1879:cf267407eabe user: Dmitry Volyntsev date: Tue Jun 07 21:30:57 2022 -0700 description: HTTP: returing undefined for Content-Type when the header is absent. This unifies empty response value type for r.headersOut. diffstat: nginx/ngx_http_js_module.c | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-) diffs (29 lines): diff -r 6ef77c24c924 -r cf267407eabe nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Mon Jun 06 23:27:11 2022 -0700 +++ b/nginx/ngx_http_js_module.c Tue Jun 07 21:30:57 2022 -0700 @@ -1978,6 +1978,12 @@ ngx_http_js_content_type(njs_vm_t *vm, n if (retval != NULL && setval == NULL) { hdr = &r->headers_out.content_type; + + if (hdr->len == 0) { + njs_value_undefined_set(retval); + return NJS_OK; + } + return njs_vm_value_string_set(vm, retval, hdr->data, hdr->len); } @@ -3764,6 +3770,12 @@ ngx_http_js_content_type(njs_vm_t *vm, n if (retval != NULL && setval == NULL) { hdr = &r->headers_out.content_type; + + if (hdr->len == 0) { + njs_value_undefined_set(retval); + return NJS_OK; + } + return njs_vm_value_string_set(vm, retval, hdr->data, hdr->len); } From xeioex at nginx.com Thu Jun 9 07:07:57 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 09 Jun 2022 07:07:57 +0000 Subject: [njs] Fixed template literal from producing byte-strings. Message-ID: details: https://hg.nginx.org/njs/rev/d4cdb9085e4d branches: changeset: 1880:d4cdb9085e4d user: Dmitry Volyntsev date: Wed Jun 08 21:06:16 2022 -0700 description: Fixed template literal from producing byte-strings. Previously, as a side effect of creating a key for the values hash a byte-string was created. This byte-string was reused internally and might appear in template literal. As a result a byte-string was produced as a value for a template literal. Byte-strings are obsolete and are scheduled for removal because they can cause issues with internal routines not prepared for them. diffstat: src/njs_generator.c | 10 ++++++++-- src/test/njs_unit_test.c | 4 ++++ 2 files changed, 12 insertions(+), 2 deletions(-) diffs (41 lines): diff -r cf267407eabe -r d4cdb9085e4d src/njs_generator.c --- a/src/njs_generator.c Tue Jun 07 21:30:57 2022 -0700 +++ b/src/njs_generator.c Wed Jun 08 21:06:16 2022 -0700 @@ -4756,6 +4756,7 @@ static njs_int_t njs_generate_global_reference(njs_vm_t *vm, njs_generator_t *generator, njs_parser_node_t *node, njs_bool_t exception) { + ssize_t length; njs_int_t ret; njs_index_t index; njs_value_t property; @@ -4783,8 +4784,13 @@ njs_generate_global_reference(njs_vm_t * return NJS_ERROR; } - ret = njs_string_set(vm, &property, lex_entry->name.start, - lex_entry->name.length); + length = njs_utf8_length(lex_entry->name.start, lex_entry->name.length); + if (njs_slow_path(length < 0)) { + return NJS_ERROR; + } + + ret = njs_string_new(vm, &property, lex_entry->name.start, + lex_entry->name.length, length); if (njs_slow_path(ret != NJS_OK)) { return NJS_ERROR; } diff -r cf267407eabe -r d4cdb9085e4d src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Tue Jun 07 21:30:57 2022 -0700 +++ b/src/test/njs_unit_test.c Wed Jun 08 21:06:16 2022 -0700 @@ -7174,6 +7174,10 @@ static njs_unit_test_t njs_test[] = { njs_str("`\\${a}bc"), njs_str("SyntaxError: Unterminated template literal in 1") }, + { njs_str("var v = undefined; var u8 = 'α';" + "[`undefined${u8}`.length, `undefineQ${u8}`.length]"), + njs_str("10,10") }, + { njs_str("`text1\ntext2`;"), njs_str("text1\ntext2") }, From xeioex at nginx.com Thu Jun 9 07:07:59 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 09 Jun 2022 07:07:59 +0000 Subject: [njs] Fixed njs_iterator_to_array() with sparse arrays. Message-ID: details: https://hg.nginx.org/njs/rev/0b75123c6ea4 branches: changeset: 1881:0b75123c6ea4 user: Dmitry Volyntsev date: Wed Jun 08 22:58:34 2022 -0700 description: Fixed njs_iterator_to_array() with sparse arrays. This closes #524 issue on Github. diffstat: src/njs_iterator.c | 10 +++++----- src/test/njs_unit_test.c | 8 ++++++++ 2 files changed, 13 insertions(+), 5 deletions(-) diffs (45 lines): diff -r d4cdb9085e4d -r 0b75123c6ea4 src/njs_iterator.c --- a/src/njs_iterator.c Wed Jun 08 21:06:16 2022 -0700 +++ b/src/njs_iterator.c Wed Jun 08 22:58:34 2022 -0700 @@ -686,7 +686,8 @@ njs_iterator_to_array(njs_vm_t *vm, njs_ return NULL; } - args.data = njs_array_alloc(vm, 1, length, 0); + args.data = njs_array_alloc(vm, 0, 0, + njs_min(length, NJS_ARRAY_LARGE_OBJECT_LENGTH)); if (njs_slow_path(args.data == NULL)) { return NULL; } @@ -708,10 +709,9 @@ static njs_int_t njs_iterator_to_array_handler(njs_vm_t *vm, njs_iterator_args_t *args, njs_value_t *value, int64_t index) { - njs_array_t *array; + njs_value_t array; - array = args->data; - array->start[index] = *value; + njs_set_array(&array, args->data); - return NJS_OK; + return njs_value_property_i64_set(vm, &array, index, value); } diff -r d4cdb9085e4d -r 0b75123c6ea4 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Wed Jun 08 21:06:16 2022 -0700 +++ b/src/test/njs_unit_test.c Wed Jun 08 22:58:34 2022 -0700 @@ -11299,6 +11299,14 @@ static njs_unit_test_t njs_test[] = { njs_str("let e = AggregateError([1, 2, 3], 'm'); e"), njs_str("AggregateError: m") }, + { njs_str("var v = Object.defineProperty([], 1025, {get: () => 1});" + "AggregateError(v).errors[23]"), + njs_str("undefined") }, + + { njs_str("var v = Object.defineProperty([], 2**20, {get: () => 1});" + "AggregateError(v).errors[2**19]"), + njs_str("undefined") }, + /* Memory object is immutable. */ { njs_str("var e = MemoryError('e'); e.name = 'E'"), From a.bavshin at f5.com Fri Jun 10 11:34:41 2022 From: a.bavshin at f5.com (Aleksei Bavshin) Date: Fri, 10 Jun 2022 11:34:41 +0000 Subject: [nginx] Stream: don't flush empty buffers created for read errors. Message-ID: details: https://hg.nginx.org/nginx/rev/457afc332c67 branches: changeset: 8044:457afc332c67 user: Aleksei Bavshin date: Mon May 23 11:29:44 2022 -0700 description: Stream: don't flush empty buffers created for read errors. When we generate the last_buf buffer for an UDP upstream recv error, it does not contain any data from the wire. ngx_stream_write_filter attempts to forward it anyways, which is incorrect (e.g., UDP upstream ECONNREFUSED will be translated to an empty packet). This happens because we mark the buffer as both 'flush' and 'last_buf', and ngx_stream_write_filter has special handling for flush with certain types of connections (see d127837c714f, 32b0ba4855a6). The flags are meant to be mutually exclusive, so the fix is to ensure that flush and last_buf are not set at the same time. Reproduction: stream { upstream unreachable { server 127.0.0.1:8880; } server { listen 127.0.0.1:8998 udp; proxy_pass unreachable; } } 1 0.000000000 127.0.0.1 → 127.0.0.1 UDP 47 45588 → 8998 Len=5 2 0.000166300 127.0.0.1 → 127.0.0.1 UDP 47 51149 → 8880 Len=5 3 0.000172600 127.0.0.1 → 127.0.0.1 ICMP 75 Destination unreachable (Port unreachable) 4 0.000202400 127.0.0.1 → 127.0.0.1 UDP 42 8998 → 45588 Len=0 Fixes d127837c714f. diffstat: src/stream/ngx_stream_proxy_module.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 1afd19dc7161 -r 457afc332c67 src/stream/ngx_stream_proxy_module.c --- a/src/stream/ngx_stream_proxy_module.c Tue Jun 07 21:58:52 2022 +0300 +++ b/src/stream/ngx_stream_proxy_module.c Mon May 23 11:29:44 2022 -0700 @@ -1737,7 +1737,7 @@ ngx_stream_proxy_process(ngx_stream_sess cl->buf->temporary = (n ? 1 : 0); cl->buf->last_buf = src->read->eof; - cl->buf->flush = 1; + cl->buf->flush = !src->read->eof; (*packets)++; *received += n; From a.bavshin at f5.com Fri Jun 10 11:34:44 2022 From: a.bavshin at f5.com (Aleksei Bavshin) Date: Fri, 10 Jun 2022 11:34:44 +0000 Subject: [nginx] Resolver: make TCP write timer event cancelable. Message-ID: details: https://hg.nginx.org/nginx/rev/aa28c802409f branches: changeset: 8045:aa28c802409f user: Aleksei Bavshin date: Wed Jun 01 20:17:23 2022 -0700 description: Resolver: make TCP write timer event cancelable. Similar to 70e65bf8dfd7, the change is made to ensure that the ability to cancel resolver tasks is fully controlled by the caller. As mentioned in the referenced commit, it is safe to make this timer cancelable because resolve tasks can have their own timeouts that are not cancelable. The scenario where this may become a problem is a periodic background resolve task (not tied to a specific request or a client connection), which receives a response with short TTL, large enough to warrant fallback to a TCP query. With each event loop wakeup, we either have a previously set write timer instance or schedule a new one. The non-cancelable write timer can delay or block graceful shutdown of a worker even if the ngx_resolver_ctx_t->cancelable flag is set by the API user, and there are no other tasks or connections. We use the resolver API in this way to maintain the list of upstream server addresses specified with the 'resolve' parameter, and there could be third-party modules implementing similar logic. diffstat: src/core/ngx_resolver.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r 457afc332c67 -r aa28c802409f src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c Mon May 23 11:29:44 2022 -0700 +++ b/src/core/ngx_resolver.c Wed Jun 01 20:17:23 2022 -0700 @@ -1389,6 +1389,7 @@ ngx_resolver_send_tcp_query(ngx_resolver rec->tcp->data = rec; rec->tcp->write->handler = ngx_resolver_tcp_write; + rec->tcp->write->cancelable = 1; rec->tcp->read->handler = ngx_resolver_tcp_read; rec->tcp->read->resolver = 1; From pluknet at nginx.com Fri Jun 10 11:37:34 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Fri, 10 Jun 2022 15:37:34 +0400 Subject: [PATCH] Upstream: prioritise Cache-Control over Expires In-Reply-To: References: <4de897b7-e1b2-dbcd-7e20-f18ea91f8463@cdnnow.ru> <151ef936-f5d2-b681-bb54-6476c654a3ae@cdnnow.ru> <00218a02-f3e7-d1fe-04ba-1e61049f17be@cdnnow.ru> <20220606134220.afco4q4u6ujj2s3h@Y9MQ9X2QVV> <6DC932A4-16AA-42D5-9D06-A3F599439908@nginx.com> Message-ID: <80E87E07-436C-48AF-AA98-DF6BC7961306@nginx.com> > On 8 Jun 2022, at 04:57, Maxim Dounin wrote: > > Hello! > > On Tue, Jun 07, 2022 at 07:58:36PM +0400, Sergey Kandaurov wrote: > >>> On 7 Jun 2022, at 01:09, Maxim Dounin wrote: >>> >>> On Mon, Jun 06, 2022 at 05:42:20PM +0400, Sergey Kandaurov wrote: >>> >>>> On Sun, Apr 24, 2022 at 06:42:47PM +0300, Maxim Dounin wrote: >>>>> Hello! >>>>> >>>>> On Sun, Apr 24, 2022 at 07:55:17AM +0300, Maxim Dounin wrote: >>>>> >>>>> [...] >>>>> >>>>>> As far as I can tell, proper behaviour, assuming we parse cache >>>>>> control extensions independently of X-Accel-Expires, can be >>>>>> implemented by using just one flag. >>>>> >>>>> No, that's wrong, as with just one flag it wouldn't be possible to >>>>> correctly disable caching of responses with: >>>>> >>>>> Cache-Control: private >>>>> Cache-Control: max-age=10 >>>>> >>>>> So it needs at least two flags. Updated patch below, review and >>>>> testing appreciated. >>>>> >>>>> # HG changeset patch >>>>> # User Maxim Dounin >>>>> # Date 1650814681 -10800 >>>>> # Sun Apr 24 18:38:01 2022 +0300 >>>>> # Node ID 940ba4317a97c72d1ee6700cbf58a543fee04c7a >>>>> # Parent a736a7a613ea6e182ff86fbadcb98bb0f8891c0b >>>>> Upstream: fixed X-Accel-Expires/Cache-Control/Expires handling. >>>>> [..] >>>> [..] >>>> As the above note covers a corner case of X-Accel-Expires, >>>> I believe it's fine to commit as is. >>> >>> This patch doesn't try to change handling of X-Accel-Expires in >>> the past. >>> >>> And, honestly, the only documented way to _disable_ caching is to >>> use 'X-Accel-Expires: 0'. The 'X-Accel-Expires: @0' only >>> documented to set the time up to which the response may be cached >>> to 0 seconds the Epoch. Expires and Cache-Control handling used >>> to disable caching in similar situations, yet this behaviour was >>> questioned more than once. X-Accel-Expires never disabled caching >>> for absolute times in the past, and this is known to be used as a >>> workaround to cache a pre-expires response[1][2]. >>> >>> Pushed to http://mdounin.ru/hg/nginx. >>> >>> [1] https://trac.nginx.org/nginx/ticket/1182 >>> [2] https://mailman.nginx.org/pipermail/nginx-ru/2013-November/052614.html >>> >> >> Tests: >> >> >> # HG changeset patch >> # User Sergey Kandaurov >> # Date 1654617432 -14400 >> # Tue Jun 07 19:57:12 2022 +0400 >> # Node ID ed5fe35807f6c4853be1eb8ac429524e04ac5d67 >> # Parent 4f238efded81e00c635c146df6ecb585fa0907ec >> Tests: added X-Accel-Expires tests. > > This probably needs to be adjusted. > >> >> diff --git a/proxy_xae.t b/proxy_xae.t >> new file mode 100644 >> --- /dev/null >> +++ b/proxy_xae.t > > Should be "proxy_cache_*.t", probably "proxy_cache_control.t" > would be a good enough name. > > Related existing tests are in proxy_cache_valid.t, though a > separate file is probably more readable given the number of tests. I didn't concentrate on Expires and Cache-Control originally, but rather on the preference order vs. X-Accel-Expires. Extending it with generic tests may be a good idea, though. > >> @@ -0,0 +1,160 @@ >> +#!/usr/bin/perl >> + >> +# (C) Sergey Kandaurov >> +# (C) Nginx, Inc. >> + >> +# Tests for proxy Expires / Cache-Control / X-Accel-Expires preference. >> + >> +############################################################################### >> + >> +use warnings; >> +use strict; >> + >> +use Test::More; >> + >> +BEGIN { use FindBin; chdir($FindBin::Bin); } >> + >> +use lib 'lib'; >> +use Test::Nginx; >> + >> +############################################################################### >> + >> +select STDERR; $| = 1; >> +select STDOUT; $| = 1; >> + >> +my $t = Test::Nginx->new()->has(qw/http proxy cache rewrite map/)->plan(19); >> + >> +$t->write_file_expand('nginx.conf', <<'EOF'); >> + >> +%%TEST_GLOBALS%% >> + >> +daemon off; >> + >> +events { >> +} >> + >> +http { >> + %%TEST_GLOBALS_HTTP%% >> + >> + proxy_cache_path %%TESTDIR%%/cache levels=1:2 >> + keys_zone=NAME:1m; >> + >> + map $arg_e $e { >> + epoch "Thu, 01 Jan 1970 00:00:01 GMT"; >> + max "Thu, 31 Dec 2037 23:55:55 GMT"; >> + } >> + >> + server { >> + listen 127.0.0.1:8080; >> + server_name localhost; >> + >> + location / { >> + proxy_pass http://127.0.0.1:8081; >> + proxy_cache NAME; >> + >> + proxy_cache_background_update on; >> + >> + add_header X-Cache-Status $upstream_cache_status; > > This probably can be moved to server level. > >> + } >> + >> + location /ignore { >> + proxy_pass http://127.0.0.1:8081/return-xae; > > Any reasons for "/return-xae"? Adjusted all the above, thnx. > >> + proxy_cache NAME; >> + >> + proxy_ignore_headers X-Accel-Expires; >> + >> + add_header X-Cache-Status $upstream_cache_status; >> + } >> + } >> + >> + server { >> + listen 127.0.0.1:8081; >> + server_name localhost; >> + >> + location / { >> + add_header Expires $e; >> + add_header Cache-Control $arg_c; >> + add_header X-Accel-Expires $arg_x; >> + add_header X-Accel-Expires $arg_xx; >> + >> + return 204; >> + } >> + >> + location /rev { >> + add_header X-Accel-Expires $arg_x; >> + add_header Cache-Control $arg_c; >> + add_header Expires $e; >> + >> + return 204; >> + } >> + } >> +} >> + >> +EOF >> + >> +$t->run(); >> + >> +############################################################################### >> + >> +# Cache-Control is preferred over Expires, and >> +# X-Accel-Expires is preferred over both Cache-Control and Expires, >> + >> +like(get('/?e=max'), qr/HIT/, 'Expires'); >> +like(get('/?c=max-age=60'), qr/HIT/, 'Cache-Control'); >> +like(get('/?x=60'), qr/HIT/, 'X-Accel-Expires'); >> +like(get('/?x=@2145902155'), qr/HIT/, 'X-Accel-Expires at'); > > It would be great to have test names in lowercase unless there are > specific reasons to write them differently. Header names are > usually written in lowercase. It's based on proxy_xar.t where headers start with capital letter. I don't mind to replace them with lowercase though and think it's generally good to have a simple description in lowercase. > >> + >> +like(get('/ignore?x=60&ign=1'), qr/MISS/, 'X-Accel-Expires ignored'); >> + >> +TODO: { >> +local $TODO = 'not yet' unless $t->has_version('1.23.0'); >> + >> +like(get('/?x=60&xx=0'), qr/HIT/, 'X-Accel-Expires duplicate'); >> + >> +} >> + > > It might be better to use explicitly returned headers in > appropriately named locations, at least for some specific tests > like the one with duplicate X-Accel-Expires, e.g.: > > location /duplicate-accel-expires { > add_header X-Accel-Expires 60; > add_header X-Accel-Expires 0; > return 204; > } > Explicit headers will require rather large configuration, but let's try and see. > Another test with multiple headers to consider, given that this > specific case requires two flags instead of just one: > > location /cache-control-no-cache-multi { > add_header Cache-Control no-cache; > add_header Cache-Control max-age=60; > return 204; > } > > Probably along with the simple variant, > > location /cache-control-no-cache-one { > add_header Cache-Control "no-cache, max-age=60"; > return 204; > } > > which is expected to be exactly equivalent. Added. > > Also, it might be a good idea to make sure that various > other factors to disable cache, such as Set-Cookie, are not > ignored when a resource is returned with Expires in the past and > valid Cache-Control max-age. There were multiple attempts to > submit patches which failed to handle this properly. E.g.: > > location /set-cookie { > add_header Set-Cookie foo; > add_header Cache-Control max-age=60; > return 204; > } If I got you right, it refers to earlier patches in January with erroneously missed Set-Cookie handling. I reproduced this with Expires in the past and X-Accel-Expires: location /set-cookie { add_header Set-Cookie foo; add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; add_header X-Accel-Expires 60; return 204; } > >> +# handling of caching parameters in Expires / Cache-Control / X-Accel-Expires >> + >> +like(get('/?e=max&c=no-cache'), qr/MISS/, 'Expires on CC off'); > > I would really like to see Cache-Control non-abbreviated, even if > it requires some line wrapping. Sure, don't like it as well. > >> +like(get('/?e=max&x=0'), qr/MISS/, 'Expires on X-Accel-Expires off'); >> +like(get('/?c=max-age=60&x=0'), qr/MISS/, 'CC on X-Accel-Expires off'); >> + >> +TODO: { >> +local $TODO = 'not yet' unless $t->has_version('1.23.0'); >> + >> +like(get('/?e=epoch&c=max-age=60'), qr/HIT/, 'Expires off CC on'); >> +like(get('/?e=epoch&x=60'), qr/HIT/, 'Expires off X-Accel-Expires on'); >> +like(get('/?c=no-cache&x=60'), qr/HIT/, 'CC off X-Accel-Expires on'); >> + >> +} > > Any tests with all three headers? > > Overall, we need to test the following basic things: > > - Expires is ignored when Cache-Control is present. This implies > that "Cache-Control: max-age=60" enables caching even if the > response has Expires header in the past, regardless of the > order, i.e.: > > location /cache-control-after-expires { > add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; > add_header Cache-Control max-age=60; > return 204; > } > > location /cache-control-before-expires { > add_header Cache-Control max-age=60; > add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; > return 204; > } > > location /cache-control-no-cache-after-expires { > add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; > add_header Cache-Control no-cache; > return 204; > } > > location /cache-control-no-cache-before-expires { > add_header Cache-Control no-cache; > add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; > return 204; > } > > - Expires and Cache-Control are ignored when X-Accel-Expires is > present. Quite similar to the above, but with Expires / > Cache-Control vs. X-Accel-Expires. > > location /x-accel-expires-after { > add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; > add_header Cache-Control no-cache; > add_header X-Accel-Expires 60; > return 204; > } > > location /x-accel-expires-before { > add_header X-Accel-Expires 60; > add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; > add_header Cache-Control no-cache; > return 204; > } > > location /x-accel-expires-0-after { > add_header Cache-Control max-age=60; > add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; > add_header X-Accel-Expires 0; > return 204; > } > > location /x-accel-expires-0-before { > add_header X-Accel-Expires 0; > add_header Cache-Control max-age=60; > add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; > return 204; > } Indeed, this results in pretty the same number of tests, even though the configuration is expanded quite a bit compared to request based configuration with variables, so it looks promising to implement that way. Updated patch is at the end. > >> + >> +# and in the reversed order: X-Accel-Expires / Cache-Control / Expires >> + >> +like(get('/rev/?e=max&c=no-cache'), qr/MISS/, 'CC off Expires on'); >> +like(get('/rev?e=max&x=0'), qr/MISS/, 'X-Accel-Expires off Expires on'); >> +like(get('/rev?c=max-age=60&x=0'), qr/MISS/, 'X-Accel-Expires off CC on'); >> + >> +like(get('/rev?e=epoch&c=max-age=60'), qr/HIT/, 'CC on Expires off'); >> +like(get('/rev?e=epoch&x=60'), qr/HIT/, 'X-Accel-Expires on Expires off'); >> +like(get('/rev?c=no-cache&x=60'), qr/HIT/, 'X-Accel-Expires on CC off'); >> + >> +# Cache-Control caching extensions preserved after X-Accel-Expires >> + >> +TODO: { >> +local $TODO = 'not yet' unless $t->has_version('1.23.0'); >> + >> +my $time = time() - 1; >> +like(get("/rev?c=max-age=60,stale-while-revalidate=60&x=\@$time"), >> + qr/STALE/, 'Cache-Control extensions after X-Accel-Expires'); > > Something like: > > X-Accel-Expires: @1 > Cache-Control: stale-while-revalidate=2145902155 > > shouldn't require any calculations (and can be placed in an > explicit location). And the opposite order probably worth > checking too. While it's better to avoid math, this one is somewhat synthetic. Though, I have no objection in principle, good enough for tests. > > Not sure if "max-age=60" is at all needed here, though probably > it can catch some potential bugs. > Since "stale-while-revalidate" is an extension, I assume that it should be given with some base parameter, such as "max-age". The assumption is supported by examples in RFC 5861. This brought me a thought that we don't respect Cache-Control preference over Expires if given with only extensions: Expires: Thu, 31 Dec 2037 23:55:55 GMT Cache-Control: stale-while-revalidate=1 In the above, Cache-Control doesn't set/override valid_sec, and as such, it is set with Expires on its own. # HG changeset patch # User Sergey Kandaurov # Date 1654860778 -14400 # Fri Jun 10 15:32:58 2022 +0400 # Node ID f8e5ab69e6edd379b2ea9e0e8f1ebbab01fd9075 # Parent 4f238efded81e00c635c146df6ecb585fa0907ec Tests: added proxy cache tests with upstream cache headers. diff --git a/proxy_cache_control.t b/proxy_cache_control.t new file mode 100644 --- /dev/null +++ b/proxy_cache_control.t @@ -0,0 +1,276 @@ +#!/usr/bin/perl + +# (C) Sergey Kandaurov +# (C) Nginx, Inc. + +# Tests for proxy Expires / Cache-Control / X-Accel-Expires. + +############################################################################### + +use warnings; +use strict; + +use Test::More; + +BEGIN { use FindBin; chdir($FindBin::Bin); } + +use lib 'lib'; +use Test::Nginx; + +############################################################################### + +select STDERR; $| = 1; +select STDOUT; $| = 1; + +my $t = Test::Nginx->new()->has(qw/http proxy cache rewrite/)->plan(19); + +$t->write_file_expand('nginx.conf', <<'EOF'); + +%%TEST_GLOBALS%% + +daemon off; + +events { +} + +http { + %%TEST_GLOBALS_HTTP%% + + proxy_cache_path %%TESTDIR%%/cache keys_zone=NAME:1m; + + server { + listen 127.0.0.1:8080; + server_name localhost; + + add_header X-Cache-Status $upstream_cache_status; + + location / { + proxy_pass http://127.0.0.1:8081; + proxy_cache NAME; + + proxy_cache_background_update on; + } + + location /ignore { + proxy_pass http://127.0.0.1:8081; + proxy_cache NAME; + + proxy_ignore_headers Cache-Control Expires; + proxy_ignore_headers X-Accel-Expires; + } + } + + server { + listen 127.0.0.1:8081; + server_name localhost; + + location /expires { + add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; + return 204; + } + + location /cache-control { + add_header Cache-Control max-age=60; + return 204; + } + + location /x-accel-expires { + add_header X-Accel-Expires 60; + return 204; + } + + location /x-accel-expires-at { + add_header X-Accel-Expires @60; + return 204; + } + + location /x-accel-expires-duplicate { + add_header X-Accel-Expires 60; + add_header X-Accel-Expires 0; + return 204; + } + + location /ignore { + add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; + add_header Cache-Control max-age=60; + add_header X-Accel-Expires 60; + return 204; + } + + location /cache-control-after-expires { + add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; + add_header Cache-Control max-age=60; + return 204; + } + + location /cache-control-before-expires { + add_header Cache-Control max-age=60; + add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; + return 204; + } + + location /cache-control-no-cache-after-expires { + add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; + add_header Cache-Control no-cache; + return 204; + } + + location /cache-control-no-cache-before-expires { + add_header Cache-Control no-cache; + add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; + return 204; + } + + location /x-accel-expires-after { + add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; + add_header Cache-Control no-cache; + add_header X-Accel-Expires 60; + return 204; + } + + location /x-accel-expires-before { + add_header X-Accel-Expires 60; + add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; + add_header Cache-Control no-cache; + return 204; + } + + location /x-accel-expires-0-after { + add_header Cache-Control max-age=60; + add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; + add_header X-Accel-Expires 0; + return 204; + } + + location /x-accel-expires-0-before { + add_header X-Accel-Expires 0; + add_header Cache-Control max-age=60; + add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; + return 204; + } + + location /cache-control-no-cache-one { + add_header Cache-Control "no-cache, max-age=60"; + return 204; + } + + location /cache-control-no-cache-multi { + add_header Cache-Control no-cache; + add_header Cache-Control max-age=60; + return 204; + } + + location /extension-before-x-accel-expires { + add_header Cache-Control + "max-age=60, stale-while-revalidate=2145902155"; + add_header X-Accel-Expires @1; + return 204; + } + + location /extension-after-x-accel-expires { + add_header X-Accel-Expires @1; + add_header Cache-Control + "max-age=60, stale-while-revalidate=2145902155"; + return 204; + } + + location /set-cookie { + add_header Set-Cookie foo; + add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; + add_header X-Accel-Expires 60; + return 204; + } + } +} + +EOF + +$t->run(); + +############################################################################### + +# upstream cache headers work + +like(get('/expires'), qr/HIT/, 'expires'); +like(get('/cache-control'), qr/HIT/, 'cache-control'); +like(get('/x-accel-expires'), qr/HIT/, 'x-accel-expires'); +like(get('/x-accel-expires-at'), qr/EXPIRED/, 'x-accel-expires at'); + +TODO: { +local $TODO = 'not yet' unless $t->has_version('1.23.0'); + +# the second header to disable cache is duplicate and ignored + +like(get('/x-accel-expires-duplicate'), qr/HIT/, 'x-accel-expires duplicate'); + +} + +# with cache headers ignored, the response will be fresh + +like(get('/ignore'), qr/MISS/, 'cache headers ignored'); + +# Cache-Control is preferred over Expires + +TODO: { +local $TODO = 'not yet' unless $t->has_version('1.23.0'); + +like(get('/cache-control-after-expires'), qr/HIT/, + 'cache-control after expires'); + +} + +like(get('/cache-control-before-expires'), qr/HIT/, + 'cache-control before expires'); + +like(get('/cache-control-no-cache-after-expires'), qr/MISS/, + 'cache-control no-cache after expires'); +like(get('/cache-control-no-cache-before-expires'), qr/MISS/, + 'cache-control no-cache before expires'); + +# X-Accel-Expires is preferred over both Cache-Control and Expires + +TODO: { +local $TODO = 'not yet' unless $t->has_version('1.23.0'); + +like(get('/x-accel-expires-after'), qr/HIT/, 'x-accel-expires after'); + +} + +like(get('/x-accel-expires-before'), qr/HIT/, 'x-accel-expires before'); + +like(get('/x-accel-expires-0-after'), qr/MISS/, 'x-accel-expires 0 after'); +like(get('/x-accel-expires-0-before'), qr/MISS/, 'x-accel-expires 0 before'); + +# "Cache-Control: no-cache" disables caching, no matter of "max-age". + +like(get('/cache-control-no-cache-one'), qr/MISS/, + 'cache-control no-cache'); +like(get('/cache-control-no-cache-multi'), qr/MISS/, + 'cache-control no-cache multi line'); + +# Set-Cookie is considered when caching with X-Accel-Expires + +like(get('/set-cookie'), qr/MISS/, 'set-cookie not cached'); + +# Cache-Control extensions are preserved with X-Accel-Expires + +like(get('/extension-before-x-accel-expires'), + qr/STALE/, 'cache-control extensions before x-accel-expires'); + +TODO: { +local $TODO = 'not yet' unless $t->has_version('1.23.0'); + +like(get('/extension-after-x-accel-expires'), + qr/STALE/, 'cache-control extensions after x-accel-expires'); + +} + +############################################################################### + +sub get { + my ($uri) = @_; + http_get($uri); + http_get($uri); +} + +############################################################################### -- Sergey Kandaurov From xeioex at nginx.com Sat Jun 11 07:16:51 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Sat, 11 Jun 2022 07:16:51 +0000 Subject: [njs] Removed unused argument from njs_function_lambda_call(). Message-ID: details: https://hg.nginx.org/njs/rev/860d74320bb4 branches: changeset: 1882:860d74320bb4 user: Dmitry Volyntsev date: Sat Jun 11 00:15:20 2022 -0700 description: Removed unused argument from njs_function_lambda_call(). diffstat: src/njs_async.c | 2 +- src/njs_function.c | 6 +++--- src/njs_function.h | 3 +-- 3 files changed, 5 insertions(+), 6 deletions(-) diffs (55 lines): diff -r 0b75123c6ea4 -r 860d74320bb4 src/njs_async.c --- a/src/njs_async.c Wed Jun 08 22:58:34 2022 -0700 +++ b/src/njs_async.c Sat Jun 11 00:15:20 2022 -0700 @@ -29,7 +29,7 @@ njs_async_function_frame_invoke(njs_vm_t return NJS_ERROR; } - ret = njs_function_lambda_call(vm, capability, NULL); + ret = njs_function_lambda_call(vm, capability); if (ret == NJS_OK) { ret = njs_function_call(vm, njs_function(&capability->resolve), diff -r 0b75123c6ea4 -r 860d74320bb4 src/njs_function.c --- a/src/njs_function.c Wed Jun 08 22:58:34 2022 -0700 +++ b/src/njs_function.c Sat Jun 11 00:15:20 2022 -0700 @@ -600,7 +600,7 @@ njs_function_call2(njs_vm_t *vm, njs_fun njs_int_t -njs_function_lambda_call(njs_vm_t *vm, void *promise_cap, void *async_ctx) +njs_function_lambda_call(njs_vm_t *vm, void *promise_cap) { uint32_t n; njs_int_t ret; @@ -690,7 +690,7 @@ njs_function_lambda_call(njs_vm_t *vm, v } } - ret = njs_vmcode_interpreter(vm, lambda->start, promise_cap, async_ctx); + ret = njs_vmcode_interpreter(vm, lambda->start, promise_cap, NULL); /* Restore current level. */ vm->levels[NJS_LEVEL_LOCAL] = cur_local; @@ -777,7 +777,7 @@ njs_function_frame_invoke(njs_vm_t *vm, return njs_function_native_call(vm); } else { - return njs_function_lambda_call(vm, NULL, NULL); + return njs_function_lambda_call(vm, NULL); } } diff -r 0b75123c6ea4 -r 860d74320bb4 src/njs_function.h --- a/src/njs_function.h Wed Jun 08 22:58:34 2022 -0700 +++ b/src/njs_function.h Sat Jun 11 00:15:20 2022 -0700 @@ -110,8 +110,7 @@ njs_int_t njs_function_lambda_frame(njs_ njs_int_t njs_function_call2(njs_vm_t *vm, njs_function_t *function, const njs_value_t *this, const njs_value_t *args, njs_uint_t nargs, njs_value_t *retval, njs_bool_t ctor); -njs_int_t njs_function_lambda_call(njs_vm_t *vm, void *promise_cap, - void *async_ctx); +njs_int_t njs_function_lambda_call(njs_vm_t *vm, void *promise_cap); njs_int_t njs_function_native_call(njs_vm_t *vm); njs_native_frame_t *njs_function_frame_alloc(njs_vm_t *vm, size_t size); void njs_function_frame_free(njs_vm_t *vm, njs_native_frame_t *frame); From xeioex at nginx.com Sat Jun 11 07:16:53 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Sat, 11 Jun 2022 07:16:53 +0000 Subject: [njs] Fixed freeing of array created using njs_arr_create(). Message-ID: details: https://hg.nginx.org/njs/rev/aa4079d36ec3 branches: changeset: 1883:aa4079d36ec3 user: Dmitry Volyntsev date: Sat Jun 11 00:15:27 2022 -0700 description: Fixed freeing of array created using njs_arr_create(). diffstat: src/njs_arr.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 860d74320bb4 -r aa4079d36ec3 src/njs_arr.c --- a/src/njs_arr.c Sat Jun 11 00:15:20 2022 -0700 +++ b/src/njs_arr.c Sat Jun 11 00:15:27 2022 -0700 @@ -23,7 +23,7 @@ njs_arr_create(njs_mp_t *mp, njs_uint_t arr->item_size = size; arr->available = n; arr->pointer = 1; - arr->separate = 1; + arr->separate = 0; arr->mem_pool = mp; return arr; From xeioex at nginx.com Sat Jun 11 07:16:55 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Sat, 11 Jun 2022 07:16:55 +0000 Subject: [njs] Fixed njs_array_expand() for empty arrays. Message-ID: details: https://hg.nginx.org/njs/rev/46ec9152a054 branches: changeset: 1884:46ec9152a054 user: Dmitry Volyntsev date: Sat Jun 11 00:15:28 2022 -0700 description: Fixed njs_array_expand() for empty arrays. diffstat: src/njs_array.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diffs (14 lines): diff -r aa4079d36ec3 -r 46ec9152a054 src/njs_array.c --- a/src/njs_array.c Sat Jun 11 00:15:27 2022 -0700 +++ b/src/njs_array.c Sat Jun 11 00:15:28 2022 -0700 @@ -408,7 +408,9 @@ njs_array_expand(njs_vm_t *vm, njs_array array->start = start; - njs_mp_free(vm->mem_pool, old); + if (old != NULL) { + njs_mp_free(vm->mem_pool, old); + } return NJS_OK; From xeioex at nginx.com Sat Jun 11 07:16:57 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Sat, 11 Jun 2022 07:16:57 +0000 Subject: [njs] Removed njs_buffer_decode_destroy(). Message-ID: details: https://hg.nginx.org/njs/rev/cb0b5507ffe6 branches: changeset: 1885:cb0b5507ffe6 user: Dmitry Volyntsev date: Sat Jun 11 00:15:28 2022 -0700 description: Removed njs_buffer_decode_destroy(). diffstat: src/njs_buffer.c | 27 +-------------------------- 1 files changed, 1 insertions(+), 26 deletions(-) diffs (72 lines): diff -r 46ec9152a054 -r cb0b5507ffe6 src/njs_buffer.c --- a/src/njs_buffer.c Sat Jun 11 00:15:28 2022 -0700 +++ b/src/njs_buffer.c Sat Jun 11 00:15:28 2022 -0700 @@ -76,8 +76,6 @@ static njs_int_t njs_buffer_fill_string( static njs_int_t njs_buffer_fill_typed_array(njs_vm_t *vm, const njs_value_t *value, njs_typed_array_t *array, uint8_t *start, uint8_t *end); -static void njs_buffer_decode_destroy(njs_vm_t *vm, const njs_value_t *source, - njs_value_t *target); static njs_int_t njs_buffer(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, @@ -528,8 +526,6 @@ njs_buffer_from_string(njs_vm_t *vm, njs memcpy(njs_typed_array_buffer(buffer)->u.u8, str.start, str.length); - njs_buffer_decode_destroy(vm, value, &dst); - njs_set_typed_array(&vm->retval, buffer); return NJS_OK; @@ -1589,8 +1585,6 @@ njs_buffer_write_string(njs_vm_t *vm, nj done: - njs_buffer_decode_destroy(vm, value, &dst); - njs_set_number(&vm->retval, length); return NJS_OK; @@ -1745,7 +1739,7 @@ njs_buffer_fill_string(njs_vm_t *vm, con if (str.length == 0) { memset(start, 0, end - start); - goto done; + return NJS_OK; } while (start < end) { @@ -1753,10 +1747,6 @@ njs_buffer_fill_string(njs_vm_t *vm, con start = njs_cpymem(start, str.start, n); } -done: - - njs_buffer_decode_destroy(vm, value, &dst); - return NJS_OK; } @@ -2345,21 +2335,6 @@ njs_buffer_decode_string(njs_vm_t *vm, c } -static void -njs_buffer_decode_destroy(njs_vm_t *vm, const njs_value_t *source, - njs_value_t *target) -{ - njs_str_t src, trg; - - njs_string_get(source, &src); - njs_string_get(target, &trg); - - if (src.start != trg.start) { - njs_mp_free(vm->mem_pool, trg.start); - } -} - - static const njs_object_prop_t njs_buffer_prototype_properties[] = { { From xeioex at nginx.com Sat Jun 11 07:16:58 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Sat, 11 Jun 2022 07:16:58 +0000 Subject: [njs] Fixed memory freeing in njs_vm_compile(). Message-ID: details: https://hg.nginx.org/njs/rev/e38e175c0063 branches: changeset: 1886:e38e175c0063 user: Dmitry Volyntsev date: Sat Jun 11 00:15:29 2022 -0700 description: Fixed memory freeing in njs_vm_compile(). diffstat: src/njs_vm.c | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diffs (12 lines): diff -r cb0b5507ffe6 -r e38e175c0063 src/njs_vm.c --- a/src/njs_vm.c Sat Jun 11 00:15:28 2022 -0700 +++ b/src/njs_vm.c Sat Jun 11 00:15:29 2022 -0700 @@ -214,8 +214,6 @@ njs_vm_compile(njs_vm_t *vm, u_char **st *new++ = *global++; } - - njs_mp_free(vm->mem_pool, global); } } From xeioex at nginx.com Sat Jun 11 07:17:00 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Sat, 11 Jun 2022 07:17:00 +0000 Subject: [njs] Catching invalid njs_mp_free() calls. Message-ID: details: https://hg.nginx.org/njs/rev/77758def397a branches: changeset: 1887:77758def397a user: Dmitry Volyntsev date: Sat Jun 11 00:15:30 2022 -0700 description: Catching invalid njs_mp_free() calls. diffstat: auto/options | 4 ++++ src/njs_assert.h | 10 ++++++++++ src/njs_mp.c | 6 ++---- 3 files changed, 16 insertions(+), 4 deletions(-) diffs (54 lines): diff -r e38e175c0063 -r 77758def397a auto/options --- a/auto/options Sat Jun 11 00:15:29 2022 -0700 +++ b/auto/options Sat Jun 11 00:15:30 2022 -0700 @@ -62,3 +62,7 @@ do NJS_CONFIGURE_OPTIONS="$NJS_CONFIGURE_OPTIONS $njs_opt" done + +if [ "$NJS_DEBUG_MEMORY" = "YES" ]; then + NJS_DEBUG=YES +fi diff -r e38e175c0063 -r 77758def397a src/njs_assert.h --- a/src/njs_assert.h Sat Jun 11 00:15:29 2022 -0700 +++ b/src/njs_assert.h Sat Jun 11 00:15:30 2022 -0700 @@ -18,9 +18,19 @@ } \ } while (0) +#define njs_assert_msg(condition, fmt, ...) \ + do { \ + if (!(condition)) { \ + njs_stderror(fmt, ##__VA_ARGS__); \ + njs_stderror(" at %s:%d\n", __FILE__, __LINE__); \ + abort(); \ + } \ + } while (0) + #else #define njs_assert(condition) (void) (condition) +#define njs_assert_msg(condition, fmt, ...) (void) (condition) #endif diff -r e38e175c0063 -r 77758def397a src/njs_mp.c --- a/src/njs_mp.c Sat Jun 11 00:15:29 2022 -0700 +++ b/src/njs_mp.c Sat Jun 11 00:15:30 2022 -0700 @@ -678,14 +678,12 @@ njs_mp_free(njs_mp_t *mp, void *p) return; } else { - err = "freed pointer points to middle of block: %p\n"; + njs_assert_msg(0, "freed pointer points to middle of blk: %p\n", p); } } else { - err = "freed pointer is out of mp: %p\n"; + njs_assert_msg(0, "freed pointer is out of mp: %p\n", p); } - - njs_debug_alloc(err, p); } From xeioex at nginx.com Sat Jun 11 07:17:02 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Sat, 11 Jun 2022 07:17:02 +0000 Subject: [njs] Fixed njs_array_convert_to_slow_array(). Message-ID: details: https://hg.nginx.org/njs/rev/c6522ba4bd53 branches: changeset: 1888:c6522ba4bd53 user: Dmitry Volyntsev date: Sat Jun 11 00:15:49 2022 -0700 description: Fixed njs_array_convert_to_slow_array(). Previously, the function might free invalid pointer, as array->start is not always points to the beginning of allocated memory block. This closes #540 issue on Github. diffstat: src/njs_array.c | 2 +- src/test/njs_unit_test.c | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletions(-) diffs (28 lines): diff -r 77758def397a -r c6522ba4bd53 src/njs_array.c --- a/src/njs_array.c Sat Jun 11 00:15:30 2022 -0700 +++ b/src/njs_array.c Sat Jun 11 00:15:49 2022 -0700 @@ -165,7 +165,7 @@ njs_array_convert_to_slow_array(njs_vm_t /* GC: release value. */ - njs_mp_free(vm->mem_pool, array->start); + njs_mp_free(vm->mem_pool, array->data); array->start = NULL; return NJS_OK; diff -r 77758def397a -r c6522ba4bd53 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Sat Jun 11 00:15:30 2022 -0700 +++ b/src/test/njs_unit_test.c Sat Jun 11 00:15:49 2022 -0700 @@ -4743,6 +4743,12 @@ static njs_unit_test_t njs_test[] = "a.shift(); a"), njs_str("2,3") }, + { njs_str("var arr = [1,2];" + "arr.shift();" + "arr[2**20] = 3;" + "arr[2**20]"), + njs_str("3") }, + { njs_str("var a = []; a.splice()"), njs_str("") }, From mdounin at mdounin.ru Sat Jun 11 22:52:50 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 12 Jun 2022 01:52:50 +0300 Subject: [PATCH] Upstream: prioritise Cache-Control over Expires In-Reply-To: <80E87E07-436C-48AF-AA98-DF6BC7961306@nginx.com> References: <151ef936-f5d2-b681-bb54-6476c654a3ae@cdnnow.ru> <00218a02-f3e7-d1fe-04ba-1e61049f17be@cdnnow.ru> <20220606134220.afco4q4u6ujj2s3h@Y9MQ9X2QVV> <6DC932A4-16AA-42D5-9D06-A3F599439908@nginx.com> <80E87E07-436C-48AF-AA98-DF6BC7961306@nginx.com> Message-ID: Hello! On Fri, Jun 10, 2022 at 03:37:34PM +0400, Sergey Kandaurov wrote: > > On 8 Jun 2022, at 04:57, Maxim Dounin wrote: > > > > On Tue, Jun 07, 2022 at 07:58:36PM +0400, Sergey Kandaurov wrote: [...] > > Also, it might be a good idea to make sure that various > > other factors to disable cache, such as Set-Cookie, are not > > ignored when a resource is returned with Expires in the past and > > valid Cache-Control max-age. There were multiple attempts to > > submit patches which failed to handle this properly. E.g.: > > > > location /set-cookie { > > add_header Set-Cookie foo; > > add_header Cache-Control max-age=60; > > return 204; > > } > > If I got you right, it refers to earlier patches in January > with erroneously missed Set-Cookie handling. > I reproduced this with Expires in the past and X-Accel-Expires: > > location /set-cookie { > add_header Set-Cookie foo; > add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; > add_header X-Accel-Expires 60; > return 204; > } I believe I've seen more than one patch which missed Set-Cookie (and Vary) handling and tried to simply set cacheable back to 1. Using "Cache-Control: max-age=60" might be better, as RFC explicitly says to ignore Expires if "Cache-Control: max-age" is present, and does not define X-Accel-Expires handling. While it is highly unlikely that at some point we'll decide that X-Accel-Expires should be used regardless of other factors, such as Set-Cookie and Vary, I don't see specific reasons to state the opposite in the tests. [...] > >> +my $time = time() - 1; > >> +like(get("/rev?c=max-age=60,stale-while-revalidate=60&x=\@$time"), > >> + qr/STALE/, 'Cache-Control extensions after X-Accel-Expires'); > > > > Something like: > > > > X-Accel-Expires: @1 > > Cache-Control: stale-while-revalidate=2145902155 > > > > shouldn't require any calculations (and can be placed in an > > explicit location). And the opposite order probably worth > > checking too. > > While it's better to avoid math, this one is somewhat synthetic. > Though, I have no objection in principle, good enough for tests. > > > > > Not sure if "max-age=60" is at all needed here, though probably > > it can catch some potential bugs. > > > > Since "stale-while-revalidate" is an extension, I assume that > it should be given with some base parameter, such as "max-age". > The assumption is supported by examples in RFC 5861. The "stale-while-revalidate" is an extension as it extends the Cache-Control header with an additional directive. Freshness lifetime might be determined by other means, not just by max-age directive in the same header field, see RFC 9111, "4.2.1. Calculating Freshness Lifetime" (https://datatracker.ietf.org/doc/html/rfc9111#section-4.2.1). In this particular case, X-Accel-Expires is certainly enough to provide freshness lifetime. In general, I see no reasons why even a heuristic freshness lifetime wouldn't be good enough. > This brought me a thought that we don't respect Cache-Control > preference over Expires if given with only extensions: > > Expires: Thu, 31 Dec 2037 23:55:55 GMT > Cache-Control: stale-while-revalidate=1 > > In the above, Cache-Control doesn't set/override valid_sec, > and as such, it is set with Expires on its own. That's the expected behaviour. With the current code, Expires is equivalent to the max-age time as set by "Cache-Control: max-age". This is in line with what RFC recommends, see the link above. Further, section "5.3. Expires" explicitly talks about "Cache-Control: max-age", not just Cache-Control (https://datatracker.ietf.org/doc/html/rfc9111#section-5.3). > # HG changeset patch > # User Sergey Kandaurov > # Date 1654860778 -14400 > # Fri Jun 10 15:32:58 2022 +0400 > # Node ID f8e5ab69e6edd379b2ea9e0e8f1ebbab01fd9075 > # Parent 4f238efded81e00c635c146df6ecb585fa0907ec > Tests: added proxy cache tests with upstream cache headers. > > diff --git a/proxy_cache_control.t b/proxy_cache_control.t > new file mode 100644 > --- /dev/null > +++ b/proxy_cache_control.t > @@ -0,0 +1,276 @@ > +#!/usr/bin/perl > + > +# (C) Sergey Kandaurov > +# (C) Nginx, Inc. > + > +# Tests for proxy Expires / Cache-Control / X-Accel-Expires. > + > +############################################################################### > + > +use warnings; > +use strict; > + > +use Test::More; > + > +BEGIN { use FindBin; chdir($FindBin::Bin); } > + > +use lib 'lib'; > +use Test::Nginx; > + > +############################################################################### > + > +select STDERR; $| = 1; > +select STDOUT; $| = 1; > + > +my $t = Test::Nginx->new()->has(qw/http proxy cache rewrite/)->plan(19); > + > +$t->write_file_expand('nginx.conf', <<'EOF'); > + > +%%TEST_GLOBALS%% > + > +daemon off; > + > +events { > +} > + > +http { > + %%TEST_GLOBALS_HTTP%% > + > + proxy_cache_path %%TESTDIR%%/cache keys_zone=NAME:1m; > + > + server { > + listen 127.0.0.1:8080; > + server_name localhost; > + > + add_header X-Cache-Status $upstream_cache_status; > + > + location / { > + proxy_pass http://127.0.0.1:8081; > + proxy_cache NAME; > + > + proxy_cache_background_update on; > + } > + > + location /ignore { > + proxy_pass http://127.0.0.1:8081; > + proxy_cache NAME; > + > + proxy_ignore_headers Cache-Control Expires; > + proxy_ignore_headers X-Accel-Expires; > + } > + } > + > + server { > + listen 127.0.0.1:8081; > + server_name localhost; > + > + location /expires { > + add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; > + return 204; > + } > + > + location /cache-control { > + add_header Cache-Control max-age=60; > + return 204; I tend to think that it would be better to remove extra spaces. It looks weird, especially combined with "return 204;" without extra spaces. > + } > + > + location /x-accel-expires { > + add_header X-Accel-Expires 60; > + return 204; > + } > + > + location /x-accel-expires-at { > + add_header X-Accel-Expires @60; > + return 204; > + } > + > + location /x-accel-expires-duplicate { > + add_header X-Accel-Expires 60; > + add_header X-Accel-Expires 0; > + return 204; > + } > + > + location /ignore { > + add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; > + add_header Cache-Control max-age=60; > + add_header X-Accel-Expires 60; > + return 204; > + } > + > + location /cache-control-after-expires { > + add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; > + add_header Cache-Control max-age=60; > + return 204; > + } > + > + location /cache-control-before-expires { > + add_header Cache-Control max-age=60; > + add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; > + return 204; > + } See below about before/after order. > + > + location /cache-control-no-cache-after-expires { > + add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; > + add_header Cache-Control no-cache; > + return 204; > + } > + > + location /cache-control-no-cache-before-expires { > + add_header Cache-Control no-cache; > + add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; > + return 204; > + } > + > + location /x-accel-expires-after { > + add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; > + add_header Cache-Control no-cache; > + add_header X-Accel-Expires 60; > + return 204; > + } > + > + location /x-accel-expires-before { > + add_header X-Accel-Expires 60; > + add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; > + add_header Cache-Control no-cache; > + return 204; > + } > + > + location /x-accel-expires-0-after { > + add_header Cache-Control max-age=60; > + add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; > + add_header X-Accel-Expires 0; > + return 204; > + } > + > + location /x-accel-expires-0-before { > + add_header X-Accel-Expires 0; > + add_header Cache-Control max-age=60; > + add_header Expires "Thu, 31 Dec 2037 23:55:55 GMT"; > + return 204; > + } > + > + location /cache-control-no-cache-one { > + add_header Cache-Control "no-cache, max-age=60"; > + return 204; > + } > + > + location /cache-control-no-cache-multi { > + add_header Cache-Control no-cache; > + add_header Cache-Control max-age=60; > + return 204; > + } > + > + location /extension-before-x-accel-expires { > + add_header Cache-Control > + "max-age=60, stale-while-revalidate=2145902155"; > + add_header X-Accel-Expires @1; > + return 204; See above about "max-age=". > + } > + > + location /extension-after-x-accel-expires { > + add_header X-Accel-Expires @1; > + add_header Cache-Control > + "max-age=60, stale-while-revalidate=2145902155"; > + return 204; > + } > + > + location /set-cookie { > + add_header Set-Cookie foo; > + add_header Expires "Thu, 01 Jan 1970 00:00:01 GMT"; > + add_header X-Accel-Expires 60; > + return 204; > + } > + } > +} > + > +EOF > + > +$t->run(); > + > +############################################################################### > + > +# upstream cache headers work > + > +like(get('/expires'), qr/HIT/, 'expires'); > +like(get('/cache-control'), qr/HIT/, 'cache-control'); > +like(get('/x-accel-expires'), qr/HIT/, 'x-accel-expires'); > +like(get('/x-accel-expires-at'), qr/EXPIRED/, 'x-accel-expires at'); > + > +TODO: { > +local $TODO = 'not yet' unless $t->has_version('1.23.0'); > + > +# the second header to disable cache is duplicate and ignored > + > +like(get('/x-accel-expires-duplicate'), qr/HIT/, 'x-accel-expires duplicate'); > + > +} > + > +# with cache headers ignored, the response will be fresh > + > +like(get('/ignore'), qr/MISS/, 'cache headers ignored'); > + > +# Cache-Control is preferred over Expires > + > +TODO: { > +local $TODO = 'not yet' unless $t->has_version('1.23.0'); > + > +like(get('/cache-control-after-expires'), qr/HIT/, > + 'cache-control after expires'); > + > +} > + > +like(get('/cache-control-before-expires'), qr/HIT/, > + 'cache-control before expires'); It might make sense to use the opposite order for before/after, to make sure TODO tests are after corresponding tests which pass in the previous versions. YMMV though. > + > +like(get('/cache-control-no-cache-after-expires'), qr/MISS/, > + 'cache-control no-cache after expires'); > +like(get('/cache-control-no-cache-before-expires'), qr/MISS/, > + 'cache-control no-cache before expires'); > + > +# X-Accel-Expires is preferred over both Cache-Control and Expires > + > +TODO: { > +local $TODO = 'not yet' unless $t->has_version('1.23.0'); > + > +like(get('/x-accel-expires-after'), qr/HIT/, 'x-accel-expires after'); > + > +} > + > +like(get('/x-accel-expires-before'), qr/HIT/, 'x-accel-expires before'); > + > +like(get('/x-accel-expires-0-after'), qr/MISS/, 'x-accel-expires 0 after'); > +like(get('/x-accel-expires-0-before'), qr/MISS/, 'x-accel-expires 0 before'); > + > +# "Cache-Control: no-cache" disables caching, no matter of "max-age". Style nit: extra dot. > + > +like(get('/cache-control-no-cache-one'), qr/MISS/, > + 'cache-control no-cache'); > +like(get('/cache-control-no-cache-multi'), qr/MISS/, > + 'cache-control no-cache multi line'); > + > +# Set-Cookie is considered when caching with X-Accel-Expires > + > +like(get('/set-cookie'), qr/MISS/, 'set-cookie not cached'); > + > +# Cache-Control extensions are preserved with X-Accel-Expires > + > +like(get('/extension-before-x-accel-expires'), > + qr/STALE/, 'cache-control extensions before x-accel-expires'); > + > +TODO: { > +local $TODO = 'not yet' unless $t->has_version('1.23.0'); > + > +like(get('/extension-after-x-accel-expires'), > + qr/STALE/, 'cache-control extensions after x-accel-expires'); > + > +} > + > +############################################################################### > + > +sub get { > + my ($uri) = @_; > + http_get($uri); > + http_get($uri); > +} > + > +############################################################################### Overall looks good, see minor comments above. -- Maxim Dounin http://mdounin.ru/ From thresh at nginx.com Mon Jun 13 09:09:35 2022 From: thresh at nginx.com (=?iso-8859-1?q?Konstantin_Pavlov?=) Date: Mon, 13 Jun 2022 13:09:35 +0400 Subject: [PATCH] Linux packages: updated the supported architectures for RHEL Message-ID: <40d40af45ac85339025b.1655111375@chr-l-00029761.olympus.f5net.com> # HG changeset patch # User Konstantin Pavlov # Date 1655109191 -14400 # Mon Jun 13 12:33:11 2022 +0400 # Node ID 40d40af45ac85339025b171c53033665a3632ee0 # Parent 5eb32d26a584e0364950390fa570595f0a2f772d Linux packages: updated the supported architectures for RHEL. diff -r 5eb32d26a584 -r 40d40af45ac8 xml/en/linux_packages.xml --- a/xml/en/linux_packages.xml Fri Jun 03 20:16:00 2022 +0400 +++ b/xml/en/linux_packages.xml Mon Jun 13 12:33:11 2022 +0400 @@ -7,7 +7,7 @@
+ rev="75">
@@ -28,7 +28,7 @@ versions: 7.4+ -x86_64, ppc64le, aarch64/arm64 +x86_64, aarch64/arm64 @@ -38,7 +38,7 @@ versions: 9.x -x86_64, aarch64/arm64 +x86_64, aarch64/arm64, s390x diff -r 5eb32d26a584 -r 40d40af45ac8 xml/ru/linux_packages.xml --- a/xml/ru/linux_packages.xml Fri Jun 03 20:16:00 2022 +0400 +++ b/xml/ru/linux_packages.xml Mon Jun 13 12:33:11 2022 +0400 @@ -7,7 +7,7 @@
+ rev="75">
@@ -28,7 +28,7 @@ 7.4+ -x86_64, ppc64le, aarch64/arm64 +x86_64, aarch64/arm64 @@ -38,7 +38,7 @@ 9.x -x86_64, aarch64/arm64 +x86_64, aarch64/arm64, s390x From maxim at nginx.com Mon Jun 13 09:16:39 2022 From: maxim at nginx.com (Maxim Konovalov) Date: Mon, 13 Jun 2022 13:16:39 +0400 Subject: [PATCH] Linux packages: updated the supported architectures for RHEL In-Reply-To: <40d40af45ac85339025b.1655111375@chr-l-00029761.olympus.f5net.com> References: <40d40af45ac85339025b.1655111375@chr-l-00029761.olympus.f5net.com> Message-ID: <9516f607-5734-1d6e-1be5-24894644ef5f@nginx.com> On 13.06.2022 13:09, Konstantin Pavlov wrote: > # HG changeset patch > # User Konstantin Pavlov > # Date 1655109191 -14400 > # Mon Jun 13 12:33:11 2022 +0400 > # Node ID 40d40af45ac85339025b171c53033665a3632ee0 > # Parent 5eb32d26a584e0364950390fa570595f0a2f772d > Linux packages: updated the supported architectures for RHEL. > [...] Looks good. -- Maxim Konovalov From pluknet at nginx.com Mon Jun 13 11:35:15 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 13 Jun 2022 15:35:15 +0400 Subject: [PATCH] Upstream: prioritise Cache-Control over Expires In-Reply-To: References: <151ef936-f5d2-b681-bb54-6476c654a3ae@cdnnow.ru> <00218a02-f3e7-d1fe-04ba-1e61049f17be@cdnnow.ru> <20220606134220.afco4q4u6ujj2s3h@Y9MQ9X2QVV> <6DC932A4-16AA-42D5-9D06-A3F599439908@nginx.com> <80E87E07-436C-48AF-AA98-DF6BC7961306@nginx.com> Message-ID: <68C50153-138A-4B1A-A1D4-902E86E2F114@nginx.com> > On 12 Jun 2022, at 02:52, Maxim Dounin wrote: > > Hello! > > On Fri, Jun 10, 2022 at 03:37:34PM +0400, Sergey Kandaurov wrote: > > [...] > >> # HG changeset patch >> # User Sergey Kandaurov >> # Date 1654860778 -14400 >> # Fri Jun 10 15:32:58 2022 +0400 >> # Node ID f8e5ab69e6edd379b2ea9e0e8f1ebbab01fd9075 >> # Parent 4f238efded81e00c635c146df6ecb585fa0907ec >> Tests: added proxy cache tests with upstream cache headers. >> [...] > > Overall looks good, see minor comments above. Pushed with the above adjustments, thanks. -- Sergey Kandaurov From pluknet at nginx.com Mon Jun 13 17:04:24 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 13 Jun 2022 21:04:24 +0400 Subject: [PATCH 06 of 20] Reworked multi headers to use linked lists In-Reply-To: <50fe52f516ff9c148aa9.1650493126@vm-bsd.mdounin.ru> References: <50fe52f516ff9c148aa9.1650493126@vm-bsd.mdounin.ru> Message-ID: > On 21 Apr 2022, at 02:18, Maxim Dounin wrote: > > # HG changeset patch > # User Maxim Dounin > # Date 1650492323 -10800 > # Thu Apr 21 01:05:23 2022 +0300 > # Node ID 50fe52f516ff9c148aa9e7dfcc1c31cc6a4929ae > # Parent 2ea48b5e4643a818cd81179f040f0b36be9050d6 > Reworked multi headers to use linked lists. > > Multi headers are now using linked lists instead of arrays. Notably, > the following fields were changed: r->headers_in.cookies (renamed > to r->headers_in.cookie), r->headers_in.x_forwarded_for, > r->headers_out.cache_control, r->headers_out.link, u->headers_in.cache_control > u->headers_in.cookies (renamed to u->headers_in.set_cookie). > > The r->headers_in.cookies and u->headers_in.cookies fields were renamed > to r->headers_in.cookie and u->headers_in.set_cookie to match header names. > > The ngx_http_parse_multi_header_lines() and ngx_http_parse_set_cookie_lines() > functions were changed accordingly. > > With this change, multi headers are now essentially equivalent to normal > headers, and following changes will further make them equivalent. > > [...] > > diff --git a/src/http/modules/perl/nginx.xs b/src/http/modules/perl/nginx.xs > --- a/src/http/modules/perl/nginx.xs > +++ b/src/http/modules/perl/nginx.xs > @@ -302,7 +302,7 @@ header_in(r, key) > > if (hh) { > > - if (hh->offset == offsetof(ngx_http_headers_in_t, cookies)) { > + if (hh->offset == offsetof(ngx_http_headers_in_t, cookie)) { > sep = ';'; > goto multi; > } > @@ -327,17 +327,13 @@ header_in(r, key) > > /* Cookie, X-Forwarded-For */ > > - a = (ngx_array_t *) ((char *) &r->headers_in + hh->offset); > + ph = (ngx_table_elt_t **) ((char *) &r->headers_in + hh->offset); > > - n = a->nelts; > - > - if (n == 0) { > + if (*ph == NULL) { > XSRETURN_UNDEF; > } > > - ph = a->elts; > - > - if (n == 1) { > + if ((*ph)->next == NULL) { > ngx_http_perl_set_targ((*ph)->value.data, (*ph)->value.len); > > goto done; > @@ -345,8 +341,8 @@ header_in(r, key) > > size = - (ssize_t) (sizeof("; ") - 1); > > - for (i = 0; i < n; i++) { > - size += ph[i]->value.len + sizeof("; ") - 1; > + for (h = *ph; h; h = h->next) { > + size += h->value.len + sizeof("; ") - 1; > } > > value = ngx_pnalloc(r->pool, size); > @@ -357,10 +353,10 @@ header_in(r, key) > > p = value; > > - for (i = 0; /* void */ ; i++) { > - p = ngx_copy(p, ph[i]->value.data, ph[i]->value.len); > + for (h = *ph; h; h = h->next) { > + p = ngx_copy(p, h->value.data, h->value.len); > > - if (i == n - 1) { > + if (h->next == NULL) { > break; > } > [...] A follow-up I would like to commit. It is caught on distributions that have -Wall in Perl's config_args: nginx.xs:273:33: warning: unused variable ‘a’ [-Wunused-variable] nginx.xs:272:36: warning: unused variable ‘n’ [-Wunused-variable] # HG changeset patch # User Sergey Kandaurov # Date 1655137187 -14400 # Mon Jun 13 20:19:47 2022 +0400 # Node ID f1d335894234ea518f4b6d7064fc3aeba89706cc # Parent aa28c802409fb7a74e3323195f859d09fe73cd6d Perl: removed unused variables, forgotten in ef6a3a99a81a. diff --git a/src/http/modules/perl/nginx.xs b/src/http/modules/perl/nginx.xs --- a/src/http/modules/perl/nginx.xs +++ b/src/http/modules/perl/nginx.xs @@ -269,8 +269,7 @@ header_in(r, key) u_char *p, *lowcase_key, *value, sep; STRLEN len; ssize_t size; - ngx_uint_t i, n, hash; - ngx_array_t *a; + ngx_uint_t i, hash; ngx_list_part_t *part; ngx_table_elt_t *h, *header, **ph; ngx_http_header_t *hh; -- Sergey Kandaurov From mdounin at mdounin.ru Mon Jun 13 22:51:28 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 14 Jun 2022 01:51:28 +0300 Subject: [PATCH 06 of 20] Reworked multi headers to use linked lists In-Reply-To: References: <50fe52f516ff9c148aa9.1650493126@vm-bsd.mdounin.ru> Message-ID: Hello! On Mon, Jun 13, 2022 at 09:04:24PM +0400, Sergey Kandaurov wrote: > > On 21 Apr 2022, at 02:18, Maxim Dounin wrote: > > > > # HG changeset patch > > # User Maxim Dounin > > # Date 1650492323 -10800 > > # Thu Apr 21 01:05:23 2022 +0300 > > # Node ID 50fe52f516ff9c148aa9e7dfcc1c31cc6a4929ae > > # Parent 2ea48b5e4643a818cd81179f040f0b36be9050d6 > > Reworked multi headers to use linked lists. > > > > Multi headers are now using linked lists instead of arrays. Notably, > > the following fields were changed: r->headers_in.cookies (renamed > > to r->headers_in.cookie), r->headers_in.x_forwarded_for, > > r->headers_out.cache_control, r->headers_out.link, u->headers_in.cache_control > > u->headers_in.cookies (renamed to u->headers_in.set_cookie). > > > > The r->headers_in.cookies and u->headers_in.cookies fields were renamed > > to r->headers_in.cookie and u->headers_in.set_cookie to match header names. > > > > The ngx_http_parse_multi_header_lines() and ngx_http_parse_set_cookie_lines() > > functions were changed accordingly. > > > > With this change, multi headers are now essentially equivalent to normal > > headers, and following changes will further make them equivalent. > > > > [...] > > > > diff --git a/src/http/modules/perl/nginx.xs b/src/http/modules/perl/nginx.xs > > --- a/src/http/modules/perl/nginx.xs > > +++ b/src/http/modules/perl/nginx.xs > > @@ -302,7 +302,7 @@ header_in(r, key) > > > > if (hh) { > > > > - if (hh->offset == offsetof(ngx_http_headers_in_t, cookies)) { > > + if (hh->offset == offsetof(ngx_http_headers_in_t, cookie)) { > > sep = ';'; > > goto multi; > > } > > @@ -327,17 +327,13 @@ header_in(r, key) > > > > /* Cookie, X-Forwarded-For */ > > > > - a = (ngx_array_t *) ((char *) &r->headers_in + hh->offset); > > + ph = (ngx_table_elt_t **) ((char *) &r->headers_in + hh->offset); > > > > - n = a->nelts; > > - > > - if (n == 0) { > > + if (*ph == NULL) { > > XSRETURN_UNDEF; > > } > > > > - ph = a->elts; > > - > > - if (n == 1) { > > + if ((*ph)->next == NULL) { > > ngx_http_perl_set_targ((*ph)->value.data, (*ph)->value.len); > > > > goto done; > > @@ -345,8 +341,8 @@ header_in(r, key) > > > > size = - (ssize_t) (sizeof("; ") - 1); > > > > - for (i = 0; i < n; i++) { > > - size += ph[i]->value.len + sizeof("; ") - 1; > > + for (h = *ph; h; h = h->next) { > > + size += h->value.len + sizeof("; ") - 1; > > } > > > > value = ngx_pnalloc(r->pool, size); > > @@ -357,10 +353,10 @@ header_in(r, key) > > > > p = value; > > > > - for (i = 0; /* void */ ; i++) { > > - p = ngx_copy(p, ph[i]->value.data, ph[i]->value.len); > > + for (h = *ph; h; h = h->next) { > > + p = ngx_copy(p, h->value.data, h->value.len); > > > > - if (i == n - 1) { > > + if (h->next == NULL) { > > break; > > } > > [...] > > A follow-up I would like to commit. > It is caught on distributions that have -Wall in Perl's config_args: > nginx.xs:273:33: warning: unused variable ‘a’ [-Wunused-variable] > nginx.xs:272:36: warning: unused variable ‘n’ [-Wunused-variable] > > # HG changeset patch > # User Sergey Kandaurov > # Date 1655137187 -14400 > # Mon Jun 13 20:19:47 2022 +0400 > # Node ID f1d335894234ea518f4b6d7064fc3aeba89706cc > # Parent aa28c802409fb7a74e3323195f859d09fe73cd6d > Perl: removed unused variables, forgotten in ef6a3a99a81a. > > diff --git a/src/http/modules/perl/nginx.xs b/src/http/modules/perl/nginx.xs > --- a/src/http/modules/perl/nginx.xs > +++ b/src/http/modules/perl/nginx.xs > @@ -269,8 +269,7 @@ header_in(r, key) > u_char *p, *lowcase_key, *value, sep; > STRLEN len; > ssize_t size; > - ngx_uint_t i, n, hash; > - ngx_array_t *a; > + ngx_uint_t i, hash; > ngx_list_part_t *part; > ngx_table_elt_t *h, *header, **ph; > ngx_http_header_t *hh; Looks good. -- Maxim Dounin http://mdounin.ru/ From pluknet at nginx.com Tue Jun 14 06:41:16 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 14 Jun 2022 06:41:16 +0000 Subject: [nginx] Perl: removed unused variables, forgotten in ef6a3a99a81a. Message-ID: details: https://hg.nginx.org/nginx/rev/861f076eab72 branches: changeset: 8046:861f076eab72 user: Sergey Kandaurov date: Tue Jun 14 10:39:58 2022 +0400 description: Perl: removed unused variables, forgotten in ef6a3a99a81a. diffstat: src/http/modules/perl/nginx.xs | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diffs (13 lines): diff -r aa28c802409f -r 861f076eab72 src/http/modules/perl/nginx.xs --- a/src/http/modules/perl/nginx.xs Wed Jun 01 20:17:23 2022 -0700 +++ b/src/http/modules/perl/nginx.xs Tue Jun 14 10:39:58 2022 +0400 @@ -269,8 +269,7 @@ header_in(r, key) u_char *p, *lowcase_key, *value, sep; STRLEN len; ssize_t size; - ngx_uint_t i, n, hash; - ngx_array_t *a; + ngx_uint_t i, hash; ngx_list_part_t *part; ngx_table_elt_t *h, *header, **ph; ngx_http_header_t *hh; From arut at nginx.com Tue Jun 14 13:59:39 2022 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 14 Jun 2022 17:59:39 +0400 Subject: [PATCH 1 of 3] QUIC: reusable and idle modes for main connection In-Reply-To: References: <617ec472690620cc75c4.1654177965@arut-laptop> Message-ID: <20220614135939.yop5v2niliptpy5y@N00W24XTQX> On Mon, Jun 06, 2022 at 02:34:16PM +0400, Sergey Kandaurov wrote: > > > On 2 Jun 2022, at 17:52, Roman Arutyunyan wrote: > > > > # HG changeset patch > > # User Roman Arutyunyan > > # Date 1652852691 -14400 > > # Wed May 18 09:44:51 2022 +0400 > > # Branch quic > > # Node ID 617ec472690620cc75c473f97555138a4bc7d38b > > # Parent a231f30606317a61efa3005fda9b43eb91d148d9 > > QUIC: reusable and idle modes for main connection. > > > > The modes are controlled by application layer. For HTTP/3, they are enabled > > when there are no active request streams. > > > > diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c > > --- a/src/event/quic/ngx_event_quic.c > > +++ b/src/event/quic/ngx_event_quic.c > > @@ -414,8 +414,8 @@ ngx_quic_input_handler(ngx_event_t *rev) > > } > > > > if (c->close) { > > - qc->error_reason = "graceful shutdown"; > > - ngx_quic_close_connection(c, NGX_OK); > > + qc->error = NGX_QUIC_ERR_NO_ERROR; > > + ngx_quic_close_connection(c, NGX_ERROR); > > Now a valid connection is closed with INTERNAL_ERROR on graceful shutdown. > What's the real intention? The intention is to close the QUIC connection while in this call. Closing it with NGX_OK will delay the actual closure. Closing with NGX_DONE will do a silent closure (no CC sent). The only option left is NGX_ERROR. Sadly, ngx_quic_close_connection() treats NGX_QUIC_ERR_NO_ERROR as a missing error code and indeed changes it to INTERNAL_ERROR. I think we need another code for a missing error. diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c --- a/src/event/quic/ngx_event_quic.c +++ b/src/event/quic/ngx_event_quic.c @@ -317,6 +317,8 @@ ngx_quic_new_connection(ngx_connection_t qc->congestion.ssthresh = (size_t) -1; qc->congestion.recovery_start = ngx_current_msec; + qc->error = (ngx_uint_t) -1; + if (pkt->validated && pkt->retried) { qc->tp.retry_scid.len = pkt->dcid.len; qc->tp.retry_scid.data = ngx_pstrdup(c->pool, &pkt->dcid); @@ -523,7 +525,7 @@ ngx_quic_close_connection(ngx_connection qc->error = NGX_QUIC_ERR_NO_ERROR; } else { - if (qc->error == 0 && !qc->error_app) { + if (qc->error == (ngx_uint_t) -1 && !qc->error_app) { qc->error = NGX_QUIC_ERR_INTERNAL_ERROR; } > > return; > > } > > > > diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c > > --- a/src/event/quic/ngx_event_quic_streams.c > > +++ b/src/event/quic/ngx_event_quic_streams.c > > @@ -161,6 +161,7 @@ ngx_quic_close_streams(ngx_connection_t > > ngx_pool_t *pool; > > ngx_queue_t *q; > > ngx_rbtree_t *tree; > > + ngx_connection_t *sc; > > ngx_rbtree_node_t *node; > > ngx_quic_stream_t *qs; > > > > @@ -185,24 +186,41 @@ ngx_quic_close_streams(ngx_connection_t > > return NGX_OK; > > } > > > > -#if (NGX_DEBUG) > > - ns = 0; > > -#endif > > - > > node = ngx_rbtree_min(tree->root, tree->sentinel); > > > > while (node) { > > qs = (ngx_quic_stream_t *) node; > > node = ngx_rbtree_next(tree, node); > > + sc = qs->connection; > > + > > + if (sc == NULL) { > > + ngx_quic_close_stream(qs); > > + continue; > > + } > > + > > + if (c->close || sc->idle || sc->reusable) { > > I'm not sure how the sc->idle test is a worthwhile > as it seems never be set on HTTP/3 stream connections. > > Originally, c->idle was used to close HTTP/1.x keep-alive > connections, and in HTTP/2 it is always set (see below). > > Testing sc->reusable here is a moot point and looks like > an abuse of what when short on worker connections. > For tracking active streams, HTTP/2 uses h2c->processing. > If implemented same way in HTTP/3, sc->reusable becomes > unnecessary as well (not sure if that's possible). QUIC layer is not (and should never be) aware of the application protocol. Here we can only rely on general conventions for connection flags. c->idle and c->reusable differ in degrees of gracefullness of shutting down the connection. c->reusable is a less graceful version of c->idle. But in both cases the connection can be quickly terminated by setting c->close and calling read handler, which is exactly what we want in this funtcion. > > + sc->close = 1; > > + sc->read->handler(sc->read); > > + } > > + } > > + > > + if (tree->root == tree->sentinel) { > > + return NGX_OK; > > + } > > + > > +#if (NGX_DEBUG) > > + ns = 0; > > +#endif > > + > > + for (node = ngx_rbtree_min(tree->root, tree->sentinel); > > + node; > > + node = ngx_rbtree_next(tree, node)) > > + { > > + qs = (ngx_quic_stream_t *) node; > > > > qs->recv_state = NGX_QUIC_STREAM_RECV_RESET_RECVD; > > qs->send_state = NGX_QUIC_STREAM_SEND_RESET_SENT; > > > > - if (qs->connection == NULL) { > > - ngx_quic_close_stream(qs); > > - continue; > > - } > > - > > ngx_quic_set_event(qs->connection->read); > > ngx_quic_set_event(qs->connection->write); > > > > @@ -587,6 +605,7 @@ ngx_quic_create_stream(ngx_connection_t > > { > > ngx_log_t *log; > > ngx_pool_t *pool; > > + ngx_uint_t reusable; > > ngx_queue_t *q; > > ngx_connection_t *sc; > > ngx_quic_stream_t *qs; > > @@ -639,7 +658,13 @@ ngx_quic_create_stream(ngx_connection_t > > *log = *c->log; > > pool->log = log; > > > > + reusable = c->reusable; > > + ngx_reusable_connection(c, 0); > > + > > sc = ngx_get_connection(c->fd, log); > > + > > + ngx_reusable_connection(c, reusable); > > + > > if (sc == NULL) { > > ngx_destroy_pool(pool); > > ngx_queue_insert_tail(&qc->streams.free, &qs->queue); > > diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c > > --- a/src/http/v3/ngx_http_v3_request.c > > +++ b/src/http/v3/ngx_http_v3_request.c > > @@ -210,6 +210,11 @@ ngx_http_v3_init_request_stream(ngx_conn > > > > h3c = ngx_http_v3_get_session(c); > > > > + if (!h3c->goaway && (ngx_terminate || ngx_exiting)) { > > + h3c->goaway = 1; > > + (void) ngx_http_v3_send_goaway(c, c->quic->id); > > + } > > + > > if (h3c->goaway) { > > c->close = 1; > > ngx_http_close_connection(c); > > @@ -258,7 +263,7 @@ ngx_http_v3_wait_request_handler(ngx_eve > > size_t size; > > ssize_t n; > > ngx_buf_t *b; > > - ngx_connection_t *c; > > + ngx_connection_t *c, *pc; > > ngx_pool_cleanup_t *cln; > > ngx_http_request_t *r; > > ngx_http_connection_t *hc; > > @@ -385,6 +390,10 @@ ngx_http_v3_wait_request_handler(ngx_eve > > h3c = ngx_http_v3_get_session(c); > > h3c->nrequests++; > > > > + pc = c->quic->parent; > > + pc->idle = 0; > > + ngx_reusable_connection(pc, 0); > > + > > if (h3c->keepalive.timer_set) { > > ngx_del_timer(&h3c->keepalive); > > } > > @@ -430,7 +439,7 @@ ngx_http_v3_cleanup_request(void *data) > > { > > ngx_http_request_t *r = data; > > > > - ngx_connection_t *c; > > + ngx_connection_t *c, *pc; > > ngx_http_v3_session_t *h3c; > > ngx_http_core_loc_conf_t *clcf; > > > > @@ -443,6 +452,16 @@ ngx_http_v3_cleanup_request(void *data) > > h3c = ngx_http_v3_get_session(c); > > > > if (--h3c->nrequests == 0) { > > + pc = c->quic->parent; > > + > > + if (ngx_terminate || ngx_exiting) { > > + ngx_quic_finalize_connection(pc, NGX_HTTP_V3_ERR_NO_ERROR, NULL); > > + return; > > + } > > + > > + pc->idle = 1; > > + ngx_reusable_connection(pc, 1); > > + > > Such way it doesn't take any action when there are requests streams. > It could send GOAWAY proactively. By the way, with the patch, > after graceful shutdown, it still allows processing of additional > new streams and rearms idle timeout, so it may never end. There's an additional check for that when a new stream is created. HTTP/3 keepalive is not rearmed for such streams and GOAWAY is sent, although it's not proactive. > HTTP/2 had set c->idle similarly before 5e95b9fb33b7. > Now it always sets c->idle and performs accordingly to the state. > Probably, HTTP/3 should be modified in somewhat the same direction. The problem with this approach is QUIC/HTTP layers separation. If we set c->idle, a graceful shutdown notification will arrive to QUIC. However, GOAWAY cannot be sent from QUIC, because it's an HTTP/3-level command. We can introduce a callback for this. As similar callback is added by the next pacth anyway. > > clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module); > > ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout); > > } > > diff --git a/src/http/v3/ngx_http_v3_uni.c b/src/http/v3/ngx_http_v3_uni.c > > --- a/src/http/v3/ngx_http_v3_uni.c > > +++ b/src/http/v3/ngx_http_v3_uni.c > > @@ -182,6 +182,11 @@ ngx_http_v3_uni_read_handler(ngx_event_t > > > > ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 read handler"); > > > > + if (c->close) { > > + ngx_http_v3_close_uni_stream(c); > > + return; > > + } > > + > > ngx_memzero(&b, sizeof(ngx_buf_t)); > > > > while (rev->ready) { > > @@ -262,6 +267,11 @@ ngx_http_v3_uni_dummy_read_handler(ngx_e > > > > ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 dummy read handler"); > > > > + if (c->close) { > > + ngx_http_v3_close_uni_stream(c); > > + return; > > + } > > + > > if (rev->ready) { > > if (c->recv(c, &ch, 1) != 0) { > > ngx_http_v3_finalize_connection(c, NGX_HTTP_V3_ERR_NO_ERROR, NULL); > > > > -- > Sergey Kandaurov > > _______________________________________________ > nginx-devel mailing list -- nginx-devel at nginx.org > To unsubscribe send an email to nginx-devel-leave at nginx.org From P.Pautov at F5.com Tue Jun 14 17:55:22 2022 From: P.Pautov at F5.com (Pavel Pautov) Date: Tue, 14 Jun 2022 17:55:22 +0000 Subject: SSL contexts reuse across locations In-Reply-To: References: Message-ID: Any comments? > -----Original Message----- > From: Pavel Pautov via nginx-devel > Sent: Tuesday, May 24, 2022 23:16 > To: nginx-devel at nginx.org; Maxim Dounin > Cc: Pavel Pautov > Subject: RE: SSL contexts reuse across locations > > EXTERNAL MAIL: nginx-devel-bounces at nginx.org > > Hi, > > Here is another iteration of patch, now it completely avoids creating SSL > contexts without SSL proxying. > Also changed some variable names and formatting in attempt to improve > readability. > Not sure about moving of merge/reuse logic into ngx_http_proxy_set_ssl(). I > assume, it could look like "ngx_http_proxy_set_ssl(cf, conf, prev, reuse_ssl)", > which seems to widen its scope too much. From tracey at archive.org Tue Jun 14 19:49:30 2022 From: tracey at archive.org (Tracey Jaquith) Date: Tue, 14 Jun 2022 12:49:30 -0700 Subject: [PATCH] Add optional "mp4_exact_start" nginx config off/on to show video between keyframes In-Reply-To: <642e8b4e-db98-97c4-dfc6-09d4512aad36@nginx.com> References: <20210628095320.px3ggmmoyjalyv5m@Romans-MacBook-Pro.local> <5F32216C-A041-454C-A73C-0E1C259E434C@archive.org> <20210930134811.epttik4joflf2qj6@Romans-MacBook-Pro.local> <20241A9E-BDF1-42D8-9848-AF628717EFE3@archive.org> <642e8b4e-db98-97c4-dfc6-09d4512aad36@nginx.com> Message-ID: Hi Maxim & (esp.) Roman, :) Sorry I didn’t get back to this — it took them until late November to get the videos up, and then I… LOL, seem to have been in forget.. procrastinate loop :p Anyway! Here’s the video (about 20 minutes): https://www.youtube.com/watch?v=yPwF2ms9MQ8 Obviously, at the time of the talk vs. the ultimate move to EDL, the solve that made it into nginx wound up being slightly different. But the talk goes over the pre-EDL solve and still is quite interesting. Demuxed is a SF-based annual A/V conference full of all the big players and super brilliant A/V engineers — so it’s a real honor to speak there (my 2nd time speaking there). It’s a single track type setup, that’s now expanded to 2 full days. To my complete surprise, my talk won the day’s Audience Award! 🎉 🥰 I’m so grateful for all the collaboration and help and work together to get this important feature in to the amazing `mp4` module for nginx. It’s been an honor to have been able to work with and contribute to nginx. As I say in my talk: “The best web server on the Planet” :) Warmly, -Tracey Jaquith > On Nov 3, 2021, at 12:48 AM, Maxim Konovalov wrote: > > Hi Tracey, > > On 03.11.2021 05:16, Tracey Jaquith wrote: > [...] >> You might find this hopefully amusing, but I gave a talk at Demuxed 2021 >> October 7th, titled: >> "30,000 fps nginx - To Russia with Love” >> while things were before the EDTS solution you all found and worked out. >> (Video should hit YouTube by end of 2021). >> > Please update us with the link then. > > Thanks! > > Maxim > > -- > Maxim Konovalov -Tracey @tracey_pooh TV Architect https://archive.org/tv -------------- next part -------------- An HTML attachment was scrubbed... URL: From rekgrpth at gmail.com Wed Jun 15 03:41:57 2022 From: rekgrpth at gmail.com (=?iso-8859-1?q?RekGRpth?=) Date: Wed, 15 Jun 2022 08:41:57 +0500 Subject: [PATCH] public upstream connect and finalize request Message-ID: <4ae4c147e3b9280e8aca.1655264517@gws.t72.ru> # HG changeset patch # User RekGRpth # Date 1655264350 -18000 # Wed Jun 15 08:39:10 2022 +0500 # Node ID 4ae4c147e3b9280e8aca76e99807bbbbf6059de5 # Parent 861f076eab72dab85bd5eaa56da770f8b10aa8ad public upstream connect and finalize request diff -r 861f076eab72 -r 4ae4c147e3b9 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Tue Jun 14 10:39:58 2022 +0400 +++ b/src/http/ngx_http_upstream.c Wed Jun 15 08:39:10 2022 +0500 @@ -35,8 +35,6 @@ static void ngx_http_upstream_wr_check_broken_connection(ngx_http_request_t *r); static void ngx_http_upstream_check_broken_connection(ngx_http_request_t *r, ngx_event_t *ev); -static void ngx_http_upstream_connect(ngx_http_request_t *r, - ngx_http_upstream_t *u); static ngx_int_t ngx_http_upstream_reinit(ngx_http_request_t *r, ngx_http_upstream_t *u); static void ngx_http_upstream_send_request(ngx_http_request_t *r, @@ -96,8 +94,6 @@ static void ngx_http_upstream_next(ngx_http_request_t *r, ngx_http_upstream_t *u, ngx_uint_t ft_type); static void ngx_http_upstream_cleanup(void *data); -static void ngx_http_upstream_finalize_request(ngx_http_request_t *r, - ngx_http_upstream_t *u, ngx_int_t rc); static ngx_int_t ngx_http_upstream_process_header_line(ngx_http_request_t *r, ngx_table_elt_t *h, ngx_uint_t offset); @@ -1504,7 +1500,7 @@ } -static void +void ngx_http_upstream_connect(ngx_http_request_t *r, ngx_http_upstream_t *u) { ngx_int_t rc; @@ -4464,7 +4460,7 @@ } -static void +void ngx_http_upstream_finalize_request(ngx_http_request_t *r, ngx_http_upstream_t *u, ngx_int_t rc) { diff -r 861f076eab72 -r 4ae4c147e3b9 src/http/ngx_http_upstream.h --- a/src/http/ngx_http_upstream.h Tue Jun 14 10:39:58 2022 +0400 +++ b/src/http/ngx_http_upstream.h Wed Jun 15 08:39:10 2022 +0500 @@ -416,6 +416,10 @@ ngx_int_t ngx_http_upstream_create(ngx_http_request_t *r); +void ngx_http_upstream_connect(ngx_http_request_t *r, + ngx_http_upstream_t *u); +void ngx_http_upstream_finalize_request(ngx_http_request_t *r, + ngx_http_upstream_t *u, ngx_int_t rc); void ngx_http_upstream_init(ngx_http_request_t *r); ngx_int_t ngx_http_upstream_non_buffered_filter_init(void *data); ngx_int_t ngx_http_upstream_non_buffered_filter(void *data, ssize_t bytes); From maxim at nginx.com Wed Jun 15 08:00:55 2022 From: maxim at nginx.com (Maxim Konovalov) Date: Wed, 15 Jun 2022 12:00:55 +0400 Subject: [PATCH] Add optional "mp4_exact_start" nginx config off/on to show video between keyframes In-Reply-To: References: <20210628095320.px3ggmmoyjalyv5m@Romans-MacBook-Pro.local> <5F32216C-A041-454C-A73C-0E1C259E434C@archive.org> <20210930134811.epttik4joflf2qj6@Romans-MacBook-Pro.local> <20241A9E-BDF1-42D8-9848-AF628717EFE3@archive.org> <642e8b4e-db98-97c4-dfc6-09d4512aad36@nginx.com> Message-ID: Haha, this one is really nice. Thanks for sharing, Tracey! Maxim On 14.06.2022 23:49, Tracey Jaquith wrote: > Hi Maxim & (esp.) Roman, :) > > Sorry I didn’t get back to this — it took them until late November to > get the videos up, and then I… LOL, seem to have been in forget.. > procrastinate loop :p > > > Anyway! > Here’s the video (about 20 minutes): > > https://www.youtube.com/watch?v=yPwF2ms9MQ8 > > > Obviously, at the time of the talk vs. the ultimate move to EDL, the > solve that made it into nginx wound up being slightly different. > But the talk goes over the pre-EDL solve and still is quite interesting. > > Demuxed is a SF-based annual A/V conference full of all the big players > and super brilliant A/V engineers — so it’s a real honor to speak there > (my 2nd time speaking there). > It’s a single track type setup, that’s now expanded to 2 full days. > > To my complete surprise, my talk won the day’s Audience Award! > 🎉 🥰 > > > I’m so grateful for all the collaboration and help and work together to > get this important feature in to the amazing `mp4` module for nginx. > It’s been an honor to have been able to work with and contribute to nginx. > As I say in my talk: “The best web server on the Planet” :) > > Warmly, > -Tracey Jaquith > > > >> On Nov 3, 2021, at 12:48 AM, Maxim Konovalov > > wrote: >> >> Hi Tracey, >> >> On 03.11.2021 05:16, Tracey Jaquith wrote: >> [...] >>> You might find this hopefully amusing, but I gave a talk at Demuxed 2021 >>> October 7th, titled: >>>   "30,000 fps nginx - To Russia with Love” >>> while things were before the EDTS solution you all found and worked out. >>> (Video should hit YouTube by end of 2021). >>> >> Please update us with the link then. >> >> Thanks! >> >> Maxim >> >> -- >> Maxim Konovalov > > -Tracey > @tracey_pooh > TV Architect https://archive.org/tv > > > > > > -- Maxim Konovalov From pluknet at nginx.com Wed Jun 15 10:27:27 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Wed, 15 Jun 2022 14:27:27 +0400 Subject: [PATCH] Add ipv4=off option in resolver like ipv6=off (ticket #1330) In-Reply-To: <20220223040934.GD18027@lo0.su> References: <20220216123055.GA18027@lo0.su> <20220223040934.GD18027@lo0.su> Message-ID: > On 23 Feb 2022, at 08:09, Ruslan Ermilov wrote: > > On Wed, Feb 16, 2022 at 03:30:55PM +0300, Ruslan Ermilov wrote: >> Hi Lukas, >> >> On Wed, Jan 19, 2022 at 07:47:44PM +0100, Lukas Lihotzki via nginx-devel wrote: >>> # HG changeset patch >>> # User Lukas Lihotzki >>> # Date 1642618053 -3600 >>> # Wed Jan 19 19:47:33 2022 +0100 >>> # Node ID e9f06dc2d6a4a1aa61c15009b84ceedcaf5983b2 >>> # Parent aeab41dfd2606dd36cabbf01f1472726e27e8aea >>> Add ipv4=off option in resolver like ipv6=off (ticket #1330). >>> >>> IPv6-only hosts (ticket #1330) and upstreams with IPv6 bind address >>> (ticket #1535) need to disable resolving to IPv4 addresses. >>> >>> Ticket #1330 mentions ipv4=off is the proper fix. >> >> There's a number of problems in your patch. Please try this >> one instead: >> >> # HG changeset patch >> # User Ruslan Ermilov >> # Date 1644873563 -10800 >> # Tue Feb 15 00:19:23 2022 +0300 >> # Node ID 5d2cb60a78dd32a10a0010ccff39974fd7605867 >> # Parent 1add55d236522616ce34ffaa4dc697a76d3d41a4 >> The "ipv4=" parameter of the "resolver" directive (ticket #2196). >> >> When set to "off", only IPv6 addresses will be resolved, and no >> A queries are ever sent. >> >> diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c >> --- a/src/core/ngx_resolver.c >> +++ b/src/core/ngx_resolver.c >> @@ -157,6 +157,8 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ >> cln->handler = ngx_resolver_cleanup; >> cln->data = r; >> >> + r->ipv4 = 1; >> + >> ngx_rbtree_init(&r->name_rbtree, &r->name_sentinel, >> ngx_resolver_rbtree_insert_value); >> >> @@ -225,6 +227,23 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ >> } >> >> #if (NGX_HAVE_INET6) >> + if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) { >> + >> + if (ngx_strcmp(&names[i].data[5], "on") == 0) { >> + r->ipv4 = 1; >> + >> + } else if (ngx_strcmp(&names[i].data[5], "off") == 0) { >> + r->ipv4 = 0; >> + >> + } else { >> + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, >> + "invalid parameter: %V", &names[i]); >> + return NULL; >> + } >> + >> + continue; >> + } >> + >> if (ngx_strncmp(names[i].data, "ipv6=", 5) == 0) { >> >> if (ngx_strcmp(&names[i].data[5], "on") == 0) { > > Addon to the patch: > > diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c > --- a/src/core/ngx_resolver.c > +++ b/src/core/ngx_resolver.c > @@ -229,10 +229,12 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > #if (NGX_HAVE_INET6) > if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) { > > - if (ngx_strcmp(&names[i].data[5], "on") == 0) { > + if (ngx_strcasecmp(&names[i].data[5], (u_char *) "on") == 0) { > r->ipv4 = 1; > > - } else if (ngx_strcmp(&names[i].data[5], "off") == 0) { > + } else if (ngx_strcasecmp(&names[i].data[5], (u_char *) "off") > + == 0) > + { > r->ipv4 = 0; > > } else { > @@ -246,10 +248,12 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > > if (ngx_strncmp(names[i].data, "ipv6=", 5) == 0) { > > - if (ngx_strcmp(&names[i].data[5], "on") == 0) { > + if (ngx_strcasecmp(&names[i].data[5], (u_char *) "on") == 0) { > r->ipv6 = 1; > > - } else if (ngx_strcmp(&names[i].data[5], "off") == 0) { > + } else if (ngx_strcasecmp(&names[i].data[5], (u_char *) "off") > + == 0) > + { > r->ipv6 = 0; > > } else { > I don't see the reasons to make the parameter value case-insensitive. Existing parameters in ngx_http_core_module.c take it case-sensitive. Otherwise, looks good to me. [..] -- Sergey Kandaurov From xeioex at nginx.com Thu Jun 16 00:11:51 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 16 Jun 2022 00:11:51 +0000 Subject: [njs] Renaming promise to promise_ctor to avoid confusion. Message-ID: details: https://hg.nginx.org/njs/rev/461b1a030acb branches: changeset: 1889:461b1a030acb user: Dmitry Volyntsev date: Tue Jun 14 17:48:54 2022 -0700 description: Renaming promise to promise_ctor to avoid confusion. diffstat: src/njs_promise.c | 20 ++++++++++---------- 1 files changed, 10 insertions(+), 10 deletions(-) diffs (67 lines): diff -r c6522ba4bd53 -r 461b1a030acb src/njs_promise.c --- a/src/njs_promise.c Sat Jun 11 00:15:49 2022 -0700 +++ b/src/njs_promise.c Tue Jun 14 17:48:54 2022 -0700 @@ -1242,18 +1242,18 @@ njs_promise_all(njs_vm_t *vm, njs_value_ njs_index_t function_type) { njs_int_t ret; - njs_value_t *promise, resolve; + njs_value_t *promise_ctor, resolve; njs_iterator_handler_t handler; njs_promise_iterator_args_t pargs; - promise = njs_argument(args, 0); + promise_ctor = njs_argument(args, 0); - pargs.capability = njs_promise_new_capability(vm, promise); + pargs.capability = njs_promise_new_capability(vm, promise_ctor); if (njs_slow_path(pargs.capability == NULL)) { return NJS_ERROR; } - ret = njs_value_property(vm, promise, njs_value_arg(&string_resolve), + ret = njs_value_property(vm, promise_ctor, njs_value_arg(&string_resolve), &resolve); if (njs_slow_path(ret != NJS_OK)) { return ret; @@ -1265,7 +1265,7 @@ njs_promise_all(njs_vm_t *vm, njs_value_ } pargs.function = njs_function(&resolve); - pargs.constructor = promise; + pargs.constructor = promise_ctor; switch (function_type) { case NJS_PROMISE_ALL_SETTLED: @@ -1702,18 +1702,18 @@ njs_promise_race(njs_vm_t *vm, njs_value { int64_t length; njs_int_t ret; - njs_value_t *promise, *iterator, resolve; + njs_value_t *promise_ctor, *iterator, resolve; njs_promise_iterator_args_t pargs; - promise = njs_argument(args, 0); + promise_ctor = njs_argument(args, 0); iterator = njs_arg(args, nargs, 1); - pargs.capability = njs_promise_new_capability(vm, promise); + pargs.capability = njs_promise_new_capability(vm, promise_ctor); if (njs_slow_path(pargs.capability == NULL)) { return NJS_ERROR; } - ret = njs_value_property(vm, promise, njs_value_arg(&string_resolve), + ret = njs_value_property(vm, promise_ctor, njs_value_arg(&string_resolve), &resolve); if (njs_slow_path(ret != NJS_OK)) { return ret; @@ -1732,7 +1732,7 @@ njs_promise_race(njs_vm_t *vm, njs_value njs_memzero(&pargs.args, sizeof(njs_iterator_args_t)); pargs.function = njs_function(&resolve); - pargs.constructor = promise; + pargs.constructor = promise_ctor; pargs.args.value = iterator; pargs.args.to = length; From xeioex at nginx.com Thu Jun 16 00:11:52 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 16 Jun 2022 00:11:52 +0000 Subject: [njs] Propertly handling NJS_DECLINE in promise native functions. Message-ID: details: https://hg.nginx.org/njs/rev/be631766cc1f branches: changeset: 1890:be631766cc1f user: Dmitry Volyntsev date: Wed Jun 15 17:10:39 2022 -0700 description: Propertly handling NJS_DECLINE in promise native functions. Previously, NJS_DECLINE was returned from a Promise.all() and friends when "resolve" property was not found in a promise constructor. NJS_DECLINE was treated as NJS_ERROR in one place, but as NJS_OK in a different place during the promise function evaluation. As a result, the VM was left in inconsistent state during stack unwinding which resulted in a garbage return value. The fix is to ensure that only NJS_ERROR or NJS_OK is returned from ordinary native functions. This closes #545 issue on Github. diffstat: src/njs_promise.c | 6 +++--- src/test/njs_unit_test.c | 22 ++++++++++++++++++++++ 2 files changed, 25 insertions(+), 3 deletions(-) diffs (62 lines): diff -r 461b1a030acb -r be631766cc1f src/njs_promise.c --- a/src/njs_promise.c Tue Jun 14 17:48:54 2022 -0700 +++ b/src/njs_promise.c Wed Jun 15 17:10:39 2022 -0700 @@ -389,7 +389,7 @@ njs_promise_value_constructor(njs_vm_t * ret = njs_value_property(vm, value, njs_value_arg(&string_constructor), dst); - if (njs_slow_path(ret != NJS_OK)) { + if (njs_slow_path(ret == NJS_ERROR)) { return ret; } @@ -1255,7 +1255,7 @@ njs_promise_all(njs_vm_t *vm, njs_value_ ret = njs_value_property(vm, promise_ctor, njs_value_arg(&string_resolve), &resolve); - if (njs_slow_path(ret != NJS_OK)) { + if (njs_slow_path(ret == NJS_ERROR)) { return ret; } @@ -1715,7 +1715,7 @@ njs_promise_race(njs_vm_t *vm, njs_value ret = njs_value_property(vm, promise_ctor, njs_value_arg(&string_resolve), &resolve); - if (njs_slow_path(ret != NJS_OK)) { + if (njs_slow_path(ret == NJS_ERROR)) { return ret; } diff -r 461b1a030acb -r be631766cc1f src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Tue Jun 14 17:48:54 2022 -0700 +++ b/src/test/njs_unit_test.c Wed Jun 15 17:10:39 2022 -0700 @@ -21211,6 +21211,28 @@ static njs_unit_test_t njs_externals_te { njs_str("$r.retval(Promise.all([async () => [await x('X')]]))"), njs_str("[object Promise]") }, + { njs_str("var r = [1].map(v => {" + " function C(a) {" + " a(a, parseInt);" + " };" + "" + " Promise.all.apply(C);" + "});" + "r[0]"), + /* TODO: RejectAbrupt() exception should not percolate */ + njs_str("TypeError: resolve is not callable") }, + + { njs_str("var r = [1].map(v => {" + " function C(a) {" + " a(a, parseInt);" + " };" + "" + " Promise.race.apply(C);" + "});" + "r[0]"), + /* TODO: RejectAbrupt() exception should not percolate */ + njs_str("TypeError: resolve is not callable") }, + { njs_str("let obj = { a: 1, b: 2};" "function cb(r) { r.retval(obj.a); }" "$r.subrequest($r)" From mdounin at mdounin.ru Thu Jun 16 00:33:51 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 16 Jun 2022 03:33:51 +0300 Subject: [PATCH] public upstream connect and finalize request In-Reply-To: <4ae4c147e3b9280e8aca.1655264517@gws.t72.ru> References: <4ae4c147e3b9280e8aca.1655264517@gws.t72.ru> Message-ID: Hello! On Wed, Jun 15, 2022 at 08:41:57AM +0500, RekGRpth wrote: > # HG changeset patch > # User RekGRpth > # Date 1655264350 -18000 > # Wed Jun 15 08:39:10 2022 +0500 > # Node ID 4ae4c147e3b9280e8aca76e99807bbbbf6059de5 > # Parent 861f076eab72dab85bd5eaa56da770f8b10aa8ad > public upstream connect and finalize request > > diff -r 861f076eab72 -r 4ae4c147e3b9 src/http/ngx_http_upstream.c > --- a/src/http/ngx_http_upstream.c Tue Jun 14 10:39:58 2022 +0400 > +++ b/src/http/ngx_http_upstream.c Wed Jun 15 08:39:10 2022 +0500 > @@ -35,8 +35,6 @@ > static void ngx_http_upstream_wr_check_broken_connection(ngx_http_request_t *r); > static void ngx_http_upstream_check_broken_connection(ngx_http_request_t *r, > ngx_event_t *ev); > -static void ngx_http_upstream_connect(ngx_http_request_t *r, > - ngx_http_upstream_t *u); > static ngx_int_t ngx_http_upstream_reinit(ngx_http_request_t *r, > ngx_http_upstream_t *u); > static void ngx_http_upstream_send_request(ngx_http_request_t *r, > @@ -96,8 +94,6 @@ > static void ngx_http_upstream_next(ngx_http_request_t *r, > ngx_http_upstream_t *u, ngx_uint_t ft_type); > static void ngx_http_upstream_cleanup(void *data); > -static void ngx_http_upstream_finalize_request(ngx_http_request_t *r, > - ngx_http_upstream_t *u, ngx_int_t rc); > > static ngx_int_t ngx_http_upstream_process_header_line(ngx_http_request_t *r, > ngx_table_elt_t *h, ngx_uint_t offset); [...] Thanks for the patch. Quoting the Contributing Changes article (http://nginx.org/en/docs/contributing_changes.html): "Try to make it clear why the suggested change is needed, and provide a use case, if possible." Hope this helps. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Thu Jun 16 00:36:46 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 16 Jun 2022 03:36:46 +0300 Subject: SSL contexts reuse across locations In-Reply-To: References: Message-ID: Hello! On Tue, Jun 14, 2022 at 05:55:22PM +0000, Pavel Pautov via nginx-devel wrote: > Any comments? As already suggested in the very first review: : You may want to focus on actually making the code more readable : and abstracting it into ngx_http_proxy_set_ssl() instead. : Something like ngx_http_upstream_hide_headers_hash() might be a : good example on how to do it properly. Hope this helps. -- Maxim Dounin http://mdounin.ru/ From P.Pautov at F5.com Thu Jun 16 08:26:48 2022 From: P.Pautov at F5.com (Pavel Pautov) Date: Thu, 16 Jun 2022 08:26:48 +0000 Subject: SSL contexts reuse across locations In-Reply-To: References: Message-ID: Looks like, we've made a full circle then... I've replied to that suggestion already and in last e-mail (with patch) I note that moving additional logic into the ngx_http_proxy_set_ssl() has its own drawbacks, but I can certainly move more stuff into it. So do you envision something like "ngx_http_proxy_set_ssl(cf, conf, prev, reuse_ssl)"? As previously we've established that directives merging stays out of ngx_http_proxy_set_ssl (and reuse_ssl calculation has to happen before it). > -----Original Message----- > From: Maxim Dounin > Sent: Wednesday, June 15, 2022 17:37 > To: Pavel Pautov via nginx-devel > Subject: Re: SSL contexts reuse across locations > > EXTERNAL MAIL: nginx-devel-bounces at nginx.org > > Hello! > > On Tue, Jun 14, 2022 at 05:55:22PM +0000, Pavel Pautov via nginx-devel wrote: > > > Any comments? > > As already suggested in the very first review: > > : You may want to focus on actually making the code more readable > : and abstracting it into ngx_http_proxy_set_ssl() instead. > : Something like ngx_http_upstream_hide_headers_hash() might be a > : good example on how to do it properly. > > Hope this helps. > > -- > Maxim Dounin From pluknet at nginx.com Thu Jun 16 15:17:13 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Thu, 16 Jun 2022 19:17:13 +0400 Subject: [PATCH] The "sort=" parameter of the "resolver" directive In-Reply-To: <8db4bbd67840e8bebb23.1645589485@1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa> References: <20220223040934.GD18027@lo0.su> <8db4bbd67840e8bebb23.1645589485@1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa> Message-ID: > On 23 Feb 2022, at 08:11, Ruslan Ermilov wrote: > > src/core/ngx_resolver.c | 38 +++++++++++++++++++++++++++++++++++++- > src/core/ngx_resolver.h | 5 +++++ > 2 files changed, 42 insertions(+), 1 deletions(-) > > > # HG changeset patch > # User Ruslan Ermilov > # Date 1645589387 -10800 > # Wed Feb 23 07:09:47 2022 +0300 > # Node ID 8db4bbd67840e8bebb23f9c6d10c0f633552e616 > # Parent 1c19779448db2309d607c74e2628ff98f84569ff > The "sort=" parameter of the "resolver" directive. > IMHO, the name isn't enough self-documenting in this context. It can be misinterpreted as sorting addresses, not families. I'd prefer the parameter name "prefer" here. > diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c > --- a/src/core/ngx_resolver.c > +++ b/src/core/ngx_resolver.c > @@ -266,6 +266,27 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > } > #endif > > + if (ngx_strncmp(names[i].data, "sort=", 5) == 0) { > + > + if (ngx_strcasecmp(&names[i].data[5], (u_char *) "ipv4") == 0) { The same question raises about case-sensitivity. I prefer to make it case-sensitive, like other similar directive parameters. > + r->sort = NGX_RESOLVE_A_FIRST; > + > +#if (NGX_HAVE_INET6) > + } else if (ngx_strcasecmp(&names[i].data[5], (u_char *) "ipv6") > + == 0) > + { > + r->sort = NGX_RESOLVE_AAAA_FIRST; > +#endif > + > + } else { > + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, > + "invalid parameter: %V", &names[i]); > + return NULL; > + } > + > + continue; > + } > + What about to put the whole block under NGX_HAVE_INET6? If built without ipv6 support it makes same little sense as for "ipv6" and "ipv4" parameters that already there. > ngx_memzero(&u, sizeof(ngx_url_t)); > > u.url = names[i]; > @@ -4253,7 +4274,22 @@ ngx_resolver_export(ngx_resolver_t *r, n > } > > i = 0; > - d = rotate ? ngx_random() % n : 0; > + > + if (r->sort == NGX_RESOLVE_A_FIRST) { > + d = 0; > + > +#if (NGX_HAVE_INET6) > + } else if (r->sort == NGX_RESOLVE_AAAA_FIRST) { > + d = rn->naddrs6; > + > + if (d == n) { > + d = 0; > + } > +#endif > + > + } else { > + d = rotate ? ngx_random() % n : 0; > + } > > if (rn->naddrs) { > j = rotate ? ngx_random() % rn->naddrs : 0; > diff --git a/src/core/ngx_resolver.h b/src/core/ngx_resolver.h > --- a/src/core/ngx_resolver.h > +++ b/src/core/ngx_resolver.h > @@ -36,6 +36,9 @@ > > #define NGX_RESOLVER_MAX_RECURSION 50 > > +#define NGX_RESOLVE_A_FIRST 1 > +#define NGX_RESOLVE_AAAA_FIRST 2 > + I'd adjust name for better sorting. > > typedef struct ngx_resolver_s ngx_resolver_t; > > @@ -185,6 +188,8 @@ struct ngx_resolver_s { > ngx_queue_t addr6_expire_queue; > #endif > > + ngx_uint_t sort; > + Since the previous patch introduces ipv4/ipv6 bit fields, it makes sense to make this one a bit field as well. > time_t resend_timeout; > time_t tcp_timeout; > time_t expire; > diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c +++ b/src/core/ngx_resolver.c @@ -227,6 +227,7 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ } #if (NGX_HAVE_INET6) + if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) { if (ngx_strcmp(&names[i].data[5], "on") == 0) { @@ -260,6 +261,24 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ continue; } + + if (ngx_strncmp(names[i].data, "prefer=", 7) == 0) { + + if (ngx_strcmp(&names[i].data[7], "ipv4") == 0) { + r->prefer = NGX_RESOLVE_PREFER_A; + + } else if (ngx_strcmp(&names[i].data[7], "ipv6") == 0) { + r->prefer = NGX_RESOLVE_PREFER_AAAA; + + } else { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid parameter: %V", &names[i]); + return NULL; + } + + continue; + } + #endif ngx_memzero(&u, sizeof(ngx_url_t)); @@ -4250,7 +4269,22 @@ ngx_resolver_export(ngx_resolver_t *r, n } i = 0; - d = rotate ? ngx_random() % n : 0; + + if (r->prefer == NGX_RESOLVE_PREFER_A) { + d = 0; + +#if (NGX_HAVE_INET6) + } else if (r->prefer == NGX_RESOLVE_PREFER_AAAA) { + d = rn->naddrs6; + + if (d == n) { + d = 0; + } +#endif + + } else { + d = rotate ? ngx_random() % n : 0; + } if (rn->naddrs) { j = rotate ? ngx_random() % rn->naddrs : 0; diff --git a/src/core/ngx_resolver.h b/src/core/ngx_resolver.h --- a/src/core/ngx_resolver.h +++ b/src/core/ngx_resolver.h @@ -36,6 +36,9 @@ #define NGX_RESOLVER_MAX_RECURSION 50 +#define NGX_RESOLVE_PREFER_A 1 +#define NGX_RESOLVE_PREFER_AAAA 2 + typedef struct ngx_resolver_s ngx_resolver_t; @@ -175,6 +178,8 @@ struct ngx_resolver_s { ngx_queue_t srv_expire_queue; ngx_queue_t addr_expire_queue; + unsigned prefer:2; + unsigned ipv4:1; #if (NGX_HAVE_INET6) -- Sergey Kandaurov From mdounin at mdounin.ru Thu Jun 16 21:33:20 2022 From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=) Date: Fri, 17 Jun 2022 00:33:20 +0300 Subject: [PATCH] Misc: win32 sources now preserved in release tarballs Message-ID: # HG changeset patch # User Maxim Dounin # Date 1655415152 -10800 # Fri Jun 17 00:32:32 2022 +0300 # Node ID e23a385cd0ec866a3eb1d8c9c956991e1ed50d78 # Parent 861f076eab72dab85bd5eaa56da770f8b10aa8ad Misc: win32 sources now preserved in release tarballs. This makes it possible to build nginx under Windows from release tarballs instead of using source code repository. diff --git a/misc/GNUmakefile b/misc/GNUmakefile --- a/misc/GNUmakefile +++ b/misc/GNUmakefile @@ -15,12 +15,6 @@ release: export mv $(TEMP)/$(NGINX)/auto/configure $(TEMP)/$(NGINX) - # delete incomplete sources - rm $(TEMP)/$(NGINX)/src/event/ngx_event_acceptex.c - rm $(TEMP)/$(NGINX)/src/event/ngx_event_connectex.c - rm $(TEMP)/$(NGINX)/src/event/modules/ngx_iocp_module.* - rm -r $(TEMP)/$(NGINX)/src/os/win32 - mv $(TEMP)/$(NGINX)/docs/text/LICENSE $(TEMP)/$(NGINX) mv $(TEMP)/$(NGINX)/docs/text/README $(TEMP)/$(NGINX) mv $(TEMP)/$(NGINX)/docs/html $(TEMP)/$(NGINX) From xeioex at nginx.com Fri Jun 17 00:36:43 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 17 Jun 2022 00:36:43 +0000 Subject: [njs] Fixed working with array-like object in Promise.all() and friends. Message-ID: details: https://hg.nginx.org/njs/rev/42a2406590c0 branches: changeset: 1891:42a2406590c0 user: Dmitry Volyntsev date: Thu Jun 16 17:33:49 2022 -0700 description: Fixed working with array-like object in Promise.all() and friends. Prevously, the code while iterating over an array-like object did not take into account objects with absent elements. As a result, the resulting array object was returning with elements containing garbage values. The fix is to allocate and fill the resulting array object on the fly. This closes #538 issue on Github. diffstat: src/njs_promise.c | 88 +++++++++++++++++++++++++++++++++-------------- src/test/njs_unit_test.c | 10 +++++ 2 files changed, 71 insertions(+), 27 deletions(-) diffs (235 lines): diff -r be631766cc1f -r 42a2406590c0 src/njs_promise.c --- a/src/njs_promise.c Wed Jun 15 17:10:39 2022 -0700 +++ b/src/njs_promise.c Thu Jun 16 17:33:49 2022 -0700 @@ -1308,7 +1308,7 @@ njs_promise_perform_all(njs_vm_t *vm, nj return ret; } - pargs->args.data = njs_array_alloc(vm, 1, length, 0); + pargs->args.data = njs_array_alloc(vm, 1, 0, NJS_ARRAY_SPARE); if (njs_slow_path(pargs->args.data == NULL)) { return NJS_ERROR; } @@ -1363,7 +1363,7 @@ njs_promise_perform_all_handler(njs_vm_t { njs_int_t ret; njs_array_t *array; - njs_value_t arguments[2], next; + njs_value_t arguments[2], next, arr_value; njs_function_t *on_fulfilled; njs_promise_capability_t *capability; njs_promise_all_context_t *context; @@ -1378,7 +1378,13 @@ njs_promise_perform_all_handler(njs_vm_t capability = pargs->capability; array = args->data; - njs_set_undefined(&array->start[index]); + njs_set_array(&arr_value, array); + + ret = njs_value_property_i64_set(vm, &arr_value, index, + njs_value_arg(&njs_value_undefined)); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } ret = njs_function_call(vm, pargs->function, pargs->constructor, value, 1, &next); @@ -1421,7 +1427,8 @@ static njs_int_t njs_promise_all_resolve_element_functions(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - njs_value_t arguments; + njs_int_t ret; + njs_value_t arr_value; njs_promise_all_context_t *context; context = vm->top_frame->function->context; @@ -1432,16 +1439,21 @@ njs_promise_all_resolve_element_function } context->already_called = 1; - context->values->start[context->index] = *njs_arg(args, nargs, 1); + + njs_set_array(&arr_value, context->values); + + ret = njs_value_property_i64_set(vm, &arr_value, context->index, + njs_arg(args, nargs, 1)); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } if (--(*context->remaining_elements) == 0) { njs_mp_free(vm->mem_pool, context->remaining_elements); - njs_set_array(&arguments, context->values); - return njs_function_call(vm, njs_function(&context->capability->resolve), - &njs_value_undefined, &arguments, 1, + &njs_value_undefined, &arr_value, 1, &vm->retval); } @@ -1457,7 +1469,7 @@ njs_promise_perform_all_settled_handler( { njs_int_t ret; njs_array_t *array; - njs_value_t arguments[2], next; + njs_value_t arguments[2], next, arr_value; njs_function_t *on_fulfilled, *on_rejected; njs_promise_capability_t *capability; njs_promise_all_context_t *context; @@ -1472,7 +1484,13 @@ njs_promise_perform_all_settled_handler( capability = pargs->capability; array = args->data; - njs_set_undefined(&array->start[index]); + njs_set_array(&arr_value, array); + + ret = njs_value_property_i64_set(vm, &arr_value, index, + njs_value_arg(&njs_value_undefined)); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } ret = njs_function_call(vm, pargs->function, pargs->constructor, value, 1, &next); @@ -1527,7 +1545,7 @@ njs_promise_all_settled_element_function njs_value_t *args, njs_uint_t nargs, njs_index_t rejected) { njs_int_t ret; - njs_value_t arguments, *value; + njs_value_t obj_value, arr_value; njs_object_t *obj; const njs_value_t *status, *set; njs_promise_all_context_t *context; @@ -1552,9 +1570,7 @@ njs_promise_all_settled_element_function return NJS_ERROR; } - value = &context->values->start[context->index]; - - njs_set_object(value, obj); + njs_set_object(&obj_value, obj); if (rejected) { status = &string_rejected; @@ -1565,26 +1581,32 @@ njs_promise_all_settled_element_function set = &string_value; } - ret = njs_value_property_set(vm, value, njs_value_arg(&string_status), + ret = njs_value_property_set(vm, &obj_value, njs_value_arg(&string_status), njs_value_arg(status)); if (njs_slow_path(ret == NJS_ERROR)) { return ret; } - ret = njs_value_property_set(vm, value, njs_value_arg(set), + ret = njs_value_property_set(vm, &obj_value, njs_value_arg(set), njs_arg(args, nargs, 1)); if (njs_slow_path(ret == NJS_ERROR)) { return ret; } + njs_set_array(&arr_value, context->values); + + ret = njs_value_property_i64_set(vm, &arr_value, context->index, + &obj_value); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + if (--(*context->remaining_elements) == 0) { njs_mp_free(vm->mem_pool, context->remaining_elements); - njs_set_array(&arguments, context->values); - return njs_function_call(vm, njs_function(&context->capability->resolve), - &njs_value_undefined, &arguments, 1, + &njs_value_undefined, &arr_value, 1, &vm->retval); } @@ -1600,7 +1622,7 @@ njs_promise_perform_any_handler(njs_vm_t { njs_int_t ret; njs_array_t *array; - njs_value_t arguments[2], next; + njs_value_t arguments[2], next, arr_value; njs_function_t *on_rejected; njs_promise_capability_t *capability; njs_promise_all_context_t *context; @@ -1614,8 +1636,14 @@ njs_promise_perform_any_handler(njs_vm_t capability = pargs->capability; - array = pargs->args.data; - njs_set_undefined(&array->start[index]); + array = args->data; + njs_set_array(&arr_value, array); + + ret = njs_value_property_i64_set(vm, &arr_value, index, + njs_value_arg(&njs_value_undefined)); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } ret = njs_function_call(vm, pargs->function, pargs->constructor, value, 1, &next); @@ -1658,7 +1686,8 @@ static njs_int_t njs_promise_any_reject_element_functions(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - njs_value_t argument; + njs_int_t ret; + njs_value_t argument, arr_value; njs_object_t *error; njs_promise_all_context_t *context; @@ -1670,15 +1699,20 @@ njs_promise_any_reject_element_functions } context->already_called = 1; - context->values->start[context->index] = *njs_arg(args, nargs, 1); + + njs_set_array(&arr_value, context->values); + + ret = njs_value_property_i64_set(vm, &arr_value, context->index, + njs_arg(args, nargs, 1)); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } if (--(*context->remaining_elements) == 0) { njs_mp_free(vm->mem_pool, context->remaining_elements); - njs_set_array(&argument, context->values); - error = njs_error_alloc(vm, NJS_OBJ_TYPE_AGGREGATE_ERROR, - NULL, &string_any_rejected, &argument); + NULL, &string_any_rejected, &arr_value); if (njs_slow_path(error == NULL)) { return NJS_ERROR; } diff -r be631766cc1f -r 42a2406590c0 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Wed Jun 15 17:10:39 2022 -0700 +++ b/src/test/njs_unit_test.c Thu Jun 16 17:33:49 2022 -0700 @@ -21222,6 +21222,16 @@ static njs_unit_test_t njs_externals_te /* TODO: RejectAbrupt() exception should not percolate */ njs_str("TypeError: resolve is not callable") }, + { njs_str("Promise.all({length: 1025}) " + ".then(v => $r.retval(v[0]))"), + /* TODO: TypeError: object is not iterable */ + njs_str("undefined") }, + + { njs_str("Promise.allSettled({length: 1025}) " + ".then(v => $r.retval(v[0]))"), + /* TODO: TypeError: object is not iterable */ + njs_str("undefined") }, + { njs_str("var r = [1].map(v => {" " function C(a) {" " a(a, parseInt);" From mdounin at mdounin.ru Fri Jun 17 01:51:19 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 17 Jun 2022 04:51:19 +0300 Subject: SSL contexts reuse across locations In-Reply-To: References: Message-ID: Hello! On Thu, Jun 16, 2022 at 08:26:48AM +0000, Pavel Pautov via nginx-devel wrote: > Looks like, we've made a full circle then... I've replied to > that suggestion already and in last e-mail (with patch) I note > that moving additional logic into the ngx_http_proxy_set_ssl() > has its own drawbacks, but I can certainly move more stuff into > it. > > So do you envision something like "ngx_http_proxy_set_ssl(cf, > conf, prev, reuse_ssl)"? As previously we've established that > directives merging stays out of ngx_http_proxy_set_ssl (and > reuse_ssl calculation has to happen before it). I don't think further attempts to explain how to write readable code worth the effort. Please see the patch below. Review and testing appreciated. # HG changeset patch # User Maxim Dounin # Date 1655429915 -10800 # Fri Jun 17 04:38:35 2022 +0300 # Node ID e4a0eeb3edba037f0d090023a2242bda2f8dcb03 # Parent e23a385cd0ec866a3eb1d8c9c956991e1ed50d78 Upstream: optimized use of SSL contexts (ticket #1234). To ensure optimal use of memory, SSL contexts for proxying are now inherited from previous levels as long as relevant proxy_ssl_* directives are not redefined. Further, when no proxy_ssl_* directives are redefined in a server block, we now preserve plcf->upstream.ssl in the "http" section configuration to inherit it to all servers. Similar changes made in uwsgi and grpc. diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c --- a/src/http/modules/ngx_http_grpc_module.c +++ b/src/http/modules/ngx_http_grpc_module.c @@ -210,7 +210,7 @@ static char *ngx_http_grpc_ssl_password_ static char *ngx_http_grpc_ssl_conf_command_check(ngx_conf_t *cf, void *post, void *data); static ngx_int_t ngx_http_grpc_set_ssl(ngx_conf_t *cf, - ngx_http_grpc_loc_conf_t *glcf); + ngx_http_grpc_loc_conf_t *glcf, ngx_http_grpc_loc_conf_t *prev); #endif @@ -562,7 +562,7 @@ ngx_http_grpc_handler(ngx_http_request_t ctx->host = glcf->host; #if (NGX_HTTP_SSL) - u->ssl = (glcf->upstream.ssl != NULL); + u->ssl = glcf->ssl; if (u->ssl) { ngx_str_set(&u->schema, "grpcs://"); @@ -4495,7 +4495,7 @@ ngx_http_grpc_merge_loc_conf(ngx_conf_t ngx_conf_merge_ptr_value(conf->ssl_conf_commands, prev->ssl_conf_commands, NULL); - if (conf->ssl && ngx_http_grpc_set_ssl(cf, conf) != NGX_OK) { + if (ngx_http_grpc_set_ssl(cf, conf, prev) != NGX_OK) { return NGX_CONF_ERROR; } @@ -4524,7 +4524,7 @@ ngx_http_grpc_merge_loc_conf(ngx_conf_t conf->grpc_values = prev->grpc_values; #if (NGX_HTTP_SSL) - conf->upstream.ssl = prev->upstream.ssl; + conf->ssl = prev->ssl; #endif } @@ -4874,10 +4874,37 @@ ngx_http_grpc_ssl_conf_command_check(ngx static ngx_int_t -ngx_http_grpc_set_ssl(ngx_conf_t *cf, ngx_http_grpc_loc_conf_t *glcf) +ngx_http_grpc_set_ssl(ngx_conf_t *cf, ngx_http_grpc_loc_conf_t *glcf, + ngx_http_grpc_loc_conf_t *prev) { + ngx_uint_t preserve; ngx_pool_cleanup_t *cln; + if (glcf->ssl_protocols == prev->ssl_protocols + && glcf->ssl_ciphers.data == prev->ssl_ciphers.data + && glcf->upstream.ssl_certificate == prev->upstream.ssl_certificate + && glcf->upstream.ssl_certificate_key + == prev->upstream.ssl_certificate_key + && glcf->upstream.ssl_passwords == prev->upstream.ssl_passwords + && glcf->upstream.ssl_verify == prev->upstream.ssl_verify + && glcf->ssl_verify_depth == prev->ssl_verify_depth + && glcf->ssl_trusted_certificate.data + == prev->ssl_trusted_certificate.data + && glcf->ssl_crl.data == prev->ssl_crl.data + && glcf->upstream.ssl_session_reuse == prev->upstream.ssl_session_reuse + && glcf->ssl_conf_commands == prev->ssl_conf_commands) + { + if (prev->upstream.ssl) { + glcf->upstream.ssl = prev->upstream.ssl; + return NGX_OK; + } + + preserve = 1; + + } else { + preserve = 0; + } + glcf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); if (glcf->upstream.ssl == NULL) { return NGX_ERROR; @@ -4984,6 +5011,15 @@ ngx_http_grpc_set_ssl(ngx_conf_t *cf, ng return NGX_ERROR; } + /* + * special handling to preserve glcf->upstream.ssl + * in the "http" section to inherit it to all servers + */ + + if (preserve) { + prev->upstream.ssl = glcf->upstream.ssl; + } + return NGX_OK; } diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c +++ b/src/http/modules/ngx_http_proxy_module.c @@ -237,7 +237,7 @@ static ngx_int_t ngx_http_proxy_rewrite_ #if (NGX_HTTP_SSL) static ngx_int_t ngx_http_proxy_set_ssl(ngx_conf_t *cf, - ngx_http_proxy_loc_conf_t *plcf); + ngx_http_proxy_loc_conf_t *plcf, ngx_http_proxy_loc_conf_t *prev); #endif static void ngx_http_proxy_set_vars(ngx_url_t *u, ngx_http_proxy_vars_t *v); @@ -959,7 +959,7 @@ ngx_http_proxy_handler(ngx_http_request_ ctx->vars = plcf->vars; u->schema = plcf->vars.schema; #if (NGX_HTTP_SSL) - u->ssl = (plcf->upstream.ssl != NULL); + u->ssl = plcf->ssl; #endif } else { @@ -3756,7 +3756,7 @@ ngx_http_proxy_merge_loc_conf(ngx_conf_t ngx_conf_merge_ptr_value(conf->ssl_conf_commands, prev->ssl_conf_commands, NULL); - if (conf->ssl && ngx_http_proxy_set_ssl(cf, conf) != NGX_OK) { + if (ngx_http_proxy_set_ssl(cf, conf, prev) != NGX_OK) { return NGX_CONF_ERROR; } @@ -3857,7 +3857,7 @@ ngx_http_proxy_merge_loc_conf(ngx_conf_t conf->proxy_values = prev->proxy_values; #if (NGX_HTTP_SSL) - conf->upstream.ssl = prev->upstream.ssl; + conf->ssl = prev->ssl; #endif } @@ -4923,10 +4923,37 @@ ngx_http_proxy_ssl_conf_command_check(ng static ngx_int_t -ngx_http_proxy_set_ssl(ngx_conf_t *cf, ngx_http_proxy_loc_conf_t *plcf) +ngx_http_proxy_set_ssl(ngx_conf_t *cf, ngx_http_proxy_loc_conf_t *plcf, + ngx_http_proxy_loc_conf_t *prev) { + ngx_uint_t preserve; ngx_pool_cleanup_t *cln; + if (plcf->ssl_protocols == prev->ssl_protocols + && plcf->ssl_ciphers.data == prev->ssl_ciphers.data + && plcf->upstream.ssl_certificate == prev->upstream.ssl_certificate + && plcf->upstream.ssl_certificate_key + == prev->upstream.ssl_certificate_key + && plcf->upstream.ssl_passwords == prev->upstream.ssl_passwords + && plcf->upstream.ssl_verify == prev->upstream.ssl_verify + && plcf->ssl_verify_depth == prev->ssl_verify_depth + && plcf->ssl_trusted_certificate.data + == prev->ssl_trusted_certificate.data + && plcf->ssl_crl.data == prev->ssl_crl.data + && plcf->upstream.ssl_session_reuse == prev->upstream.ssl_session_reuse + && plcf->ssl_conf_commands == prev->ssl_conf_commands) + { + if (prev->upstream.ssl) { + plcf->upstream.ssl = prev->upstream.ssl; + return NGX_OK; + } + + preserve = 1; + + } else { + preserve = 0; + } + plcf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); if (plcf->upstream.ssl == NULL) { return NGX_ERROR; @@ -5020,6 +5047,15 @@ ngx_http_proxy_set_ssl(ngx_conf_t *cf, n return NGX_ERROR; } + /* + * special handling to preserve plcf->upstream.ssl + * in the "http" section to inherit it to all servers + */ + + if (preserve) { + prev->upstream.ssl = plcf->upstream.ssl; + } + return NGX_OK; } diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c +++ b/src/http/modules/ngx_http_uwsgi_module.c @@ -97,7 +97,7 @@ static char *ngx_http_uwsgi_ssl_password static char *ngx_http_uwsgi_ssl_conf_command_check(ngx_conf_t *cf, void *post, void *data); static ngx_int_t ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, - ngx_http_uwsgi_loc_conf_t *uwcf); + ngx_http_uwsgi_loc_conf_t *uwcf, ngx_http_uwsgi_loc_conf_t *prev); #endif @@ -668,7 +668,7 @@ ngx_http_uwsgi_handler(ngx_http_request_ if (uwcf->uwsgi_lengths == NULL) { #if (NGX_HTTP_SSL) - u->ssl = (uwcf->upstream.ssl != NULL); + u->ssl = uwcf->ssl; if (u->ssl) { ngx_str_set(&u->schema, "suwsgi://"); @@ -1897,7 +1897,7 @@ ngx_http_uwsgi_merge_loc_conf(ngx_conf_t ngx_conf_merge_ptr_value(conf->ssl_conf_commands, prev->ssl_conf_commands, NULL); - if (conf->ssl && ngx_http_uwsgi_set_ssl(cf, conf) != NGX_OK) { + if (ngx_http_uwsgi_set_ssl(cf, conf, prev) != NGX_OK) { return NGX_CONF_ERROR; } @@ -1927,7 +1927,7 @@ ngx_http_uwsgi_merge_loc_conf(ngx_conf_t conf->uwsgi_values = prev->uwsgi_values; #if (NGX_HTTP_SSL) - conf->upstream.ssl = prev->upstream.ssl; + conf->ssl = prev->ssl; #endif } @@ -2455,10 +2455,37 @@ ngx_http_uwsgi_ssl_conf_command_check(ng static ngx_int_t -ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, ngx_http_uwsgi_loc_conf_t *uwcf) +ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, ngx_http_uwsgi_loc_conf_t *uwcf, + ngx_http_uwsgi_loc_conf_t *prev) { + ngx_uint_t preserve; ngx_pool_cleanup_t *cln; + if (uwcf->ssl_protocols == prev->ssl_protocols + && uwcf->ssl_ciphers.data == prev->ssl_ciphers.data + && uwcf->upstream.ssl_certificate == prev->upstream.ssl_certificate + && uwcf->upstream.ssl_certificate_key + == prev->upstream.ssl_certificate_key + && uwcf->upstream.ssl_passwords == prev->upstream.ssl_passwords + && uwcf->upstream.ssl_verify == prev->upstream.ssl_verify + && uwcf->ssl_verify_depth == prev->ssl_verify_depth + && uwcf->ssl_trusted_certificate.data + == prev->ssl_trusted_certificate.data + && uwcf->ssl_crl.data == prev->ssl_crl.data + && uwcf->upstream.ssl_session_reuse == prev->upstream.ssl_session_reuse + && uwcf->ssl_conf_commands == prev->ssl_conf_commands) + { + if (prev->upstream.ssl) { + uwcf->upstream.ssl = prev->upstream.ssl; + return NGX_OK; + } + + preserve = 1; + + } else { + preserve = 0; + } + uwcf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); if (uwcf->upstream.ssl == NULL) { return NGX_ERROR; @@ -2552,6 +2579,15 @@ ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, n return NGX_ERROR; } + /* + * special handling to preserve uwcf->upstream.ssl + * in the "http" section to inherit it to all servers + */ + + if (preserve) { + prev->upstream.ssl = uwcf->upstream.ssl; + } + return NGX_OK; } -- Maxim Dounin http://mdounin.ru/ From rekgrpth at gmail.com Fri Jun 17 03:09:25 2022 From: rekgrpth at gmail.com (RekGRpth) Date: Fri, 17 Jun 2022 08:09:25 +0500 Subject: [PATCH] public upstream connect and finalize request Message-ID: It is needed for an upstream plugin, which has its own read and write infrastructure. (And, of cause, I forgotten publicate ngx_http_upstream_next) From pluknet at nginx.com Fri Jun 17 15:31:27 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Fri, 17 Jun 2022 19:31:27 +0400 Subject: [PATCH] Misc: win32 sources now preserved in release tarballs In-Reply-To: References: Message-ID: > On 17 Jun 2022, at 01:33, Maxim Dounin wrote: > > # HG changeset patch > # User Maxim Dounin > # Date 1655415152 -10800 > # Fri Jun 17 00:32:32 2022 +0300 > # Node ID e23a385cd0ec866a3eb1d8c9c956991e1ed50d78 > # Parent 861f076eab72dab85bd5eaa56da770f8b10aa8ad > Misc: win32 sources now preserved in release tarballs. > > This makes it possible to build nginx under Windows from release tarballs > instead of using source code repository. > > diff --git a/misc/GNUmakefile b/misc/GNUmakefile > --- a/misc/GNUmakefile > +++ b/misc/GNUmakefile > @@ -15,12 +15,6 @@ release: export > > mv $(TEMP)/$(NGINX)/auto/configure $(TEMP)/$(NGINX) > Do we need the empty line here? It could be collapsed if removing all the below. > - # delete incomplete sources > - rm $(TEMP)/$(NGINX)/src/event/ngx_event_acceptex.c > - rm $(TEMP)/$(NGINX)/src/event/ngx_event_connectex.c I wonder if we need to touch ngx_event_connectex.c. It's not used nor connected to the build since introduction, so it does look incomplete. > - rm $(TEMP)/$(NGINX)/src/event/modules/ngx_iocp_module.* > - rm -r $(TEMP)/$(NGINX)/src/os/win32 > - > mv $(TEMP)/$(NGINX)/docs/text/LICENSE $(TEMP)/$(NGINX) > mv $(TEMP)/$(NGINX)/docs/text/README $(TEMP)/$(NGINX) > mv $(TEMP)/$(NGINX)/docs/html $(TEMP)/$(NGINX) > -- Sergey Kandaurov From mdounin at mdounin.ru Fri Jun 17 20:37:57 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 17 Jun 2022 23:37:57 +0300 Subject: [PATCH] Misc: win32 sources now preserved in release tarballs In-Reply-To: References: Message-ID: Hello! On Fri, Jun 17, 2022 at 07:31:27PM +0400, Sergey Kandaurov wrote: > > On 17 Jun 2022, at 01:33, Maxim Dounin wrote: > > > > # HG changeset patch > > # User Maxim Dounin > > # Date 1655415152 -10800 > > # Fri Jun 17 00:32:32 2022 +0300 > > # Node ID e23a385cd0ec866a3eb1d8c9c956991e1ed50d78 > > # Parent 861f076eab72dab85bd5eaa56da770f8b10aa8ad > > Misc: win32 sources now preserved in release tarballs. > > > > This makes it possible to build nginx under Windows from release tarballs > > instead of using source code repository. > > > > diff --git a/misc/GNUmakefile b/misc/GNUmakefile > > --- a/misc/GNUmakefile > > +++ b/misc/GNUmakefile > > @@ -15,12 +15,6 @@ release: export > > > > mv $(TEMP)/$(NGINX)/auto/configure $(TEMP)/$(NGINX) > > > > Do we need the empty line here? > It could be collapsed if removing all the below. It clearly separates two distinct blocks: configure and docs. I don't think this distinction have to be removed. > > - # delete incomplete sources > > - rm $(TEMP)/$(NGINX)/src/event/ngx_event_acceptex.c > > - rm $(TEMP)/$(NGINX)/src/event/ngx_event_connectex.c > > I wonder if we need to touch ngx_event_connectex.c. > It's not used nor connected to the build since introduction, > so it does look incomplete. I don't think it is a good idea to keep any of these removals, even if some files are not connected to the build. Rather, I think that release tarballs should be as close as possible to the source code repository. Just in case, ngx_service.c and ngx_stat.c in src/os/win32/ also not connected to the build. > > - rm $(TEMP)/$(NGINX)/src/event/modules/ngx_iocp_module.* > > - rm -r $(TEMP)/$(NGINX)/src/os/win32 > > - > > mv $(TEMP)/$(NGINX)/docs/text/LICENSE $(TEMP)/$(NGINX) > > mv $(TEMP)/$(NGINX)/docs/text/README $(TEMP)/$(NGINX) > > mv $(TEMP)/$(NGINX)/docs/html $(TEMP)/$(NGINX) > > > > -- > Sergey Kandaurov -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Fri Jun 17 22:01:17 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 18 Jun 2022 01:01:17 +0300 Subject: [PATCH] public upstream connect and finalize request In-Reply-To: References: Message-ID: Hello! On Fri, Jun 17, 2022 at 08:09:25AM +0500, RekGRpth wrote: > It is needed for an upstream plugin, which has its own read and write > infrastructure. (And, of cause, I forgotten publicate > ngx_http_upstream_next) Thanks for the details. The suggested changes do not look in line with how upstream module works: it manages connections itself, and provides protocol-specific callbacks and filters to make it possible to implement various protocols. You may want to reconsider how your module works and change it to work with existing upstream interfaces instead. -- Maxim Dounin http://mdounin.ru/ From rekgrpth at gmail.com Sat Jun 18 05:13:04 2022 From: rekgrpth at gmail.com (RekGRpth) Date: Sat, 18 Jun 2022 10:13:04 +0500 Subject: [PATCH] public upstream connect and finalize request Message-ID: >"Try to make it clear why the suggested change is needed, >and provide a use case, if possible." It is needed for an upstream plugin, which has its own read and write infrastructure. (And, of cause, I forgotten publicate ngx_http_upstream_next) >Thanks for the details. The suggested changes do not look in line with how upstream >module works: it manages connections itself, and provides protocol-specific callbacks and >filters to make it possible to implement various protocols. You may want to reconsider how >your module works and change it to work with existing upstream interfaces instead. Unfortunately, I can not modify used third-party library (such as libpq) and then I need copy some private functions from ngx_http_upstream.c https://github.com/RekGRpth/ngx_pq_module From gmm at csdoc.com Sat Jun 18 13:02:17 2022 From: gmm at csdoc.com (Gena Makhomed) Date: Sat, 18 Jun 2022 16:02:17 +0300 Subject: [PATCH] Contrib: vim syntax, update core and 3rd party module directives. Message-ID: # HG changeset patch # User Gena Makhomed # Date 1655556880 -10800 # Sat Jun 18 15:54:40 2022 +0300 # Node ID 81096862f8f13438ca059779cbc2097db8c0ad68 # Parent 861f076eab72dab85bd5eaa56da770f8b10aa8ad Contrib: vim syntax, update core and 3rd party module directives. List of 3rd party modules github repositories are obtained from https://github.com/freebsd/freebsd-ports/blob/main/www/nginx-devel/Makefile.extmod diff -r 861f076eab72 -r 81096862f8f1 contrib/vim/syntax/nginx.vim --- a/contrib/vim/syntax/nginx.vim Tue Jun 14 10:39:58 2022 +0400 +++ b/contrib/vim/syntax/nginx.vim Sat Jun 18 15:54:40 2022 +0300 @@ -111,19 +111,14 @@ syn keyword ngxDirectiveError contained error_page syn keyword ngxDirectiveError contained post_action -syn keyword ngxDirectiveDeprecated contained limit_zone syn keyword ngxDirectiveDeprecated contained proxy_downstream_buffer syn keyword ngxDirectiveDeprecated contained proxy_upstream_buffer -syn keyword ngxDirectiveDeprecated contained spdy_chunk_size -syn keyword ngxDirectiveDeprecated contained spdy_headers_comp -syn keyword ngxDirectiveDeprecated contained spdy_keepalive_timeout -syn keyword ngxDirectiveDeprecated contained spdy_max_concurrent_streams -syn keyword ngxDirectiveDeprecated contained spdy_pool_size -syn keyword ngxDirectiveDeprecated contained spdy_recv_buffer_size -syn keyword ngxDirectiveDeprecated contained spdy_recv_timeout -syn keyword ngxDirectiveDeprecated contained spdy_streams_index_size syn keyword ngxDirectiveDeprecated contained ssl -syn keyword ngxDirectiveDeprecated contained upstream_conf +syn keyword ngxDirectiveDeprecated contained http2_idle_timeout +syn keyword ngxDirectiveDeprecated contained http2_max_field_size +syn keyword ngxDirectiveDeprecated contained http2_max_header_size +syn keyword ngxDirectiveDeprecated contained http2_max_requests +syn keyword ngxDirectiveDeprecated contained http2_recv_timeout syn keyword ngxDirective contained absolute_redirect syn keyword ngxDirective contained accept_mutex @@ -152,6 +147,7 @@ syn keyword ngxDirective contained auth_jwt syn keyword ngxDirective contained auth_jwt_claim_set syn keyword ngxDirective contained auth_jwt_header_set +syn keyword ngxDirective contained auth_jwt_key_cache syn keyword ngxDirective contained auth_jwt_key_file syn keyword ngxDirective contained auth_jwt_key_request syn keyword ngxDirective contained auth_jwt_leeway @@ -309,17 +305,12 @@ syn keyword ngxDirective contained hls_mp4_max_buffer_size syn keyword ngxDirective contained http2_body_preread_size syn keyword ngxDirective contained http2_chunk_size -syn keyword ngxDirective contained http2_idle_timeout syn keyword ngxDirective contained http2_max_concurrent_pushes syn keyword ngxDirective contained http2_max_concurrent_streams -syn keyword ngxDirective contained http2_max_field_size -syn keyword ngxDirective contained http2_max_header_size -syn keyword ngxDirective contained http2_max_requests syn keyword ngxDirective contained http2_pool_size syn keyword ngxDirective contained http2_push syn keyword ngxDirective contained http2_push_preload syn keyword ngxDirective contained http2_recv_buffer_size -syn keyword ngxDirective contained http2_recv_timeout syn keyword ngxDirective contained http2_streams_index_size syn keyword ngxDirective contained if_modified_since syn keyword ngxDirective contained ignore_invalid_headers @@ -339,14 +330,17 @@ syn keyword ngxDirective contained js_access syn keyword ngxDirective contained js_body_filter syn keyword ngxDirective contained js_content +syn keyword ngxDirective contained js_fetch_buffer_size syn keyword ngxDirective contained js_fetch_ciphers +syn keyword ngxDirective contained js_fetch_max_response_buffer_size syn keyword ngxDirective contained js_fetch_protocols +syn keyword ngxDirective contained js_fetch_timeout syn keyword ngxDirective contained js_fetch_trusted_certificate +syn keyword ngxDirective contained js_fetch_verify syn keyword ngxDirective contained js_fetch_verify_depth syn keyword ngxDirective contained js_filter syn keyword ngxDirective contained js_header_filter syn keyword ngxDirective contained js_import -syn keyword ngxDirective contained js_include syn keyword ngxDirective contained js_path syn keyword ngxDirective contained js_preread syn keyword ngxDirective contained js_set @@ -391,7 +385,6 @@ syn keyword ngxDirective contained memcached_bind syn keyword ngxDirective contained memcached_buffer_size syn keyword ngxDirective contained memcached_connect_timeout -syn keyword ngxDirective contained memcached_force_ranges syn keyword ngxDirective contained memcached_gzip_flag syn keyword ngxDirective contained memcached_next_upstream syn keyword ngxDirective contained memcached_next_upstream_timeout @@ -645,7 +638,6 @@ syn keyword ngxDirective contained status_format syn keyword ngxDirective contained status_zone syn keyword ngxDirective contained sticky -syn keyword ngxDirective contained sticky_cookie_insert syn keyword ngxDirective contained stub_status syn keyword ngxDirective contained sub_filter syn keyword ngxDirective contained sub_filter_last_modified @@ -774,62 +766,14 @@ syn keyword ngxDirective contained zone_sync_ssl_verify_depth syn keyword ngxDirective contained zone_sync_timeout -" 3rd party modules list taken from -" https://github.com/freebsd/freebsd-ports/blob/master/www/nginx-devel/Makefile -" ----------------------------------------------------------------------------- - -" Accept Language -" https://github.com/giom/nginx_accept_language_module -syn keyword ngxDirectiveThirdParty contained set_from_accept_language - -" Digest Authentication -" https://github.com/atomx/nginx-http-auth-digest -syn keyword ngxDirectiveThirdParty contained auth_digest -syn keyword ngxDirectiveThirdParty contained auth_digest_drop_time -syn keyword ngxDirectiveThirdParty contained auth_digest_evasion_time -syn keyword ngxDirectiveThirdParty contained auth_digest_expires -syn keyword ngxDirectiveThirdParty contained auth_digest_maxtries -syn keyword ngxDirectiveThirdParty contained auth_digest_replays -syn keyword ngxDirectiveThirdParty contained auth_digest_shm_size -syn keyword ngxDirectiveThirdParty contained auth_digest_timeout -syn keyword ngxDirectiveThirdParty contained auth_digest_user_file -" SPNEGO Authentication -" https://github.com/stnoonan/spnego-http-auth-nginx-module -syn keyword ngxDirectiveThirdParty contained auth_gss -syn keyword ngxDirectiveThirdParty contained auth_gss_allow_basic_fallback -syn keyword ngxDirectiveThirdParty contained auth_gss_authorized_principal -syn keyword ngxDirectiveThirdParty contained auth_gss_authorized_principal_regex -syn keyword ngxDirectiveThirdParty contained auth_gss_constrained_delegation -syn keyword ngxDirectiveThirdParty contained auth_gss_delegate_credentials -syn keyword ngxDirectiveThirdParty contained auth_gss_force_realm -syn keyword ngxDirectiveThirdParty contained auth_gss_format_full -syn keyword ngxDirectiveThirdParty contained auth_gss_keytab -syn keyword ngxDirectiveThirdParty contained auth_gss_map_to_local -syn keyword ngxDirectiveThirdParty contained auth_gss_realm -syn keyword ngxDirectiveThirdParty contained auth_gss_service_ccache -syn keyword ngxDirectiveThirdParty contained auth_gss_service_name +" 3rd party modules list taken from +" https://github.com/freebsd/freebsd-ports/blob/main/www/nginx-devel/Makefile.extmod +" ---------------------------------------------------------------------------------- -" LDAP Authentication -" https://github.com/kvspb/nginx-auth-ldap -syn keyword ngxDirectiveThirdParty contained auth_ldap -syn keyword ngxDirectiveThirdParty contained auth_ldap_cache_enabled -syn keyword ngxDirectiveThirdParty contained auth_ldap_cache_expiration_time -syn keyword ngxDirectiveThirdParty contained auth_ldap_cache_size -syn keyword ngxDirectiveThirdParty contained auth_ldap_servers -syn keyword ngxDirectiveThirdParty contained auth_ldap_servers_size -syn keyword ngxDirectiveThirdParty contained ldap_server - -" PAM Authentication -" https://github.com/sto/ngx_http_auth_pam_module -syn keyword ngxDirectiveThirdParty contained auth_pam -syn keyword ngxDirectiveThirdParty contained auth_pam_service_name -syn keyword ngxDirectiveThirdParty contained auth_pam_set_pam_env - -" AJP protocol proxy -" https://github.com/yaoweibin/nginx_ajp_module +" https://github.com/msva/nginx_ajp_module +syn keyword ngxDirectiveThirdParty contained ajp_buffer_size syn keyword ngxDirectiveThirdParty contained ajp_buffers -syn keyword ngxDirectiveThirdParty contained ajp_buffer_size syn keyword ngxDirectiveThirdParty contained ajp_busy_buffers_size syn keyword ngxDirectiveThirdParty contained ajp_cache syn keyword ngxDirectiveThirdParty contained ajp_cache_key @@ -850,11 +794,13 @@ syn keyword ngxDirectiveThirdParty contained ajp_max_data_packet_size syn keyword ngxDirectiveThirdParty contained ajp_max_temp_file_size syn keyword ngxDirectiveThirdParty contained ajp_next_upstream +syn keyword ngxDirectiveThirdParty contained ajp_param syn keyword ngxDirectiveThirdParty contained ajp_pass syn keyword ngxDirectiveThirdParty contained ajp_pass_header syn keyword ngxDirectiveThirdParty contained ajp_pass_request_body syn keyword ngxDirectiveThirdParty contained ajp_pass_request_headers syn keyword ngxDirectiveThirdParty contained ajp_read_timeout +syn keyword ngxDirectiveThirdParty contained ajp_script_url syn keyword ngxDirectiveThirdParty contained ajp_secret syn keyword ngxDirectiveThirdParty contained ajp_send_lowat syn keyword ngxDirectiveThirdParty contained ajp_send_timeout @@ -865,7 +811,12 @@ syn keyword ngxDirectiveThirdParty contained ajp_upstream_fail_timeout syn keyword ngxDirectiveThirdParty contained ajp_upstream_max_fails -" AWS proxy +" https://github.com/openresty/array-var-nginx-module +syn keyword ngxDirectiveThirdParty contained array_join +syn keyword ngxDirectiveThirdParty contained array_map +syn keyword ngxDirectiveThirdParty contained array_map_op +syn keyword ngxDirectiveThirdParty contained array_split + " https://github.com/anomalizer/ngx_aws_auth syn keyword ngxDirectiveThirdParty contained aws_access_key syn keyword ngxDirectiveThirdParty contained aws_endpoint @@ -874,7 +825,18 @@ syn keyword ngxDirectiveThirdParty contained aws_sign syn keyword ngxDirectiveThirdParty contained aws_signing_key -" embedding Clojure or Java or Groovy programs +" https://github.com/google/ngx_brotli +syn keyword ngxDirectiveThirdParty contained brotli +syn keyword ngxDirectiveThirdParty contained brotli_buffers +syn keyword ngxDirectiveThirdParty contained brotli_comp_level +syn keyword ngxDirectiveThirdParty contained brotli_min_length +syn keyword ngxDirectiveThirdParty contained brotli_static +syn keyword ngxDirectiveThirdParty contained brotli_types +syn keyword ngxDirectiveThirdParty contained brotli_window + +" https://github.com/torden/ngx_cache_purge +syn keyword ngxDirectiveThirdParty contained cache_purge_response_type + " https://github.com/nginx-clojure/nginx-clojure syn keyword ngxDirectiveThirdParty contained access_handler_code syn keyword ngxDirectiveThirdParty contained access_handler_name @@ -892,8 +854,8 @@ syn keyword ngxDirectiveThirdParty contained content_handler_type syn keyword ngxDirectiveThirdParty contained handler_code syn keyword ngxDirectiveThirdParty contained handler_name +syn keyword ngxDirectiveThirdParty contained handler_type syn keyword ngxDirectiveThirdParty contained handlers_lazy_init -syn keyword ngxDirectiveThirdParty contained handler_type syn keyword ngxDirectiveThirdParty contained header_filter_code syn keyword ngxDirectiveThirdParty contained header_filter_name syn keyword ngxDirectiveThirdParty contained header_filter_property @@ -921,18 +883,20 @@ syn keyword ngxDirectiveThirdParty contained shared_map syn keyword ngxDirectiveThirdParty contained write_page_size +" https://github.com/AirisX/nginx_cookie_flag_module +syn keyword ngxDirectiveThirdParty contained set_cookie_flag -" Certificate Transparency " https://github.com/grahamedgecombe/nginx-ct syn keyword ngxDirectiveThirdParty contained ssl_ct syn keyword ngxDirectiveThirdParty contained ssl_ct_static_scts -" ngx_echo " https://github.com/openresty/echo-nginx-module +syn keyword ngxDirectiveThirdParty contained echo syn keyword ngxDirectiveThirdParty contained echo_abort_parent syn keyword ngxDirectiveThirdParty contained echo_after_body syn keyword ngxDirectiveThirdParty contained echo_before_body syn keyword ngxDirectiveThirdParty contained echo_blocking_sleep +syn keyword ngxDirectiveThirdParty contained echo_duplicate syn keyword ngxDirectiveThirdParty contained echo_end syn keyword ngxDirectiveThirdParty contained echo_exec syn keyword ngxDirectiveThirdParty contained echo_flush @@ -942,28 +906,124 @@ syn keyword ngxDirectiveThirdParty contained echo_read_request_body syn keyword ngxDirectiveThirdParty contained echo_request_body syn keyword ngxDirectiveThirdParty contained echo_reset_timer +syn keyword ngxDirectiveThirdParty contained echo_sleep syn keyword ngxDirectiveThirdParty contained echo_status syn keyword ngxDirectiveThirdParty contained echo_subrequest syn keyword ngxDirectiveThirdParty contained echo_subrequest_async -" FastDFS -" https://github.com/happyfish100/fastdfs-nginx-module -syn keyword ngxDirectiveThirdParty contained ngx_fastdfs_module +" https://github.com/openresty/drizzle-nginx-module +syn keyword ngxDirectiveThirdParty contained drizzle_buffer_size +syn keyword ngxDirectiveThirdParty contained drizzle_connect_timeout +syn keyword ngxDirectiveThirdParty contained drizzle_dbname +syn keyword ngxDirectiveThirdParty contained drizzle_keepalive +syn keyword ngxDirectiveThirdParty contained drizzle_module_header +syn keyword ngxDirectiveThirdParty contained drizzle_pass +syn keyword ngxDirectiveThirdParty contained drizzle_query +syn keyword ngxDirectiveThirdParty contained drizzle_recv_cols_timeout +syn keyword ngxDirectiveThirdParty contained drizzle_recv_rows_timeout +syn keyword ngxDirectiveThirdParty contained drizzle_send_query_timeout +syn keyword ngxDirectiveThirdParty contained drizzle_server +syn keyword ngxDirectiveThirdParty contained drizzle_status + +" https://github.com/ZigzagAK/ngx_dynamic_upstream +syn keyword ngxDirectiveThirdParty contained dns_add_down +syn keyword ngxDirectiveThirdParty contained dns_ipv6 +syn keyword ngxDirectiveThirdParty contained dns_update +syn keyword ngxDirectiveThirdParty contained dynamic_state_file +syn keyword ngxDirectiveThirdParty contained dynamic_upstream -" ngx_headers_more +" https://github.com/ZigzagAK/ngx_dynamic_healthcheck +syn keyword ngxDirectiveThirdParty contained check +syn keyword ngxDirectiveThirdParty contained check_disable_host +syn keyword ngxDirectiveThirdParty contained check_exclude_host +syn keyword ngxDirectiveThirdParty contained check_persistent +syn keyword ngxDirectiveThirdParty contained check_request_body +syn keyword ngxDirectiveThirdParty contained check_request_headers +syn keyword ngxDirectiveThirdParty contained check_request_uri +syn keyword ngxDirectiveThirdParty contained check_response_body +syn keyword ngxDirectiveThirdParty contained check_response_codes +syn keyword ngxDirectiveThirdParty contained healthcheck +syn keyword ngxDirectiveThirdParty contained healthcheck_buffer_size +syn keyword ngxDirectiveThirdParty contained healthcheck_disable_host +syn keyword ngxDirectiveThirdParty contained healthcheck_get +syn keyword ngxDirectiveThirdParty contained healthcheck_persistent +syn keyword ngxDirectiveThirdParty contained healthcheck_request_body +syn keyword ngxDirectiveThirdParty contained healthcheck_request_headers +syn keyword ngxDirectiveThirdParty contained healthcheck_request_uri +syn keyword ngxDirectiveThirdParty contained healthcheck_response_body +syn keyword ngxDirectiveThirdParty contained healthcheck_response_codes +syn keyword ngxDirectiveThirdParty contained healthcheck_status +syn keyword ngxDirectiveThirdParty contained healthcheck_update + +" https://github.com/openresty/encrypted-session-nginx-module +syn keyword ngxDirectiveThirdParty contained encrypted_session_expires +syn keyword ngxDirectiveThirdParty contained encrypted_session_iv +syn keyword ngxDirectiveThirdParty contained encrypted_session_key +syn keyword ngxDirectiveThirdParty contained set_decrypt_session +syn keyword ngxDirectiveThirdParty contained set_encrypt_session + +" https://github.com/calio/form-input-nginx-module +syn keyword ngxDirectiveThirdParty contained set_form_input +syn keyword ngxDirectiveThirdParty contained set_form_input_multi + +" https://github.com/nieoding/nginx-gridfs +syn keyword ngxDirectiveThirdParty contained gridfs +syn keyword ngxDirectiveThirdParty contained mongo + " https://github.com/openresty/headers-more-nginx-module syn keyword ngxDirectiveThirdParty contained more_clear_headers syn keyword ngxDirectiveThirdParty contained more_clear_input_headers syn keyword ngxDirectiveThirdParty contained more_set_headers syn keyword ngxDirectiveThirdParty contained more_set_input_headers -" NGINX WebDAV missing commands support (PROPFIND & OPTIONS) +" https://github.com/dvershinin/nginx_accept_language_module +syn keyword ngxDirectiveThirdParty contained set_from_accept_language + +" https://github.com/atomx/nginx-http-auth-digest +syn keyword ngxDirectiveThirdParty contained auth_digest +syn keyword ngxDirectiveThirdParty contained auth_digest_drop_time +syn keyword ngxDirectiveThirdParty contained auth_digest_evasion_time +syn keyword ngxDirectiveThirdParty contained auth_digest_expires +syn keyword ngxDirectiveThirdParty contained auth_digest_maxtries +syn keyword ngxDirectiveThirdParty contained auth_digest_replays +syn keyword ngxDirectiveThirdParty contained auth_digest_shm_size +syn keyword ngxDirectiveThirdParty contained auth_digest_timeout +syn keyword ngxDirectiveThirdParty contained auth_digest_user_file + +" https://github.com/stnoonan/spnego-http-auth-nginx-module +syn keyword ngxDirectiveThirdParty contained auth_gss +syn keyword ngxDirectiveThirdParty contained auth_gss_allow_basic_fallback +syn keyword ngxDirectiveThirdParty contained auth_gss_authorized_principal +syn keyword ngxDirectiveThirdParty contained auth_gss_authorized_principal_regex +syn keyword ngxDirectiveThirdParty contained auth_gss_constrained_delegation +syn keyword ngxDirectiveThirdParty contained auth_gss_delegate_credentials +syn keyword ngxDirectiveThirdParty contained auth_gss_force_realm +syn keyword ngxDirectiveThirdParty contained auth_gss_format_full +syn keyword ngxDirectiveThirdParty contained auth_gss_keytab +syn keyword ngxDirectiveThirdParty contained auth_gss_map_to_local +syn keyword ngxDirectiveThirdParty contained auth_gss_realm +syn keyword ngxDirectiveThirdParty contained auth_gss_service_ccache +syn keyword ngxDirectiveThirdParty contained auth_gss_service_name + +" https://github.com/kvspb/nginx-auth-ldap +syn keyword ngxDirectiveThirdParty contained auth_ldap +syn keyword ngxDirectiveThirdParty contained auth_ldap_cache_enabled +syn keyword ngxDirectiveThirdParty contained auth_ldap_cache_expiration_time +syn keyword ngxDirectiveThirdParty contained auth_ldap_cache_size +syn keyword ngxDirectiveThirdParty contained auth_ldap_servers +syn keyword ngxDirectiveThirdParty contained auth_ldap_servers_size +syn keyword ngxDirectiveThirdParty contained ldap_server + +" https://github.com/sto/ngx_http_auth_pam_module +syn keyword ngxDirectiveThirdParty contained auth_pam +syn keyword ngxDirectiveThirdParty contained auth_pam_service_name +syn keyword ngxDirectiveThirdParty contained auth_pam_set_pam_env + " https://github.com/arut/nginx-dav-ext-module syn keyword ngxDirectiveThirdParty contained dav_ext_lock syn keyword ngxDirectiveThirdParty contained dav_ext_lock_zone syn keyword ngxDirectiveThirdParty contained dav_ext_methods -" ngx_eval " https://github.com/openresty/nginx-eval-module syn keyword ngxDirectiveThirdParty contained eval syn keyword ngxDirectiveThirdParty contained eval_buffer_size @@ -971,7 +1031,6 @@ syn keyword ngxDirectiveThirdParty contained eval_override_content_type syn keyword ngxDirectiveThirdParty contained eval_subrequest_in_memory -" Fancy Index " https://github.com/aperezdc/ngx-fancyindex syn keyword ngxDirectiveThirdParty contained fancyindex syn keyword ngxDirectiveThirdParty contained fancyindex_css_href @@ -988,40 +1047,29 @@ syn keyword ngxDirectiveThirdParty contained fancyindex_show_path syn keyword ngxDirectiveThirdParty contained fancyindex_time_format -" Footer filter " https://github.com/alibaba/nginx-http-footer-filter syn keyword ngxDirectiveThirdParty contained footer syn keyword ngxDirectiveThirdParty contained footer_types -" ngx_http_geoip2_module " https://github.com/leev/ngx_http_geoip2_module syn keyword ngxDirectiveThirdParty contained geoip2 syn keyword ngxDirectiveThirdParty contained geoip2_proxy syn keyword ngxDirectiveThirdParty contained geoip2_proxy_recursive -" A version of the Nginx HTTP stub status module that outputs in JSON format -" https://github.com/nginx-modules/nginx-json-status-module -syn keyword ngxDirectiveThirdParty contained json_status -syn keyword ngxDirectiveThirdParty contained json_status_type +" https://github.com/ip2location/ip2location-nginx +syn keyword ngxDirectiveThirdParty contained ip2location_database +syn keyword ngxDirectiveThirdParty contained ip2location_proxy +syn keyword ngxDirectiveThirdParty contained ip2location_proxy_recursive -" MogileFS client for nginx -" https://github.com/vkholodkov/nginx-mogilefs-module -syn keyword ngxDirectiveThirdParty contained mogilefs_class -syn keyword ngxDirectiveThirdParty contained mogilefs_connect_timeout -syn keyword ngxDirectiveThirdParty contained mogilefs_domain -syn keyword ngxDirectiveThirdParty contained mogilefs_methods -syn keyword ngxDirectiveThirdParty contained mogilefs_noverify -syn keyword ngxDirectiveThirdParty contained mogilefs_pass -syn keyword ngxDirectiveThirdParty contained mogilefs_read_timeout -syn keyword ngxDirectiveThirdParty contained mogilefs_send_timeout -syn keyword ngxDirectiveThirdParty contained mogilefs_tracker +" https://github.com/ip2location/ip2proxy-nginx +syn keyword ngxDirectiveThirdParty contained ip2proxy_database +syn keyword ngxDirectiveThirdParty contained ip2proxy_proxy +syn keyword ngxDirectiveThirdParty contained ip2proxy_proxy_recursive -" Ancient nginx plugin; probably not useful to anyone " https://github.com/kr/nginx-notice syn keyword ngxDirectiveThirdParty contained notice syn keyword ngxDirectiveThirdParty contained notice_type -" nchan " https://github.com/slact/nchan syn keyword ngxDirectiveThirdParty contained nchan_access_control_allow_credentials syn keyword ngxDirectiveThirdParty contained nchan_access_control_allow_origin @@ -1034,8 +1082,8 @@ syn keyword ngxDirectiveThirdParty contained nchan_benchmark_subscriber_distribution syn keyword ngxDirectiveThirdParty contained nchan_benchmark_subscribers_per_channel syn keyword ngxDirectiveThirdParty contained nchan_benchmark_time +syn keyword ngxDirectiveThirdParty contained nchan_channel_event_string syn keyword ngxDirectiveThirdParty contained nchan_channel_events_channel_id -syn keyword ngxDirectiveThirdParty contained nchan_channel_event_string syn keyword ngxDirectiveThirdParty contained nchan_channel_group syn keyword ngxDirectiveThirdParty contained nchan_channel_group_accounting syn keyword ngxDirectiveThirdParty contained nchan_channel_id @@ -1074,11 +1122,25 @@ syn keyword ngxDirectiveThirdParty contained nchan_pubsub_channel_id syn keyword ngxDirectiveThirdParty contained nchan_pubsub_location syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_check_interval +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_check_interval_backoff +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_check_interval_jitter +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_check_interval_max +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_check_interval_min +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_connect_timeout +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_max_failing_time +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_recovery_delay +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_recovery_delay_backoff +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_recovery_delay_jitter +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_recovery_delay_max +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_recovery_delay_min +syn keyword ngxDirectiveThirdParty contained nchan_redis_command_timeout syn keyword ngxDirectiveThirdParty contained nchan_redis_connect_timeout syn keyword ngxDirectiveThirdParty contained nchan_redis_discovered_ip_range_blacklist syn keyword ngxDirectiveThirdParty contained nchan_redis_fakesub_timer_interval syn keyword ngxDirectiveThirdParty contained nchan_redis_idle_channel_cache_timeout +syn keyword ngxDirectiveThirdParty contained nchan_redis_load_scripts_unconditionally syn keyword ngxDirectiveThirdParty contained nchan_redis_namespace +syn keyword ngxDirectiveThirdParty contained nchan_redis_node_connect_timeout syn keyword ngxDirectiveThirdParty contained nchan_redis_nostore_fastpublish syn keyword ngxDirectiveThirdParty contained nchan_redis_optimize_target syn keyword ngxDirectiveThirdParty contained nchan_redis_pass @@ -1086,6 +1148,13 @@ syn keyword ngxDirectiveThirdParty contained nchan_redis_password syn keyword ngxDirectiveThirdParty contained nchan_redis_ping_interval syn keyword ngxDirectiveThirdParty contained nchan_redis_publish_msgpacked_max_size +syn keyword ngxDirectiveThirdParty contained nchan_redis_reconnect_delay +syn keyword ngxDirectiveThirdParty contained nchan_redis_reconnect_delay_backoff +syn keyword ngxDirectiveThirdParty contained nchan_redis_reconnect_delay_jitter +syn keyword ngxDirectiveThirdParty contained nchan_redis_reconnect_delay_max +syn keyword ngxDirectiveThirdParty contained nchan_redis_reconnect_delay_min +syn keyword ngxDirectiveThirdParty contained nchan_redis_retry_commands +syn keyword ngxDirectiveThirdParty contained nchan_redis_retry_commands_max_wait syn keyword ngxDirectiveThirdParty contained nchan_redis_server syn keyword ngxDirectiveThirdParty contained nchan_redis_ssl syn keyword ngxDirectiveThirdParty contained nchan_redis_ssl_ciphers @@ -1113,10 +1182,10 @@ syn keyword ngxDirectiveThirdParty contained nchan_stub_status syn keyword ngxDirectiveThirdParty contained nchan_sub_channel_id syn keyword ngxDirectiveThirdParty contained nchan_subscribe_existing_channels_only +syn keyword ngxDirectiveThirdParty contained nchan_subscribe_request syn keyword ngxDirectiveThirdParty contained nchan_subscriber syn keyword ngxDirectiveThirdParty contained nchan_subscriber_channel_id syn keyword ngxDirectiveThirdParty contained nchan_subscriber_compound_etag_message_id -syn keyword ngxDirectiveThirdParty contained nchan_subscribe_request syn keyword ngxDirectiveThirdParty contained nchan_subscriber_first_message syn keyword ngxDirectiveThirdParty contained nchan_subscriber_http_raw_stream_separator syn keyword ngxDirectiveThirdParty contained nchan_subscriber_info @@ -1145,7 +1214,6 @@ syn keyword ngxDirectiveThirdParty contained push_subscriber_concurrency syn keyword ngxDirectiveThirdParty contained push_subscriber_timeout -" Push Stream " https://github.com/wandenberg/nginx-push-stream-module syn keyword ngxDirectiveThirdParty contained push_stream_allow_connections_to_events_channel syn keyword ngxDirectiveThirdParty contained push_stream_allowed_origins @@ -1184,23 +1252,6 @@ syn keyword ngxDirectiveThirdParty contained push_stream_wildcard_channel_max_qtd syn keyword ngxDirectiveThirdParty contained push_stream_wildcard_channel_prefix -" redis module -" https://www.nginx.com/resources/wiki/modules/redis/ -syn keyword ngxDirectiveThirdParty contained redis_bind -syn keyword ngxDirectiveThirdParty contained redis_buffer_size -syn keyword ngxDirectiveThirdParty contained redis_connect_timeout -syn keyword ngxDirectiveThirdParty contained redis_gzip_flag -syn keyword ngxDirectiveThirdParty contained redis_next_upstream -syn keyword ngxDirectiveThirdParty contained redis_pass -syn keyword ngxDirectiveThirdParty contained redis_read_timeout -syn keyword ngxDirectiveThirdParty contained redis_send_timeout - -" ngx_http_response -" http://catap.ru/downloads/nginx/ -syn keyword ngxDirectiveThirdParty contained response -syn keyword ngxDirectiveThirdParty contained response_type - -" nginx_substitutions_filter " https://github.com/yaoweibin/ngx_http_substitutions_filter_module syn keyword ngxDirectiveThirdParty contained subs_buffers syn keyword ngxDirectiveThirdParty contained subs_filter @@ -1208,7 +1259,6 @@ syn keyword ngxDirectiveThirdParty contained subs_filter_types syn keyword ngxDirectiveThirdParty contained subs_line_buffer_size -" Tarantool nginx upstream module " https://github.com/tarantool/nginx_upstream_module syn keyword ngxDirectiveThirdParty contained tnt_allowed_indexes syn keyword ngxDirectiveThirdParty contained tnt_allowed_spaces @@ -1238,44 +1288,28 @@ syn keyword ngxDirectiveThirdParty contained tnt_update syn keyword ngxDirectiveThirdParty contained tnt_upsert -" A module for nginx web server for handling file uploads using multipart/form-data encoding (RFC 1867) -" https://github.com/Austinb/nginx-upload-module +" https://github.com/fdintino/nginx-upload-module +syn keyword ngxDirectiveThirdParty contained upload_add_header syn keyword ngxDirectiveThirdParty contained upload_aggregate_form_field -syn keyword ngxDirectiveThirdParty contained upload_archive_elm -syn keyword ngxDirectiveThirdParty contained upload_archive_elm_separator -syn keyword ngxDirectiveThirdParty contained upload_archive_path -syn keyword ngxDirectiveThirdParty contained upload_archive_path_separator syn keyword ngxDirectiveThirdParty contained upload_buffer_size syn keyword ngxDirectiveThirdParty contained upload_cleanup -syn keyword ngxDirectiveThirdParty contained upload_content_type -syn keyword ngxDirectiveThirdParty contained upload_discard -syn keyword ngxDirectiveThirdParty contained upload_field_name -syn keyword ngxDirectiveThirdParty contained upload_file_crc32 -syn keyword ngxDirectiveThirdParty contained upload_file_md5 -syn keyword ngxDirectiveThirdParty contained upload_file_md5_uc -syn keyword ngxDirectiveThirdParty contained upload_file_name -syn keyword ngxDirectiveThirdParty contained upload_file_sha1 -syn keyword ngxDirectiveThirdParty contained upload_file_sha1_uc -syn keyword ngxDirectiveThirdParty contained upload_file_size -syn keyword ngxDirectiveThirdParty contained upload_filter +syn keyword ngxDirectiveThirdParty contained upload_empty_fiels_names +syn keyword ngxDirectiveThirdParty contained upload_limit_rate syn keyword ngxDirectiveThirdParty contained upload_max_file_size syn keyword ngxDirectiveThirdParty contained upload_max_output_body_len syn keyword ngxDirectiveThirdParty contained upload_max_part_header_len +syn keyword ngxDirectiveThirdParty contained upload_merge_buffer_size syn keyword ngxDirectiveThirdParty contained upload_pass syn keyword ngxDirectiveThirdParty contained upload_pass_args syn keyword ngxDirectiveThirdParty contained upload_pass_form_field +syn keyword ngxDirectiveThirdParty contained upload_range_header_buffer_size +syn keyword ngxDirectiveThirdParty contained upload_resumable syn keyword ngxDirectiveThirdParty contained upload_set_form_field +syn keyword ngxDirectiveThirdParty contained upload_state_store syn keyword ngxDirectiveThirdParty contained upload_store syn keyword ngxDirectiveThirdParty contained upload_store_access -syn keyword ngxDirectiveThirdParty contained upload_tmp_path -syn keyword ngxDirectiveThirdParty contained upload_unzip -syn keyword ngxDirectiveThirdParty contained upload_unzip_buffers -syn keyword ngxDirectiveThirdParty contained upload_unzip_hash -syn keyword ngxDirectiveThirdParty contained upload_unzip_max_file_name_len -syn keyword ngxDirectiveThirdParty contained upload_unzip_window -syn keyword ngxDirectiveThirdParty contained upload_void_content_type +syn keyword ngxDirectiveThirdParty contained upload_tame_arrays -" nginx-upload-progress-module " https://github.com/masterzen/nginx-upload-progress-module syn keyword ngxDirectiveThirdParty contained report_uploads syn keyword ngxDirectiveThirdParty contained track_uploads @@ -1288,9 +1322,7 @@ syn keyword ngxDirectiveThirdParty contained upload_progress_jsonp_parameter syn keyword ngxDirectiveThirdParty contained upload_progress_template -" Health checks upstreams for nginx " https://github.com/yaoweibin/nginx_upstream_check_module -syn keyword ngxDirectiveThirdParty contained check syn keyword ngxDirectiveThirdParty contained check_fastcgi_param syn keyword ngxDirectiveThirdParty contained check_http_expect_alive syn keyword ngxDirectiveThirdParty contained check_http_send @@ -1298,13 +1330,14 @@ syn keyword ngxDirectiveThirdParty contained check_shm_size syn keyword ngxDirectiveThirdParty contained check_status -" The fair load balancer module for nginx -" https://github.com/cryptofuture/nginx-upstream-fair +" https://github.com/jaygooby/nginx-upstream-fair syn keyword ngxDirectiveThirdParty contained fair syn keyword ngxDirectiveThirdParty contained upstream_fair_shm_size -" Nginx Video Thumb Extractor Module -" https://github.com/wandenberg/nginx-video-thumbextractor-module +" https://github.com/ayty-adrianomartins/nginx-sticky-module-ng +syn keyword ngxDirectiveThirdParty contained sticky_no_fallback + +" https://github.com/Novetta/nginx-video-thumbextractor-module syn keyword ngxDirectiveThirdParty contained video_thumbextractor syn keyword ngxDirectiveThirdParty contained video_thumbextractor_image_height syn keyword ngxDirectiveThirdParty contained video_thumbextractor_image_width @@ -1329,43 +1362,14 @@ syn keyword ngxDirectiveThirdParty contained video_thumbextractor_video_filename syn keyword ngxDirectiveThirdParty contained video_thumbextractor_video_second -" drizzle-nginx-module - Upstream module for talking to MySQL and Drizzle directly -" https://github.com/openresty/drizzle-nginx-module -syn keyword ngxDirectiveThirdParty contained drizzle_buffer_size -syn keyword ngxDirectiveThirdParty contained drizzle_connect_timeout -syn keyword ngxDirectiveThirdParty contained drizzle_dbname -syn keyword ngxDirectiveThirdParty contained drizzle_keepalive -syn keyword ngxDirectiveThirdParty contained drizzle_module_header -syn keyword ngxDirectiveThirdParty contained drizzle_pass -syn keyword ngxDirectiveThirdParty contained drizzle_query -syn keyword ngxDirectiveThirdParty contained drizzle_recv_cols_timeout -syn keyword ngxDirectiveThirdParty contained drizzle_recv_rows_timeout -syn keyword ngxDirectiveThirdParty contained drizzle_send_query_timeout -syn keyword ngxDirectiveThirdParty contained drizzle_server -syn keyword ngxDirectiveThirdParty contained drizzle_status +" https://github.com/calio/iconv-nginx-module +syn keyword ngxDirectiveThirdParty contained iconv_buffer_size +syn keyword ngxDirectiveThirdParty contained iconv_filter +syn keyword ngxDirectiveThirdParty contained set_iconv -" ngx_dynamic_upstream -" https://github.com/cubicdaiya/ngx_dynamic_upstream -syn keyword ngxDirectiveThirdParty contained dynamic_upstream - -" encrypt and decrypt nginx variable values -" https://github.com/openresty/encrypted-session-nginx-module -syn keyword ngxDirectiveThirdParty contained encrypted_session_expires -syn keyword ngxDirectiveThirdParty contained encrypted_session_iv -syn keyword ngxDirectiveThirdParty contained encrypted_session_key -syn keyword ngxDirectiveThirdParty contained set_decrypt_session -syn keyword ngxDirectiveThirdParty contained set_encrypt_session - -" serve content directly from MongoDB's GridFS -" https://github.com/mdirolf/nginx-gridfs -syn keyword ngxDirectiveThirdParty contained gridfs -syn keyword ngxDirectiveThirdParty contained mongo - -" Adds support for arithmetic operations to NGINX config -" https://github.com/arut/nginx-let-module +" https://github.com/baysao/nginx-let-module syn keyword ngxDirectiveThirdParty contained let -" ngx_http_lua_module - Embed the power of Lua into Nginx HTTP Servers " https://github.com/openresty/lua-nginx-module syn keyword ngxDirectiveThirdParty contained access_by_lua syn keyword ngxDirectiveThirdParty contained access_by_lua_block @@ -1431,6 +1435,8 @@ syn keyword ngxDirectiveThirdParty contained rewrite_by_lua_block syn keyword ngxDirectiveThirdParty contained rewrite_by_lua_file syn keyword ngxDirectiveThirdParty contained rewrite_by_lua_no_postpone +syn keyword ngxDirectiveThirdParty contained server_rewrite_by_lua_block +syn keyword ngxDirectiveThirdParty contained server_rewrite_by_lua_file syn keyword ngxDirectiveThirdParty contained set_by_lua syn keyword ngxDirectiveThirdParty contained set_by_lua_block syn keyword ngxDirectiveThirdParty contained set_by_lua_file @@ -1443,7 +1449,16 @@ syn keyword ngxDirectiveThirdParty contained ssl_session_store_by_lua_block syn keyword ngxDirectiveThirdParty contained ssl_session_store_by_lua_file -" ngx_memc - An extended version of the standard memcached module +" https://github.com/Taymindis/nginx-link-function +syn keyword ngxDirectiveThirdParty contained ngx_link_func_add_prop +syn keyword ngxDirectiveThirdParty contained ngx_link_func_add_req_header +syn keyword ngxDirectiveThirdParty contained ngx_link_func_ca_cert +syn keyword ngxDirectiveThirdParty contained ngx_link_func_call +syn keyword ngxDirectiveThirdParty contained ngx_link_func_download_link_lib +syn keyword ngxDirectiveThirdParty contained ngx_link_func_lib +syn keyword ngxDirectiveThirdParty contained ngx_link_func_shm_size +syn keyword ngxDirectiveThirdParty contained ngx_link_func_subrequest + " https://github.com/openresty/memc-nginx-module syn keyword ngxDirectiveThirdParty contained memc_buffer_size syn keyword ngxDirectiveThirdParty contained memc_cmds_allowed @@ -1457,21 +1472,24 @@ syn keyword ngxDirectiveThirdParty contained memc_upstream_fail_timeout syn keyword ngxDirectiveThirdParty contained memc_upstream_max_fails -" ModSecurity web application firewall -" https://github.com/SpiderLabs/ModSecurity/tree/master -syn keyword ngxDirectiveThirdParty contained ModSecurityConfig -syn keyword ngxDirectiveThirdParty contained ModSecurityEnabled -syn keyword ngxDirectiveThirdParty contained pool_context_hash_size +" https://github.com/SpiderLabs/ModSecurity-nginx +syn keyword ngxDirectiveThirdParty contained modsecurity +syn keyword ngxDirectiveThirdParty contained modsecurity_rules +syn keyword ngxDirectiveThirdParty contained modsecurity_rules_file +syn keyword ngxDirectiveThirdParty contained modsecurity_rules_remote +syn keyword ngxDirectiveThirdParty contained modsecurity_transaction_id -" NAXSI is an open-source, high performance, low rules maintenance WAF for NGINX " https://github.com/nbs-system/naxsi syn keyword ngxDirectiveThirdParty contained BasicRule syn keyword ngxDirectiveThirdParty contained CheckRule syn keyword ngxDirectiveThirdParty contained DeniedUrl +syn keyword ngxDirectiveThirdParty contained IgnoreCIDR +syn keyword ngxDirectiveThirdParty contained IgnoreIP syn keyword ngxDirectiveThirdParty contained LearningMode syn keyword ngxDirectiveThirdParty contained LibInjectionSql syn keyword ngxDirectiveThirdParty contained LibInjectionXss syn keyword ngxDirectiveThirdParty contained MainRule +syn keyword ngxDirectiveThirdParty contained NaxsiLogFile syn keyword ngxDirectiveThirdParty contained SecRulesDisabled syn keyword ngxDirectiveThirdParty contained SecRulesEnabled syn keyword ngxDirectiveThirdParty contained basic_rule @@ -1481,17 +1499,31 @@ syn keyword ngxDirectiveThirdParty contained libinjection_sql syn keyword ngxDirectiveThirdParty contained libinjection_xss syn keyword ngxDirectiveThirdParty contained main_rule +syn keyword ngxDirectiveThirdParty contained naxsi_log syn keyword ngxDirectiveThirdParty contained rules_disabled syn keyword ngxDirectiveThirdParty contained rules_enabled -" Phusion Passenger -" https://www.phusionpassenger.com/library/config/nginx/reference/ +" https://github.com/opentracing-contrib/nginx-opentracing +syn keyword ngxDirectiveThirdParty contained opentracing +syn keyword ngxDirectiveThirdParty contained opentracing_fastcgi_propagate_context +syn keyword ngxDirectiveThirdParty contained opentracing_grpc_propagate_context +syn keyword ngxDirectiveThirdParty contained opentracing_load_tracer +syn keyword ngxDirectiveThirdParty contained opentracing_location_operation_name +syn keyword ngxDirectiveThirdParty contained opentracing_operation_name +syn keyword ngxDirectiveThirdParty contained opentracing_propagate_context +syn keyword ngxDirectiveThirdParty contained opentracing_tag +syn keyword ngxDirectiveThirdParty contained opentracing_trace_locations +syn keyword ngxDirectiveThirdParty contained opentracing_trust_incoming_span + +" https://github.com/phusion/passenger syn keyword ngxDirectiveThirdParty contained passenger_abort_on_startup_error syn keyword ngxDirectiveThirdParty contained passenger_abort_websockets_on_process_shutdown syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_auth_type syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_password syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_url syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_username +syn keyword ngxDirectiveThirdParty contained passenger_analytics_log_group +syn keyword ngxDirectiveThirdParty contained passenger_analytics_log_user syn keyword ngxDirectiveThirdParty contained passenger_anonymous_telemetry_proxy syn keyword ngxDirectiveThirdParty contained passenger_app_env syn keyword ngxDirectiveThirdParty contained passenger_app_file_descriptor_ulimit @@ -1499,20 +1531,25 @@ syn keyword ngxDirectiveThirdParty contained passenger_app_log_file syn keyword ngxDirectiveThirdParty contained passenger_app_rights syn keyword ngxDirectiveThirdParty contained passenger_app_root +syn keyword ngxDirectiveThirdParty contained passenger_app_start_command syn keyword ngxDirectiveThirdParty contained passenger_app_type syn keyword ngxDirectiveThirdParty contained passenger_base_uri syn keyword ngxDirectiveThirdParty contained passenger_buffer_response syn keyword ngxDirectiveThirdParty contained passenger_buffer_size +syn keyword ngxDirectiveThirdParty contained passenger_buffer_upload syn keyword ngxDirectiveThirdParty contained passenger_buffers syn keyword ngxDirectiveThirdParty contained passenger_busy_buffers_size syn keyword ngxDirectiveThirdParty contained passenger_concurrency_model syn keyword ngxDirectiveThirdParty contained passenger_core_file_descriptor_ulimit syn keyword ngxDirectiveThirdParty contained passenger_ctl syn keyword ngxDirectiveThirdParty contained passenger_data_buffer_dir +syn keyword ngxDirectiveThirdParty contained passenger_debug_log_file syn keyword ngxDirectiveThirdParty contained passenger_debugger syn keyword ngxDirectiveThirdParty contained passenger_default_group syn keyword ngxDirectiveThirdParty contained passenger_default_user +syn keyword ngxDirectiveThirdParty contained passenger_direct_instance_request_address syn keyword ngxDirectiveThirdParty contained passenger_disable_anonymous_telemetry +syn keyword ngxDirectiveThirdParty contained passenger_disable_log_prefix syn keyword ngxDirectiveThirdParty contained passenger_disable_security_update_check syn keyword ngxDirectiveThirdParty contained passenger_document_root syn keyword ngxDirectiveThirdParty contained passenger_dump_config_manifest @@ -1548,8 +1585,10 @@ syn keyword ngxDirectiveThirdParty contained passenger_pass_header syn keyword ngxDirectiveThirdParty contained passenger_pool_idle_time syn keyword ngxDirectiveThirdParty contained passenger_pre_start +syn keyword ngxDirectiveThirdParty contained passenger_preload_bundler syn keyword ngxDirectiveThirdParty contained passenger_python syn keyword ngxDirectiveThirdParty contained passenger_read_timeout +syn keyword ngxDirectiveThirdParty contained passenger_request_buffering syn keyword ngxDirectiveThirdParty contained passenger_request_queue_overflow_status_code syn keyword ngxDirectiveThirdParty contained passenger_resist_deployment_errors syn keyword ngxDirectiveThirdParty contained passenger_response_buffer_high_watermark @@ -1561,36 +1600,36 @@ syn keyword ngxDirectiveThirdParty contained passenger_set_header syn keyword ngxDirectiveThirdParty contained passenger_show_version_in_header syn keyword ngxDirectiveThirdParty contained passenger_socket_backlog +syn keyword ngxDirectiveThirdParty contained passenger_spawn_dir +syn keyword ngxDirectiveThirdParty contained passenger_spawn_exception_status_code syn keyword ngxDirectiveThirdParty contained passenger_spawn_method syn keyword ngxDirectiveThirdParty contained passenger_start_timeout syn keyword ngxDirectiveThirdParty contained passenger_startup_file syn keyword ngxDirectiveThirdParty contained passenger_stat_throttle_rate syn keyword ngxDirectiveThirdParty contained passenger_sticky_sessions +syn keyword ngxDirectiveThirdParty contained passenger_sticky_sessions_cookie_attributes syn keyword ngxDirectiveThirdParty contained passenger_sticky_sessions_cookie_name +syn keyword ngxDirectiveThirdParty contained passenger_temp_path syn keyword ngxDirectiveThirdParty contained passenger_thread_count syn keyword ngxDirectiveThirdParty contained passenger_turbocaching +syn keyword ngxDirectiveThirdParty contained passenger_use_global_queue syn keyword ngxDirectiveThirdParty contained passenger_user syn keyword ngxDirectiveThirdParty contained passenger_user_switching syn keyword ngxDirectiveThirdParty contained passenger_vary_turbocache_by_cookie -syn keyword ngxDirectiveThirdPartyDeprecated contained passenger_analytics_log_group -syn keyword ngxDirectiveThirdPartyDeprecated contained passenger_analytics_log_user -syn keyword ngxDirectiveThirdPartyDeprecated contained passenger_debug_log_file -syn keyword ngxDirectiveThirdPartyDeprecated contained passenger_use_global_queue -syn keyword ngxDirectiveThirdPartyDeprecated contained rack_env -syn keyword ngxDirectiveThirdPartyDeprecated contained rails_app_spawner_idle_time -syn keyword ngxDirectiveThirdPartyDeprecated contained rails_env -syn keyword ngxDirectiveThirdPartyDeprecated contained rails_framework_spawner_idle_time -syn keyword ngxDirectiveThirdPartyDeprecated contained rails_spawn_method -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_filter -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_gateway_address -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_gateway_cert -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_gateway_port -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_key -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_proxy_address -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_support +syn keyword ngxDirectiveThirdParty contained rack_env +syn keyword ngxDirectiveThirdParty contained rails_app_spawner_idle_time +syn keyword ngxDirectiveThirdParty contained rails_env +syn keyword ngxDirectiveThirdParty contained rails_framework_spawner_idle_time +syn keyword ngxDirectiveThirdParty contained rails_spawn_method +syn keyword ngxDirectiveThirdParty contained union_station_filter +syn keyword ngxDirectiveThirdParty contained union_station_gateway_address +syn keyword ngxDirectiveThirdParty contained union_station_gateway_cert +syn keyword ngxDirectiveThirdParty contained union_station_gateway_port +syn keyword ngxDirectiveThirdParty contained union_station_key +syn keyword ngxDirectiveThirdParty contained union_station_proxy_address +syn keyword ngxDirectiveThirdParty contained union_station_support -" ngx_postgres is an upstream module that allows nginx to communicate directly with PostgreSQL database -" https://github.com/FRiCKLE/ngx_postgres +" https://github.com/konstruxi/ngx_postgres syn keyword ngxDirectiveThirdParty contained postgres_connect_timeout syn keyword ngxDirectiveThirdParty contained postgres_escape syn keyword ngxDirectiveThirdParty contained postgres_keepalive @@ -1602,7 +1641,6 @@ syn keyword ngxDirectiveThirdParty contained postgres_server syn keyword ngxDirectiveThirdParty contained postgres_set -" ngx_rds_csv - Nginx output filter module to convert Resty-DBD-Streams (RDS) to Comma-Separated Values (CSV) " https://github.com/openresty/rds-csv-nginx-module syn keyword ngxDirectiveThirdParty contained rds_csv syn keyword ngxDirectiveThirdParty contained rds_csv_buffer_size @@ -1611,7 +1649,6 @@ syn keyword ngxDirectiveThirdParty contained rds_csv_field_separator syn keyword ngxDirectiveThirdParty contained rds_csv_row_terminator -" ngx_rds_json - an output filter that formats Resty DBD Streams generated by ngx_drizzle and others to JSON " https://github.com/openresty/rds-json-nginx-module syn keyword ngxDirectiveThirdParty contained rds_json syn keyword ngxDirectiveThirdParty contained rds_json_buffer_size @@ -1624,7 +1661,6 @@ syn keyword ngxDirectiveThirdParty contained rds_json_success_property syn keyword ngxDirectiveThirdParty contained rds_json_user_property -" ngx_redis2 - Nginx upstream module for the Redis 2.0 protocol " https://github.com/openresty/redis2-nginx-module syn keyword ngxDirectiveThirdParty contained redis2_bind syn keyword ngxDirectiveThirdParty contained redis2_buffer_size @@ -1638,7 +1674,6 @@ syn keyword ngxDirectiveThirdParty contained redis2_read_timeout syn keyword ngxDirectiveThirdParty contained redis2_send_timeout -" NGINX-based Media Streaming Server " https://github.com/arut/nginx-rtmp-module syn keyword ngxDirectiveThirdParty contained ack_window syn keyword ngxDirectiveThirdParty contained application @@ -1750,7 +1785,6 @@ syn keyword ngxDirectiveThirdParty contained wait_key syn keyword ngxDirectiveThirdParty contained wait_video -" ngx_set_misc - Various set_xxx directives added to nginx's rewrite module (md5/sha1, sql/json quoting, and many more) " https://github.com/openresty/set-misc-nginx-module syn keyword ngxDirectiveThirdParty contained set_base32_alphabet syn keyword ngxDirectiveThirdParty contained set_base32_padding @@ -1770,6 +1804,7 @@ syn keyword ngxDirectiveThirdParty contained set_hmac_sha256 syn keyword ngxDirectiveThirdParty contained set_if_empty syn keyword ngxDirectiveThirdParty contained set_local_today +syn keyword ngxDirectiveThirdParty contained set_md5 syn keyword ngxDirectiveThirdParty contained set_misc_base32_padding syn keyword ngxDirectiveThirdParty contained set_quote_json_str syn keyword ngxDirectiveThirdParty contained set_quote_pgsql_str @@ -1778,20 +1813,18 @@ syn keyword ngxDirectiveThirdParty contained set_rotate syn keyword ngxDirectiveThirdParty contained set_secure_random_alphanum syn keyword ngxDirectiveThirdParty contained set_secure_random_lcalpha +syn keyword ngxDirectiveThirdParty contained set_sha1 syn keyword ngxDirectiveThirdParty contained set_unescape_uri -" nginx-sflow-module " https://github.com/sflow/nginx-sflow-module syn keyword ngxDirectiveThirdParty contained sflow -" Shibboleth auth request module for Nginx " https://github.com/nginx-shib/nginx-http-shibboleth syn keyword ngxDirectiveThirdParty contained shib_request syn keyword ngxDirectiveThirdParty contained shib_request_set syn keyword ngxDirectiveThirdParty contained shib_request_use_headers -" nginx module which adds ability to cache static files -" https://github.com/FRiCKLE/ngx_slowfs_cache +" https://github.com/baysao/ngx_slowfs_cache syn keyword ngxDirectiveThirdParty contained slowfs_big_file_size syn keyword ngxDirectiveThirdParty contained slowfs_cache syn keyword ngxDirectiveThirdParty contained slowfs_cache_key @@ -1801,8 +1834,7 @@ syn keyword ngxDirectiveThirdParty contained slowfs_cache_valid syn keyword ngxDirectiveThirdParty contained slowfs_temp_path -" Dynamic Image Transformation Module For nginx -" https://github.com/cubicdaiya/ngx_small_light +" https://github.com/kawakibi/ngx_small_light syn keyword ngxDirectiveThirdParty contained small_light syn keyword ngxDirectiveThirdParty contained small_light_buffer syn keyword ngxDirectiveThirdParty contained small_light_getparam_mode @@ -1812,7 +1844,6 @@ syn keyword ngxDirectiveThirdParty contained small_light_radius_max syn keyword ngxDirectiveThirdParty contained small_light_sigma_max -" ngx_srcache - Transparent subrequest-based caching layout for arbitrary nginx locations " https://github.com/openresty/srcache-nginx-module syn keyword ngxDirectiveThirdParty contained srcache_buffer syn keyword ngxDirectiveThirdParty contained srcache_default_expire @@ -1835,7 +1866,6 @@ syn keyword ngxDirectiveThirdParty contained srcache_store_skip syn keyword ngxDirectiveThirdParty contained srcache_store_statuses -" NGINX-based VOD Packager " https://github.com/kaltura/nginx-vod-module syn keyword ngxDirectiveThirdParty contained vod syn keyword ngxDirectiveThirdParty contained vod_align_segments_to_key_frames @@ -1875,6 +1905,7 @@ syn keyword ngxDirectiveThirdParty contained vod_manifest_duration_policy syn keyword ngxDirectiveThirdParty contained vod_manifest_segment_durations_mode syn keyword ngxDirectiveThirdParty contained vod_mapping_cache +syn keyword ngxDirectiveThirdParty contained vod_max_frame_count syn keyword ngxDirectiveThirdParty contained vod_max_frames_size syn keyword ngxDirectiveThirdParty contained vod_max_mapping_response_size syn keyword ngxDirectiveThirdParty contained vod_max_metadata_size @@ -1901,6 +1932,7 @@ syn keyword ngxDirectiveThirdParty contained vod_secret_key syn keyword ngxDirectiveThirdParty contained vod_segment_count_policy syn keyword ngxDirectiveThirdParty contained vod_segment_duration +syn keyword ngxDirectiveThirdParty contained vod_segment_max_frame_count syn keyword ngxDirectiveThirdParty contained vod_segments_base_url syn keyword ngxDirectiveThirdParty contained vod_source_clip_map_uri syn keyword ngxDirectiveThirdParty contained vod_speed_param_name @@ -1910,7 +1942,6 @@ syn keyword ngxDirectiveThirdParty contained vod_upstream_extra_args syn keyword ngxDirectiveThirdParty contained vod_upstream_location -" Nginx virtual host traffic status module " https://github.com/vozlt/nginx-module-vts syn keyword ngxDirectiveThirdParty contained vhost_traffic_status syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_average_method @@ -1934,7 +1965,6 @@ syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_set_by_filter syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_zone -" xss-nginx-module - Native cross-site scripting support in nginx " https://github.com/openresty/xss-nginx-module syn keyword ngxDirectiveThirdParty contained xss_callback_arg syn keyword ngxDirectiveThirdParty contained xss_check_status @@ -1943,471 +1973,6 @@ syn keyword ngxDirectiveThirdParty contained xss_output_type syn keyword ngxDirectiveThirdParty contained xss_override_status -" Add support for array-typed variables to nginx config files -" https://github.com/openresty/array-var-nginx-module -syn keyword ngxDirectiveThirdParty contained array_join -syn keyword ngxDirectiveThirdParty contained array_map -syn keyword ngxDirectiveThirdParty contained array_map_op -syn keyword ngxDirectiveThirdParty contained array_split - -" NGINX module for Brotli compression -" https://github.com/eustas/ngx_brotli -syn keyword ngxDirectiveThirdParty contained brotli -syn keyword ngxDirectiveThirdParty contained brotli_buffers -syn keyword ngxDirectiveThirdParty contained brotli_comp_level -syn keyword ngxDirectiveThirdParty contained brotli_min_length -syn keyword ngxDirectiveThirdParty contained brotli_static -syn keyword ngxDirectiveThirdParty contained brotli_types -syn keyword ngxDirectiveThirdParty contained brotli_window - -" form-input-nginx-module -" https://github.com/calio/form-input-nginx-module -syn keyword ngxDirectiveThirdParty contained set_form_input -syn keyword ngxDirectiveThirdParty contained set_form_input_multi - -" character conversion nginx module using libiconv -" https://github.com/calio/iconv-nginx-module -syn keyword ngxDirectiveThirdParty contained iconv_buffer_size -syn keyword ngxDirectiveThirdParty contained iconv_filter -syn keyword ngxDirectiveThirdParty contained set_iconv - -" 3rd party modules list taken from -" https://www.nginx.com/resources/wiki/modules/ -" --------------------------------------------- - -" Nginx Module for Authenticating Akamai G2O requests -" https://github.com/kaltura/nginx_mod_akamai_g2o -syn keyword ngxDirectiveThirdParty contained g2o -syn keyword ngxDirectiveThirdParty contained g2o_data_header -syn keyword ngxDirectiveThirdParty contained g2o_hash_function -syn keyword ngxDirectiveThirdParty contained g2o_key -syn keyword ngxDirectiveThirdParty contained g2o_log_level -syn keyword ngxDirectiveThirdParty contained g2o_nonce -syn keyword ngxDirectiveThirdParty contained g2o_sign_header -syn keyword ngxDirectiveThirdParty contained g2o_time_window -syn keyword ngxDirectiveThirdParty contained g2o_version - -" nginx_lua_module -" https://github.com/alacner/nginx_lua_module -syn keyword ngxDirectiveThirdParty contained lua_file - -" Nginx Audio Track for HTTP Live Streaming -" https://github.com/flavioribeiro/nginx-audio-track-for-hls-module -syn keyword ngxDirectiveThirdParty contained ngx_hls_audio_track -syn keyword ngxDirectiveThirdParty contained ngx_hls_audio_track_output_format -syn keyword ngxDirectiveThirdParty contained ngx_hls_audio_track_output_header -syn keyword ngxDirectiveThirdParty contained ngx_hls_audio_track_rootpath - -" A Nginx module to dump backtrace when a worker process exits abnormally -" https://github.com/alibaba/nginx-backtrace -syn keyword ngxDirectiveThirdParty contained backtrace_log -syn keyword ngxDirectiveThirdParty contained backtrace_max_stack_size - -" circle_gif module -" https://github.com/evanmiller/nginx_circle_gif -syn keyword ngxDirectiveThirdParty contained circle_gif -syn keyword ngxDirectiveThirdParty contained circle_gif_max_radius -syn keyword ngxDirectiveThirdParty contained circle_gif_min_radius -syn keyword ngxDirectiveThirdParty contained circle_gif_step_radius - -" Upstream Consistent Hash -" https://github.com/replay/ngx_http_consistent_hash -syn keyword ngxDirectiveThirdParty contained consistent_hash - -" Nginx module for etags on dynamic content -" https://github.com/kali/nginx-dynamic-etags -syn keyword ngxDirectiveThirdParty contained dynamic_etags - -" Enhanced Nginx Memcached Module -" https://github.com/bpaquet/ngx_http_enhanced_memcached_module -syn keyword ngxDirectiveThirdParty contained enhanced_memcached_allow_delete -syn keyword ngxDirectiveThirdParty contained enhanced_memcached_allow_put -syn keyword ngxDirectiveThirdParty contained enhanced_memcached_bind -syn keyword ngxDirectiveThirdParty contained enhanced_memcached_buffer_size -syn keyword ngxDirectiveThirdParty contained enhanced_memcached_connect_timeout -syn keyword ngxDirectiveThirdParty contained enhanced_memcached_flush -syn keyword ngxDirectiveThirdParty contained enhanced_memcached_flush_namespace -syn keyword ngxDirectiveThirdParty contained enhanced_memcached_hash_keys_with_md5 -syn keyword ngxDirectiveThirdParty contained enhanced_memcached_pass -syn keyword ngxDirectiveThirdParty contained enhanced_memcached_read_timeout -syn keyword ngxDirectiveThirdParty contained enhanced_memcached_send_timeout -syn keyword ngxDirectiveThirdParty contained enhanced_memcached_stats - -" nginx max connections queue -" https://github.com/ezmobius/nginx-ey-balancer -syn keyword ngxDirectiveThirdParty contained max_connections_max_queue_length -syn keyword ngxDirectiveThirdParty contained max_connections_queue_timeout - -" Nginx module for POST authentication and authorization -" https://github.com/veruu/ngx_form_auth -syn keyword ngxDirectiveThirdParty contained form_auth -syn keyword ngxDirectiveThirdParty contained form_auth_login -syn keyword ngxDirectiveThirdParty contained form_auth_pam_service -syn keyword ngxDirectiveThirdParty contained form_auth_password -syn keyword ngxDirectiveThirdParty contained form_auth_remote_user - -" ngx_http_accounting_module -" https://github.com/Lax/ngx_http_accounting_module -syn keyword ngxDirectiveThirdParty contained accounting -syn keyword ngxDirectiveThirdParty contained accounting_id -syn keyword ngxDirectiveThirdParty contained accounting_interval -syn keyword ngxDirectiveThirdParty contained accounting_log -syn keyword ngxDirectiveThirdParty contained accounting_perturb - -" concatenating files in a given context: CSS and JS files usually -" https://github.com/alibaba/nginx-http-concat -syn keyword ngxDirectiveThirdParty contained concat -syn keyword ngxDirectiveThirdParty contained concat_delimiter -syn keyword ngxDirectiveThirdParty contained concat_ignore_file_error -syn keyword ngxDirectiveThirdParty contained concat_max_files -syn keyword ngxDirectiveThirdParty contained concat_types -syn keyword ngxDirectiveThirdParty contained concat_unique - -" update upstreams' config by restful interface -" https://github.com/yzprofile/ngx_http_dyups_module -syn keyword ngxDirectiveThirdParty contained dyups_interface -syn keyword ngxDirectiveThirdParty contained dyups_shm_zone_size - -" add given content to the end of the response according to the condition specified -" https://github.com/flygoast/ngx_http_footer_if_filter -syn keyword ngxDirectiveThirdParty contained footer_if - -" NGINX HTTP Internal Redirect Module -" https://github.com/flygoast/ngx_http_internal_redirect -syn keyword ngxDirectiveThirdParty contained internal_redirect_if -syn keyword ngxDirectiveThirdParty contained internal_redirect_if_no_postpone - -" nginx-ip-blocker -" https://github.com/tmthrgd/nginx-ip-blocker -syn keyword ngxDirectiveThirdParty contained ip_blocker - -" IP2Location Nginx -" https://github.com/chrislim2888/ip2location-nginx -syn keyword ngxDirectiveThirdParty contained ip2location_database - -" Limit upload rate -" https://github.com/cfsego/limit_upload_rate -syn keyword ngxDirectiveThirdParty contained limit_upload_rate -syn keyword ngxDirectiveThirdParty contained limit_upload_rate_after -syn keyword ngxDirectiveThirdParty contained limit_upload_rate_log_level - -" limit the number of connections to upstream -" https://github.com/cfsego/nginx-limit-upstream -syn keyword ngxDirectiveThirdParty contained limit_upstream_conn -syn keyword ngxDirectiveThirdParty contained limit_upstream_log_level -syn keyword ngxDirectiveThirdParty contained limit_upstream_zone - -" conditional accesslog for nginx -" https://github.com/cfsego/ngx_log_if -syn keyword ngxDirectiveThirdParty contained access_log_bypass_if - -" log messages over ZeroMQ -" https://github.com/alticelabs/nginx-log-zmq -syn keyword ngxDirectiveThirdParty contained log_zmq_endpoint -syn keyword ngxDirectiveThirdParty contained log_zmq_format -syn keyword ngxDirectiveThirdParty contained log_zmq_off -syn keyword ngxDirectiveThirdParty contained log_zmq_server - -" simple module to uppercase/lowercase strings in the nginx config -" https://github.com/replay/ngx_http_lower_upper_case -syn keyword ngxDirectiveThirdParty contained lower -syn keyword ngxDirectiveThirdParty contained upper - -" content filter for nginx, which returns the md5 hash of the content otherwise returned -" https://github.com/kainswor/nginx_md5_filter -syn keyword ngxDirectiveThirdParty contained md5_filter - -" Non-blocking upstream module for Nginx to connect to MongoDB -" https://github.com/simpl/ngx_mongo -syn keyword ngxDirectiveThirdParty contained mongo_auth -syn keyword ngxDirectiveThirdParty contained mongo_bind -syn keyword ngxDirectiveThirdParty contained mongo_buffer_size -syn keyword ngxDirectiveThirdParty contained mongo_buffering -syn keyword ngxDirectiveThirdParty contained mongo_buffers -syn keyword ngxDirectiveThirdParty contained mongo_busy_buffers_size -syn keyword ngxDirectiveThirdParty contained mongo_connect_timeout -syn keyword ngxDirectiveThirdParty contained mongo_json -syn keyword ngxDirectiveThirdParty contained mongo_next_upstream -syn keyword ngxDirectiveThirdParty contained mongo_pass -syn keyword ngxDirectiveThirdParty contained mongo_query -syn keyword ngxDirectiveThirdParty contained mongo_read_timeout -syn keyword ngxDirectiveThirdParty contained mongo_send_timeout - -" Nginx OCSP processing module designed for response caching -" https://github.com/kyprizel/nginx_ocsp_proxy-module -syn keyword ngxDirectiveThirdParty contained ocsp_cache_timeout -syn keyword ngxDirectiveThirdParty contained ocsp_proxy - -" Nginx OpenSSL version check at startup -" https://github.com/apcera/nginx-openssl-version -syn keyword ngxDirectiveThirdParty contained openssl_builddate_minimum -syn keyword ngxDirectiveThirdParty contained openssl_version_minimum - -" Automatic PageSpeed optimization module for Nginx -" https://github.com/pagespeed/ngx_pagespeed -syn keyword ngxDirectiveThirdParty contained pagespeed - -" PECL Memcache standard hashing compatible loadbalancer for Nginx -" https://github.com/replay/ngx_http_php_memcache_standard_balancer -syn keyword ngxDirectiveThirdParty contained hash_key - -" nginx module to parse php sessions -" https://github.com/replay/ngx_http_php_session -syn keyword ngxDirectiveThirdParty contained php_session_parse -syn keyword ngxDirectiveThirdParty contained php_session_strip_formatting - -" Nginx HTTP rDNS module -" https://github.com/flant/nginx-http-rdns -syn keyword ngxDirectiveThirdParty contained rdns -syn keyword ngxDirectiveThirdParty contained rdns_allow -syn keyword ngxDirectiveThirdParty contained rdns_deny - -" Streaming regular expression replacement in response bodies -" https://github.com/openresty/replace-filter-nginx-module -syn keyword ngxDirectiveThirdParty contained replace_filter -syn keyword ngxDirectiveThirdParty contained replace_filter_last_modified -syn keyword ngxDirectiveThirdParty contained replace_filter_max_buffered_size -syn keyword ngxDirectiveThirdParty contained replace_filter_skip -syn keyword ngxDirectiveThirdParty contained replace_filter_types - -" Link RRDtool's graphing facilities directly into nginx -" https://github.com/evanmiller/mod_rrd_graph -syn keyword ngxDirectiveThirdParty contained rrd_graph -syn keyword ngxDirectiveThirdParty contained rrd_graph_root - -" Module for nginx to proxy rtmp using http protocol -" https://github.com/kwojtek/nginx-rtmpt-proxy-module -syn keyword ngxDirectiveThirdParty contained rtmpt_proxy -syn keyword ngxDirectiveThirdParty contained rtmpt_proxy_http_timeout -syn keyword ngxDirectiveThirdParty contained rtmpt_proxy_rtmp_timeout -syn keyword ngxDirectiveThirdParty contained rtmpt_proxy_stat -syn keyword ngxDirectiveThirdParty contained rtmpt_proxy_stylesheet -syn keyword ngxDirectiveThirdParty contained rtmpt_proxy_target - -" Syntactically Awesome NGINX Module -" https://github.com/mneudert/sass-nginx-module -syn keyword ngxDirectiveThirdParty contained sass_compile -syn keyword ngxDirectiveThirdParty contained sass_error_log -syn keyword ngxDirectiveThirdParty contained sass_include_path -syn keyword ngxDirectiveThirdParty contained sass_indent -syn keyword ngxDirectiveThirdParty contained sass_is_indented_syntax -syn keyword ngxDirectiveThirdParty contained sass_linefeed -syn keyword ngxDirectiveThirdParty contained sass_output_style -syn keyword ngxDirectiveThirdParty contained sass_precision -syn keyword ngxDirectiveThirdParty contained sass_source_comments -syn keyword ngxDirectiveThirdParty contained sass_source_map_embed - -" Nginx Selective Cache Purge Module -" https://github.com/wandenberg/nginx-selective-cache-purge-module -syn keyword ngxDirectiveThirdParty contained selective_cache_purge_query -syn keyword ngxDirectiveThirdParty contained selective_cache_purge_redis_database -syn keyword ngxDirectiveThirdParty contained selective_cache_purge_redis_host -syn keyword ngxDirectiveThirdParty contained selective_cache_purge_redis_password -syn keyword ngxDirectiveThirdParty contained selective_cache_purge_redis_port -syn keyword ngxDirectiveThirdParty contained selective_cache_purge_redis_unix_socket - -" cconv nginx module -" https://github.com/liseen/set-cconv-nginx-module -syn keyword ngxDirectiveThirdParty contained set_cconv_to_simp -syn keyword ngxDirectiveThirdParty contained set_cconv_to_trad -syn keyword ngxDirectiveThirdParty contained set_pinyin_to_normal - -" Nginx module that allows the setting of variables to the value of a variety of hashes -" https://github.com/simpl/ngx_http_set_hash -syn keyword ngxDirectiveThirdParty contained set_md5 -syn keyword ngxDirectiveThirdParty contained set_md5_upper -syn keyword ngxDirectiveThirdParty contained set_murmur2 -syn keyword ngxDirectiveThirdParty contained set_murmur2_upper -syn keyword ngxDirectiveThirdParty contained set_sha1 -syn keyword ngxDirectiveThirdParty contained set_sha1_upper - -" Nginx module to set the language of a request based on a number of options -" https://github.com/simpl/ngx_http_set_lang -syn keyword ngxDirectiveThirdParty contained lang_cookie -syn keyword ngxDirectiveThirdParty contained lang_get_var -syn keyword ngxDirectiveThirdParty contained lang_host -syn keyword ngxDirectiveThirdParty contained lang_list -syn keyword ngxDirectiveThirdParty contained lang_post_var -syn keyword ngxDirectiveThirdParty contained lang_referer -syn keyword ngxDirectiveThirdParty contained set_lang -syn keyword ngxDirectiveThirdParty contained set_lang_method - -" Nginx Sorted Querystring Module -" https://github.com/wandenberg/nginx-sorted-querystring-module -syn keyword ngxDirectiveThirdParty contained sorted_querysting_filter_parameter - -" Nginx upstream module for Sphinx 2.x search daemon -" https://github.com/reeteshranjan/sphinx2-nginx-module -syn keyword ngxDirectiveThirdParty contained sphinx2_bind -syn keyword ngxDirectiveThirdParty contained sphinx2_buffer_size -syn keyword ngxDirectiveThirdParty contained sphinx2_connect_timeout -syn keyword ngxDirectiveThirdParty contained sphinx2_next_upstream -syn keyword ngxDirectiveThirdParty contained sphinx2_pass -syn keyword ngxDirectiveThirdParty contained sphinx2_read_timeout -syn keyword ngxDirectiveThirdParty contained sphinx2_send_timeout - -" Nginx module for retrieving user attributes and groups from SSSD -" https://github.com/veruu/ngx_sssd_info -syn keyword ngxDirectiveThirdParty contained sssd_info -syn keyword ngxDirectiveThirdParty contained sssd_info_attribute -syn keyword ngxDirectiveThirdParty contained sssd_info_attribute_separator -syn keyword ngxDirectiveThirdParty contained sssd_info_attributes -syn keyword ngxDirectiveThirdParty contained sssd_info_group -syn keyword ngxDirectiveThirdParty contained sssd_info_group_separator -syn keyword ngxDirectiveThirdParty contained sssd_info_groups -syn keyword ngxDirectiveThirdParty contained sssd_info_output_to - -" An nginx module for sending statistics to statsd -" https://github.com/zebrafishlabs/nginx-statsd -syn keyword ngxDirectiveThirdParty contained statsd_count -syn keyword ngxDirectiveThirdParty contained statsd_sample_rate -syn keyword ngxDirectiveThirdParty contained statsd_server -syn keyword ngxDirectiveThirdParty contained statsd_timing - -" ngx_stream_echo - TCP/stream echo module for NGINX (a port of the ngx_http_echo module) -" https://github.com/openresty/stream-echo-nginx-module -syn keyword ngxDirectiveThirdParty contained echo -syn keyword ngxDirectiveThirdParty contained echo_client_error_log_level -syn keyword ngxDirectiveThirdParty contained echo_discard_request -syn keyword ngxDirectiveThirdParty contained echo_duplicate -syn keyword ngxDirectiveThirdParty contained echo_flush_wait -syn keyword ngxDirectiveThirdParty contained echo_lingering_close -syn keyword ngxDirectiveThirdParty contained echo_lingering_time -syn keyword ngxDirectiveThirdParty contained echo_lingering_timeout -syn keyword ngxDirectiveThirdParty contained echo_read_buffer_size -syn keyword ngxDirectiveThirdParty contained echo_read_bytes -syn keyword ngxDirectiveThirdParty contained echo_read_line -syn keyword ngxDirectiveThirdParty contained echo_read_timeout -syn keyword ngxDirectiveThirdParty contained echo_request_data -syn keyword ngxDirectiveThirdParty contained echo_send_timeout -syn keyword ngxDirectiveThirdParty contained echo_sleep - -" Embed the power of Lua into NGINX TCP/UDP servers -" https://github.com/openresty/stream-lua-nginx-module -syn keyword ngxDirectiveThirdParty contained lua_add_variable -syn keyword ngxDirectiveThirdParty contained preread_by_lua_block -syn keyword ngxDirectiveThirdParty contained preread_by_lua_file -syn keyword ngxDirectiveThirdParty contained preread_by_lua_no_postpone - -" nginx-upsync-module -" https://github.com/weibocom/nginx-upsync-module -syn keyword ngxDirectiveThirdParty contained upstream_show -syn keyword ngxDirectiveThirdParty contained upsync -syn keyword ngxDirectiveThirdParty contained upsync_dump_path -syn keyword ngxDirectiveThirdParty contained upsync_lb - -" Whitespace stripper for nginx -" https://github.com/evanmiller/mod_strip -syn keyword ngxDirectiveThirdParty contained strip - -" Split one big HTTP/Range request to multiple subrange requesets -" https://github.com/Qihoo360/ngx_http_subrange_module -syn keyword ngxDirectiveThirdParty contained subrange - -" summarizer-nginx-module -" https://github.com/reeteshranjan/summarizer-nginx-module -syn keyword ngxDirectiveThirdParty contained summarizer_bind -syn keyword ngxDirectiveThirdParty contained summarizer_buffer_size -syn keyword ngxDirectiveThirdParty contained summarizer_connect_timeout -syn keyword ngxDirectiveThirdParty contained summarizer_next_upstream -syn keyword ngxDirectiveThirdParty contained summarizer_pass -syn keyword ngxDirectiveThirdParty contained summarizer_read_timeout -syn keyword ngxDirectiveThirdParty contained summarizer_send_timeout - -" nginx module providing API to communicate with supervisord and manage (start/stop) backends on-demand -" https://github.com/FRiCKLE/ngx_supervisord -syn keyword ngxDirectiveThirdParty contained supervisord -syn keyword ngxDirectiveThirdParty contained supervisord_inherit_backend_status -syn keyword ngxDirectiveThirdParty contained supervisord_name -syn keyword ngxDirectiveThirdParty contained supervisord_start -syn keyword ngxDirectiveThirdParty contained supervisord_stop - -" simple robot mitigation module using cookie based challenge/response technique. Not supported any more. -" https://github.com/kyprizel/testcookie-nginx-module -syn keyword ngxDirectiveThirdParty contained testcookie -syn keyword ngxDirectiveThirdParty contained testcookie_arg -syn keyword ngxDirectiveThirdParty contained testcookie_deny_keepalive -syn keyword ngxDirectiveThirdParty contained testcookie_domain -syn keyword ngxDirectiveThirdParty contained testcookie_expires -syn keyword ngxDirectiveThirdParty contained testcookie_fallback -syn keyword ngxDirectiveThirdParty contained testcookie_get_only -syn keyword ngxDirectiveThirdParty contained testcookie_httponly_flag -syn keyword ngxDirectiveThirdParty contained testcookie_https_location -syn keyword ngxDirectiveThirdParty contained testcookie_internal -syn keyword ngxDirectiveThirdParty contained testcookie_max_attempts -syn keyword ngxDirectiveThirdParty contained testcookie_name -syn keyword ngxDirectiveThirdParty contained testcookie_p3p -syn keyword ngxDirectiveThirdParty contained testcookie_pass -syn keyword ngxDirectiveThirdParty contained testcookie_path -syn keyword ngxDirectiveThirdParty contained testcookie_port_in_redirect -syn keyword ngxDirectiveThirdParty contained testcookie_redirect_via_refresh -syn keyword ngxDirectiveThirdParty contained testcookie_refresh_encrypt_cookie -syn keyword ngxDirectiveThirdParty contained testcookie_refresh_encrypt_cookie_iv -syn keyword ngxDirectiveThirdParty contained testcookie_refresh_encrypt_cookie_key -syn keyword ngxDirectiveThirdParty contained testcookie_refresh_status -syn keyword ngxDirectiveThirdParty contained testcookie_refresh_template -syn keyword ngxDirectiveThirdParty contained testcookie_samesite -syn keyword ngxDirectiveThirdParty contained testcookie_secret -syn keyword ngxDirectiveThirdParty contained testcookie_secure_flag -syn keyword ngxDirectiveThirdParty contained testcookie_session -syn keyword ngxDirectiveThirdParty contained testcookie_whitelist - -" ngx_http_types_filter_module -" https://github.com/flygoast/ngx_http_types_filter -syn keyword ngxDirectiveThirdParty contained types_filter -syn keyword ngxDirectiveThirdParty contained types_filter_use_default - -" A module allowing the nginx to use files embedded in a zip file -" https://github.com/youzee/nginx-unzip-module -syn keyword ngxDirectiveThirdParty contained file_in_unzip -syn keyword ngxDirectiveThirdParty contained file_in_unzip_archivefile -syn keyword ngxDirectiveThirdParty contained file_in_unzip_extract - -" An asynchronous domain name resolve module for nginx upstream -" https://github.com/wdaike/ngx_upstream_jdomain -syn keyword ngxDirectiveThirdParty contained jdomain - -" Nginx url encoding converting module -" https://github.com/vozlt/nginx-module-url -syn keyword ngxDirectiveThirdParty contained url_encoding_convert -syn keyword ngxDirectiveThirdParty contained url_encoding_convert_alloc_size -syn keyword ngxDirectiveThirdParty contained url_encoding_convert_alloc_size_x -syn keyword ngxDirectiveThirdParty contained url_encoding_convert_from -syn keyword ngxDirectiveThirdParty contained url_encoding_convert_phase -syn keyword ngxDirectiveThirdParty contained url_encoding_convert_to - -" A nginx module to match browsers and crawlers -" https://github.com/alibaba/nginx-http-user-agent -syn keyword ngxDirectiveThirdParty contained user_agent - -" nginx load-balancer module implementing ketama consistent hashing -" https://github.com/flygoast/ngx_http_upstream_ketama_chash -syn keyword ngxDirectiveThirdParty contained ketama_chash - -" nginx-sticky-module-ng -" https://github.com/ayty-adrianomartins/nginx-sticky-module-ng -syn keyword ngxDirectiveThirdParty contained sticky_no_fallback - -" dynamic linking and call the function of your application -" https://github.com/Taymindis/nginx-link-function -syn keyword ngxDirectiveThirdParty contained ngx_link_func_add_prop -syn keyword ngxDirectiveThirdParty contained ngx_link_func_add_req_header -syn keyword ngxDirectiveThirdParty contained ngx_link_func_ca_cert -syn keyword ngxDirectiveThirdParty contained ngx_link_func_call -syn keyword ngxDirectiveThirdParty contained ngx_link_func_download_link_lib -syn keyword ngxDirectiveThirdParty contained ngx_link_func_lib -syn keyword ngxDirectiveThirdParty contained ngx_link_func_shm_size -syn keyword ngxDirectiveThirdParty contained ngx_link_func_subrequest - -" purge content from FastCGI, proxy, SCGI and uWSGI caches -" https://github.com/torden/ngx_cache_purge -syn keyword ngxDirectiveThirdParty contained cache_purge_response_type - -" set the flags "HttpOnly", "secure" and "SameSite" for cookies -" https://github.com/AirisX/nginx_cookie_flag_module -syn keyword ngxDirectiveThirdParty contained set_cookie_flag - -" Embed websockify into Nginx (convert any tcp connection into websocket) " https://github.com/tg123/websockify-nginx-module syn keyword ngxDirectiveThirdParty contained websockify_buffer_size syn keyword ngxDirectiveThirdParty contained websockify_connect_timeout @@ -2415,55 +1980,6 @@ syn keyword ngxDirectiveThirdParty contained websockify_read_timeout syn keyword ngxDirectiveThirdParty contained websockify_send_timeout -" IP2Location Nginx -" https://github.com/ip2location/ip2location-nginx -syn keyword ngxDirectiveThirdParty contained ip2location_addresstype -syn keyword ngxDirectiveThirdParty contained ip2location_areacode -syn keyword ngxDirectiveThirdParty contained ip2location_category -syn keyword ngxDirectiveThirdParty contained ip2location_city -syn keyword ngxDirectiveThirdParty contained ip2location_country_long -syn keyword ngxDirectiveThirdParty contained ip2location_country_short -syn keyword ngxDirectiveThirdParty contained ip2location_domain -syn keyword ngxDirectiveThirdParty contained ip2location_elevation -syn keyword ngxDirectiveThirdParty contained ip2location_iddcode -syn keyword ngxDirectiveThirdParty contained ip2location_isp -syn keyword ngxDirectiveThirdParty contained ip2location_latitude -syn keyword ngxDirectiveThirdParty contained ip2location_longitude -syn keyword ngxDirectiveThirdParty contained ip2location_mcc -syn keyword ngxDirectiveThirdParty contained ip2location_mnc -syn keyword ngxDirectiveThirdParty contained ip2location_mobilebrand -syn keyword ngxDirectiveThirdParty contained ip2location_netspeed -syn keyword ngxDirectiveThirdParty contained ip2location_proxy -syn keyword ngxDirectiveThirdParty contained ip2location_proxy_recursive -syn keyword ngxDirectiveThirdParty contained ip2location_region -syn keyword ngxDirectiveThirdParty contained ip2location_timezone -syn keyword ngxDirectiveThirdParty contained ip2location_usagetype -syn keyword ngxDirectiveThirdParty contained ip2location_weatherstationcode -syn keyword ngxDirectiveThirdParty contained ip2location_weatherstationname -syn keyword ngxDirectiveThirdParty contained ip2location_zipcode - -" IP2Proxy module for Nginx -" https://github.com/ip2location/ip2proxy-nginx -syn keyword ngxDirectiveThirdParty contained ip2proxy_as -syn keyword ngxDirectiveThirdParty contained ip2proxy_asn -syn keyword ngxDirectiveThirdParty contained ip2proxy_city -syn keyword ngxDirectiveThirdParty contained ip2proxy_country_long -syn keyword ngxDirectiveThirdParty contained ip2proxy_country_short -syn keyword ngxDirectiveThirdParty contained ip2proxy_database -syn keyword ngxDirectiveThirdParty contained ip2proxy_domain -syn keyword ngxDirectiveThirdParty contained ip2proxy_isp -syn keyword ngxDirectiveThirdParty contained ip2proxy_is_proxy -syn keyword ngxDirectiveThirdParty contained ip2proxy_last_seen -syn keyword ngxDirectiveThirdParty contained ip2proxy_provider -syn keyword ngxDirectiveThirdParty contained ip2proxy_proxy -syn keyword ngxDirectiveThirdParty contained ip2proxy_proxy_recursive -syn keyword ngxDirectiveThirdParty contained ip2proxy_proxy_type -syn keyword ngxDirectiveThirdParty contained ip2proxy_region -syn keyword ngxDirectiveThirdParty contained ip2proxy_threat -syn keyword ngxDirectiveThirdParty contained ip2proxy_usage_type - - - " highlight hi def link ngxComment Comment From pluknet at nginx.com Mon Jun 20 06:10:39 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 20 Jun 2022 10:10:39 +0400 Subject: [PATCH] Misc: win32 sources now preserved in release tarballs In-Reply-To: References: Message-ID: > On 18 Jun 2022, at 00:37, Maxim Dounin wrote: > > Hello! > > On Fri, Jun 17, 2022 at 07:31:27PM +0400, Sergey Kandaurov wrote: > >>> On 17 Jun 2022, at 01:33, Maxim Dounin wrote: >>> >>> # HG changeset patch >>> # User Maxim Dounin >>> # Date 1655415152 -10800 >>> # Fri Jun 17 00:32:32 2022 +0300 >>> # Node ID e23a385cd0ec866a3eb1d8c9c956991e1ed50d78 >>> # Parent 861f076eab72dab85bd5eaa56da770f8b10aa8ad >>> Misc: win32 sources now preserved in release tarballs. >>> >>> This makes it possible to build nginx under Windows from release tarballs >>> instead of using source code repository. >>> >>> diff --git a/misc/GNUmakefile b/misc/GNUmakefile >>> --- a/misc/GNUmakefile >>> +++ b/misc/GNUmakefile >>> @@ -15,12 +15,6 @@ release: export >>> >>> mv $(TEMP)/$(NGINX)/auto/configure $(TEMP)/$(NGINX) >>> >> >> Do we need the empty line here? >> It could be collapsed if removing all the below. > > It clearly separates two distinct blocks: configure and docs. I > don't think this distinction have to be removed. > Ok. >>> - # delete incomplete sources >>> - rm $(TEMP)/$(NGINX)/src/event/ngx_event_acceptex.c >>> - rm $(TEMP)/$(NGINX)/src/event/ngx_event_connectex.c >> >> I wonder if we need to touch ngx_event_connectex.c. >> It's not used nor connected to the build since introduction, >> so it does look incomplete. > > I don't think it is a good idea to keep any of these removals, > even if some files are not connected to the build. Rather, I > think that release tarballs should be as close as possible to the > source code repository. Agreed, it makes sense. > > Just in case, ngx_service.c and ngx_stat.c in src/os/win32/ also > not connected to the build. > >>> - rm $(TEMP)/$(NGINX)/src/event/modules/ngx_iocp_module.* >>> - rm -r $(TEMP)/$(NGINX)/src/os/win32 >>> - >>> mv $(TEMP)/$(NGINX)/docs/text/LICENSE $(TEMP)/$(NGINX) >>> mv $(TEMP)/$(NGINX)/docs/text/README $(TEMP)/$(NGINX) >>> mv $(TEMP)/$(NGINX)/docs/html $(TEMP)/$(NGINX) >>> -- Sergey Kandaurov From shanlei at asiainfo.com Mon Jun 20 08:05:53 2022 From: shanlei at asiainfo.com (shanlei at asiainfo.com) Date: Mon, 20 Jun 2022 16:05:53 +0800 Subject: what's the proper way to get module specific configuration in init_process Message-ID: <4e653e995eda3fbe41ac592d465b6709@asiainfo.com> Hello, guys, Recently I have been working on an nginx c module, which have to leverage on init_process routine.The logic is based on instruction command which may be configured inside stream.server, I have to do some work in the init_process routine, for example, to schedule a timer. I did google a while ,and decide to using code snippet like below: ============================ ngx_stream_conf_ctx_t conf_ctx = (ngx_stream_conf_ctx_t *)ngx_get_conf(cycle->conf_ctx, ngx_stream_module); "module_spec_conf = conf_ctx->srv_conf["my_module".ctx_index]; ============================= However, the module_spec_conf I got is the top_level ,that is created when nginx parse the stream section, not the one created when nginx parse the stream.server section, the later has the proper values. Currently to got the later structure, I have to copy the configure value back from child to parent in configuration merger function, which is inconvenient and buggy when the stream includes multi servers. So I wonder what's the proper way to get module specific configuration in the routine: init_process Thanks Lei From mwachasu at cisco.com Mon Jun 20 11:07:52 2022 From: mwachasu at cisco.com (Mithilesh Wachasunder (mwachasu)) Date: Mon, 20 Jun 2022 11:07:52 +0000 Subject: Require some information regarding the support for nginx-1.14.2 Message-ID: Hello team Is the stable version for nginx-1.14.2 declared as EOL(End of life)? Thanks Mithilesh -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon Jun 20 12:41:36 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 20 Jun 2022 15:41:36 +0300 Subject: Require some information regarding the support for nginx-1.14.2 In-Reply-To: References: Message-ID: Hello! On Mon, Jun 20, 2022 at 11:07:52AM +0000, Mithilesh Wachasunder (mwachasu) via nginx-devel wrote: > Hello team > > Is the stable version for nginx-1.14.2 declared as EOL(End of life)? This mailing list is about nginx development. Please avoid asking user-level questions here, use the nginx@ mailing list instead. Thank you. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Mon Jun 20 16:32:35 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 20 Jun 2022 19:32:35 +0300 Subject: [PATCH] Contrib: vim syntax, update core and 3rd party module directives. In-Reply-To: References: Message-ID: Hello! On Sat, Jun 18, 2022 at 04:02:17PM +0300, Gena Makhomed wrote: > # HG changeset patch > # User Gena Makhomed > # Date 1655556880 -10800 > # Sat Jun 18 15:54:40 2022 +0300 > # Node ID 81096862f8f13438ca059779cbc2097db8c0ad68 > # Parent 861f076eab72dab85bd5eaa56da770f8b10aa8ad > Contrib: vim syntax, update core and 3rd party module directives. > > List of 3rd party modules github repositories are obtained from > https://github.com/freebsd/freebsd-ports/blob/main/www/nginx-devel/Makefile.extmod [...] Queued, thanks. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Mon Jun 20 16:45:08 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 20 Jun 2022 19:45:08 +0300 Subject: [PATCH] Misc: win32 sources now preserved in release tarballs In-Reply-To: References: Message-ID: Hello! On Mon, Jun 20, 2022 at 10:10:39AM +0400, Sergey Kandaurov wrote: > > On 18 Jun 2022, at 00:37, Maxim Dounin wrote: > > > > On Fri, Jun 17, 2022 at 07:31:27PM +0400, Sergey Kandaurov wrote: > > > >>> On 17 Jun 2022, at 01:33, Maxim Dounin wrote: > >>> > >>> # HG changeset patch > >>> # User Maxim Dounin > >>> # Date 1655415152 -10800 > >>> # Fri Jun 17 00:32:32 2022 +0300 > >>> # Node ID e23a385cd0ec866a3eb1d8c9c956991e1ed50d78 > >>> # Parent 861f076eab72dab85bd5eaa56da770f8b10aa8ad > >>> Misc: win32 sources now preserved in release tarballs. > >>> > >>> This makes it possible to build nginx under Windows from release tarballs > >>> instead of using source code repository. > >>> > >>> diff --git a/misc/GNUmakefile b/misc/GNUmakefile > >>> --- a/misc/GNUmakefile > >>> +++ b/misc/GNUmakefile > >>> @@ -15,12 +15,6 @@ release: export > >>> > >>> mv $(TEMP)/$(NGINX)/auto/configure $(TEMP)/$(NGINX) > >>> > >> > >> Do we need the empty line here? > >> It could be collapsed if removing all the below. > > > > It clearly separates two distinct blocks: configure and docs. I > > don't think this distinction have to be removed. > > > > Ok. > > >>> - # delete incomplete sources > >>> - rm $(TEMP)/$(NGINX)/src/event/ngx_event_acceptex.c > >>> - rm $(TEMP)/$(NGINX)/src/event/ngx_event_connectex.c > >> > >> I wonder if we need to touch ngx_event_connectex.c. > >> It's not used nor connected to the build since introduction, > >> so it does look incomplete. > > > > I don't think it is a good idea to keep any of these removals, > > even if some files are not connected to the build. Rather, I > > think that release tarballs should be as close as possible to the > > source code repository. > > Agreed, it makes sense. Pushed to http://mdounin.ru/hg/nginx/. -- Maxim Dounin http://mdounin.ru/ From pluknet at nginx.com Mon Jun 20 17:48:57 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 20 Jun 2022 17:48:57 +0000 Subject: [nginx] Contrib: vim syntax, update core and 3rd party module directives. Message-ID: details: https://hg.nginx.org/nginx/rev/1b7c7c3af6d9 branches: changeset: 8047:1b7c7c3af6d9 user: Gena Makhomed date: Sat Jun 18 15:54:40 2022 +0300 description: Contrib: vim syntax, update core and 3rd party module directives. List of 3rd party modules github repositories are obtained from https://github.com/freebsd/freebsd-ports/blob/main/www/nginx-devel/Makefile.extmod diffstat: contrib/vim/syntax/nginx.vim | 1004 ++++++++++------------------------------- 1 files changed, 260 insertions(+), 744 deletions(-) diffs (truncated from 1501 to 1000 lines): diff -r 861f076eab72 -r 1b7c7c3af6d9 contrib/vim/syntax/nginx.vim --- a/contrib/vim/syntax/nginx.vim Tue Jun 14 10:39:58 2022 +0400 +++ b/contrib/vim/syntax/nginx.vim Sat Jun 18 15:54:40 2022 +0300 @@ -111,19 +111,14 @@ syn keyword ngxDirectiveControl containe syn keyword ngxDirectiveError contained error_page syn keyword ngxDirectiveError contained post_action -syn keyword ngxDirectiveDeprecated contained limit_zone syn keyword ngxDirectiveDeprecated contained proxy_downstream_buffer syn keyword ngxDirectiveDeprecated contained proxy_upstream_buffer -syn keyword ngxDirectiveDeprecated contained spdy_chunk_size -syn keyword ngxDirectiveDeprecated contained spdy_headers_comp -syn keyword ngxDirectiveDeprecated contained spdy_keepalive_timeout -syn keyword ngxDirectiveDeprecated contained spdy_max_concurrent_streams -syn keyword ngxDirectiveDeprecated contained spdy_pool_size -syn keyword ngxDirectiveDeprecated contained spdy_recv_buffer_size -syn keyword ngxDirectiveDeprecated contained spdy_recv_timeout -syn keyword ngxDirectiveDeprecated contained spdy_streams_index_size syn keyword ngxDirectiveDeprecated contained ssl -syn keyword ngxDirectiveDeprecated contained upstream_conf +syn keyword ngxDirectiveDeprecated contained http2_idle_timeout +syn keyword ngxDirectiveDeprecated contained http2_max_field_size +syn keyword ngxDirectiveDeprecated contained http2_max_header_size +syn keyword ngxDirectiveDeprecated contained http2_max_requests +syn keyword ngxDirectiveDeprecated contained http2_recv_timeout syn keyword ngxDirective contained absolute_redirect syn keyword ngxDirective contained accept_mutex @@ -152,6 +147,7 @@ syn keyword ngxDirective contained auth_ syn keyword ngxDirective contained auth_jwt syn keyword ngxDirective contained auth_jwt_claim_set syn keyword ngxDirective contained auth_jwt_header_set +syn keyword ngxDirective contained auth_jwt_key_cache syn keyword ngxDirective contained auth_jwt_key_file syn keyword ngxDirective contained auth_jwt_key_request syn keyword ngxDirective contained auth_jwt_leeway @@ -309,17 +305,12 @@ syn keyword ngxDirective contained hls_m syn keyword ngxDirective contained hls_mp4_max_buffer_size syn keyword ngxDirective contained http2_body_preread_size syn keyword ngxDirective contained http2_chunk_size -syn keyword ngxDirective contained http2_idle_timeout syn keyword ngxDirective contained http2_max_concurrent_pushes syn keyword ngxDirective contained http2_max_concurrent_streams -syn keyword ngxDirective contained http2_max_field_size -syn keyword ngxDirective contained http2_max_header_size -syn keyword ngxDirective contained http2_max_requests syn keyword ngxDirective contained http2_pool_size syn keyword ngxDirective contained http2_push syn keyword ngxDirective contained http2_push_preload syn keyword ngxDirective contained http2_recv_buffer_size -syn keyword ngxDirective contained http2_recv_timeout syn keyword ngxDirective contained http2_streams_index_size syn keyword ngxDirective contained if_modified_since syn keyword ngxDirective contained ignore_invalid_headers @@ -339,14 +330,17 @@ syn keyword ngxDirective contained ip_ha syn keyword ngxDirective contained js_access syn keyword ngxDirective contained js_body_filter syn keyword ngxDirective contained js_content +syn keyword ngxDirective contained js_fetch_buffer_size syn keyword ngxDirective contained js_fetch_ciphers +syn keyword ngxDirective contained js_fetch_max_response_buffer_size syn keyword ngxDirective contained js_fetch_protocols +syn keyword ngxDirective contained js_fetch_timeout syn keyword ngxDirective contained js_fetch_trusted_certificate +syn keyword ngxDirective contained js_fetch_verify syn keyword ngxDirective contained js_fetch_verify_depth syn keyword ngxDirective contained js_filter syn keyword ngxDirective contained js_header_filter syn keyword ngxDirective contained js_import -syn keyword ngxDirective contained js_include syn keyword ngxDirective contained js_path syn keyword ngxDirective contained js_preread syn keyword ngxDirective contained js_set @@ -391,7 +385,6 @@ syn keyword ngxDirective contained max_r syn keyword ngxDirective contained memcached_bind syn keyword ngxDirective contained memcached_buffer_size syn keyword ngxDirective contained memcached_connect_timeout -syn keyword ngxDirective contained memcached_force_ranges syn keyword ngxDirective contained memcached_gzip_flag syn keyword ngxDirective contained memcached_next_upstream syn keyword ngxDirective contained memcached_next_upstream_timeout @@ -645,7 +638,6 @@ syn keyword ngxDirective contained statu syn keyword ngxDirective contained status_format syn keyword ngxDirective contained status_zone syn keyword ngxDirective contained sticky -syn keyword ngxDirective contained sticky_cookie_insert syn keyword ngxDirective contained stub_status syn keyword ngxDirective contained sub_filter syn keyword ngxDirective contained sub_filter_last_modified @@ -774,62 +766,14 @@ syn keyword ngxDirective contained zone_ syn keyword ngxDirective contained zone_sync_ssl_verify_depth syn keyword ngxDirective contained zone_sync_timeout -" 3rd party modules list taken from -" https://github.com/freebsd/freebsd-ports/blob/master/www/nginx-devel/Makefile -" ----------------------------------------------------------------------------- - -" Accept Language -" https://github.com/giom/nginx_accept_language_module -syn keyword ngxDirectiveThirdParty contained set_from_accept_language - -" Digest Authentication -" https://github.com/atomx/nginx-http-auth-digest -syn keyword ngxDirectiveThirdParty contained auth_digest -syn keyword ngxDirectiveThirdParty contained auth_digest_drop_time -syn keyword ngxDirectiveThirdParty contained auth_digest_evasion_time -syn keyword ngxDirectiveThirdParty contained auth_digest_expires -syn keyword ngxDirectiveThirdParty contained auth_digest_maxtries -syn keyword ngxDirectiveThirdParty contained auth_digest_replays -syn keyword ngxDirectiveThirdParty contained auth_digest_shm_size -syn keyword ngxDirectiveThirdParty contained auth_digest_timeout -syn keyword ngxDirectiveThirdParty contained auth_digest_user_file -" SPNEGO Authentication -" https://github.com/stnoonan/spnego-http-auth-nginx-module -syn keyword ngxDirectiveThirdParty contained auth_gss -syn keyword ngxDirectiveThirdParty contained auth_gss_allow_basic_fallback -syn keyword ngxDirectiveThirdParty contained auth_gss_authorized_principal -syn keyword ngxDirectiveThirdParty contained auth_gss_authorized_principal_regex -syn keyword ngxDirectiveThirdParty contained auth_gss_constrained_delegation -syn keyword ngxDirectiveThirdParty contained auth_gss_delegate_credentials -syn keyword ngxDirectiveThirdParty contained auth_gss_force_realm -syn keyword ngxDirectiveThirdParty contained auth_gss_format_full -syn keyword ngxDirectiveThirdParty contained auth_gss_keytab -syn keyword ngxDirectiveThirdParty contained auth_gss_map_to_local -syn keyword ngxDirectiveThirdParty contained auth_gss_realm -syn keyword ngxDirectiveThirdParty contained auth_gss_service_ccache -syn keyword ngxDirectiveThirdParty contained auth_gss_service_name +" 3rd party modules list taken from +" https://github.com/freebsd/freebsd-ports/blob/main/www/nginx-devel/Makefile.extmod +" ---------------------------------------------------------------------------------- -" LDAP Authentication -" https://github.com/kvspb/nginx-auth-ldap -syn keyword ngxDirectiveThirdParty contained auth_ldap -syn keyword ngxDirectiveThirdParty contained auth_ldap_cache_enabled -syn keyword ngxDirectiveThirdParty contained auth_ldap_cache_expiration_time -syn keyword ngxDirectiveThirdParty contained auth_ldap_cache_size -syn keyword ngxDirectiveThirdParty contained auth_ldap_servers -syn keyword ngxDirectiveThirdParty contained auth_ldap_servers_size -syn keyword ngxDirectiveThirdParty contained ldap_server - -" PAM Authentication -" https://github.com/sto/ngx_http_auth_pam_module -syn keyword ngxDirectiveThirdParty contained auth_pam -syn keyword ngxDirectiveThirdParty contained auth_pam_service_name -syn keyword ngxDirectiveThirdParty contained auth_pam_set_pam_env - -" AJP protocol proxy -" https://github.com/yaoweibin/nginx_ajp_module +" https://github.com/msva/nginx_ajp_module +syn keyword ngxDirectiveThirdParty contained ajp_buffer_size syn keyword ngxDirectiveThirdParty contained ajp_buffers -syn keyword ngxDirectiveThirdParty contained ajp_buffer_size syn keyword ngxDirectiveThirdParty contained ajp_busy_buffers_size syn keyword ngxDirectiveThirdParty contained ajp_cache syn keyword ngxDirectiveThirdParty contained ajp_cache_key @@ -850,11 +794,13 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained ajp_max_data_packet_size syn keyword ngxDirectiveThirdParty contained ajp_max_temp_file_size syn keyword ngxDirectiveThirdParty contained ajp_next_upstream +syn keyword ngxDirectiveThirdParty contained ajp_param syn keyword ngxDirectiveThirdParty contained ajp_pass syn keyword ngxDirectiveThirdParty contained ajp_pass_header syn keyword ngxDirectiveThirdParty contained ajp_pass_request_body syn keyword ngxDirectiveThirdParty contained ajp_pass_request_headers syn keyword ngxDirectiveThirdParty contained ajp_read_timeout +syn keyword ngxDirectiveThirdParty contained ajp_script_url syn keyword ngxDirectiveThirdParty contained ajp_secret syn keyword ngxDirectiveThirdParty contained ajp_send_lowat syn keyword ngxDirectiveThirdParty contained ajp_send_timeout @@ -865,7 +811,12 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained ajp_upstream_fail_timeout syn keyword ngxDirectiveThirdParty contained ajp_upstream_max_fails -" AWS proxy +" https://github.com/openresty/array-var-nginx-module +syn keyword ngxDirectiveThirdParty contained array_join +syn keyword ngxDirectiveThirdParty contained array_map +syn keyword ngxDirectiveThirdParty contained array_map_op +syn keyword ngxDirectiveThirdParty contained array_split + " https://github.com/anomalizer/ngx_aws_auth syn keyword ngxDirectiveThirdParty contained aws_access_key syn keyword ngxDirectiveThirdParty contained aws_endpoint @@ -874,7 +825,18 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained aws_sign syn keyword ngxDirectiveThirdParty contained aws_signing_key -" embedding Clojure or Java or Groovy programs +" https://github.com/google/ngx_brotli +syn keyword ngxDirectiveThirdParty contained brotli +syn keyword ngxDirectiveThirdParty contained brotli_buffers +syn keyword ngxDirectiveThirdParty contained brotli_comp_level +syn keyword ngxDirectiveThirdParty contained brotli_min_length +syn keyword ngxDirectiveThirdParty contained brotli_static +syn keyword ngxDirectiveThirdParty contained brotli_types +syn keyword ngxDirectiveThirdParty contained brotli_window + +" https://github.com/torden/ngx_cache_purge +syn keyword ngxDirectiveThirdParty contained cache_purge_response_type + " https://github.com/nginx-clojure/nginx-clojure syn keyword ngxDirectiveThirdParty contained access_handler_code syn keyword ngxDirectiveThirdParty contained access_handler_name @@ -892,8 +854,8 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained content_handler_type syn keyword ngxDirectiveThirdParty contained handler_code syn keyword ngxDirectiveThirdParty contained handler_name +syn keyword ngxDirectiveThirdParty contained handler_type syn keyword ngxDirectiveThirdParty contained handlers_lazy_init -syn keyword ngxDirectiveThirdParty contained handler_type syn keyword ngxDirectiveThirdParty contained header_filter_code syn keyword ngxDirectiveThirdParty contained header_filter_name syn keyword ngxDirectiveThirdParty contained header_filter_property @@ -921,18 +883,20 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained shared_map syn keyword ngxDirectiveThirdParty contained write_page_size +" https://github.com/AirisX/nginx_cookie_flag_module +syn keyword ngxDirectiveThirdParty contained set_cookie_flag -" Certificate Transparency " https://github.com/grahamedgecombe/nginx-ct syn keyword ngxDirectiveThirdParty contained ssl_ct syn keyword ngxDirectiveThirdParty contained ssl_ct_static_scts -" ngx_echo " https://github.com/openresty/echo-nginx-module +syn keyword ngxDirectiveThirdParty contained echo syn keyword ngxDirectiveThirdParty contained echo_abort_parent syn keyword ngxDirectiveThirdParty contained echo_after_body syn keyword ngxDirectiveThirdParty contained echo_before_body syn keyword ngxDirectiveThirdParty contained echo_blocking_sleep +syn keyword ngxDirectiveThirdParty contained echo_duplicate syn keyword ngxDirectiveThirdParty contained echo_end syn keyword ngxDirectiveThirdParty contained echo_exec syn keyword ngxDirectiveThirdParty contained echo_flush @@ -942,28 +906,124 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained echo_read_request_body syn keyword ngxDirectiveThirdParty contained echo_request_body syn keyword ngxDirectiveThirdParty contained echo_reset_timer +syn keyword ngxDirectiveThirdParty contained echo_sleep syn keyword ngxDirectiveThirdParty contained echo_status syn keyword ngxDirectiveThirdParty contained echo_subrequest syn keyword ngxDirectiveThirdParty contained echo_subrequest_async -" FastDFS -" https://github.com/happyfish100/fastdfs-nginx-module -syn keyword ngxDirectiveThirdParty contained ngx_fastdfs_module +" https://github.com/openresty/drizzle-nginx-module +syn keyword ngxDirectiveThirdParty contained drizzle_buffer_size +syn keyword ngxDirectiveThirdParty contained drizzle_connect_timeout +syn keyword ngxDirectiveThirdParty contained drizzle_dbname +syn keyword ngxDirectiveThirdParty contained drizzle_keepalive +syn keyword ngxDirectiveThirdParty contained drizzle_module_header +syn keyword ngxDirectiveThirdParty contained drizzle_pass +syn keyword ngxDirectiveThirdParty contained drizzle_query +syn keyword ngxDirectiveThirdParty contained drizzle_recv_cols_timeout +syn keyword ngxDirectiveThirdParty contained drizzle_recv_rows_timeout +syn keyword ngxDirectiveThirdParty contained drizzle_send_query_timeout +syn keyword ngxDirectiveThirdParty contained drizzle_server +syn keyword ngxDirectiveThirdParty contained drizzle_status + +" https://github.com/ZigzagAK/ngx_dynamic_upstream +syn keyword ngxDirectiveThirdParty contained dns_add_down +syn keyword ngxDirectiveThirdParty contained dns_ipv6 +syn keyword ngxDirectiveThirdParty contained dns_update +syn keyword ngxDirectiveThirdParty contained dynamic_state_file +syn keyword ngxDirectiveThirdParty contained dynamic_upstream -" ngx_headers_more +" https://github.com/ZigzagAK/ngx_dynamic_healthcheck +syn keyword ngxDirectiveThirdParty contained check +syn keyword ngxDirectiveThirdParty contained check_disable_host +syn keyword ngxDirectiveThirdParty contained check_exclude_host +syn keyword ngxDirectiveThirdParty contained check_persistent +syn keyword ngxDirectiveThirdParty contained check_request_body +syn keyword ngxDirectiveThirdParty contained check_request_headers +syn keyword ngxDirectiveThirdParty contained check_request_uri +syn keyword ngxDirectiveThirdParty contained check_response_body +syn keyword ngxDirectiveThirdParty contained check_response_codes +syn keyword ngxDirectiveThirdParty contained healthcheck +syn keyword ngxDirectiveThirdParty contained healthcheck_buffer_size +syn keyword ngxDirectiveThirdParty contained healthcheck_disable_host +syn keyword ngxDirectiveThirdParty contained healthcheck_get +syn keyword ngxDirectiveThirdParty contained healthcheck_persistent +syn keyword ngxDirectiveThirdParty contained healthcheck_request_body +syn keyword ngxDirectiveThirdParty contained healthcheck_request_headers +syn keyword ngxDirectiveThirdParty contained healthcheck_request_uri +syn keyword ngxDirectiveThirdParty contained healthcheck_response_body +syn keyword ngxDirectiveThirdParty contained healthcheck_response_codes +syn keyword ngxDirectiveThirdParty contained healthcheck_status +syn keyword ngxDirectiveThirdParty contained healthcheck_update + +" https://github.com/openresty/encrypted-session-nginx-module +syn keyword ngxDirectiveThirdParty contained encrypted_session_expires +syn keyword ngxDirectiveThirdParty contained encrypted_session_iv +syn keyword ngxDirectiveThirdParty contained encrypted_session_key +syn keyword ngxDirectiveThirdParty contained set_decrypt_session +syn keyword ngxDirectiveThirdParty contained set_encrypt_session + +" https://github.com/calio/form-input-nginx-module +syn keyword ngxDirectiveThirdParty contained set_form_input +syn keyword ngxDirectiveThirdParty contained set_form_input_multi + +" https://github.com/nieoding/nginx-gridfs +syn keyword ngxDirectiveThirdParty contained gridfs +syn keyword ngxDirectiveThirdParty contained mongo + " https://github.com/openresty/headers-more-nginx-module syn keyword ngxDirectiveThirdParty contained more_clear_headers syn keyword ngxDirectiveThirdParty contained more_clear_input_headers syn keyword ngxDirectiveThirdParty contained more_set_headers syn keyword ngxDirectiveThirdParty contained more_set_input_headers -" NGINX WebDAV missing commands support (PROPFIND & OPTIONS) +" https://github.com/dvershinin/nginx_accept_language_module +syn keyword ngxDirectiveThirdParty contained set_from_accept_language + +" https://github.com/atomx/nginx-http-auth-digest +syn keyword ngxDirectiveThirdParty contained auth_digest +syn keyword ngxDirectiveThirdParty contained auth_digest_drop_time +syn keyword ngxDirectiveThirdParty contained auth_digest_evasion_time +syn keyword ngxDirectiveThirdParty contained auth_digest_expires +syn keyword ngxDirectiveThirdParty contained auth_digest_maxtries +syn keyword ngxDirectiveThirdParty contained auth_digest_replays +syn keyword ngxDirectiveThirdParty contained auth_digest_shm_size +syn keyword ngxDirectiveThirdParty contained auth_digest_timeout +syn keyword ngxDirectiveThirdParty contained auth_digest_user_file + +" https://github.com/stnoonan/spnego-http-auth-nginx-module +syn keyword ngxDirectiveThirdParty contained auth_gss +syn keyword ngxDirectiveThirdParty contained auth_gss_allow_basic_fallback +syn keyword ngxDirectiveThirdParty contained auth_gss_authorized_principal +syn keyword ngxDirectiveThirdParty contained auth_gss_authorized_principal_regex +syn keyword ngxDirectiveThirdParty contained auth_gss_constrained_delegation +syn keyword ngxDirectiveThirdParty contained auth_gss_delegate_credentials +syn keyword ngxDirectiveThirdParty contained auth_gss_force_realm +syn keyword ngxDirectiveThirdParty contained auth_gss_format_full +syn keyword ngxDirectiveThirdParty contained auth_gss_keytab +syn keyword ngxDirectiveThirdParty contained auth_gss_map_to_local +syn keyword ngxDirectiveThirdParty contained auth_gss_realm +syn keyword ngxDirectiveThirdParty contained auth_gss_service_ccache +syn keyword ngxDirectiveThirdParty contained auth_gss_service_name + +" https://github.com/kvspb/nginx-auth-ldap +syn keyword ngxDirectiveThirdParty contained auth_ldap +syn keyword ngxDirectiveThirdParty contained auth_ldap_cache_enabled +syn keyword ngxDirectiveThirdParty contained auth_ldap_cache_expiration_time +syn keyword ngxDirectiveThirdParty contained auth_ldap_cache_size +syn keyword ngxDirectiveThirdParty contained auth_ldap_servers +syn keyword ngxDirectiveThirdParty contained auth_ldap_servers_size +syn keyword ngxDirectiveThirdParty contained ldap_server + +" https://github.com/sto/ngx_http_auth_pam_module +syn keyword ngxDirectiveThirdParty contained auth_pam +syn keyword ngxDirectiveThirdParty contained auth_pam_service_name +syn keyword ngxDirectiveThirdParty contained auth_pam_set_pam_env + " https://github.com/arut/nginx-dav-ext-module syn keyword ngxDirectiveThirdParty contained dav_ext_lock syn keyword ngxDirectiveThirdParty contained dav_ext_lock_zone syn keyword ngxDirectiveThirdParty contained dav_ext_methods -" ngx_eval " https://github.com/openresty/nginx-eval-module syn keyword ngxDirectiveThirdParty contained eval syn keyword ngxDirectiveThirdParty contained eval_buffer_size @@ -971,7 +1031,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained eval_override_content_type syn keyword ngxDirectiveThirdParty contained eval_subrequest_in_memory -" Fancy Index " https://github.com/aperezdc/ngx-fancyindex syn keyword ngxDirectiveThirdParty contained fancyindex syn keyword ngxDirectiveThirdParty contained fancyindex_css_href @@ -988,40 +1047,29 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained fancyindex_show_path syn keyword ngxDirectiveThirdParty contained fancyindex_time_format -" Footer filter " https://github.com/alibaba/nginx-http-footer-filter syn keyword ngxDirectiveThirdParty contained footer syn keyword ngxDirectiveThirdParty contained footer_types -" ngx_http_geoip2_module " https://github.com/leev/ngx_http_geoip2_module syn keyword ngxDirectiveThirdParty contained geoip2 syn keyword ngxDirectiveThirdParty contained geoip2_proxy syn keyword ngxDirectiveThirdParty contained geoip2_proxy_recursive -" A version of the Nginx HTTP stub status module that outputs in JSON format -" https://github.com/nginx-modules/nginx-json-status-module -syn keyword ngxDirectiveThirdParty contained json_status -syn keyword ngxDirectiveThirdParty contained json_status_type +" https://github.com/ip2location/ip2location-nginx +syn keyword ngxDirectiveThirdParty contained ip2location_database +syn keyword ngxDirectiveThirdParty contained ip2location_proxy +syn keyword ngxDirectiveThirdParty contained ip2location_proxy_recursive -" MogileFS client for nginx -" https://github.com/vkholodkov/nginx-mogilefs-module -syn keyword ngxDirectiveThirdParty contained mogilefs_class -syn keyword ngxDirectiveThirdParty contained mogilefs_connect_timeout -syn keyword ngxDirectiveThirdParty contained mogilefs_domain -syn keyword ngxDirectiveThirdParty contained mogilefs_methods -syn keyword ngxDirectiveThirdParty contained mogilefs_noverify -syn keyword ngxDirectiveThirdParty contained mogilefs_pass -syn keyword ngxDirectiveThirdParty contained mogilefs_read_timeout -syn keyword ngxDirectiveThirdParty contained mogilefs_send_timeout -syn keyword ngxDirectiveThirdParty contained mogilefs_tracker +" https://github.com/ip2location/ip2proxy-nginx +syn keyword ngxDirectiveThirdParty contained ip2proxy_database +syn keyword ngxDirectiveThirdParty contained ip2proxy_proxy +syn keyword ngxDirectiveThirdParty contained ip2proxy_proxy_recursive -" Ancient nginx plugin; probably not useful to anyone " https://github.com/kr/nginx-notice syn keyword ngxDirectiveThirdParty contained notice syn keyword ngxDirectiveThirdParty contained notice_type -" nchan " https://github.com/slact/nchan syn keyword ngxDirectiveThirdParty contained nchan_access_control_allow_credentials syn keyword ngxDirectiveThirdParty contained nchan_access_control_allow_origin @@ -1034,8 +1082,8 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained nchan_benchmark_subscriber_distribution syn keyword ngxDirectiveThirdParty contained nchan_benchmark_subscribers_per_channel syn keyword ngxDirectiveThirdParty contained nchan_benchmark_time +syn keyword ngxDirectiveThirdParty contained nchan_channel_event_string syn keyword ngxDirectiveThirdParty contained nchan_channel_events_channel_id -syn keyword ngxDirectiveThirdParty contained nchan_channel_event_string syn keyword ngxDirectiveThirdParty contained nchan_channel_group syn keyword ngxDirectiveThirdParty contained nchan_channel_group_accounting syn keyword ngxDirectiveThirdParty contained nchan_channel_id @@ -1074,11 +1122,25 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained nchan_pubsub_channel_id syn keyword ngxDirectiveThirdParty contained nchan_pubsub_location syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_check_interval +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_check_interval_backoff +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_check_interval_jitter +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_check_interval_max +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_check_interval_min +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_connect_timeout +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_max_failing_time +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_recovery_delay +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_recovery_delay_backoff +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_recovery_delay_jitter +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_recovery_delay_max +syn keyword ngxDirectiveThirdParty contained nchan_redis_cluster_recovery_delay_min +syn keyword ngxDirectiveThirdParty contained nchan_redis_command_timeout syn keyword ngxDirectiveThirdParty contained nchan_redis_connect_timeout syn keyword ngxDirectiveThirdParty contained nchan_redis_discovered_ip_range_blacklist syn keyword ngxDirectiveThirdParty contained nchan_redis_fakesub_timer_interval syn keyword ngxDirectiveThirdParty contained nchan_redis_idle_channel_cache_timeout +syn keyword ngxDirectiveThirdParty contained nchan_redis_load_scripts_unconditionally syn keyword ngxDirectiveThirdParty contained nchan_redis_namespace +syn keyword ngxDirectiveThirdParty contained nchan_redis_node_connect_timeout syn keyword ngxDirectiveThirdParty contained nchan_redis_nostore_fastpublish syn keyword ngxDirectiveThirdParty contained nchan_redis_optimize_target syn keyword ngxDirectiveThirdParty contained nchan_redis_pass @@ -1086,6 +1148,13 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained nchan_redis_password syn keyword ngxDirectiveThirdParty contained nchan_redis_ping_interval syn keyword ngxDirectiveThirdParty contained nchan_redis_publish_msgpacked_max_size +syn keyword ngxDirectiveThirdParty contained nchan_redis_reconnect_delay +syn keyword ngxDirectiveThirdParty contained nchan_redis_reconnect_delay_backoff +syn keyword ngxDirectiveThirdParty contained nchan_redis_reconnect_delay_jitter +syn keyword ngxDirectiveThirdParty contained nchan_redis_reconnect_delay_max +syn keyword ngxDirectiveThirdParty contained nchan_redis_reconnect_delay_min +syn keyword ngxDirectiveThirdParty contained nchan_redis_retry_commands +syn keyword ngxDirectiveThirdParty contained nchan_redis_retry_commands_max_wait syn keyword ngxDirectiveThirdParty contained nchan_redis_server syn keyword ngxDirectiveThirdParty contained nchan_redis_ssl syn keyword ngxDirectiveThirdParty contained nchan_redis_ssl_ciphers @@ -1113,10 +1182,10 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained nchan_stub_status syn keyword ngxDirectiveThirdParty contained nchan_sub_channel_id syn keyword ngxDirectiveThirdParty contained nchan_subscribe_existing_channels_only +syn keyword ngxDirectiveThirdParty contained nchan_subscribe_request syn keyword ngxDirectiveThirdParty contained nchan_subscriber syn keyword ngxDirectiveThirdParty contained nchan_subscriber_channel_id syn keyword ngxDirectiveThirdParty contained nchan_subscriber_compound_etag_message_id -syn keyword ngxDirectiveThirdParty contained nchan_subscribe_request syn keyword ngxDirectiveThirdParty contained nchan_subscriber_first_message syn keyword ngxDirectiveThirdParty contained nchan_subscriber_http_raw_stream_separator syn keyword ngxDirectiveThirdParty contained nchan_subscriber_info @@ -1145,7 +1214,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained push_subscriber_concurrency syn keyword ngxDirectiveThirdParty contained push_subscriber_timeout -" Push Stream " https://github.com/wandenberg/nginx-push-stream-module syn keyword ngxDirectiveThirdParty contained push_stream_allow_connections_to_events_channel syn keyword ngxDirectiveThirdParty contained push_stream_allowed_origins @@ -1184,23 +1252,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained push_stream_wildcard_channel_max_qtd syn keyword ngxDirectiveThirdParty contained push_stream_wildcard_channel_prefix -" redis module -" https://www.nginx.com/resources/wiki/modules/redis/ -syn keyword ngxDirectiveThirdParty contained redis_bind -syn keyword ngxDirectiveThirdParty contained redis_buffer_size -syn keyword ngxDirectiveThirdParty contained redis_connect_timeout -syn keyword ngxDirectiveThirdParty contained redis_gzip_flag -syn keyword ngxDirectiveThirdParty contained redis_next_upstream -syn keyword ngxDirectiveThirdParty contained redis_pass -syn keyword ngxDirectiveThirdParty contained redis_read_timeout -syn keyword ngxDirectiveThirdParty contained redis_send_timeout - -" ngx_http_response -" http://catap.ru/downloads/nginx/ -syn keyword ngxDirectiveThirdParty contained response -syn keyword ngxDirectiveThirdParty contained response_type - -" nginx_substitutions_filter " https://github.com/yaoweibin/ngx_http_substitutions_filter_module syn keyword ngxDirectiveThirdParty contained subs_buffers syn keyword ngxDirectiveThirdParty contained subs_filter @@ -1208,7 +1259,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained subs_filter_types syn keyword ngxDirectiveThirdParty contained subs_line_buffer_size -" Tarantool nginx upstream module " https://github.com/tarantool/nginx_upstream_module syn keyword ngxDirectiveThirdParty contained tnt_allowed_indexes syn keyword ngxDirectiveThirdParty contained tnt_allowed_spaces @@ -1238,44 +1288,28 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained tnt_update syn keyword ngxDirectiveThirdParty contained tnt_upsert -" A module for nginx web server for handling file uploads using multipart/form-data encoding (RFC 1867) -" https://github.com/Austinb/nginx-upload-module +" https://github.com/fdintino/nginx-upload-module +syn keyword ngxDirectiveThirdParty contained upload_add_header syn keyword ngxDirectiveThirdParty contained upload_aggregate_form_field -syn keyword ngxDirectiveThirdParty contained upload_archive_elm -syn keyword ngxDirectiveThirdParty contained upload_archive_elm_separator -syn keyword ngxDirectiveThirdParty contained upload_archive_path -syn keyword ngxDirectiveThirdParty contained upload_archive_path_separator syn keyword ngxDirectiveThirdParty contained upload_buffer_size syn keyword ngxDirectiveThirdParty contained upload_cleanup -syn keyword ngxDirectiveThirdParty contained upload_content_type -syn keyword ngxDirectiveThirdParty contained upload_discard -syn keyword ngxDirectiveThirdParty contained upload_field_name -syn keyword ngxDirectiveThirdParty contained upload_file_crc32 -syn keyword ngxDirectiveThirdParty contained upload_file_md5 -syn keyword ngxDirectiveThirdParty contained upload_file_md5_uc -syn keyword ngxDirectiveThirdParty contained upload_file_name -syn keyword ngxDirectiveThirdParty contained upload_file_sha1 -syn keyword ngxDirectiveThirdParty contained upload_file_sha1_uc -syn keyword ngxDirectiveThirdParty contained upload_file_size -syn keyword ngxDirectiveThirdParty contained upload_filter +syn keyword ngxDirectiveThirdParty contained upload_empty_fiels_names +syn keyword ngxDirectiveThirdParty contained upload_limit_rate syn keyword ngxDirectiveThirdParty contained upload_max_file_size syn keyword ngxDirectiveThirdParty contained upload_max_output_body_len syn keyword ngxDirectiveThirdParty contained upload_max_part_header_len +syn keyword ngxDirectiveThirdParty contained upload_merge_buffer_size syn keyword ngxDirectiveThirdParty contained upload_pass syn keyword ngxDirectiveThirdParty contained upload_pass_args syn keyword ngxDirectiveThirdParty contained upload_pass_form_field +syn keyword ngxDirectiveThirdParty contained upload_range_header_buffer_size +syn keyword ngxDirectiveThirdParty contained upload_resumable syn keyword ngxDirectiveThirdParty contained upload_set_form_field +syn keyword ngxDirectiveThirdParty contained upload_state_store syn keyword ngxDirectiveThirdParty contained upload_store syn keyword ngxDirectiveThirdParty contained upload_store_access -syn keyword ngxDirectiveThirdParty contained upload_tmp_path -syn keyword ngxDirectiveThirdParty contained upload_unzip -syn keyword ngxDirectiveThirdParty contained upload_unzip_buffers -syn keyword ngxDirectiveThirdParty contained upload_unzip_hash -syn keyword ngxDirectiveThirdParty contained upload_unzip_max_file_name_len -syn keyword ngxDirectiveThirdParty contained upload_unzip_window -syn keyword ngxDirectiveThirdParty contained upload_void_content_type +syn keyword ngxDirectiveThirdParty contained upload_tame_arrays -" nginx-upload-progress-module " https://github.com/masterzen/nginx-upload-progress-module syn keyword ngxDirectiveThirdParty contained report_uploads syn keyword ngxDirectiveThirdParty contained track_uploads @@ -1288,9 +1322,7 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained upload_progress_jsonp_parameter syn keyword ngxDirectiveThirdParty contained upload_progress_template -" Health checks upstreams for nginx " https://github.com/yaoweibin/nginx_upstream_check_module -syn keyword ngxDirectiveThirdParty contained check syn keyword ngxDirectiveThirdParty contained check_fastcgi_param syn keyword ngxDirectiveThirdParty contained check_http_expect_alive syn keyword ngxDirectiveThirdParty contained check_http_send @@ -1298,13 +1330,14 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained check_shm_size syn keyword ngxDirectiveThirdParty contained check_status -" The fair load balancer module for nginx -" https://github.com/cryptofuture/nginx-upstream-fair +" https://github.com/jaygooby/nginx-upstream-fair syn keyword ngxDirectiveThirdParty contained fair syn keyword ngxDirectiveThirdParty contained upstream_fair_shm_size -" Nginx Video Thumb Extractor Module -" https://github.com/wandenberg/nginx-video-thumbextractor-module +" https://github.com/ayty-adrianomartins/nginx-sticky-module-ng +syn keyword ngxDirectiveThirdParty contained sticky_no_fallback + +" https://github.com/Novetta/nginx-video-thumbextractor-module syn keyword ngxDirectiveThirdParty contained video_thumbextractor syn keyword ngxDirectiveThirdParty contained video_thumbextractor_image_height syn keyword ngxDirectiveThirdParty contained video_thumbextractor_image_width @@ -1329,43 +1362,14 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained video_thumbextractor_video_filename syn keyword ngxDirectiveThirdParty contained video_thumbextractor_video_second -" drizzle-nginx-module - Upstream module for talking to MySQL and Drizzle directly -" https://github.com/openresty/drizzle-nginx-module -syn keyword ngxDirectiveThirdParty contained drizzle_buffer_size -syn keyword ngxDirectiveThirdParty contained drizzle_connect_timeout -syn keyword ngxDirectiveThirdParty contained drizzle_dbname -syn keyword ngxDirectiveThirdParty contained drizzle_keepalive -syn keyword ngxDirectiveThirdParty contained drizzle_module_header -syn keyword ngxDirectiveThirdParty contained drizzle_pass -syn keyword ngxDirectiveThirdParty contained drizzle_query -syn keyword ngxDirectiveThirdParty contained drizzle_recv_cols_timeout -syn keyword ngxDirectiveThirdParty contained drizzle_recv_rows_timeout -syn keyword ngxDirectiveThirdParty contained drizzle_send_query_timeout -syn keyword ngxDirectiveThirdParty contained drizzle_server -syn keyword ngxDirectiveThirdParty contained drizzle_status +" https://github.com/calio/iconv-nginx-module +syn keyword ngxDirectiveThirdParty contained iconv_buffer_size +syn keyword ngxDirectiveThirdParty contained iconv_filter +syn keyword ngxDirectiveThirdParty contained set_iconv -" ngx_dynamic_upstream -" https://github.com/cubicdaiya/ngx_dynamic_upstream -syn keyword ngxDirectiveThirdParty contained dynamic_upstream - -" encrypt and decrypt nginx variable values -" https://github.com/openresty/encrypted-session-nginx-module -syn keyword ngxDirectiveThirdParty contained encrypted_session_expires -syn keyword ngxDirectiveThirdParty contained encrypted_session_iv -syn keyword ngxDirectiveThirdParty contained encrypted_session_key -syn keyword ngxDirectiveThirdParty contained set_decrypt_session -syn keyword ngxDirectiveThirdParty contained set_encrypt_session - -" serve content directly from MongoDB's GridFS -" https://github.com/mdirolf/nginx-gridfs -syn keyword ngxDirectiveThirdParty contained gridfs -syn keyword ngxDirectiveThirdParty contained mongo - -" Adds support for arithmetic operations to NGINX config -" https://github.com/arut/nginx-let-module +" https://github.com/baysao/nginx-let-module syn keyword ngxDirectiveThirdParty contained let -" ngx_http_lua_module - Embed the power of Lua into Nginx HTTP Servers " https://github.com/openresty/lua-nginx-module syn keyword ngxDirectiveThirdParty contained access_by_lua syn keyword ngxDirectiveThirdParty contained access_by_lua_block @@ -1431,6 +1435,8 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained rewrite_by_lua_block syn keyword ngxDirectiveThirdParty contained rewrite_by_lua_file syn keyword ngxDirectiveThirdParty contained rewrite_by_lua_no_postpone +syn keyword ngxDirectiveThirdParty contained server_rewrite_by_lua_block +syn keyword ngxDirectiveThirdParty contained server_rewrite_by_lua_file syn keyword ngxDirectiveThirdParty contained set_by_lua syn keyword ngxDirectiveThirdParty contained set_by_lua_block syn keyword ngxDirectiveThirdParty contained set_by_lua_file @@ -1443,7 +1449,16 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained ssl_session_store_by_lua_block syn keyword ngxDirectiveThirdParty contained ssl_session_store_by_lua_file -" ngx_memc - An extended version of the standard memcached module +" https://github.com/Taymindis/nginx-link-function +syn keyword ngxDirectiveThirdParty contained ngx_link_func_add_prop +syn keyword ngxDirectiveThirdParty contained ngx_link_func_add_req_header +syn keyword ngxDirectiveThirdParty contained ngx_link_func_ca_cert +syn keyword ngxDirectiveThirdParty contained ngx_link_func_call +syn keyword ngxDirectiveThirdParty contained ngx_link_func_download_link_lib +syn keyword ngxDirectiveThirdParty contained ngx_link_func_lib +syn keyword ngxDirectiveThirdParty contained ngx_link_func_shm_size +syn keyword ngxDirectiveThirdParty contained ngx_link_func_subrequest + " https://github.com/openresty/memc-nginx-module syn keyword ngxDirectiveThirdParty contained memc_buffer_size syn keyword ngxDirectiveThirdParty contained memc_cmds_allowed @@ -1457,21 +1472,24 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained memc_upstream_fail_timeout syn keyword ngxDirectiveThirdParty contained memc_upstream_max_fails -" ModSecurity web application firewall -" https://github.com/SpiderLabs/ModSecurity/tree/master -syn keyword ngxDirectiveThirdParty contained ModSecurityConfig -syn keyword ngxDirectiveThirdParty contained ModSecurityEnabled -syn keyword ngxDirectiveThirdParty contained pool_context_hash_size +" https://github.com/SpiderLabs/ModSecurity-nginx +syn keyword ngxDirectiveThirdParty contained modsecurity +syn keyword ngxDirectiveThirdParty contained modsecurity_rules +syn keyword ngxDirectiveThirdParty contained modsecurity_rules_file +syn keyword ngxDirectiveThirdParty contained modsecurity_rules_remote +syn keyword ngxDirectiveThirdParty contained modsecurity_transaction_id -" NAXSI is an open-source, high performance, low rules maintenance WAF for NGINX " https://github.com/nbs-system/naxsi syn keyword ngxDirectiveThirdParty contained BasicRule syn keyword ngxDirectiveThirdParty contained CheckRule syn keyword ngxDirectiveThirdParty contained DeniedUrl +syn keyword ngxDirectiveThirdParty contained IgnoreCIDR +syn keyword ngxDirectiveThirdParty contained IgnoreIP syn keyword ngxDirectiveThirdParty contained LearningMode syn keyword ngxDirectiveThirdParty contained LibInjectionSql syn keyword ngxDirectiveThirdParty contained LibInjectionXss syn keyword ngxDirectiveThirdParty contained MainRule +syn keyword ngxDirectiveThirdParty contained NaxsiLogFile syn keyword ngxDirectiveThirdParty contained SecRulesDisabled syn keyword ngxDirectiveThirdParty contained SecRulesEnabled syn keyword ngxDirectiveThirdParty contained basic_rule @@ -1481,17 +1499,31 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained libinjection_sql syn keyword ngxDirectiveThirdParty contained libinjection_xss syn keyword ngxDirectiveThirdParty contained main_rule +syn keyword ngxDirectiveThirdParty contained naxsi_log syn keyword ngxDirectiveThirdParty contained rules_disabled syn keyword ngxDirectiveThirdParty contained rules_enabled -" Phusion Passenger -" https://www.phusionpassenger.com/library/config/nginx/reference/ +" https://github.com/opentracing-contrib/nginx-opentracing +syn keyword ngxDirectiveThirdParty contained opentracing +syn keyword ngxDirectiveThirdParty contained opentracing_fastcgi_propagate_context +syn keyword ngxDirectiveThirdParty contained opentracing_grpc_propagate_context +syn keyword ngxDirectiveThirdParty contained opentracing_load_tracer +syn keyword ngxDirectiveThirdParty contained opentracing_location_operation_name +syn keyword ngxDirectiveThirdParty contained opentracing_operation_name +syn keyword ngxDirectiveThirdParty contained opentracing_propagate_context +syn keyword ngxDirectiveThirdParty contained opentracing_tag +syn keyword ngxDirectiveThirdParty contained opentracing_trace_locations +syn keyword ngxDirectiveThirdParty contained opentracing_trust_incoming_span + +" https://github.com/phusion/passenger syn keyword ngxDirectiveThirdParty contained passenger_abort_on_startup_error syn keyword ngxDirectiveThirdParty contained passenger_abort_websockets_on_process_shutdown syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_auth_type syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_password syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_url syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_username +syn keyword ngxDirectiveThirdParty contained passenger_analytics_log_group +syn keyword ngxDirectiveThirdParty contained passenger_analytics_log_user syn keyword ngxDirectiveThirdParty contained passenger_anonymous_telemetry_proxy syn keyword ngxDirectiveThirdParty contained passenger_app_env syn keyword ngxDirectiveThirdParty contained passenger_app_file_descriptor_ulimit @@ -1499,20 +1531,25 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained passenger_app_log_file syn keyword ngxDirectiveThirdParty contained passenger_app_rights syn keyword ngxDirectiveThirdParty contained passenger_app_root +syn keyword ngxDirectiveThirdParty contained passenger_app_start_command syn keyword ngxDirectiveThirdParty contained passenger_app_type syn keyword ngxDirectiveThirdParty contained passenger_base_uri syn keyword ngxDirectiveThirdParty contained passenger_buffer_response syn keyword ngxDirectiveThirdParty contained passenger_buffer_size +syn keyword ngxDirectiveThirdParty contained passenger_buffer_upload syn keyword ngxDirectiveThirdParty contained passenger_buffers syn keyword ngxDirectiveThirdParty contained passenger_busy_buffers_size syn keyword ngxDirectiveThirdParty contained passenger_concurrency_model syn keyword ngxDirectiveThirdParty contained passenger_core_file_descriptor_ulimit syn keyword ngxDirectiveThirdParty contained passenger_ctl syn keyword ngxDirectiveThirdParty contained passenger_data_buffer_dir +syn keyword ngxDirectiveThirdParty contained passenger_debug_log_file syn keyword ngxDirectiveThirdParty contained passenger_debugger syn keyword ngxDirectiveThirdParty contained passenger_default_group syn keyword ngxDirectiveThirdParty contained passenger_default_user +syn keyword ngxDirectiveThirdParty contained passenger_direct_instance_request_address syn keyword ngxDirectiveThirdParty contained passenger_disable_anonymous_telemetry +syn keyword ngxDirectiveThirdParty contained passenger_disable_log_prefix syn keyword ngxDirectiveThirdParty contained passenger_disable_security_update_check syn keyword ngxDirectiveThirdParty contained passenger_document_root syn keyword ngxDirectiveThirdParty contained passenger_dump_config_manifest @@ -1548,8 +1585,10 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained passenger_pass_header syn keyword ngxDirectiveThirdParty contained passenger_pool_idle_time syn keyword ngxDirectiveThirdParty contained passenger_pre_start +syn keyword ngxDirectiveThirdParty contained passenger_preload_bundler syn keyword ngxDirectiveThirdParty contained passenger_python syn keyword ngxDirectiveThirdParty contained passenger_read_timeout +syn keyword ngxDirectiveThirdParty contained passenger_request_buffering syn keyword ngxDirectiveThirdParty contained passenger_request_queue_overflow_status_code syn keyword ngxDirectiveThirdParty contained passenger_resist_deployment_errors syn keyword ngxDirectiveThirdParty contained passenger_response_buffer_high_watermark @@ -1561,36 +1600,36 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained passenger_set_header syn keyword ngxDirectiveThirdParty contained passenger_show_version_in_header syn keyword ngxDirectiveThirdParty contained passenger_socket_backlog +syn keyword ngxDirectiveThirdParty contained passenger_spawn_dir +syn keyword ngxDirectiveThirdParty contained passenger_spawn_exception_status_code syn keyword ngxDirectiveThirdParty contained passenger_spawn_method syn keyword ngxDirectiveThirdParty contained passenger_start_timeout syn keyword ngxDirectiveThirdParty contained passenger_startup_file syn keyword ngxDirectiveThirdParty contained passenger_stat_throttle_rate syn keyword ngxDirectiveThirdParty contained passenger_sticky_sessions +syn keyword ngxDirectiveThirdParty contained passenger_sticky_sessions_cookie_attributes syn keyword ngxDirectiveThirdParty contained passenger_sticky_sessions_cookie_name +syn keyword ngxDirectiveThirdParty contained passenger_temp_path syn keyword ngxDirectiveThirdParty contained passenger_thread_count syn keyword ngxDirectiveThirdParty contained passenger_turbocaching +syn keyword ngxDirectiveThirdParty contained passenger_use_global_queue syn keyword ngxDirectiveThirdParty contained passenger_user syn keyword ngxDirectiveThirdParty contained passenger_user_switching syn keyword ngxDirectiveThirdParty contained passenger_vary_turbocache_by_cookie -syn keyword ngxDirectiveThirdPartyDeprecated contained passenger_analytics_log_group -syn keyword ngxDirectiveThirdPartyDeprecated contained passenger_analytics_log_user -syn keyword ngxDirectiveThirdPartyDeprecated contained passenger_debug_log_file -syn keyword ngxDirectiveThirdPartyDeprecated contained passenger_use_global_queue -syn keyword ngxDirectiveThirdPartyDeprecated contained rack_env -syn keyword ngxDirectiveThirdPartyDeprecated contained rails_app_spawner_idle_time -syn keyword ngxDirectiveThirdPartyDeprecated contained rails_env -syn keyword ngxDirectiveThirdPartyDeprecated contained rails_framework_spawner_idle_time -syn keyword ngxDirectiveThirdPartyDeprecated contained rails_spawn_method -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_filter -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_gateway_address -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_gateway_cert -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_gateway_port -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_key -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_proxy_address -syn keyword ngxDirectiveThirdPartyDeprecated contained union_station_support +syn keyword ngxDirectiveThirdParty contained rack_env +syn keyword ngxDirectiveThirdParty contained rails_app_spawner_idle_time +syn keyword ngxDirectiveThirdParty contained rails_env +syn keyword ngxDirectiveThirdParty contained rails_framework_spawner_idle_time +syn keyword ngxDirectiveThirdParty contained rails_spawn_method +syn keyword ngxDirectiveThirdParty contained union_station_filter +syn keyword ngxDirectiveThirdParty contained union_station_gateway_address +syn keyword ngxDirectiveThirdParty contained union_station_gateway_cert +syn keyword ngxDirectiveThirdParty contained union_station_gateway_port +syn keyword ngxDirectiveThirdParty contained union_station_key +syn keyword ngxDirectiveThirdParty contained union_station_proxy_address +syn keyword ngxDirectiveThirdParty contained union_station_support -" ngx_postgres is an upstream module that allows nginx to communicate directly with PostgreSQL database -" https://github.com/FRiCKLE/ngx_postgres +" https://github.com/konstruxi/ngx_postgres syn keyword ngxDirectiveThirdParty contained postgres_connect_timeout syn keyword ngxDirectiveThirdParty contained postgres_escape syn keyword ngxDirectiveThirdParty contained postgres_keepalive @@ -1602,7 +1641,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained postgres_server syn keyword ngxDirectiveThirdParty contained postgres_set -" ngx_rds_csv - Nginx output filter module to convert Resty-DBD-Streams (RDS) to Comma-Separated Values (CSV) " https://github.com/openresty/rds-csv-nginx-module syn keyword ngxDirectiveThirdParty contained rds_csv syn keyword ngxDirectiveThirdParty contained rds_csv_buffer_size @@ -1611,7 +1649,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained rds_csv_field_separator syn keyword ngxDirectiveThirdParty contained rds_csv_row_terminator -" ngx_rds_json - an output filter that formats Resty DBD Streams generated by ngx_drizzle and others to JSON " https://github.com/openresty/rds-json-nginx-module syn keyword ngxDirectiveThirdParty contained rds_json syn keyword ngxDirectiveThirdParty contained rds_json_buffer_size @@ -1624,7 +1661,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained rds_json_success_property syn keyword ngxDirectiveThirdParty contained rds_json_user_property -" ngx_redis2 - Nginx upstream module for the Redis 2.0 protocol " https://github.com/openresty/redis2-nginx-module syn keyword ngxDirectiveThirdParty contained redis2_bind syn keyword ngxDirectiveThirdParty contained redis2_buffer_size @@ -1638,7 +1674,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained redis2_read_timeout syn keyword ngxDirectiveThirdParty contained redis2_send_timeout -" NGINX-based Media Streaming Server " https://github.com/arut/nginx-rtmp-module syn keyword ngxDirectiveThirdParty contained ack_window syn keyword ngxDirectiveThirdParty contained application @@ -1750,7 +1785,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained wait_key syn keyword ngxDirectiveThirdParty contained wait_video -" ngx_set_misc - Various set_xxx directives added to nginx's rewrite module (md5/sha1, sql/json quoting, and many more) " https://github.com/openresty/set-misc-nginx-module syn keyword ngxDirectiveThirdParty contained set_base32_alphabet syn keyword ngxDirectiveThirdParty contained set_base32_padding @@ -1770,6 +1804,7 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained set_hmac_sha256 syn keyword ngxDirectiveThirdParty contained set_if_empty syn keyword ngxDirectiveThirdParty contained set_local_today +syn keyword ngxDirectiveThirdParty contained set_md5 syn keyword ngxDirectiveThirdParty contained set_misc_base32_padding syn keyword ngxDirectiveThirdParty contained set_quote_json_str syn keyword ngxDirectiveThirdParty contained set_quote_pgsql_str @@ -1778,20 +1813,18 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained set_rotate syn keyword ngxDirectiveThirdParty contained set_secure_random_alphanum syn keyword ngxDirectiveThirdParty contained set_secure_random_lcalpha +syn keyword ngxDirectiveThirdParty contained set_sha1 syn keyword ngxDirectiveThirdParty contained set_unescape_uri -" nginx-sflow-module " https://github.com/sflow/nginx-sflow-module syn keyword ngxDirectiveThirdParty contained sflow -" Shibboleth auth request module for Nginx " https://github.com/nginx-shib/nginx-http-shibboleth syn keyword ngxDirectiveThirdParty contained shib_request syn keyword ngxDirectiveThirdParty contained shib_request_set syn keyword ngxDirectiveThirdParty contained shib_request_use_headers -" nginx module which adds ability to cache static files -" https://github.com/FRiCKLE/ngx_slowfs_cache +" https://github.com/baysao/ngx_slowfs_cache syn keyword ngxDirectiveThirdParty contained slowfs_big_file_size syn keyword ngxDirectiveThirdParty contained slowfs_cache syn keyword ngxDirectiveThirdParty contained slowfs_cache_key @@ -1801,8 +1834,7 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained slowfs_cache_valid syn keyword ngxDirectiveThirdParty contained slowfs_temp_path -" Dynamic Image Transformation Module For nginx -" https://github.com/cubicdaiya/ngx_small_light +" https://github.com/kawakibi/ngx_small_light syn keyword ngxDirectiveThirdParty contained small_light syn keyword ngxDirectiveThirdParty contained small_light_buffer syn keyword ngxDirectiveThirdParty contained small_light_getparam_mode @@ -1812,7 +1844,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained small_light_radius_max syn keyword ngxDirectiveThirdParty contained small_light_sigma_max -" ngx_srcache - Transparent subrequest-based caching layout for arbitrary nginx locations " https://github.com/openresty/srcache-nginx-module syn keyword ngxDirectiveThirdParty contained srcache_buffer syn keyword ngxDirectiveThirdParty contained srcache_default_expire @@ -1835,7 +1866,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained srcache_store_skip syn keyword ngxDirectiveThirdParty contained srcache_store_statuses -" NGINX-based VOD Packager " https://github.com/kaltura/nginx-vod-module syn keyword ngxDirectiveThirdParty contained vod syn keyword ngxDirectiveThirdParty contained vod_align_segments_to_key_frames @@ -1875,6 +1905,7 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained vod_manifest_duration_policy syn keyword ngxDirectiveThirdParty contained vod_manifest_segment_durations_mode syn keyword ngxDirectiveThirdParty contained vod_mapping_cache +syn keyword ngxDirectiveThirdParty contained vod_max_frame_count syn keyword ngxDirectiveThirdParty contained vod_max_frames_size syn keyword ngxDirectiveThirdParty contained vod_max_mapping_response_size syn keyword ngxDirectiveThirdParty contained vod_max_metadata_size @@ -1901,6 +1932,7 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained vod_secret_key syn keyword ngxDirectiveThirdParty contained vod_segment_count_policy syn keyword ngxDirectiveThirdParty contained vod_segment_duration +syn keyword ngxDirectiveThirdParty contained vod_segment_max_frame_count syn keyword ngxDirectiveThirdParty contained vod_segments_base_url syn keyword ngxDirectiveThirdParty contained vod_source_clip_map_uri syn keyword ngxDirectiveThirdParty contained vod_speed_param_name @@ -1910,7 +1942,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained vod_upstream_extra_args syn keyword ngxDirectiveThirdParty contained vod_upstream_location -" Nginx virtual host traffic status module " https://github.com/vozlt/nginx-module-vts syn keyword ngxDirectiveThirdParty contained vhost_traffic_status syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_average_method @@ -1934,7 +1965,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_set_by_filter syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_zone -" xss-nginx-module - Native cross-site scripting support in nginx " https://github.com/openresty/xss-nginx-module syn keyword ngxDirectiveThirdParty contained xss_callback_arg syn keyword ngxDirectiveThirdParty contained xss_check_status @@ -1943,471 +1973,6 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained xss_output_type syn keyword ngxDirectiveThirdParty contained xss_override_status -" Add support for array-typed variables to nginx config files -" https://github.com/openresty/array-var-nginx-module -syn keyword ngxDirectiveThirdParty contained array_join -syn keyword ngxDirectiveThirdParty contained array_map -syn keyword ngxDirectiveThirdParty contained array_map_op -syn keyword ngxDirectiveThirdParty contained array_split - -" NGINX module for Brotli compression -" https://github.com/eustas/ngx_brotli -syn keyword ngxDirectiveThirdParty contained brotli -syn keyword ngxDirectiveThirdParty contained brotli_buffers -syn keyword ngxDirectiveThirdParty contained brotli_comp_level -syn keyword ngxDirectiveThirdParty contained brotli_min_length -syn keyword ngxDirectiveThirdParty contained brotli_static -syn keyword ngxDirectiveThirdParty contained brotli_types -syn keyword ngxDirectiveThirdParty contained brotli_window - -" form-input-nginx-module -" https://github.com/calio/form-input-nginx-module -syn keyword ngxDirectiveThirdParty contained set_form_input -syn keyword ngxDirectiveThirdParty contained set_form_input_multi - -" character conversion nginx module using libiconv From pluknet at nginx.com Mon Jun 20 17:49:00 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 20 Jun 2022 17:49:00 +0000 Subject: [nginx] Misc: win32 sources now preserved in release tarballs. Message-ID: details: https://hg.nginx.org/nginx/rev/70365b6fe5f4 branches: changeset: 8048:70365b6fe5f4 user: Maxim Dounin date: Mon Jun 20 19:30:50 2022 +0300 description: Misc: win32 sources now preserved in release tarballs. This makes it possible to build nginx under Windows from release tarballs instead of using source code repository. diffstat: misc/GNUmakefile | 6 ------ 1 files changed, 0 insertions(+), 6 deletions(-) diffs (16 lines): diff -r 1b7c7c3af6d9 -r 70365b6fe5f4 misc/GNUmakefile --- a/misc/GNUmakefile Sat Jun 18 15:54:40 2022 +0300 +++ b/misc/GNUmakefile Mon Jun 20 19:30:50 2022 +0300 @@ -15,12 +15,6 @@ release: export mv $(TEMP)/$(NGINX)/auto/configure $(TEMP)/$(NGINX) - # delete incomplete sources - rm $(TEMP)/$(NGINX)/src/event/ngx_event_acceptex.c - rm $(TEMP)/$(NGINX)/src/event/ngx_event_connectex.c - rm $(TEMP)/$(NGINX)/src/event/modules/ngx_iocp_module.* - rm -r $(TEMP)/$(NGINX)/src/os/win32 - mv $(TEMP)/$(NGINX)/docs/text/LICENSE $(TEMP)/$(NGINX) mv $(TEMP)/$(NGINX)/docs/text/README $(TEMP)/$(NGINX) mv $(TEMP)/$(NGINX)/docs/html $(TEMP)/$(NGINX) From pluknet at nginx.com Mon Jun 20 19:05:23 2022 From: pluknet at nginx.com (=?iso-8859-1?q?Sergey_Kandaurov?=) Date: Mon, 20 Jun 2022 23:05:23 +0400 Subject: [PATCH] Removed documentation for nonexistent memcached_force_ranges Message-ID: <53734009b0c36276e667.1655751923@enoparse.local> # HG changeset patch # User Sergey Kandaurov # Date 1655748656 -14400 # Mon Jun 20 22:10:56 2022 +0400 # Node ID 53734009b0c36276e66792e23180ee9665c95419 # Parent 40d40af45ac85339025b171c53033665a3632ee0 Removed documentation for nonexistent memcached_force_ranges. diff --git a/xml/en/docs/http/ngx_http_memcached_module.xml b/xml/en/docs/http/ngx_http_memcached_module.xml --- a/xml/en/docs/http/ngx_http_memcached_module.xml +++ b/xml/en/docs/http/ngx_http_memcached_module.xml @@ -10,7 +10,7 @@ + rev="18">
@@ -121,23 +121,6 @@ It should be noted that this timeout can - -on | off -off -http -server -location -1.7.7 - - -Enables byte-range support -for both cached and uncached responses from the memcached server -regardless of the
Accept-Ranges
field in these responses. - - - - - flag diff --git a/xml/ru/docs/http/ngx_http_memcached_module.xml b/xml/ru/docs/http/ngx_http_memcached_module.xml --- a/xml/ru/docs/http/ngx_http_memcached_module.xml +++ b/xml/ru/docs/http/ngx_http_memcached_module.xml @@ -10,7 +10,7 @@ + rev="18">
@@ -122,24 +122,6 @@ memcached_bind $remote_addr transparent; - -on | off -off -http -server -location -1.7.7 - - -Включает поддержку диапазонов запрашиваемых байт (byte-range) -для кэшированных и некэшированных ответов сервера memcached -вне зависимости от наличия поля
Accept-Ranges
-в заголовках этих ответов. - - - - - флаг From pluknet at nginx.com Mon Jun 20 19:33:43 2022 From: pluknet at nginx.com (=?iso-8859-1?q?Sergey_Kandaurov?=) Date: Mon, 20 Jun 2022 23:33:43 +0400 Subject: [PATCH] Corrected syntax of ssl_conf_command and friends Message-ID: <3e1af31b0ecd9e0ae5a8.1655753623@enoparse.local> # HG changeset patch # User Sergey Kandaurov # Date 1655753604 -14400 # Mon Jun 20 23:33:24 2022 +0400 # Node ID 3e1af31b0ecd9e0ae5a8a93a15ff15cc0b1f98f6 # Parent 53734009b0c36276e66792e23180ee9665c95419 Corrected syntax of ssl_conf_command and friends. Each directive takes exactly two arguments: command name and value. The parameter names follow SSL_CONF_cmd(3) documentation of OpenSSL 3.0. diff --git a/xml/en/docs/http/ngx_http_grpc_module.xml b/xml/en/docs/http/ngx_http_grpc_module.xml --- a/xml/en/docs/http/ngx_http_grpc_module.xml +++ b/xml/en/docs/http/ngx_http_grpc_module.xml @@ -10,7 +10,7 @@ + rev="8">
@@ -535,7 +535,7 @@ The full list can be viewed using the -command ... +option value http server diff --git a/xml/en/docs/http/ngx_http_proxy_module.xml b/xml/en/docs/http/ngx_http_proxy_module.xml --- a/xml/en/docs/http/ngx_http_proxy_module.xml +++ b/xml/en/docs/http/ngx_http_proxy_module.xml @@ -10,7 +10,7 @@ + rev="75">
@@ -1995,7 +1995,7 @@ The full list can be viewed using the -command ... +option value http server diff --git a/xml/en/docs/http/ngx_http_ssl_module.xml b/xml/en/docs/http/ngx_http_ssl_module.xml --- a/xml/en/docs/http/ngx_http_ssl_module.xml +++ b/xml/en/docs/http/ngx_http_ssl_module.xml @@ -10,7 +10,7 @@ + rev="58">
@@ -297,7 +297,7 @@ directive can be used. -command ... +option value http server diff --git a/xml/en/docs/http/ngx_http_uwsgi_module.xml b/xml/en/docs/http/ngx_http_uwsgi_module.xml --- a/xml/en/docs/http/ngx_http_uwsgi_module.xml +++ b/xml/en/docs/http/ngx_http_uwsgi_module.xml @@ -10,7 +10,7 @@ + rev="49">
@@ -1445,7 +1445,7 @@ The full list can be viewed using the -command ... +option value http server diff --git a/xml/en/docs/mail/ngx_mail_ssl_module.xml b/xml/en/docs/mail/ngx_mail_ssl_module.xml --- a/xml/en/docs/mail/ngx_mail_ssl_module.xml +++ b/xml/en/docs/mail/ngx_mail_ssl_module.xml @@ -10,7 +10,7 @@ + rev="25">
@@ -239,7 +239,7 @@ directive can be used. -command ... +option value mail server diff --git a/xml/en/docs/stream/ngx_stream_proxy_module.xml b/xml/en/docs/stream/ngx_stream_proxy_module.xml --- a/xml/en/docs/stream/ngx_stream_proxy_module.xml +++ b/xml/en/docs/stream/ngx_stream_proxy_module.xml @@ -9,7 +9,7 @@ + rev="30">
@@ -448,7 +448,7 @@ The full list can be viewed using the -command ... +option value stream server diff --git a/xml/en/docs/stream/ngx_stream_ssl_module.xml b/xml/en/docs/stream/ngx_stream_ssl_module.xml --- a/xml/en/docs/stream/ngx_stream_ssl_module.xml +++ b/xml/en/docs/stream/ngx_stream_ssl_module.xml @@ -9,7 +9,7 @@ + rev="30">
@@ -254,7 +254,7 @@ directive can be used. -command ... +option value stream server diff --git a/xml/en/docs/stream/ngx_stream_zone_sync_module.xml b/xml/en/docs/stream/ngx_stream_zone_sync_module.xml --- a/xml/en/docs/stream/ngx_stream_zone_sync_module.xml +++ b/xml/en/docs/stream/ngx_stream_zone_sync_module.xml @@ -9,7 +9,7 @@ + rev="7">
@@ -321,7 +321,7 @@ The full list can be viewed using the -command ... +option value stream server diff --git a/xml/ru/docs/http/ngx_http_grpc_module.xml b/xml/ru/docs/http/ngx_http_grpc_module.xml --- a/xml/ru/docs/http/ngx_http_grpc_module.xml +++ b/xml/ru/docs/http/ngx_http_grpc_module.xml @@ -10,7 +10,7 @@ + rev="8">
@@ -534,7 +534,7 @@ grpc_set_header Accept-Encoding ""; -command ... +параметр значение http server diff --git a/xml/ru/docs/http/ngx_http_proxy_module.xml b/xml/ru/docs/http/ngx_http_proxy_module.xml --- a/xml/ru/docs/http/ngx_http_proxy_module.xml +++ b/xml/ru/docs/http/ngx_http_proxy_module.xml @@ -10,7 +10,7 @@ + rev="75">
@@ -1996,7 +1996,7 @@ proxy_set_header Accept-Encoding ""; -command ... +параметр значение http server diff --git a/xml/ru/docs/http/ngx_http_ssl_module.xml b/xml/ru/docs/http/ngx_http_ssl_module.xml --- a/xml/ru/docs/http/ngx_http_ssl_module.xml +++ b/xml/ru/docs/http/ngx_http_ssl_module.xml @@ -10,7 +10,7 @@ + rev="58">
@@ -300,7 +300,7 @@ PEM, которые используются для -command ... +параметр значение http server diff --git a/xml/ru/docs/http/ngx_http_uwsgi_module.xml b/xml/ru/docs/http/ngx_http_uwsgi_module.xml --- a/xml/ru/docs/http/ngx_http_uwsgi_module.xml +++ b/xml/ru/docs/http/ngx_http_uwsgi_module.xml @@ -10,7 +10,7 @@ + rev="49">
@@ -1441,7 +1441,7 @@ uwsgi-сервер. -command ... +параметр значение http server diff --git a/xml/ru/docs/mail/ngx_mail_ssl_module.xml b/xml/ru/docs/mail/ngx_mail_ssl_module.xml --- a/xml/ru/docs/mail/ngx_mail_ssl_module.xml +++ b/xml/ru/docs/mail/ngx_mail_ssl_module.xml @@ -10,7 +10,7 @@ + rev="25">
@@ -241,7 +241,7 @@ PEM, которые используются для -command ... +параметр значение mail server diff --git a/xml/ru/docs/stream/ngx_stream_proxy_module.xml b/xml/ru/docs/stream/ngx_stream_proxy_module.xml --- a/xml/ru/docs/stream/ngx_stream_proxy_module.xml +++ b/xml/ru/docs/stream/ngx_stream_proxy_module.xml @@ -9,7 +9,7 @@ + rev="30">
@@ -447,7 +447,7 @@ Cессия завершится после отправки всех принятых датаграмм -command ... +параметр значение stream server diff --git a/xml/ru/docs/stream/ngx_stream_ssl_module.xml b/xml/ru/docs/stream/ngx_stream_ssl_module.xml --- a/xml/ru/docs/stream/ngx_stream_ssl_module.xml +++ b/xml/ru/docs/stream/ngx_stream_ssl_module.xml @@ -9,7 +9,7 @@ + rev="30">
@@ -257,7 +257,7 @@ PEM, которые используются для -command ... +параметр значение stream server From mdounin at mdounin.ru Mon Jun 20 20:27:47 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 20 Jun 2022 23:27:47 +0300 Subject: [PATCH] Removed documentation for nonexistent memcached_force_ranges In-Reply-To: <53734009b0c36276e667.1655751923@enoparse.local> References: <53734009b0c36276e667.1655751923@enoparse.local> Message-ID: Hello! On Mon, Jun 20, 2022 at 11:05:23PM +0400, Sergey Kandaurov wrote: > # HG changeset patch > # User Sergey Kandaurov > # Date 1655748656 -14400 > # Mon Jun 20 22:10:56 2022 +0400 > # Node ID 53734009b0c36276e66792e23180ee9665c95419 > # Parent 40d40af45ac85339025b171c53033665a3632ee0 > Removed documentation for nonexistent memcached_force_ranges. > > diff --git a/xml/en/docs/http/ngx_http_memcached_module.xml b/xml/en/docs/http/ngx_http_memcached_module.xml > --- a/xml/en/docs/http/ngx_http_memcached_module.xml > +++ b/xml/en/docs/http/ngx_http_memcached_module.xml > @@ -10,7 +10,7 @@ > link="/en/docs/http/ngx_http_memcached_module.html" > lang="en" > - rev="17"> > + rev="18"> > >
> > @@ -121,23 +121,6 @@ It should be noted that this timeout can > > > > - > -on | off > -off > -http > -server > -location > -1.7.7 > - > - > -Enables byte-range support > -for both cached and uncached responses from the memcached server > -regardless of the
Accept-Ranges
field in these responses. > - > - > - > - > - > > flag > > diff --git a/xml/ru/docs/http/ngx_http_memcached_module.xml b/xml/ru/docs/http/ngx_http_memcached_module.xml > --- a/xml/ru/docs/http/ngx_http_memcached_module.xml > +++ b/xml/ru/docs/http/ngx_http_memcached_module.xml > @@ -10,7 +10,7 @@ > link="/ru/docs/http/ngx_http_memcached_module.html" > lang="ru" > - rev="17"> > + rev="18"> > >
> > @@ -122,24 +122,6 @@ memcached_bind $remote_addr transparent; > > > > - > -on | off > -off > -http > -server > -location > -1.7.7 > - > - > -Включает поддержку диапазонов запрашиваемых байт (byte-range) > -для кэшированных и некэшированных ответов сервера memcached > -вне зависимости от наличия поля
Accept-Ranges
> -в заголовках этих ответов. > - > - > - > - > - > > флаг > Looks good. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Mon Jun 20 21:18:37 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 Jun 2022 00:18:37 +0300 Subject: [PATCH] Corrected syntax of ssl_conf_command and friends In-Reply-To: <3e1af31b0ecd9e0ae5a8.1655753623@enoparse.local> References: <3e1af31b0ecd9e0ae5a8.1655753623@enoparse.local> Message-ID: Hello! On Mon, Jun 20, 2022 at 11:33:43PM +0400, Sergey Kandaurov wrote: > # HG changeset patch > # User Sergey Kandaurov > # Date 1655753604 -14400 > # Mon Jun 20 23:33:24 2022 +0400 > # Node ID 3e1af31b0ecd9e0ae5a8a93a15ff15cc0b1f98f6 > # Parent 53734009b0c36276e66792e23180ee9665c95419 > Corrected syntax of ssl_conf_command and friends. > > Each directive takes exactly two arguments: command name and value. > The parameter names follow SSL_CONF_cmd(3) documentation of OpenSSL 3.0. > > diff --git a/xml/en/docs/http/ngx_http_grpc_module.xml b/xml/en/docs/http/ngx_http_grpc_module.xml > --- a/xml/en/docs/http/ngx_http_grpc_module.xml > +++ b/xml/en/docs/http/ngx_http_grpc_module.xml > @@ -10,7 +10,7 @@ > link="/en/docs/http/ngx_http_grpc_module.html" > lang="en" > - rev="7"> > + rev="8"> > >
> > @@ -535,7 +535,7 @@ The full list can be viewed using the > > > > -command ... > +option value I don't think such wording improves readability, since it basically contradicts the "command" in the name of the directive. The OpenSSL 3.0 documentation of SSL_CONF_cmd() does not seem to be self-consistent, and uses terms "command", "operation" and "option" in various places. Further, the term "option" is already used by SSL_CTX_set_options() in a completely different meaning, and it is also configurable with SSL_CONF_cmd("Options"). Overall, I would rather suggest to stick with the ssl_conf_command name value; syntax, which seems to be much more consistent and self-explanatory. [...] -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Tue Jun 21 01:27:56 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 Jun 2022 04:27:56 +0300 Subject: nginx-1.23.0 changes draft Message-ID: Hello! Changes with nginx 1.23.0 21 Jun 2022 *) Change in internal API: now header lines are represented as linked lists. *) Change: now nginx combines arbitrary header lines with identical names when sending to FastCGI, SCGI, and uwsgi backends, in the $r->header_in() method of the ngx_http_perl_module, and during lookup of the "$http_...", "$sent_http_...", "$sent_trailer_...", "$upstream_http_...", and "$upstream_trailer_..." variables. *) Bugfix: if there were multiple "Vary" header lines in the backend response, nginx only used the last of them when caching. *) Bugfix: if there were multiple "WWW-Authenticate" header lines in the backend response and errors with code 401 were intercepted or the "auth_request" directive was used, nginx only sent the first of the header lines to the client. *) Change: the logging level of the "application data after close notify" SSL errors has been lowered from "crit" to "info". *) Bugfix: connections might hang if nginx was build on Linux 2.6.17 or newer, but was used on systems without EPOLLRDHUP support, notably with epoll emulation layers; the bug had appeared in 1.17.5. Thanks to Marcus Ball. *) Bugfix: nginx did not cache the response if the "Expires" response header line disabled caching, but following "Cache-Control" header line enabled caching. Изменения в nginx 1.23.0 21.06.2022 *) Изменение во внутреннем API: теперь строки заголовков представлены связными списками. *) Изменение: теперь nginx объединяет произвольные строки заголовков с одинаковыми именами при отправке на FastCGI-, SCGI- и uwsgi- бэкенды, в методе $r->header_in() модуля ngx_http_perl_module, и при доступе через переменные "$http_...", "$sent_http_...", "$sent_trailer_...", "$upstream_http_..." и "$upstream_trailer_...". *) Исправление: если в заголовке ответа бэкенда было несколько строк "Vary", при кэшировании nginx учитывал только последнюю из них. *) Исправление: если в заголовке ответа бэкенда было несколько строк "WWW-Authenticate" и использовался перехват ошибок с кодом 401 от бэкенда или директива auth_request, nginx пересылал клиенту только первую из этих строк. *) Изменение: уровень логгирования ошибок SSL "application data after close notify" понижен с уровня crit до info. *) Исправление: соединения могли зависать, если nginx был собран на Linux 2.6.17 и новее, а использовался на системах без поддержки EPOLLRDHUP, в частности, на системах с эмуляцией epoll; ошибка появилась в 1.17.5. Спасибо Marcus Ball. *) Исправление: nginx не кэшировал ответ, если строка заголовка ответа "Expires" запрещала кэширование, а последующая строка заголовка "Cache-Control" разрешала кэширование. -- Maxim Dounin http://mdounin.ru/ From xeioex at nginx.com Tue Jun 21 05:45:35 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 21 Jun 2022 05:45:35 +0000 Subject: [njs] Version 0.7.5. Message-ID: details: https://hg.nginx.org/njs/rev/63c258c456ca branches: changeset: 1892:63c258c456ca user: Dmitry Volyntsev date: Mon Jun 20 21:38:49 2022 -0700 description: Version 0.7.5. diffstat: CHANGES | 40 ++++++++++++++++++++++++++++++++++++++++ 1 files changed, 40 insertions(+), 0 deletions(-) diffs (47 lines): diff -r 42a2406590c0 -r 63c258c456ca CHANGES --- a/CHANGES Thu Jun 16 17:33:49 2022 -0700 +++ b/CHANGES Mon Jun 20 21:38:49 2022 -0700 @@ -1,3 +1,43 @@ +Changes with njs 0.7.5 21 Jun 2022 + + nginx modules: + + *) Change: adapting to changes in nginx header structures. + + *) Bugfix: fixed r.headersOut special getters when value + is absent. + + *) Change: returning undefined value instead of an empty string + for Content-Type when the header is absent. + + Core: + + *) Bugfix: fixed catching of the exception thrown from an + awaited function. + + *) Bugfix: fixed function value initialization. + + *) Bugfix: fixed interpreter when await fails. + + *) Bugfix: fixed typed-array constructor when source array + is changed while iterating. + + *) Bugfix: fixed String.prototype.replace() with byte strings. + + *) Bugfix: fixed template literal from producing byte-strings. + + *) Bugfix: fixed array iterator with sparse arrays. + + *) Bugfix: fixed memory free while converting a flat array to + a slow array. + + *) Bugfix: properly handling NJS_DECLINE in promise native + functions. + + *) Bugfix: fixed working with an array-like object in Promise.all() + and friends. + + Changes with njs 0.7.4 24 May 2022 nginx modules: From xeioex at nginx.com Tue Jun 21 05:45:37 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 21 Jun 2022 05:45:37 +0000 Subject: [njs] Added tag 0.7.5 for changeset 63c258c456ca Message-ID: details: https://hg.nginx.org/njs/rev/90f0e772e13f branches: changeset: 1893:90f0e772e13f user: Dmitry Volyntsev date: Mon Jun 20 22:26:55 2022 -0700 description: Added tag 0.7.5 for changeset 63c258c456ca diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 63c258c456ca -r 90f0e772e13f .hgtags --- a/.hgtags Mon Jun 20 21:38:49 2022 -0700 +++ b/.hgtags Mon Jun 20 22:26:55 2022 -0700 @@ -50,3 +50,4 @@ 35aca5cc5ea7582b80947caa1e3f4a4fb8ee232d 3dd315b80bab10b6ac475ee25dd207d2eb759881 0.7.2 f15d039cf625fb92e061f21c9f28a788032a0faa 0.7.3 b5198f7f11a3b5e174f9e75a7bd50394fa354fb0 0.7.4 +63c258c456ca018385b13f352faefdf25c7bd3bb 0.7.5 From thresh at nginx.com Tue Jun 21 10:14:35 2022 From: thresh at nginx.com (Konstantin Pavlov) Date: Tue, 21 Jun 2022 14:14:35 +0400 Subject: nginx-1.23.0 changes draft In-Reply-To: References: Message-ID: <1fb67c7a-90f7-2ae8-fde8-e8207a8d9432@nginx.com> Hello, On 21/06/2022 5:27 AM, Maxim Dounin wrote: > *) Bugfix: connections might hang if nginx was build on Linux 2.6.17 or was built -- Konstantin Pavlov Principal Consultant https://www.nginx.com From pluknet at nginx.com Tue Jun 21 11:20:31 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 21 Jun 2022 15:20:31 +0400 Subject: nginx-1.23.0 changes draft In-Reply-To: References: Message-ID: > On 21 Jun 2022, at 05:27, Maxim Dounin wrote: > > Hello! > > > Changes with nginx 1.23.0 21 Jun 2022 > > *) Change in internal API: now header lines are represented as linked > lists. > > *) Change: now nginx combines arbitrary header lines with identical > names when sending to FastCGI, SCGI, and uwsgi backends, in the > $r->header_in() method of the ngx_http_perl_module, and during lookup > of the "$http_...", "$sent_http_...", "$sent_trailer_...", > "$upstream_http_...", and "$upstream_trailer_..." variables. > > *) Bugfix: if there were multiple "Vary" header lines in the backend > response, nginx only used the last of them when caching. > > *) Bugfix: if there were multiple "WWW-Authenticate" header lines in the > backend response and errors with code 401 were intercepted or the > "auth_request" directive was used, nginx only sent the first of the > header lines to the client. > > *) Change: the logging level of the "application data after close > notify" SSL errors has been lowered from "crit" to "info". > > *) Bugfix: connections might hang if nginx was build on Linux 2.6.17 or > newer, but was used on systems without EPOLLRDHUP support, notably > with epoll emulation layers; the bug had appeared in 1.17.5. > Thanks to Marcus Ball. Just wonder how such emulation layer is widespread enough to step on the bug and to deserve an entry in changes, comparing to the bugfix for "ignore long locked" alert with two cache managers that's not there. > > *) Bugfix: nginx did not cache the response if the "Expires" response > header line disabled caching, but following "Cache-Control" header > line enabled caching. > > > Изменения в nginx 1.23.0 21.06.2022 > > *) Изменение во внутреннем API: теперь строки заголовков представлены > связными списками. > > *) Изменение: теперь nginx объединяет произвольные строки заголовков с > одинаковыми именами при отправке на FastCGI-, SCGI- и uwsgi- бэкенды, a stray space in "uwsgi- "? > в методе $r->header_in() модуля ngx_http_perl_module, и при доступе > через переменные "$http_...", "$sent_http_...", "$sent_trailer_...", > "$upstream_http_..." и "$upstream_trailer_...". > > *) Исправление: если в заголовке ответа бэкенда было несколько строк > "Vary", при кэшировании nginx учитывал только последнюю из них. > > *) Исправление: если в заголовке ответа бэкенда было несколько строк > "WWW-Authenticate" и использовался перехват ошибок с кодом 401 от > бэкенда или директива auth_request, nginx пересылал клиенту только > первую из этих строк. > > *) Изменение: уровень логгирования ошибок SSL "application data after > close notify" понижен с уровня crit до info. > > *) Исправление: соединения могли зависать, если nginx был собран на > Linux 2.6.17 и новее, а использовался на системах без поддержки > EPOLLRDHUP, в частности, на системах с эмуляцией epoll; ошибка > появилась в 1.17.5. > Спасибо Marcus Ball. > > *) Исправление: nginx не кэшировал ответ, если строка заголовка ответа > "Expires" запрещала кэширование, а последующая строка заголовка > "Cache-Control" разрешала кэширование. > Otherwise, looks good. -- Sergey Kandaurov From mdounin at mdounin.ru Tue Jun 21 12:55:39 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 Jun 2022 15:55:39 +0300 Subject: nginx-1.23.0 changes draft In-Reply-To: <1fb67c7a-90f7-2ae8-fde8-e8207a8d9432@nginx.com> References: <1fb67c7a-90f7-2ae8-fde8-e8207a8d9432@nginx.com> Message-ID: Hello! On Tue, Jun 21, 2022 at 02:14:35PM +0400, Konstantin Pavlov wrote: > Hello, > > On 21/06/2022 5:27 AM, Maxim Dounin wrote: > > *) Bugfix: connections might hang if nginx was build on Linux 2.6.17 or > > was built Fixed, thnx. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Tue Jun 21 13:19:30 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 Jun 2022 16:19:30 +0300 Subject: nginx-1.23.0 changes draft In-Reply-To: References: Message-ID: Hello! On Tue, Jun 21, 2022 at 03:20:31PM +0400, Sergey Kandaurov wrote: > > On 21 Jun 2022, at 05:27, Maxim Dounin wrote: > > > > Changes with nginx 1.23.0 21 Jun 2022 > > > > *) Change in internal API: now header lines are represented as linked > > lists. > > > > *) Change: now nginx combines arbitrary header lines with identical > > names when sending to FastCGI, SCGI, and uwsgi backends, in the > > $r->header_in() method of the ngx_http_perl_module, and during lookup > > of the "$http_...", "$sent_http_...", "$sent_trailer_...", > > "$upstream_http_...", and "$upstream_trailer_..." variables. > > > > *) Bugfix: if there were multiple "Vary" header lines in the backend > > response, nginx only used the last of them when caching. > > > > *) Bugfix: if there were multiple "WWW-Authenticate" header lines in the > > backend response and errors with code 401 were intercepted or the > > "auth_request" directive was used, nginx only sent the first of the > > header lines to the client. > > > > *) Change: the logging level of the "application data after close > > notify" SSL errors has been lowered from "crit" to "info". > > > > *) Bugfix: connections might hang if nginx was build on Linux 2.6.17 or > > newer, but was used on systems without EPOLLRDHUP support, notably > > with epoll emulation layers; the bug had appeared in 1.17.5. > > Thanks to Marcus Ball. > > Just wonder how such emulation layer is widespread enough to step on > the bug and to deserve an entry in changes, comparing to the bugfix for > "ignore long locked" alert with two cache managers that's not there. The "ignore long locked" alert is something caught with artificial stress testing and never reported in practice (or at least never attributed to the particular race condition). Further, there is no real issue, just a race condition which results in an unwanted alert. As such, I've skipped it as something minor, mostly internal and very unlikely to be seen by users. If you think it worth an entry I can add one though. The EPOLLRDHUP issue is a real one, which results in connection hangs. And, as far as I understand, it was observed by Marcus in a real world use case on DigitalOcean's App Platform. As such, it certainly deserves to be mentioned in CHANGES. > > *) Bugfix: nginx did not cache the response if the "Expires" response > > header line disabled caching, but following "Cache-Control" header > > line enabled caching. > > > > > > Изменения в nginx 1.23.0 21.06.2022 > > > > *) Изменение во внутреннем API: теперь строки заголовков представлены > > связными списками. > > > > *) Изменение: теперь nginx объединяет произвольные строки заголовков с > > одинаковыми именами при отправке на FastCGI-, SCGI- и uwsgi- бэкенды, > > a stray space in "uwsgi- "? Yes, fixed, thnx. [...] -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Tue Jun 21 16:11:58 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 Jun 2022 19:11:58 +0300 Subject: nginx-1.23.0 changes draft In-Reply-To: References: Message-ID: Hello! On Tue, Jun 21, 2022 at 04:19:30PM +0300, Maxim Dounin wrote: > Hello! > > On Tue, Jun 21, 2022 at 03:20:31PM +0400, Sergey Kandaurov wrote: > > > > On 21 Jun 2022, at 05:27, Maxim Dounin wrote: > > > > > > Changes with nginx 1.23.0 21 Jun 2022 > > > > > > *) Change in internal API: now header lines are represented as linked > > > lists. > > > > > > *) Change: now nginx combines arbitrary header lines with identical > > > names when sending to FastCGI, SCGI, and uwsgi backends, in the > > > $r->header_in() method of the ngx_http_perl_module, and during lookup > > > of the "$http_...", "$sent_http_...", "$sent_trailer_...", > > > "$upstream_http_...", and "$upstream_trailer_..." variables. > > > > > > *) Bugfix: if there were multiple "Vary" header lines in the backend > > > response, nginx only used the last of them when caching. > > > > > > *) Bugfix: if there were multiple "WWW-Authenticate" header lines in the > > > backend response and errors with code 401 were intercepted or the > > > "auth_request" directive was used, nginx only sent the first of the > > > header lines to the client. > > > > > > *) Change: the logging level of the "application data after close > > > notify" SSL errors has been lowered from "crit" to "info". > > > > > > *) Bugfix: connections might hang if nginx was build on Linux 2.6.17 or > > > newer, but was used on systems without EPOLLRDHUP support, notably > > > with epoll emulation layers; the bug had appeared in 1.17.5. > > > Thanks to Marcus Ball. > > > > Just wonder how such emulation layer is widespread enough to step on > > the bug and to deserve an entry in changes, comparing to the bugfix for > > "ignore long locked" alert with two cache managers that's not there. > > The "ignore long locked" alert is something caught with artificial > stress testing and never reported in practice (or at least never > attributed to the particular race condition). Further, there is > no real issue, just a race condition which results in an unwanted > alert. As such, I've skipped it as something minor, mostly > internal and very unlikely to be seen by users. If you think it > worth an entry I can add one though. > > The EPOLLRDHUP issue is a real one, which results in connection > hangs. And, as far as I understand, it was observed by Marcus in a > real world use case on DigitalOcean's App Platform. As such, it > certainly deserves to be mentioned in CHANGES. > > > > *) Bugfix: nginx did not cache the response if the "Expires" response > > > header line disabled caching, but following "Cache-Control" header > > > line enabled caching. > > > > > > > > > Изменения в nginx 1.23.0 21.06.2022 > > > > > > *) Изменение во внутреннем API: теперь строки заголовков представлены > > > связными списками. > > > > > > *) Изменение: теперь nginx объединяет произвольные строки заголовков с > > > одинаковыми именами при отправке на FastCGI-, SCGI- и uwsgi- бэкенды, > > > > a stray space in "uwsgi- "? > > Yes, fixed, thnx. Pushed to: http://mdounin.ru/hg/nginx http://mdounin.ru/hg/nginx.org Release files as build by me: http://mdounin.ru/temp/nginx-1.23.0.tar.gz http://mdounin.ru/temp/nginx-1.23.0.tar.gz.asc http://mdounin.ru/temp/nginx-1.23.0.zip http://mdounin.ru/temp/nginx-1.23.0.zip.asc -- Maxim Dounin http://mdounin.ru/ From thresh at nginx.com Tue Jun 21 16:19:36 2022 From: thresh at nginx.com (Konstantin Pavlov) Date: Tue, 21 Jun 2022 16:19:36 +0000 Subject: [nginx] Updated OpenSSL used for win32 builds. Message-ID: details: https://hg.nginx.org/nginx/rev/c4e1c97098e1 branches: changeset: 8049:c4e1c97098e1 user: Maxim Dounin date: Tue Jun 21 17:09:34 2022 +0300 description: Updated OpenSSL used for win32 builds. diffstat: misc/GNUmakefile | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 70365b6fe5f4 -r c4e1c97098e1 misc/GNUmakefile --- a/misc/GNUmakefile Mon Jun 20 19:30:50 2022 +0300 +++ b/misc/GNUmakefile Tue Jun 21 17:09:34 2022 +0300 @@ -6,7 +6,7 @@ TEMP = tmp CC = cl OBJS = objs.msvc8 -OPENSSL = openssl-1.1.1o +OPENSSL = openssl-1.1.1p ZLIB = zlib-1.2.12 PCRE = pcre2-10.39 From thresh at nginx.com Tue Jun 21 16:19:39 2022 From: thresh at nginx.com (Konstantin Pavlov) Date: Tue, 21 Jun 2022 16:19:39 +0000 Subject: [nginx] nginx-1.23.0-RELEASE Message-ID: details: https://hg.nginx.org/nginx/rev/5da2c0902e8e branches: changeset: 8050:5da2c0902e8e user: Maxim Dounin date: Tue Jun 21 17:25:36 2022 +0300 description: nginx-1.23.0-RELEASE diffstat: docs/xml/nginx/changes.xml | 101 +++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 101 insertions(+), 0 deletions(-) diffs (111 lines): diff -r c4e1c97098e1 -r 5da2c0902e8e docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml Tue Jun 21 17:09:34 2022 +0300 +++ b/docs/xml/nginx/changes.xml Tue Jun 21 17:25:36 2022 +0300 @@ -5,6 +5,107 @@ + + + + +Изменение во внутреннем API: +теперь строки заголовков представлены связными списками. + + +Change in internal API: +now header lines are represented as linked lists. + + + + + +теперь nginx объединяет произвольные строки заголовков с одинаковыми именами +при отправке на FastCGI-, SCGI- и uwsgi-бэкенды, +в методе $r->header_in() модуля ngx_http_perl_module, +и при доступе через переменные "$http_...", "$sent_http_...", +"$sent_trailer_...", "$upstream_http_..." и "$upstream_trailer_...". + + +now nginx combines arbitrary header lines with identical names +when sending to FastCGI, SCGI, and uwsgi backends, +in the $r->header_in() method of the ngx_http_perl_module, +and during lookup of the "$http_...", "$sent_http_...", +"$sent_trailer_...", "$upstream_http_...", and "$upstream_trailer_..." +variables. + + + + + +если в заголовке ответа бэкенда было несколько строк "Vary", +при кэшировании nginx учитывал только последнюю из них. + + +if there were multiple "Vary" header lines in the backend response, +nginx only used the last of them when caching. + + + + + +если в заголовке ответа бэкенда было несколько строк "WWW-Authenticate" +и использовался перехват ошибок с кодом 401 от бэкенда +или директива auth_request, +nginx пересылал клиенту только первую из этих строк. + + +if there were multiple "WWW-Authenticate" header lines in the backend response +and errors with code 401 were intercepted +or the "auth_request" directive was used, +nginx only sent the first of the header lines to the client. + + + + + +уровень логгирования ошибок SSL "application data after close notify" +понижен с уровня crit до info. + + +the logging level of the "application data after close notify" SSL errors +has been lowered from "crit" to "info". + + + + + +соединения могли зависать, если nginx был собран на Linux 2.6.17 и новее, +а использовался на системах без поддержки EPOLLRDHUP, в частности, на +системах с эмуляцией epoll; +ошибка появилась в 1.17.5.
+Спасибо Marcus Ball. + + +connections might hang if nginx was built on Linux 2.6.17 or newer, +but was used on systems without EPOLLRDHUP support, notably with epoll +emulation layers; +the bug had appeared in 1.17.5.
+Thanks to Marcus Ball. + + + + + +nginx не кэшировал ответ, +если строка заголовка ответа "Expires" запрещала кэширование, +а последующая строка заголовка "Cache-Control" разрешала кэширование. + + +nginx did not cache the response +if the "Expires" response header line disabled caching, +but following "Cache-Control" header line enabled caching. + + + + + + From thresh at nginx.com Tue Jun 21 16:19:42 2022 From: thresh at nginx.com (Konstantin Pavlov) Date: Tue, 21 Jun 2022 16:19:42 +0000 Subject: [nginx] release-1.23.0 tag Message-ID: details: https://hg.nginx.org/nginx/rev/fecd73db563f branches: changeset: 8051:fecd73db563f user: Maxim Dounin date: Tue Jun 21 17:25:37 2022 +0300 description: release-1.23.0 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 5da2c0902e8e -r fecd73db563f .hgtags --- a/.hgtags Tue Jun 21 17:25:36 2022 +0300 +++ b/.hgtags Tue Jun 21 17:25:37 2022 +0300 @@ -467,3 +467,4 @@ 2217a9c1d0b86026f22700b3c089545db1964f55 39be8a682c58308d9399cddd57e37f9fdb7bdf3e release-1.21.4 d986378168fd4d70e0121cabac274c560cca9bdf release-1.21.5 714eb4b2c09e712fb2572a2164ce2bf67638ccac release-1.21.6 +5da2c0902e8e2aa4534008a582a60c61c135960e release-1.23.0 From xeioex at nginx.com Wed Jun 22 05:14:57 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 22 Jun 2022 05:14:57 +0000 Subject: [njs] Version bump. Message-ID: details: https://hg.nginx.org/njs/rev/8ff4c3126a89 branches: changeset: 1894:8ff4c3126a89 user: Dmitry Volyntsev date: Tue Jun 21 15:29:00 2022 -0700 description: Version bump. diffstat: src/njs.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r 90f0e772e13f -r 8ff4c3126a89 src/njs.h --- a/src/njs.h Mon Jun 20 22:26:55 2022 -0700 +++ b/src/njs.h Tue Jun 21 15:29:00 2022 -0700 @@ -11,8 +11,8 @@ #include -#define NJS_VERSION "0.7.5" -#define NJS_VERSION_NUMBER 0x000705 +#define NJS_VERSION "0.7.6" +#define NJS_VERSION_NUMBER 0x000706 #include /* STDOUT_FILENO, STDERR_FILENO */ From xeioex at nginx.com Wed Jun 22 05:14:59 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 22 Jun 2022 05:14:59 +0000 Subject: [njs] Added Symbol.for() and Symbol.keyfor(). Message-ID: details: https://hg.nginx.org/njs/rev/f7fec5be38a0 branches: changeset: 1895:f7fec5be38a0 user: Dmitry Volyntsev date: Tue Jun 21 21:54:14 2022 -0700 description: Added Symbol.for() and Symbol.keyfor(). diffstat: src/njs_extern.c | 2 +- src/njs_symbol.c | 146 +++++++++++++++++++++++++++++++++++----------- src/njs_symbol.h | 10 +++ src/njs_value.h | 3 +- src/njs_vm.c | 2 + src/njs_vm.h | 1 + src/test/njs_unit_test.c | 37 +++++++++++ 7 files changed, 162 insertions(+), 39 deletions(-) diffs (336 lines): diff -r 8ff4c3126a89 -r f7fec5be38a0 src/njs_extern.c --- a/src/njs_extern.c Tue Jun 21 15:29:00 2022 -0700 +++ b/src/njs_extern.c Tue Jun 21 21:54:14 2022 -0700 @@ -49,7 +49,7 @@ njs_external_add(njs_vm_t *vm, njs_arr_t prop->enumerable = external->enumerable; if (external->flags & 4) { - njs_set_symbol(&prop->name, external->name.symbol); + njs_set_symbol(&prop->name, external->name.symbol, NULL); lhq.key_hash = external->name.symbol; diff -r 8ff4c3126a89 -r f7fec5be38a0 src/njs_symbol.c --- a/src/njs_symbol.c Tue Jun 21 15:29:00 2022 -0700 +++ b/src/njs_symbol.c Tue Jun 21 21:54:14 2022 -0700 @@ -57,20 +57,16 @@ static const njs_value_t *njs_symbol_na const njs_value_t * njs_symbol_description(const njs_value_t *value) { - const njs_value_t *name; - - if (njs_symbol_key(value) < NJS_SYMBOL_KNOWN_MAX) { - name = njs_symbol_names[njs_symbol_key(value)]; + uint32_t key; - } else { - name = value->data.u.value; + key = njs_symbol_key(value); - if (name == NULL) { - return &njs_value_undefined; - } + if (key < NJS_SYMBOL_KNOWN_MAX) { + return njs_symbol_names[key]; } - return name; + return value->data.u.value != NULL ? value->data.u.value + : &njs_value_undefined; } @@ -109,9 +105,9 @@ static njs_int_t njs_symbol_constructor(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { + uint64_t key; njs_int_t ret; njs_value_t *value, *name; - uint64_t key; if (njs_slow_path(vm->top_frame->ctor)) { njs_type_error(vm, "Symbol is not a constructor"); @@ -120,25 +116,13 @@ njs_symbol_constructor(njs_vm_t *vm, njs value = njs_arg(args, nargs, 1); - if (njs_is_undefined(value)) { - name = NULL; - - } else { + if (njs_is_defined(value)) { if (njs_slow_path(!njs_is_string(value))) { ret = njs_value_to_string(vm, value, value); if (njs_slow_path(ret != NJS_OK)) { return ret; } } - - name = njs_mp_alloc(vm->mem_pool, sizeof(njs_value_t)); - if (njs_slow_path(name == NULL)) { - njs_memory_error(vm); - return NJS_ERROR; - } - - /* GC: retain */ - *name = *value; } key = ++vm->symbol_generator; @@ -148,32 +132,100 @@ njs_symbol_constructor(njs_vm_t *vm, njs return NJS_ERROR; } - vm->retval.type = NJS_SYMBOL; - vm->retval.data.truth = 1; - vm->retval.data.magic32 = key; - vm->retval.data.u.value = name; + name = njs_mp_alloc(vm->mem_pool, sizeof(njs_value_t)); + if (njs_slow_path(name == NULL)) { + njs_memory_error(vm); + return NJS_ERROR; + } + + njs_value_assign(name, value); + njs_set_symbol(&vm->retval, key, name); return NJS_OK; } static njs_int_t -njs_symbol_for(njs_vm_t *vm, njs_value_t *args, - njs_uint_t nargs, njs_index_t unused) +njs_symbol_for(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t unused) { - njs_internal_error(vm, "not implemented"); + uint64_t key; + njs_int_t ret; + njs_value_t *value; + njs_rbtree_node_t *rb_node; + njs_rb_symbol_node_t *node; + + value = njs_arg(args, nargs, 1); + + if (njs_slow_path(!njs_is_string(value))) { + ret = njs_value_to_string(vm, value, value); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + } + + rb_node = njs_rbtree_min(&vm->global_symbols); + + while (njs_rbtree_is_there_successor(&vm->global_symbols, rb_node)) { + + node = (njs_rb_symbol_node_t *) rb_node; - return NJS_ERROR; + if (njs_is_string(&node->name) + && njs_string_cmp(value, &node->name) == 0) + { + njs_set_symbol(&vm->retval, node->key, &node->name); + return NJS_OK; + } + + rb_node = njs_rbtree_node_successor(&vm->global_symbols, rb_node); + } + + key = ++vm->symbol_generator; + + if (njs_slow_path(key >= UINT32_MAX)) { + njs_internal_error(vm, "Symbol generator overflow"); + return NJS_ERROR; + } + + node = njs_mp_alloc(vm->mem_pool, sizeof(njs_rb_symbol_node_t)); + if (njs_slow_path(node == NULL)) { + njs_memory_error(vm); + return NJS_ERROR; + } + + node->key = key; + njs_value_assign(&node->name, value); + + njs_rbtree_insert(&vm->global_symbols, &node->node); + + njs_set_symbol(&vm->retval, key, &node->name); + + return NJS_OK; } static njs_int_t -njs_symbol_key_for(njs_vm_t *vm, njs_value_t *args, - njs_uint_t nargs, njs_index_t unused) +njs_symbol_key_for(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t unused) { - njs_internal_error(vm, "not implemented"); + njs_value_t *value; + njs_rb_symbol_node_t query, *node; + + value = njs_arg(args, nargs, 1); - return NJS_ERROR; + if (njs_slow_path(!njs_is_symbol(value))) { + njs_type_error(vm, "is not a symbol"); + return NJS_ERROR; + } + + query.key = njs_symbol_key(value); + node = (njs_rb_symbol_node_t *) njs_rbtree_find(&vm->global_symbols, + &query.node); + + njs_value_assign(&vm->retval, + node != NULL ? &node->name : &njs_value_undefined); + + return NJS_OK; } @@ -355,7 +407,7 @@ njs_symbol_prototype_description(njs_vm_ return ret; } - vm->retval = *njs_symbol_description(&vm->retval); + njs_value_assign(&vm->retval, njs_symbol_description(&vm->retval)); return NJS_OK; } @@ -426,3 +478,23 @@ const njs_object_type_init_t njs_symbol .prototype_props = &njs_symbol_prototype_init, .prototype_value = { .object = { .type = NJS_OBJECT } }, }; + + +intptr_t +njs_symbol_rbtree_cmp(njs_rbtree_node_t *node1, njs_rbtree_node_t *node2) +{ + njs_rb_symbol_node_t *item1, *item2; + + item1 = (njs_rb_symbol_node_t *) node1; + item2 = (njs_rb_symbol_node_t *) node2; + + if (item1->key < item2->key) { + return -1; + } + + if (item1->key == item2->key) { + return 0; + } + + return 1; +} diff -r 8ff4c3126a89 -r f7fec5be38a0 src/njs_symbol.h --- a/src/njs_symbol.h Tue Jun 21 15:29:00 2022 -0700 +++ b/src/njs_symbol.h Tue Jun 21 21:54:14 2022 -0700 @@ -7,9 +7,19 @@ #ifndef _NJS_SYMBOL_H_INCLUDED_ #define _NJS_SYMBOL_H_INCLUDED_ + +typedef struct { + NJS_RBTREE_NODE (node); + uint32_t key; + njs_value_t name; +} njs_rb_symbol_node_t; + + const njs_value_t *njs_symbol_description(const njs_value_t *value); njs_int_t njs_symbol_descriptive_string(njs_vm_t *vm, njs_value_t *dst, const njs_value_t *value); +intptr_t njs_symbol_rbtree_cmp(njs_rbtree_node_t *node1, + njs_rbtree_node_t *node2); extern const njs_object_type_init_t njs_symbol_type_init; diff -r 8ff4c3126a89 -r f7fec5be38a0 src/njs_value.h --- a/src/njs_value.h Tue Jun 21 15:29:00 2022 -0700 +++ b/src/njs_value.h Tue Jun 21 21:54:14 2022 -0700 @@ -875,11 +875,12 @@ njs_set_uint32(njs_value_t *value, uint3 njs_inline void -njs_set_symbol(njs_value_t *value, uint32_t symbol) +njs_set_symbol(njs_value_t *value, uint32_t symbol, njs_value_t *name) { value->data.magic32 = symbol; value->type = NJS_SYMBOL; value->data.truth = 1; + value->data.u.value = name; } diff -r 8ff4c3126a89 -r f7fec5be38a0 src/njs_vm.c --- a/src/njs_vm.c Tue Jun 21 15:29:00 2022 -0700 +++ b/src/njs_vm.c Tue Jun 21 21:54:14 2022 -0700 @@ -396,6 +396,8 @@ njs_vm_init(njs_vm_t *vm) njs_lvlhsh_init(&vm->modules_hash); njs_lvlhsh_init(&vm->events_hash); + njs_rbtree_init(&vm->global_symbols, njs_symbol_rbtree_cmp); + njs_queue_init(&vm->posted_events); njs_queue_init(&vm->promise_events); diff -r 8ff4c3126a89 -r f7fec5be38a0 src/njs_vm.h --- a/src/njs_vm.h Tue Jun 21 15:29:00 2022 -0700 +++ b/src/njs_vm.h Tue Jun 21 21:54:14 2022 -0700 @@ -197,6 +197,7 @@ struct njs_vm_s { njs_trace_t trace; njs_random_t random; + njs_rbtree_t global_symbols; uint64_t symbol_generator; }; diff -r 8ff4c3126a89 -r f7fec5be38a0 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Tue Jun 21 15:29:00 2022 -0700 +++ b/src/test/njs_unit_test.c Tue Jun 21 21:54:14 2022 -0700 @@ -12866,6 +12866,43 @@ static njs_unit_test_t njs_test[] = ".every((x) => 'Symbol(Symbol.' + x.k + ')' == String(x.v))"), njs_str("true") }, + { njs_str("Symbol.for('desc').toString()"), + njs_str("Symbol(desc)") }, + + { njs_str("Symbol.for({toString: () => 'desc'}).description"), + njs_str("desc") }, + + { njs_str("Symbol.for('desc') === Symbol.for('desc')"), + njs_str("true") }, + + { njs_str("Symbol.keyFor(Symbol())"), + njs_str("undefined") }, + + { njs_str("Symbol.keyFor(Symbol('desc'))"), + njs_str("undefined") }, + + { njs_str("Symbol.keyFor(1)"), + njs_str("TypeError: is not a symbol") }, + + { njs_str("Symbol.keyFor(Symbol.for('desc'))"), + njs_str("desc") }, + + { njs_str("[ 'asyncIterator'," + " 'hasInstance'," + " 'isConcatSpreadable'," + " 'iterator'," + " 'match'," + " 'matchAll'," + " 'replace'," + " 'search'," + " 'species'," + " 'split'," + " 'toPrimitive'," + " 'toStringTag'," + " 'unscopables' ]" + ".every(v => Symbol.keyFor(Symbol[v]) === undefined)"), + njs_str("true") }, + { njs_str("typeof Symbol.prototype.description"), njs_str("TypeError: unexpected value type:object") }, From thresh at nginx.com Wed Jun 22 10:38:24 2022 From: thresh at nginx.com (=?iso-8859-1?q?Konstantin_Pavlov?=) Date: Wed, 22 Jun 2022 14:38:24 +0400 Subject: [PATCH] Linux packages: updated the supported architectures for Ubuntu 22.04 Message-ID: # HG changeset patch # User Konstantin Pavlov # Date 1655894271 -14400 # Wed Jun 22 14:37:51 2022 +0400 # Node ID b2e6057d99afc0cd843928a6f20b20ab5b8b92e2 # Parent cf91f42cf7e7a952860ecfd65fdd48f7b3729da1 Linux packages: updated the supported architectures for Ubuntu 22.04. diff -r cf91f42cf7e7 -r b2e6057d99af xml/en/linux_packages.xml --- a/xml/en/linux_packages.xml Tue Jun 21 17:49:58 2022 +0100 +++ b/xml/en/linux_packages.xml Wed Jun 22 14:37:51 2022 +0400 @@ -7,7 +7,7 @@
+ rev="76">
@@ -94,7 +94,7 @@ versions: 22.04 “jammy” -x86_64, aarch64/arm64 +x86_64, aarch64/arm64, s390x diff -r cf91f42cf7e7 -r b2e6057d99af xml/ru/linux_packages.xml --- a/xml/ru/linux_packages.xml Tue Jun 21 17:49:58 2022 +0100 +++ b/xml/ru/linux_packages.xml Wed Jun 22 14:37:51 2022 +0400 @@ -7,7 +7,7 @@
+ rev="76">
@@ -94,7 +94,7 @@ 22.04 “jammy” -x86_64, aarch64/arm64 +x86_64, aarch64/arm64, s390x From maxim at nginx.com Wed Jun 22 11:13:26 2022 From: maxim at nginx.com (Maxim Konovalov) Date: Wed, 22 Jun 2022 14:13:26 +0300 Subject: [PATCH] Linux packages: updated the supported architectures for Ubuntu 22.04 In-Reply-To: References: Message-ID: <5bfc8e6e-4b9a-c62e-2d34-e644b83d7aca@nginx.com> On 22.06.2022 13:38, Konstantin Pavlov wrote: > # HG changeset patch > # User Konstantin Pavlov > # Date 1655894271 -14400 > # Wed Jun 22 14:37:51 2022 +0400 > # Node ID b2e6057d99afc0cd843928a6f20b20ab5b8b92e2 > # Parent cf91f42cf7e7a952860ecfd65fdd48f7b3729da1 > Linux packages: updated the supported architectures for Ubuntu 22.04. > [...] Looks good. -- Maxim Konovalov From mdounin at mdounin.ru Wed Jun 22 17:51:07 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 22 Jun 2022 20:51:07 +0300 Subject: what's the proper way to get module specific configuration in init_process In-Reply-To: <4e653e995eda3fbe41ac592d465b6709@asiainfo.com> References: <4e653e995eda3fbe41ac592d465b6709@asiainfo.com> Message-ID: Hello! On Mon, Jun 20, 2022 at 04:05:53PM +0800, shanlei at asiainfo.com wrote: > Hello, guys, > > Recently I have been working on an nginx c module, which have to > leverage on init_process routine.The logic is based on instruction > command which may be configured inside stream.server, I have to do some > work in the init_process routine, for example, to schedule a timer. > I did google a while ,and decide to using code snippet like below: > > ============================ > > ngx_stream_conf_ctx_t conf_ctx = (ngx_stream_conf_ctx_t > *)ngx_get_conf(cycle->conf_ctx, ngx_stream_module); > > "module_spec_conf = conf_ctx->srv_conf["my_module".ctx_index]; > > ============================= > > > However, the module_spec_conf I got is the top_level ,that is created > when nginx parse the stream section, not the one created when nginx > parse the stream.server section, the later has the proper values. > Currently to got the later structure, I have to copy the configure value > back from child to parent in configuration merger function, which is > inconvenient and buggy when the stream includes multi servers. > > So I wonder what's the proper way to get module specific configuration > in the routine: init_process For the purpose of the init_process callback, the most simple approach would be to use your module's main configuration. For example, the embedded Perl module uses the init process callback to set correct PID in Perl interpreter: Perl interpreter is stored in the module's main configuration, and the init process callback uses the ngx_http_cycle_get_module_main_conf() macro to access it. See src/http/modules/perl/ngx_http_perl_module.c for details. (Just in case, corresponding macro in the stream module is ngx_stream_cycle_get_module_main_conf().) -- Maxim Dounin http://mdounin.ru/ From xeioex at nginx.com Thu Jun 23 06:40:52 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 23 Jun 2022 06:40:52 +0000 Subject: [njs] Stream: throwing an exception when s.done() is used while filtering. Message-ID: details: https://hg.nginx.org/njs/rev/9f544b8e28c5 branches: changeset: 1896:9f544b8e28c5 user: Dmitry Volyntsev date: Wed Jun 22 23:37:15 2022 -0700 description: Stream: throwing an exception when s.done() is used while filtering. diffstat: nginx/ngx_stream_js_module.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diffs (15 lines): diff -r f7fec5be38a0 -r 9f544b8e28c5 nginx/ngx_stream_js_module.c --- a/nginx/ngx_stream_js_module.c Tue Jun 21 21:54:14 2022 -0700 +++ b/nginx/ngx_stream_js_module.c Wed Jun 22 23:37:15 2022 -0700 @@ -1114,6 +1114,11 @@ ngx_stream_js_ext_done(njs_vm_t *vm, njs ctx = ngx_stream_get_module_ctx(s, ngx_stream_js_module); + if (ctx->filter) { + njs_vm_error(vm, "should not be called while filtering"); + return NJS_ERROR; + } + ngx_log_debug1(NGX_LOG_DEBUG_STREAM, s->connection->log, 0, "stream js set status: %i", status); From xeioex at nginx.com Thu Jun 23 06:40:54 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 23 Jun 2022 06:40:54 +0000 Subject: [njs] Fixed large non-decimal literals. Message-ID: details: https://hg.nginx.org/njs/rev/33b1637aad33 branches: changeset: 1897:33b1637aad33 user: Dmitry Volyntsev date: Wed Jun 22 23:37:16 2022 -0700 description: Fixed large non-decimal literals. This closes #551 issue on Github. diffstat: src/njs_number.c | 27 +++++++++++---------------- src/njs_number.h | 8 +++----- src/test/njs_unit_test.c | 21 +++++++++++++++++++++ 3 files changed, 35 insertions(+), 21 deletions(-) diffs (163 lines): diff -r 9f544b8e28c5 -r 33b1637aad33 src/njs_number.c --- a/src/njs_number.c Wed Jun 22 23:37:15 2022 -0700 +++ b/src/njs_number.c Wed Jun 22 23:37:16 2022 -0700 @@ -61,11 +61,11 @@ njs_number_dec_parse(const u_char **star } -uint64_t +double njs_number_oct_parse(const u_char **start, const u_char *end) { u_char c; - uint64_t num; + double num; const u_char *p, *_; p = *start; @@ -95,11 +95,11 @@ njs_number_oct_parse(const u_char **star } -uint64_t +double njs_number_bin_parse(const u_char **start, const u_char *end) { u_char c; - uint64_t num; + double num; const u_char *p, *_; p = *start; @@ -129,11 +129,11 @@ njs_number_bin_parse(const u_char **star } -uint64_t +double njs_number_hex_parse(const u_char **start, const u_char *end, njs_bool_t literal) { - uint64_t num; + double num; njs_int_t n; const u_char *p, *_; @@ -163,12 +163,11 @@ njs_number_hex_parse(const u_char **star } -int64_t +static double njs_number_radix_parse(const u_char **start, const u_char *end, uint8_t radix) { uint8_t d; - int64_t num; - uint64_t n; + double num, n; const u_char *p; static const int8_t digits[256] @@ -192,7 +191,7 @@ njs_number_radix_parse(const u_char **st -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, }; - num = -1; + num = NAN; n = 0; for (p = *start; p < end; p++) { @@ -1066,7 +1065,6 @@ njs_number_parse_int(njs_vm_t *vm, njs_v njs_index_t unused) { double num; - int64_t n; int32_t radix; njs_int_t ret; njs_str_t string; @@ -1142,12 +1140,9 @@ found: radix = 16; } - n = njs_number_radix_parse(&p, end, radix); + num = njs_number_radix_parse(&p, end, radix); - if (n >= 0) { - num = n; - num = minus ? -num : num; - } + num = minus ? -num : num; done: diff -r 9f544b8e28c5 -r 33b1637aad33 src/njs_number.h --- a/src/njs_number.h Wed Jun 22 23:37:15 2022 -0700 +++ b/src/njs_number.h Wed Jun 22 23:37:16 2022 -0700 @@ -16,12 +16,10 @@ double njs_key_to_index(const njs_value_t *value); double njs_number_dec_parse(const u_char **start, const u_char *end, njs_bool_t literal); -uint64_t njs_number_oct_parse(const u_char **start, const u_char *end); -uint64_t njs_number_bin_parse(const u_char **start, const u_char *end); -uint64_t njs_number_hex_parse(const u_char **start, const u_char *end, +double njs_number_oct_parse(const u_char **start, const u_char *end); +double njs_number_bin_parse(const u_char **start, const u_char *end); +double njs_number_hex_parse(const u_char **start, const u_char *end, njs_bool_t literal); -int64_t njs_number_radix_parse(const u_char **start, const u_char *end, - uint8_t radix); njs_int_t njs_number_to_string(njs_vm_t *vm, njs_value_t *string, const njs_value_t *number); njs_int_t njs_number_to_chain(njs_vm_t *vm, njs_chb_t *chain, diff -r 9f544b8e28c5 -r 33b1637aad33 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Wed Jun 22 23:37:15 2022 -0700 +++ b/src/test/njs_unit_test.c Wed Jun 22 23:37:16 2022 -0700 @@ -338,6 +338,9 @@ static njs_unit_test_t njs_test[] = { njs_str("-0O7_7_7"), njs_str("-511") }, + { njs_str("0o7777777777777777777777777777777777700000000000000000000000000000000"), + njs_str("3.2138760885179806e+60") }, + { njs_str("0o"), njs_str("SyntaxError: Unexpected token \"0o\" in 1") }, @@ -390,6 +393,9 @@ static njs_unit_test_t njs_test[] = { njs_str("-0B1111_1111"), njs_str("-255") }, + { njs_str("0b111111111111111111111111111111111111111111111111111111111111111111111111111110000000000000"), + njs_str("1.2379400392853803e+27") }, + { njs_str("0b"), njs_str("SyntaxError: Unexpected token \"0b\" in 1") }, @@ -419,6 +425,12 @@ static njs_unit_test_t njs_test[] = { njs_str("0X00_00_BE_EF"), njs_str("48879") }, + { njs_str("0x21bc2b266d3a3600000000000000000000000000000000000000000000000000000"), + njs_str("6.25e+79") }, + + { njs_str("0x21bc2b266d3a36000000000000000000000000000000000000000000000000000000"), + njs_str("1e+81") }, + { njs_str("0x"), njs_str("SyntaxError: Unexpected token \"0x\" in 1") }, @@ -16735,6 +16747,15 @@ static njs_unit_test_t njs_test[] = { njs_str("parseInt('1010', 2)"), njs_str("10") }, + { njs_str("parseInt('111111111111111111111111111111111111111111111111111111111111111111111111111110000000000000', 2)"), + njs_str("1.2379400392853803e+27") }, + + { njs_str("parseInt('7777777777777777777777777777777777700000000000000000000000000000000', 8)"), + njs_str("3.2138760885179806e+60") }, + + { njs_str("parseInt('0x21bc2b266d3a3600000000000000000000000000000000000000000000000000000')"), + njs_str("6.25e+79") }, + { njs_str("parseInt('aBc', 16)"), njs_str("2748") }, From xeioex at nginx.com Thu Jun 23 06:40:56 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 23 Jun 2022 06:40:56 +0000 Subject: [njs] Fixed unicode argument trimming in parseInt(). Message-ID: details: https://hg.nginx.org/njs/rev/20ee5213e3c4 branches: changeset: 1898:20ee5213e3c4 user: Dmitry Volyntsev date: Wed Jun 22 23:37:27 2022 -0700 description: Fixed unicode argument trimming in parseInt(). diffstat: src/njs_number.c | 67 ++++++++------------- src/njs_string.c | 144 ++++++++++++++++++++++------------------------ src/njs_string.h | 8 ++ src/test/njs_unit_test.c | 6 + 4 files changed, 110 insertions(+), 115 deletions(-) diffs (355 lines): diff -r 33b1637aad33 -r 20ee5213e3c4 src/njs_number.c --- a/src/njs_number.c Wed Jun 22 23:37:16 2022 -0700 +++ b/src/njs_number.c Wed Jun 22 23:37:27 2022 -0700 @@ -1064,42 +1064,31 @@ njs_int_t njs_number_parse_int(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - double num; - int32_t radix; - njs_int_t ret; - njs_str_t string; - njs_bool_t minus, test_prefix; - njs_value_t *value; - const u_char *p, *end; + double num; + int32_t radix; + njs_int_t ret; + njs_bool_t minus, test_prefix; + njs_value_t *value; + const u_char *p, *end; + njs_string_prop_t string; num = NAN; - if (nargs < 2) { + value = njs_arg(args, nargs, 1); + + ret = njs_value_to_string(vm, value, value); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + (void) njs_string_trim(value, &string, NJS_TRIM_START); + + if (string.size == 0) { goto done; } - value = njs_argument(args, 1); - - if (!njs_is_string(value)) { - ret = njs_value_to_string(vm, value, value); - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } - } - - njs_string_get(value, &string); - - end = string.start + string.length; - - for (p = string.start; p < end; p++) { - if (*p != ' ') { - goto found; - } - } - - goto done; - -found: + p = string.start; + end = p + string.size; minus = 0; @@ -1156,21 +1145,17 @@ njs_int_t njs_number_parse_float(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - double num; - njs_int_t ret; - - num = NAN; + njs_int_t ret; + njs_value_t *value; - if (nargs > 1) { - ret = njs_value_to_string(vm, &args[1], &args[1]); - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } + value = njs_arg(args, nargs, 1); - num = njs_string_to_number(&args[1], 1); + ret = njs_value_to_string(vm, value, value); + if (njs_slow_path(ret != NJS_OK)) { + return ret; } - njs_set_number(&vm->retval, num); + njs_set_number(&vm->retval, njs_string_to_number(value, 1)); return NJS_OK; } diff -r 33b1637aad33 -r 20ee5213e3c4 src/njs_string.c --- a/src/njs_string.c Wed Jun 22 23:37:16 2022 -0700 +++ b/src/njs_string.c Wed Jun 22 23:37:27 2022 -0700 @@ -8,10 +8,6 @@ #include -#define NJS_TRIM_START 1 -#define NJS_TRIM_END 2 - - static u_char njs_basis64[] = { 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, @@ -2761,36 +2757,27 @@ njs_string_prototype_to_upper_case(njs_v } -static njs_int_t -njs_string_prototype_trim(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, - njs_index_t mode) +uint32_t +njs_string_trim(const njs_value_t *value, njs_string_prop_t *string, + unsigned mode) { - uint32_t u, trim, length; - njs_int_t ret; - njs_value_t *value; + uint32_t cp, trim; const u_char *p, *prev, *start, *end; - njs_string_prop_t string; njs_unicode_decode_t ctx; - value = njs_argument(args, 0); - ret = njs_string_object_validate(vm, value); - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } - trim = 0; - njs_string_prop(&string, value); - - start = string.start; - end = string.start + string.size; - - if (njs_is_byte_or_ascii_string(&string)) { + njs_string_prop(string, value); + + start = string->start; + end = string->start + string->size; + + if (njs_is_byte_or_ascii_string(string)) { if (mode & NJS_TRIM_START) { for ( ;; ) { if (start == end) { - goto empty; + break; } if (njs_is_whitespace(*start)) { @@ -2806,7 +2793,7 @@ njs_string_prototype_trim(njs_vm_t *vm, if (mode & NJS_TRIM_END) { for ( ;; ) { if (start == end) { - goto empty; + break; } end--; @@ -2829,13 +2816,13 @@ njs_string_prototype_trim(njs_vm_t *vm, for ( ;; ) { if (start == end) { - goto empty; + break; } p = start; - u = njs_utf8_decode(&ctx, &start, end); - - if (njs_utf8_is_whitespace(u)) { + cp = njs_utf8_decode(&ctx, &start, end); + + if (njs_utf8_is_whitespace(cp)) { trim++; continue; } @@ -2852,14 +2839,14 @@ njs_string_prototype_trim(njs_vm_t *vm, for ( ;; ) { if (start == prev) { - goto empty; + break; } prev = njs_utf8_prev(prev); p = prev; - u = njs_utf8_decode(&ctx, &p, end); - - if (njs_utf8_is_whitespace(u)) { + cp = njs_utf8_decode(&ctx, &p, end); + + if (njs_utf8_is_whitespace(cp)) { trim++; continue; } @@ -2870,22 +2857,52 @@ njs_string_prototype_trim(njs_vm_t *vm, } } + if (start == end) { + string->length = 0; + string->size = 0; + return trim; + } + + string->start = (u_char *) start; + string->size = end - start; + + if (string->length != 0) { + string->length -= trim; + } + + return trim; +} + + +static njs_int_t +njs_string_prototype_trim(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t mode) +{ + uint32_t trim; + njs_int_t ret; + njs_value_t *value; + njs_string_prop_t string; + + value = njs_argument(args, 0); + ret = njs_string_object_validate(vm, value); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + trim = njs_string_trim(value, &string, mode); + if (trim == 0) { - /* GC: retain. */ - vm->retval = *value; - + njs_value_assign(&vm->retval, value); return NJS_OK; } - length = (string.length != 0) ? string.length - trim : 0; - - return njs_string_new(vm, &vm->retval, start, end - start, length); - -empty: - - vm->retval = njs_string_empty; - - return NJS_OK; + if (string.size == 0) { + njs_value_assign(&vm->retval, &njs_string_empty); + return NJS_OK; + } + + return njs_string_new(vm, &vm->retval, string.start, string.size, + string.length); } @@ -3811,38 +3828,17 @@ njs_string_prototype_iterator_obj(njs_vm double njs_string_to_number(const njs_value_t *value, njs_bool_t parse_float) { - double num; - size_t size; - uint32_t u; - njs_bool_t minus; - const u_char *p, *start, *end; - njs_unicode_decode_t ctx; + double num; + njs_bool_t minus; + const u_char *p, *start, *end; + njs_string_prop_t string; const size_t infinity = njs_length("Infinity"); - size = value->short_string.size; - - if (size != NJS_STRING_LONG) { - p = value->short_string.start; - - } else { - size = value->long_string.size; - p = value->long_string.data->start; - } - - end = p + size; - - njs_utf8_decode_init(&ctx); - - while (p < end) { - start = p; - u = njs_utf8_decode(&ctx, &p, end); - - if (!njs_utf8_is_whitespace(u)) { - p = start; - break; - } - } + (void) njs_string_trim(value, &string, NJS_TRIM_START); + + p = string.start; + end = p + string.size; if (p == end) { return parse_float ? NAN : 0.0; diff -r 33b1637aad33 -r 20ee5213e3c4 src/njs_string.h --- a/src/njs_string.h Wed Jun 22 23:37:16 2022 -0700 +++ b/src/njs_string.h Wed Jun 22 23:37:27 2022 -0700 @@ -98,6 +98,12 @@ typedef enum { } njs_utf8_t; +typedef enum { + NJS_TRIM_START = 1, + NJS_TRIM_END = 2, +} njs_trim_t; + + njs_inline njs_bool_t njs_is_byte_string(njs_string_prop_t *string) { @@ -224,6 +230,8 @@ njs_int_t njs_string_decode_base64(njs_v njs_int_t njs_string_decode_base64url(njs_vm_t *vm, njs_value_t *value, const njs_str_t *src); void njs_string_truncate(njs_value_t *value, uint32_t size, uint32_t length); +uint32_t njs_string_trim(const njs_value_t *value, njs_string_prop_t *string, + unsigned mode); void njs_string_copy(njs_value_t *dst, njs_value_t *src); njs_int_t njs_string_validate(njs_vm_t *vm, njs_string_prop_t *string, njs_value_t *value); diff -r 33b1637aad33 -r 20ee5213e3c4 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Wed Jun 22 23:37:16 2022 -0700 +++ b/src/test/njs_unit_test.c Wed Jun 22 23:37:27 2022 -0700 @@ -16744,6 +16744,12 @@ static njs_unit_test_t njs_test[] = { njs_str("parseInt(' 123')"), njs_str("123") }, + { njs_str("parseInt('\\u0009123')"), + njs_str("123") }, + + { njs_str("parseInt('\\u200A123')"), + njs_str("123") }, + { njs_str("parseInt('1010', 2)"), njs_str("10") }, From shanlei at asiainfo.com Thu Jun 23 08:26:39 2022 From: shanlei at asiainfo.com (lei shan) Date: Thu, 23 Jun 2022 16:26:39 +0800 Subject: what's the proper way to get module specific configuration in init_process In-Reply-To: References: <4e653e995eda3fbe41ac592d465b6709@asiainfo.com> Message-ID: Thanks, That means we need move some configuration commands form server to main section. We will review the logic Lei On Thu, Jun 23, 2022, 01:51 Maxim Dounin wrote: > Hello! > > On Mon, Jun 20, 2022 at 04:05:53PM +0800, shanlei at asiainfo.com wrote: > > > Hello, guys, > > > > Recently I have been working on an nginx c module, which have to > > leverage on init_process routine.The logic is based on instruction > > command which may be configured inside stream.server, I have to do some > > work in the init_process routine, for example, to schedule a timer. > > I did google a while ,and decide to using code snippet like below: > > > > ============================ > > > > ngx_stream_conf_ctx_t conf_ctx = (ngx_stream_conf_ctx_t > > *)ngx_get_conf(cycle->conf_ctx, ngx_stream_module); > > > > "module_spec_conf = conf_ctx->srv_conf["my_module".ctx_index]; > > > > ============================= > > > > > > However, the module_spec_conf I got is the top_level ,that is created > > when nginx parse the stream section, not the one created when nginx > > parse the stream.server section, the later has the proper values. > > Currently to got the later structure, I have to copy the configure value > > back from child to parent in configuration merger function, which is > > inconvenient and buggy when the stream includes multi servers. > > > > So I wonder what's the proper way to get module specific configuration > > in the routine: init_process > > For the purpose of the init_process callback, the most simple > approach would be to use your module's main configuration. For > example, the embedded Perl module uses the init process callback > to set correct PID in Perl interpreter: Perl interpreter is stored > in the module's main configuration, and the init process callback > uses the ngx_http_cycle_get_module_main_conf() macro to access it. > See src/http/modules/perl/ngx_http_perl_module.c for details. > > (Just in case, corresponding macro in the stream module is > ngx_stream_cycle_get_module_main_conf().) > > -- > Maxim Dounin > http://mdounin.ru/ > _______________________________________________ > nginx-devel mailing list -- nginx-devel at nginx.org > To unsubscribe send an email to nginx-devel-leave at nginx.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pluknet at nginx.com Thu Jun 23 10:02:10 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Thu, 23 Jun 2022 14:02:10 +0400 Subject: [PATCH] Corrected syntax of ssl_conf_command and friends In-Reply-To: References: <3e1af31b0ecd9e0ae5a8.1655753623@enoparse.local> Message-ID: <3DDB233D-A7EF-4263-BC99-4E39CA6CEC1B@nginx.com> > On 21 Jun 2022, at 01:18, Maxim Dounin wrote: > > Hello! > > On Mon, Jun 20, 2022 at 11:33:43PM +0400, Sergey Kandaurov wrote: > >> # HG changeset patch >> # User Sergey Kandaurov >> # Date 1655753604 -14400 >> # Mon Jun 20 23:33:24 2022 +0400 >> # Node ID 3e1af31b0ecd9e0ae5a8a93a15ff15cc0b1f98f6 >> # Parent 53734009b0c36276e66792e23180ee9665c95419 >> Corrected syntax of ssl_conf_command and friends. >> >> Each directive takes exactly two arguments: command name and value. >> The parameter names follow SSL_CONF_cmd(3) documentation of OpenSSL 3.0. >> >> diff --git a/xml/en/docs/http/ngx_http_grpc_module.xml b/xml/en/docs/http/ngx_http_grpc_module.xml >> --- a/xml/en/docs/http/ngx_http_grpc_module.xml >> +++ b/xml/en/docs/http/ngx_http_grpc_module.xml >> @@ -10,7 +10,7 @@ >> > link="/en/docs/http/ngx_http_grpc_module.html" >> lang="en" >> - rev="7"> >> + rev="8"> >> >>
>> >> @@ -535,7 +535,7 @@ The full list can be viewed using the >> >> >> >> -command ... >> +option value > > I don't think such wording improves readability, since it > basically contradicts the "command" in the name of the directive. Yeah, that's true. > > The OpenSSL 3.0 documentation of SSL_CONF_cmd() does not seem to > be self-consistent, and uses terms "command", "operation" and > "option" in various places. Further, the term "option" is already > used by SSL_CTX_set_options() in a completely different meaning, > and it is also configurable with SSL_CONF_cmd("Options"). > > Overall, I would rather suggest to stick with the > > ssl_conf_command name value; > > syntax, which seems to be much more consistent and > self-explanatory. > > [...] Committed with adjustments, thanks. -- Sergey Kandaurov From gwj1235 at yeah.net Thu Jun 23 11:47:09 2022 From: gwj1235 at yeah.net (Wenjun) Date: Thu, 23 Jun 2022 19:47:09 +0800 (CST) Subject: nginx abort connection during reload when proxy_cache enabled Message-ID: We use nginx as the cluster ingress provider, and hit the case when: 1. enabled proxy_cache, and the specific cache is expired 2. nginx is reloading config then for incoming request, the backend servers can still receive the request and send back response to nginx, while client will receive "curl: (52) Empty reply from server". It seems nginx reload may cause the connection break? attached debug log below: -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: vmware-view_fp17OqOhvZ.png Type: image/png Size: 297952 bytes Desc: not available URL: From arut at nginx.com Thu Jun 23 15:58:38 2022 From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=) Date: Thu, 23 Jun 2022 19:58:38 +0400 Subject: [PATCH 0 of 8] QUIC connection reuse In-Reply-To: References: Message-ID: A new series with persistent c->idle, shutdown() callback and application-level graceful shutdown. From arut at nginx.com Thu Jun 23 15:58:39 2022 From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=) Date: Thu, 23 Jun 2022 19:58:39 +0400 Subject: [PATCH 1 of 8] QUIC: treat qc->error == -1 as a missing error In-Reply-To: References: Message-ID: <951d7116f37dc39d9eba.1655999919@arut-laptop> # HG changeset patch # User Roman Arutyunyan # Date 1655889315 -14400 # Wed Jun 22 13:15:15 2022 +0400 # Branch quic # Node ID 951d7116f37dc39d9eba20ceae49434592ce4677 # Parent 5b1011b5702b5c5db2ba3d392a4da25596183cc2 QUIC: treat qc->error == -1 as a missing error. Previously, zero was used for this purpose. However, NGX_QUIC_ERR_NO_ERROR is zero too. As a result, NGX_QUIC_ERR_NO_ERROR was changed to NGX_QUIC_ERR_INTERNAL_ERROR when closing a QUIC connection. diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c --- a/src/event/quic/ngx_event_quic.c +++ b/src/event/quic/ngx_event_quic.c @@ -317,6 +317,8 @@ ngx_quic_new_connection(ngx_connection_t qc->congestion.ssthresh = (size_t) -1; qc->congestion.recovery_start = ngx_current_msec; + qc->error = (ngx_uint_t) -1; + if (pkt->validated && pkt->retried) { qc->tp.retry_scid.len = pkt->dcid.len; qc->tp.retry_scid.data = ngx_pstrdup(c->pool, &pkt->dcid); @@ -523,7 +525,7 @@ ngx_quic_close_connection(ngx_connection qc->error = NGX_QUIC_ERR_NO_ERROR; } else { - if (qc->error == 0 && !qc->error_app) { + if (qc->error == (ngx_uint_t) -1 && !qc->error_app) { qc->error = NGX_QUIC_ERR_INTERNAL_ERROR; } From arut at nginx.com Thu Jun 23 15:58:40 2022 From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=) Date: Thu, 23 Jun 2022 19:58:40 +0400 Subject: [PATCH 2 of 8] QUIC: ngx_quic_terminate_connection() function In-Reply-To: References: Message-ID: <5edae53d1fbfcd2eedab.1655999920@arut-laptop> # HG changeset patch # User Roman Arutyunyan # Date 1655903226 -14400 # Wed Jun 22 17:07:06 2022 +0400 # Branch quic # Node ID 5edae53d1fbfcd2eedab36252d9a2522dce5c3c8 # Parent 951d7116f37dc39d9eba20ceae49434592ce4677 QUIC: ngx_quic_terminate_connection() function. Previously, ngx_quic_finalize_connection() terminated the connection. Now it shuts down the connection gracefully with a close timeout. The new function terminates the connection now. diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c --- a/src/event/quic/ngx_event_quic.c +++ b/src/event/quic/ngx_event_quic.c @@ -416,6 +416,7 @@ ngx_quic_input_handler(ngx_event_t *rev) } if (c->close) { + qc->error = NGX_QUIC_ERR_NO_ERROR; qc->error_reason = "graceful shutdown"; ngx_quic_close_connection(c, NGX_OK); return; @@ -508,31 +509,26 @@ ngx_quic_close_connection(ngx_connection qc->error_level = c->ssl ? SSL_quic_read_level(c->ssl->connection) : ssl_encryption_initial; + if (qc->error == (ngx_uint_t) -1) { + qc->error = NGX_QUIC_ERR_INTERNAL_ERROR; + qc->error_app = 0; + } + + ngx_log_debug5(NGX_LOG_DEBUG_EVENT, c->log, 0, + "quic close immediate term:%d drain:%d " + "%serror:%ui \"%s\"", + rc == NGX_ERROR ? 1 : 0, qc->draining, + qc->error_app ? "app " : "", qc->error, + qc->error_reason ? qc->error_reason : ""); + if (rc == NGX_OK) { - ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, - "quic close immediate drain:%d", - qc->draining); - qc->close.log = c->log; qc->close.data = c; qc->close.handler = ngx_quic_close_timer_handler; qc->close.cancelable = 1; ctx = ngx_quic_get_send_ctx(qc, qc->error_level); - ngx_add_timer(&qc->close, 3 * ngx_quic_pto(c, ctx)); - - qc->error = NGX_QUIC_ERR_NO_ERROR; - - } else { - if (qc->error == (ngx_uint_t) -1 && !qc->error_app) { - qc->error = NGX_QUIC_ERR_INTERNAL_ERROR; - } - - ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0, - "quic close immediate due to %serror: %ui %s", - qc->error_app ? "app " : "", qc->error, - qc->error_reason ? qc->error_reason : ""); } (void) ngx_quic_send_cc(c); @@ -608,6 +604,22 @@ quic_done: void +ngx_quic_terminate_connection(ngx_connection_t *c, ngx_uint_t err, + const char *reason) +{ + ngx_quic_connection_t *qc; + + qc = ngx_quic_get_connection(c); + qc->error = err; + qc->error_reason = reason; + qc->error_app = 1; + qc->error_ftype = 0; + + ngx_quic_close_connection(c, NGX_ERROR); +} + + +void ngx_quic_finalize_connection(ngx_connection_t *c, ngx_uint_t err, const char *reason) { @@ -619,7 +631,7 @@ ngx_quic_finalize_connection(ngx_connect qc->error_app = 1; qc->error_ftype = 0; - ngx_quic_close_connection(c, NGX_ERROR); + ngx_quic_close_connection(c, NGX_OK); } diff --git a/src/event/quic/ngx_event_quic.h b/src/event/quic/ngx_event_quic.h --- a/src/event/quic/ngx_event_quic.h +++ b/src/event/quic/ngx_event_quic.h @@ -107,6 +107,8 @@ void ngx_quic_rbtree_insert_value(ngx_rb ngx_rbtree_node_t *node, ngx_rbtree_node_t *sentinel); void ngx_quic_run(ngx_connection_t *c, ngx_quic_conf_t *conf); ngx_connection_t *ngx_quic_open_stream(ngx_connection_t *c, ngx_uint_t bidi); +void ngx_quic_terminate_connection(ngx_connection_t *c, ngx_uint_t err, + const char *reason); void ngx_quic_finalize_connection(ngx_connection_t *c, ngx_uint_t err, const char *reason); void ngx_quic_shutdown_connection(ngx_connection_t *c, ngx_uint_t err, From arut at nginx.com Thu Jun 23 15:58:41 2022 From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=) Date: Thu, 23 Jun 2022 19:58:41 +0400 Subject: [PATCH 3 of 8] QUIC: reusable mode for main connection In-Reply-To: References: Message-ID: # HG changeset patch # User Roman Arutyunyan # Date 1655995431 -14400 # Thu Jun 23 18:43:51 2022 +0400 # Branch quic # Node ID ea555f7caec372e93e1b9a11ed822002e97ed58b # Parent 5edae53d1fbfcd2eedab36252d9a2522dce5c3c8 QUIC: reusable mode for main connection. The mode is controlled by application layer. For HTTP/3, it is enabled when there are no active request streams. diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c --- a/src/event/quic/ngx_event_quic.c +++ b/src/event/quic/ngx_event_quic.c @@ -418,7 +418,7 @@ ngx_quic_input_handler(ngx_event_t *rev) if (c->close) { qc->error = NGX_QUIC_ERR_NO_ERROR; qc->error_reason = "graceful shutdown"; - ngx_quic_close_connection(c, NGX_OK); + ngx_quic_close_connection(c, NGX_ERROR); return; } diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c --- a/src/event/quic/ngx_event_quic_streams.c +++ b/src/event/quic/ngx_event_quic_streams.c @@ -161,6 +161,7 @@ ngx_quic_close_streams(ngx_connection_t ngx_pool_t *pool; ngx_queue_t *q; ngx_rbtree_t *tree; + ngx_connection_t *sc; ngx_rbtree_node_t *node; ngx_quic_stream_t *qs; @@ -185,24 +186,41 @@ ngx_quic_close_streams(ngx_connection_t return NGX_OK; } -#if (NGX_DEBUG) - ns = 0; -#endif - node = ngx_rbtree_min(tree->root, tree->sentinel); while (node) { qs = (ngx_quic_stream_t *) node; node = ngx_rbtree_next(tree, node); + sc = qs->connection; + + if (sc == NULL) { + ngx_quic_close_stream(qs); + continue; + } + + if (qs->cancelable || sc->idle || sc->reusable) { + sc->close = 1; + sc->read->handler(sc->read); + } + } + + if (tree->root == tree->sentinel) { + return NGX_OK; + } + +#if (NGX_DEBUG) + ns = 0; +#endif + + for (node = ngx_rbtree_min(tree->root, tree->sentinel); + node; + node = ngx_rbtree_next(tree, node)) + { + qs = (ngx_quic_stream_t *) node; qs->recv_state = NGX_QUIC_STREAM_RECV_RESET_RECVD; qs->send_state = NGX_QUIC_STREAM_SEND_RESET_SENT; - if (qs->connection == NULL) { - ngx_quic_close_stream(qs); - continue; - } - ngx_quic_set_event(qs->connection->read); ngx_quic_set_event(qs->connection->write); @@ -587,6 +605,7 @@ ngx_quic_create_stream(ngx_connection_t { ngx_log_t *log; ngx_pool_t *pool; + ngx_uint_t reusable; ngx_queue_t *q; ngx_connection_t *sc; ngx_quic_stream_t *qs; @@ -639,7 +658,13 @@ ngx_quic_create_stream(ngx_connection_t *log = *c->log; pool->log = log; + reusable = c->reusable; + ngx_reusable_connection(c, 0); + sc = ngx_get_connection(c->fd, log); + + ngx_reusable_connection(c, reusable); + if (sc == NULL) { ngx_destroy_pool(pool); ngx_queue_insert_tail(&qc->streams.free, &qs->queue); diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c --- a/src/http/v3/ngx_http_v3_request.c +++ b/src/http/v3/ngx_http_v3_request.c @@ -210,6 +210,11 @@ ngx_http_v3_init_request_stream(ngx_conn h3c = ngx_http_v3_get_session(c); + if (!h3c->goaway && (ngx_terminate || ngx_exiting)) { + h3c->goaway = 1; + (void) ngx_http_v3_send_goaway(c, c->quic->id); + } + if (h3c->goaway) { c->close = 1; ngx_http_close_connection(c); @@ -258,7 +263,7 @@ ngx_http_v3_wait_request_handler(ngx_eve size_t size; ssize_t n; ngx_buf_t *b; - ngx_connection_t *c; + ngx_connection_t *c, *pc; ngx_pool_cleanup_t *cln; ngx_http_request_t *r; ngx_http_connection_t *hc; @@ -385,6 +390,9 @@ ngx_http_v3_wait_request_handler(ngx_eve h3c = ngx_http_v3_get_session(c); h3c->nrequests++; + pc = c->quic->parent; + ngx_reusable_connection(pc, 0); + if (h3c->keepalive.timer_set) { ngx_del_timer(&h3c->keepalive); } @@ -430,7 +438,7 @@ ngx_http_v3_cleanup_request(void *data) { ngx_http_request_t *r = data; - ngx_connection_t *c; + ngx_connection_t *c, *pc; ngx_http_v3_session_t *h3c; ngx_http_core_loc_conf_t *clcf; @@ -443,6 +451,15 @@ ngx_http_v3_cleanup_request(void *data) h3c = ngx_http_v3_get_session(c); if (--h3c->nrequests == 0) { + pc = c->quic->parent; + + if (ngx_terminate || ngx_exiting) { + ngx_quic_finalize_connection(pc, NGX_HTTP_V3_ERR_NO_ERROR, NULL); + return; + } + + ngx_reusable_connection(pc, 1); + clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module); ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout); } diff --git a/src/http/v3/ngx_http_v3_uni.c b/src/http/v3/ngx_http_v3_uni.c --- a/src/http/v3/ngx_http_v3_uni.c +++ b/src/http/v3/ngx_http_v3_uni.c @@ -182,6 +182,11 @@ ngx_http_v3_uni_read_handler(ngx_event_t ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 read handler"); + if (c->close) { + ngx_http_v3_close_uni_stream(c); + return; + } + ngx_memzero(&b, sizeof(ngx_buf_t)); while (rev->ready) { @@ -262,6 +267,11 @@ ngx_http_v3_uni_dummy_read_handler(ngx_e ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 dummy read handler"); + if (c->close) { + ngx_http_v3_close_uni_stream(c); + return; + } + if (rev->ready) { if (c->recv(c, &ch, 1) != 0) { ngx_http_v3_finalize_connection(c, NGX_HTTP_V3_ERR_NO_ERROR, NULL); From arut at nginx.com Thu Jun 23 15:58:42 2022 From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=) Date: Thu, 23 Jun 2022 19:58:42 +0400 Subject: [PATCH 4 of 8] QUIC: removed ngx_quic_shutdown_connection() In-Reply-To: References: Message-ID: <1912c09e0e4d746ec0a4.1655999922@arut-laptop> # HG changeset patch # User Roman Arutyunyan # Date 1655904279 -14400 # Wed Jun 22 17:24:39 2022 +0400 # Branch quic # Node ID 1912c09e0e4d746ec0a4c2140b6d2046b283b647 # Parent ea555f7caec372e93e1b9a11ed822002e97ed58b QUIC: removed ngx_quic_shutdown_connection(). HTTP/3 shutdown is now controlled by HTTP/3 layer. QUIC connection is finalized after reaching requests limit when all request streams have finished. diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c --- a/src/event/quic/ngx_event_quic.c +++ b/src/event/quic/ngx_event_quic.c @@ -82,7 +82,6 @@ ngx_quic_connstate_dbg(ngx_connection_t } } - p = ngx_slprintf(p, last, "%s", qc->shutdown ? " shutdown" : ""); p = ngx_slprintf(p, last, "%s", qc->closing ? " closing" : ""); p = ngx_slprintf(p, last, "%s", qc->draining ? " draining" : ""); p = ngx_slprintf(p, last, "%s", qc->key_phase ? " kp" : ""); @@ -425,9 +424,6 @@ ngx_quic_input_handler(ngx_event_t *rev) if (!rev->ready) { if (qc->closing) { ngx_quic_close_connection(c, NGX_OK); - - } else if (qc->shutdown) { - ngx_quic_shutdown_quic(c); } return; @@ -635,21 +631,6 @@ ngx_quic_finalize_connection(ngx_connect } -void -ngx_quic_shutdown_connection(ngx_connection_t *c, ngx_uint_t err, - const char *reason) -{ - ngx_quic_connection_t *qc; - - qc = ngx_quic_get_connection(c); - qc->shutdown = 1; - qc->shutdown_code = err; - qc->shutdown_reason = reason; - - ngx_quic_shutdown_quic(c); -} - - static void ngx_quic_close_timer_handler(ngx_event_t *ev) { @@ -1438,36 +1419,3 @@ ngx_quic_push_handler(ngx_event_t *ev) ngx_quic_connstate_dbg(c); } - - -void -ngx_quic_shutdown_quic(ngx_connection_t *c) -{ - ngx_rbtree_t *tree; - ngx_rbtree_node_t *node; - ngx_quic_stream_t *qs; - ngx_quic_connection_t *qc; - - qc = ngx_quic_get_connection(c); - - if (qc->closing) { - return; - } - - tree = &qc->streams.tree; - - if (tree->root != tree->sentinel) { - for (node = ngx_rbtree_min(tree->root, tree->sentinel); - node; - node = ngx_rbtree_next(tree, node)) - { - qs = (ngx_quic_stream_t *) node; - - if (!qs->cancelable) { - return; - } - } - } - - ngx_quic_finalize_connection(c, qc->shutdown_code, qc->shutdown_reason); -} diff --git a/src/event/quic/ngx_event_quic.h b/src/event/quic/ngx_event_quic.h --- a/src/event/quic/ngx_event_quic.h +++ b/src/event/quic/ngx_event_quic.h @@ -111,8 +111,6 @@ void ngx_quic_terminate_connection(ngx_c const char *reason); void ngx_quic_finalize_connection(ngx_connection_t *c, ngx_uint_t err, const char *reason); -void ngx_quic_shutdown_connection(ngx_connection_t *c, ngx_uint_t err, - const char *reason); ngx_int_t ngx_quic_reset_stream(ngx_connection_t *c, ngx_uint_t err); ngx_int_t ngx_quic_shutdown_stream(ngx_connection_t *c, int how); ngx_int_t ngx_quic_handle_read_event(ngx_event_t *rev, ngx_uint_t flags); diff --git a/src/event/quic/ngx_event_quic_connection.h b/src/event/quic/ngx_event_quic_connection.h --- a/src/event/quic/ngx_event_quic_connection.h +++ b/src/event/quic/ngx_event_quic_connection.h @@ -246,13 +246,9 @@ struct ngx_quic_connection_s { ngx_uint_t error_ftype; const char *error_reason; - ngx_uint_t shutdown_code; - const char *shutdown_reason; - unsigned error_app:1; unsigned send_timer_set:1; unsigned closing:1; - unsigned shutdown:1; unsigned draining:1; unsigned key_phase:1; unsigned validated:1; @@ -265,7 +261,6 @@ ngx_int_t ngx_quic_apply_transport_param void ngx_quic_discard_ctx(ngx_connection_t *c, enum ssl_encryption_level_t level); void ngx_quic_close_connection(ngx_connection_t *c, ngx_int_t rc); -void ngx_quic_shutdown_quic(ngx_connection_t *c); #if (NGX_DEBUG) void ngx_quic_connstate_dbg(ngx_connection_t *c); diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c --- a/src/event/quic/ngx_event_quic_streams.c +++ b/src/event/quic/ngx_event_quic_streams.c @@ -385,7 +385,7 @@ ngx_quic_get_stream(ngx_connection_t *c, return qs; } - if (qc->shutdown || qc->closing) { + if (qc->closing) { return NGX_QUIC_STREAM_GONE; } @@ -1081,10 +1081,6 @@ ngx_quic_close_stream(ngx_quic_stream_t ngx_quic_queue_frame(qc, frame); } - if (qc->shutdown) { - ngx_post_event(pc->read, &ngx_posted_events); - } - return NGX_OK; } diff --git a/src/http/v3/ngx_http_v3.h b/src/http/v3/ngx_http_v3.h --- a/src/http/v3/ngx_http_v3.h +++ b/src/http/v3/ngx_http_v3.h @@ -95,10 +95,6 @@ ngx_quic_finalize_connection((c)->quic ? (c)->quic->parent : (c), \ code, reason) -#define ngx_http_v3_shutdown_connection(c, code, reason) \ - ngx_quic_shutdown_connection((c)->quic ? (c)->quic->parent : (c), \ - code, reason) - typedef struct { size_t max_table_capacity; diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c --- a/src/http/v3/ngx_http_v3_request.c +++ b/src/http/v3/ngx_http_v3_request.c @@ -14,6 +14,7 @@ static void ngx_http_v3_init_hq_stream(ngx_connection_t *c); #endif static void ngx_http_v3_init_request_stream(ngx_connection_t *c); +static void ngx_http_v3_cleanup_connection(void *data); static void ngx_http_v3_wait_request_handler(ngx_event_t *rev); static void ngx_http_v3_cleanup_request(void *data); static void ngx_http_v3_process_request(ngx_event_t *rev); @@ -184,6 +185,8 @@ ngx_http_v3_init_request_stream(ngx_conn { uint64_t n; ngx_event_t *rev; + ngx_connection_t *pc; + ngx_pool_cleanup_t *cln; ngx_http_connection_t *hc; ngx_http_v3_session_t *h3c; ngx_http_core_loc_conf_t *clcf; @@ -221,9 +224,10 @@ ngx_http_v3_init_request_stream(ngx_conn return; } + pc = c->quic->parent; + if (n + 1 == clcf->keepalive_requests - || ngx_current_msec - c->quic->parent->start_time - > clcf->keepalive_time) + || ngx_current_msec - pc->start_time > clcf->keepalive_time) { h3c->goaway = 1; @@ -231,9 +235,25 @@ ngx_http_v3_init_request_stream(ngx_conn ngx_http_close_connection(c); return; } + } - ngx_http_v3_shutdown_connection(c, NGX_HTTP_V3_ERR_NO_ERROR, - "reached maximum number of requests"); + cln = ngx_pool_cleanup_add(c->pool, 0); + if (cln == NULL) { + ngx_quic_finalize_connection(pc, NGX_HTTP_V3_ERR_INTERNAL_ERROR, + "internal error"); + ngx_http_close_connection(c); + return; + } + + cln->handler = ngx_http_v3_cleanup_connection; + cln->data = c; + + if (h3c->nrequests++ == 0) { + ngx_reusable_connection(pc, 0); + } + + if (h3c->keepalive.timer_set) { + ngx_del_timer(&h3c->keepalive); } rev = c->read; @@ -258,16 +278,43 @@ ngx_http_v3_init_request_stream(ngx_conn static void +ngx_http_v3_cleanup_connection(void *data) +{ + ngx_connection_t *c = data; + + ngx_connection_t *pc; + ngx_http_v3_session_t *h3c; + ngx_http_core_loc_conf_t *clcf; + + h3c = ngx_http_v3_get_session(c); + + if (--h3c->nrequests == 0) { + pc = c->quic->parent; + + if (h3c->goaway) { + ngx_quic_finalize_connection(pc, NGX_HTTP_V3_ERR_NO_ERROR, + "connection shutdown"); + return; + } + + ngx_reusable_connection(pc, 1); + + clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module); + ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout); + } +} + + +static void ngx_http_v3_wait_request_handler(ngx_event_t *rev) { size_t size; ssize_t n; ngx_buf_t *b; - ngx_connection_t *c, *pc; + ngx_connection_t *c; ngx_pool_cleanup_t *cln; ngx_http_request_t *r; ngx_http_connection_t *hc; - ngx_http_v3_session_t *h3c; ngx_http_core_srv_conf_t *cscf; c = rev->data; @@ -387,16 +434,6 @@ ngx_http_v3_wait_request_handler(ngx_eve cln->handler = ngx_http_v3_cleanup_request; cln->data = r; - h3c = ngx_http_v3_get_session(c); - h3c->nrequests++; - - pc = c->quic->parent; - ngx_reusable_connection(pc, 0); - - if (h3c->keepalive.timer_set) { - ngx_del_timer(&h3c->keepalive); - } - rev->handler = ngx_http_v3_process_request; ngx_http_v3_process_request(rev); } @@ -438,30 +475,8 @@ ngx_http_v3_cleanup_request(void *data) { ngx_http_request_t *r = data; - ngx_connection_t *c, *pc; - ngx_http_v3_session_t *h3c; - ngx_http_core_loc_conf_t *clcf; - - c = r->connection; - if (!r->response_sent) { - c->error = 1; - } - - h3c = ngx_http_v3_get_session(c); - - if (--h3c->nrequests == 0) { - pc = c->quic->parent; - - if (ngx_terminate || ngx_exiting) { - ngx_quic_finalize_connection(pc, NGX_HTTP_V3_ERR_NO_ERROR, NULL); - return; - } - - ngx_reusable_connection(pc, 1); - - clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module); - ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout); + r->connection->error = 1; } } From arut at nginx.com Thu Jun 23 15:58:43 2022 From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=) Date: Thu, 23 Jun 2022 19:58:43 +0400 Subject: [PATCH 5 of 8] HTTP/3: keepalive timer for hq mode In-Reply-To: References: Message-ID: <56b6cd5bc78bba1085a1.1655999923@arut-laptop> # HG changeset patch # User Roman Arutyunyan # Date 1655998736 -14400 # Thu Jun 23 19:38:56 2022 +0400 # Branch quic # Node ID 56b6cd5bc78bba1085a168c0f5ea9a03d34646c5 # Parent 1912c09e0e4d746ec0a4c2140b6d2046b283b647 HTTP/3: keepalive timer for hq mode. diff --git a/src/http/v3/ngx_http_v3.c b/src/http/v3/ngx_http_v3.c --- a/src/http/v3/ngx_http_v3.c +++ b/src/http/v3/ngx_http_v3.c @@ -61,7 +61,7 @@ ngx_http_v3_init_session(ngx_connection_ hc->v3_session = h3c; - return ngx_http_v3_send_settings(c); + return NGX_OK; failed: diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c --- a/src/http/v3/ngx_http_v3_request.c +++ b/src/http/v3/ngx_http_v3_request.c @@ -61,6 +61,7 @@ static const struct { void ngx_http_v3_init(ngx_connection_t *c) { + ngx_http_v3_session_t *h3c; ngx_http_connection_t *hc, *phc; ngx_http_v3_srv_conf_t *h3scf; ngx_http_core_loc_conf_t *clcf; @@ -87,6 +88,13 @@ ngx_http_v3_init(ngx_connection_t *c) ngx_set_connection_log(c, clcf->error_log); } + h3c = hc->v3_session; + + if (h3c == NULL && ngx_http_v3_init_session(c) != NGX_OK) { + ngx_http_close_connection(c); + return; + } + #if (NGX_HTTP_V3_HQ) if (h3scf->hq) { ngx_http_v3_init_hq_stream(c); @@ -94,7 +102,7 @@ ngx_http_v3_init(ngx_connection_t *c) } #endif - if (ngx_http_v3_init_session(c) != NGX_OK) { + if (h3c == NULL && ngx_http_v3_send_settings(c)) { ngx_http_close_connection(c); return; } @@ -115,6 +123,9 @@ ngx_http_v3_init_hq_stream(ngx_connectio { uint64_t n; ngx_event_t *rev; + ngx_connection_t *pc; + ngx_pool_cleanup_t *cln; + ngx_http_v3_session_t *h3c; ngx_http_connection_t *hc; ngx_http_core_loc_conf_t *clcf; ngx_http_core_srv_conf_t *cscf; @@ -159,6 +170,28 @@ ngx_http_v3_init_hq_stream(ngx_connectio return; } + pc = c->quic->parent; + h3c = ngx_http_v3_get_session(c); + + cln = ngx_pool_cleanup_add(c->pool, 0); + if (cln == NULL) { + ngx_quic_finalize_connection(pc, NGX_HTTP_V3_ERR_INTERNAL_ERROR, + "internal error"); + ngx_http_close_connection(c); + return; + } + + cln->handler = ngx_http_v3_cleanup_connection; + cln->data = c; + + if (h3c->nrequests++ == 0) { + ngx_reusable_connection(pc, 0); + } + + if (h3c->keepalive.timer_set) { + ngx_del_timer(&h3c->keepalive); + } + rev = c->read; if (rev->ready) { From arut at nginx.com Thu Jun 23 15:58:44 2022 From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=) Date: Thu, 23 Jun 2022 19:58:44 +0400 Subject: [PATCH 6 of 8] QUIC: idle mode for main connection In-Reply-To: References: Message-ID: <1fad6f74168bdceaf3cd.1655999924@arut-laptop> # HG changeset patch # User Roman Arutyunyan # Date 1655995473 -14400 # Thu Jun 23 18:44:33 2022 +0400 # Branch quic # Node ID 1fad6f74168bdceaf3cd59ae72f7c6128dfaa84d # Parent 56b6cd5bc78bba1085a168c0f5ea9a03d34646c5 QUIC: idle mode for main connection. Now main QUIC connection for HTTP/3 always has c->idle flag set. This allows the connection to receive worker shutdown notification. It is passed to application level via a new conf->shutdown() callback. The HTTP/3 shutdown callback sends GOAWAY to client and enables rejecting new client request streams. diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c --- a/src/event/quic/ngx_event_quic.c +++ b/src/event/quic/ngx_event_quic.c @@ -415,9 +415,9 @@ ngx_quic_input_handler(ngx_event_t *rev) } if (c->close) { - qc->error = NGX_QUIC_ERR_NO_ERROR; - qc->error_reason = "graceful shutdown"; - ngx_quic_close_connection(c, NGX_ERROR); + if (qc->conf->shutdown) { + qc->conf->shutdown(c); + } return; } diff --git a/src/event/quic/ngx_event_quic.h b/src/event/quic/ngx_event_quic.h --- a/src/event/quic/ngx_event_quic.h +++ b/src/event/quic/ngx_event_quic.h @@ -28,6 +28,9 @@ #define NGX_QUIC_STREAM_UNIDIRECTIONAL 0x02 +typedef void (*ngx_quic_shutdown_pt)(ngx_connection_t *c); + + typedef enum { NGX_QUIC_STREAM_SEND_READY = 0, NGX_QUIC_STREAM_SEND_SEND, @@ -74,6 +77,8 @@ typedef struct { ngx_int_t stream_reject_code_uni; ngx_int_t stream_reject_code_bidi; + ngx_quic_shutdown_pt shutdown; + u_char av_token_key[NGX_QUIC_AV_KEY_LEN]; u_char sr_token_key[NGX_QUIC_SR_KEY_LEN]; } ngx_quic_conf_t; diff --git a/src/http/v3/ngx_http_v3.h b/src/http/v3/ngx_http_v3.h --- a/src/http/v3/ngx_http_v3.h +++ b/src/http/v3/ngx_http_v3.h @@ -137,6 +137,7 @@ struct ngx_http_v3_session_s { uint64_t next_push_id; uint64_t max_push_id; uint64_t goaway_push_id; + uint64_t next_request_id; off_t total_bytes; off_t payload_bytes; @@ -151,6 +152,7 @@ void ngx_http_v3_init(ngx_connection_t * void ngx_http_v3_reset_connection(ngx_connection_t *c); ngx_int_t ngx_http_v3_init_session(ngx_connection_t *c); ngx_int_t ngx_http_v3_check_flood(ngx_connection_t *c); +void ngx_http_v3_shutdown(ngx_connection_t *c); ngx_int_t ngx_http_v3_read_request_body(ngx_http_request_t *r); ngx_int_t ngx_http_v3_read_unbuffered_request_body(ngx_http_request_t *r); diff --git a/src/http/v3/ngx_http_v3_module.c b/src/http/v3/ngx_http_v3_module.c --- a/src/http/v3/ngx_http_v3_module.c +++ b/src/http/v3/ngx_http_v3_module.c @@ -249,6 +249,8 @@ ngx_http_v3_create_srv_conf(ngx_conf_t * h3scf->quic.stream_reject_code_bidi = NGX_HTTP_V3_ERR_REQUEST_REJECTED; h3scf->quic.active_connection_id_limit = NGX_CONF_UNSET_UINT; + h3scf->quic.shutdown = ngx_http_v3_shutdown; + return h3scf; } diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c --- a/src/http/v3/ngx_http_v3_request.c +++ b/src/http/v3/ngx_http_v3_request.c @@ -74,6 +74,7 @@ ngx_http_v3_init(ngx_connection_t *c) h3scf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_v3_module); if (c->quic == NULL) { + c->idle = 1; h3scf->quic.timeout = clcf->keepalive_timeout; ngx_quic_run(c, &h3scf->quic); return; @@ -116,6 +117,40 @@ ngx_http_v3_init(ngx_connection_t *c) } +void +ngx_http_v3_shutdown(ngx_connection_t *c) +{ + ngx_http_v3_session_t *h3c; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 shutdown"); + + if (c->reusable) { + ngx_quic_terminate_connection(c, NGX_HTTP_V3_ERR_NO_ERROR, + "connection shutdown"); + return; + } + + h3c = ngx_http_v3_get_session(c); + + if (h3c == NULL) { + ngx_quic_finalize_connection(c, NGX_HTTP_V3_ERR_NO_ERROR, + "connection shutdown"); + return; + } + + if (!h3c->goaway) { + h3c->goaway = 1; + + (void) ngx_http_v3_send_goaway(c, h3c->next_request_id); + } + + if (h3c->nrequests == 0) { + ngx_quic_finalize_connection(c, NGX_HTTP_V3_ERR_NO_ERROR, + "connection shutdown"); + } +} + + #if (NGX_HTTP_V3_HQ) static void @@ -246,17 +281,14 @@ ngx_http_v3_init_request_stream(ngx_conn h3c = ngx_http_v3_get_session(c); - if (!h3c->goaway && (ngx_terminate || ngx_exiting)) { - h3c->goaway = 1; - (void) ngx_http_v3_send_goaway(c, c->quic->id); - } - if (h3c->goaway) { c->close = 1; ngx_http_close_connection(c); return; } + h3c->next_request_id = c->quic->id + 0x04; + pc = c->quic->parent; if (n + 1 == clcf->keepalive_requests @@ -264,7 +296,7 @@ ngx_http_v3_init_request_stream(ngx_conn { h3c->goaway = 1; - if (ngx_http_v3_send_goaway(c, (n + 1) << 2) != NGX_OK) { + if (ngx_http_v3_send_goaway(c, h3c->next_request_id) != NGX_OK) { ngx_http_close_connection(c); return; } From arut at nginx.com Thu Jun 23 15:58:45 2022 From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=) Date: Thu, 23 Jun 2022 19:58:45 +0400 Subject: [PATCH 7 of 8] HTTP/3: renamed functions In-Reply-To: References: Message-ID: <03f1661db9601dd552c8.1655999925@arut-laptop> # HG changeset patch # User Roman Arutyunyan # Date 1655740875 -14400 # Mon Jun 20 20:01:15 2022 +0400 # Branch quic # Node ID 03f1661db9601dd552c83643a6198375762b0615 # Parent 1fad6f74168bdceaf3cd59ae72f7c6128dfaa84d HTTP/3: renamed functions. ngx_http_v3_init() is renamed ngx_http_v3_init_stream(). ngx_http_v3_reset_connection() is renamed to ngx_http_v3_reset_stream(). diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c +++ b/src/http/ngx_http_request.c @@ -328,7 +328,7 @@ ngx_http_init_connection(ngx_connection_ #if (NGX_HTTP_V3) if (hc->addr_conf->http3) { - ngx_http_v3_init(c); + ngx_http_v3_init_stream(c); return; } #endif @@ -3812,7 +3812,7 @@ ngx_http_close_connection(ngx_connection #if (NGX_HTTP_V3) if (c->quic) { - ngx_http_v3_reset_connection(c); + ngx_http_v3_reset_stream(c); } #endif diff --git a/src/http/v3/ngx_http_v3.h b/src/http/v3/ngx_http_v3.h --- a/src/http/v3/ngx_http_v3.h +++ b/src/http/v3/ngx_http_v3.h @@ -148,8 +148,8 @@ struct ngx_http_v3_session_s { }; -void ngx_http_v3_init(ngx_connection_t *c); -void ngx_http_v3_reset_connection(ngx_connection_t *c); +void ngx_http_v3_init_stream(ngx_connection_t *c); +void ngx_http_v3_reset_stream(ngx_connection_t *c); ngx_int_t ngx_http_v3_init_session(ngx_connection_t *c); ngx_int_t ngx_http_v3_check_flood(ngx_connection_t *c); void ngx_http_v3_shutdown(ngx_connection_t *c); diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c --- a/src/http/v3/ngx_http_v3_request.c +++ b/src/http/v3/ngx_http_v3_request.c @@ -59,7 +59,7 @@ static const struct { void -ngx_http_v3_init(ngx_connection_t *c) +ngx_http_v3_init_stream(ngx_connection_t *c) { ngx_http_v3_session_t *h3c; ngx_http_connection_t *hc, *phc; @@ -505,7 +505,7 @@ ngx_http_v3_wait_request_handler(ngx_eve void -ngx_http_v3_reset_connection(ngx_connection_t *c) +ngx_http_v3_reset_stream(ngx_connection_t *c) { ngx_http_v3_srv_conf_t *h3scf; From arut at nginx.com Thu Jun 23 15:58:46 2022 From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=) Date: Thu, 23 Jun 2022 19:58:46 +0400 Subject: [PATCH 8 of 8] QUIC: application init() callback In-Reply-To: References: Message-ID: <9be89f499d1f4f550fd9.1655999926@arut-laptop> # HG changeset patch # User Roman Arutyunyan # Date 1655999826 -14400 # Thu Jun 23 19:57:06 2022 +0400 # Branch quic # Node ID 9be89f499d1f4f550fd995aaa59866edcf386bac # Parent 03f1661db9601dd552c83643a6198375762b0615 QUIC: application init() callback. It's called after handshake completion or prior to the first early data stream creation. The callback should initialize application-level data before creating streams. HTTP/3 callback implementation sets keepalive timer and marks connection as reusable. Also, this allows to limit max handshake time in ngx_http_v3_init_stream(). diff --git a/src/event/quic/ngx_event_quic.h b/src/event/quic/ngx_event_quic.h --- a/src/event/quic/ngx_event_quic.h +++ b/src/event/quic/ngx_event_quic.h @@ -28,6 +28,7 @@ #define NGX_QUIC_STREAM_UNIDIRECTIONAL 0x02 +typedef ngx_int_t (*ngx_quic_init_pt)(ngx_connection_t *c); typedef void (*ngx_quic_shutdown_pt)(ngx_connection_t *c); @@ -77,6 +78,7 @@ typedef struct { ngx_int_t stream_reject_code_uni; ngx_int_t stream_reject_code_bidi; + ngx_quic_init_pt init; ngx_quic_shutdown_pt shutdown; u_char av_token_key[NGX_QUIC_AV_KEY_LEN]; diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c --- a/src/event/quic/ngx_event_quic_streams.c +++ b/src/event/quic/ngx_event_quic_streams.c @@ -21,6 +21,7 @@ static ngx_quic_stream_t *ngx_quic_get_s static ngx_int_t ngx_quic_reject_stream(ngx_connection_t *c, uint64_t id); static void ngx_quic_init_stream_handler(ngx_event_t *ev); static void ngx_quic_init_streams_handler(ngx_connection_t *c); +static ngx_int_t ngx_quic_do_init_streams(ngx_connection_t *c); static ngx_quic_stream_t *ngx_quic_create_stream(ngx_connection_t *c, uint64_t id); static void ngx_quic_empty_handler(ngx_event_t *ev); @@ -571,15 +572,22 @@ ngx_quic_init_streams(ngx_connection_t * return NGX_OK; } - ngx_quic_init_streams_handler(c); - - return NGX_OK; + return ngx_quic_do_init_streams(c); } static void ngx_quic_init_streams_handler(ngx_connection_t *c) { + if (ngx_quic_do_init_streams(c) != NGX_OK) { + ngx_quic_close_connection(c, NGX_ERROR); + } +} + + +static ngx_int_t +ngx_quic_do_init_streams(ngx_connection_t *c) +{ ngx_queue_t *q; ngx_quic_stream_t *qs; ngx_quic_connection_t *qc; @@ -588,6 +596,12 @@ ngx_quic_init_streams_handler(ngx_connec qc = ngx_quic_get_connection(c); + if (qc->conf->init) { + if (qc->conf->init(c) != NGX_OK) { + return NGX_ERROR; + } + } + for (q = ngx_queue_head(&qc->streams.uninitialized); q != ngx_queue_sentinel(&qc->streams.uninitialized); q = ngx_queue_next(q)) @@ -597,6 +611,8 @@ ngx_quic_init_streams_handler(ngx_connec } qc->streams.initialized = 1; + + return NGX_OK; } diff --git a/src/http/v3/ngx_http_v3.c b/src/http/v3/ngx_http_v3.c --- a/src/http/v3/ngx_http_v3.c +++ b/src/http/v3/ngx_http_v3.c @@ -17,21 +17,15 @@ static void ngx_http_v3_cleanup_session( ngx_int_t ngx_http_v3_init_session(ngx_connection_t *c) { - ngx_connection_t *pc; ngx_pool_cleanup_t *cln; ngx_http_connection_t *hc; ngx_http_v3_session_t *h3c; - pc = c->quic->parent; - hc = pc->data; - - if (hc->v3_session) { - return NGX_OK; - } + hc = c->data; ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 init session"); - h3c = ngx_pcalloc(pc->pool, sizeof(ngx_http_v3_session_t)); + h3c = ngx_pcalloc(c->pool, sizeof(ngx_http_v3_session_t)); if (h3c == NULL) { goto failed; } @@ -42,16 +36,16 @@ ngx_http_v3_init_session(ngx_connection_ ngx_queue_init(&h3c->blocked); ngx_queue_init(&h3c->pushing); - h3c->keepalive.log = pc->log; - h3c->keepalive.data = pc; + h3c->keepalive.log = c->log; + h3c->keepalive.data = c; h3c->keepalive.handler = ngx_http_v3_keepalive_handler; h3c->keepalive.cancelable = 1; - h3c->table.send_insert_count.log = pc->log; - h3c->table.send_insert_count.data = pc; + h3c->table.send_insert_count.log = c->log; + h3c->table.send_insert_count.data = c; h3c->table.send_insert_count.handler = ngx_http_v3_inc_insert_count_handler; - cln = ngx_pool_cleanup_add(pc->pool, 0); + cln = ngx_pool_cleanup_add(c->pool, 0); if (cln == NULL) { goto failed; } @@ -83,7 +77,7 @@ ngx_http_v3_keepalive_handler(ngx_event_ ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 keepalive handler"); ngx_http_v3_finalize_connection(c, NGX_HTTP_V3_ERR_NO_ERROR, - "keepalive timeout"); + "client timed out"); } diff --git a/src/http/v3/ngx_http_v3.h b/src/http/v3/ngx_http_v3.h --- a/src/http/v3/ngx_http_v3.h +++ b/src/http/v3/ngx_http_v3.h @@ -152,6 +152,7 @@ void ngx_http_v3_init_stream(ngx_connect void ngx_http_v3_reset_stream(ngx_connection_t *c); ngx_int_t ngx_http_v3_init_session(ngx_connection_t *c); ngx_int_t ngx_http_v3_check_flood(ngx_connection_t *c); +ngx_int_t ngx_http_v3_init(ngx_connection_t *c); void ngx_http_v3_shutdown(ngx_connection_t *c); ngx_int_t ngx_http_v3_read_request_body(ngx_http_request_t *r); diff --git a/src/http/v3/ngx_http_v3_module.c b/src/http/v3/ngx_http_v3_module.c --- a/src/http/v3/ngx_http_v3_module.c +++ b/src/http/v3/ngx_http_v3_module.c @@ -249,6 +249,7 @@ ngx_http_v3_create_srv_conf(ngx_conf_t * h3scf->quic.stream_reject_code_bidi = NGX_HTTP_V3_ERR_REQUEST_REJECTED; h3scf->quic.active_connection_id_limit = NGX_CONF_UNSET_UINT; + h3scf->quic.init = ngx_http_v3_init; h3scf->quic.shutdown = ngx_http_v3_shutdown; return h3scf; diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c --- a/src/http/v3/ngx_http_v3_request.c +++ b/src/http/v3/ngx_http_v3_request.c @@ -61,19 +61,29 @@ static const struct { void ngx_http_v3_init_stream(ngx_connection_t *c) { + ngx_http_connection_t *hc, *phc; ngx_http_v3_session_t *h3c; - ngx_http_connection_t *hc, *phc; ngx_http_v3_srv_conf_t *h3scf; ngx_http_core_loc_conf_t *clcf; + ngx_http_core_srv_conf_t *cscf; hc = c->data; hc->ssl = 1; clcf = ngx_http_get_module_loc_conf(hc->conf_ctx, ngx_http_core_module); + cscf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_core_module); h3scf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_v3_module); if (c->quic == NULL) { + if (ngx_http_v3_init_session(c) != NGX_OK) { + ngx_http_close_connection(c); + return; + } + + h3c = hc->v3_session; + ngx_add_timer(&h3c->keepalive, cscf->client_header_timeout); + c->idle = 1; h3scf->quic.timeout = clcf->keepalive_timeout; ngx_quic_run(c, &h3scf->quic); @@ -89,13 +99,6 @@ ngx_http_v3_init_stream(ngx_connection_t ngx_set_connection_log(c, clcf->error_log); } - h3c = hc->v3_session; - - if (h3c == NULL && ngx_http_v3_init_session(c) != NGX_OK) { - ngx_http_close_connection(c); - return; - } - #if (NGX_HTTP_V3_HQ) if (h3scf->hq) { ngx_http_v3_init_hq_stream(c); @@ -103,11 +106,6 @@ ngx_http_v3_init_stream(ngx_connection_t } #endif - if (h3c == NULL && ngx_http_v3_send_settings(c)) { - ngx_http_close_connection(c); - return; - } - if (c->quic->id & NGX_QUIC_STREAM_UNIDIRECTIONAL) { ngx_http_v3_init_uni_stream(c); @@ -117,6 +115,43 @@ ngx_http_v3_init_stream(ngx_connection_t } +ngx_int_t +ngx_http_v3_init(ngx_connection_t *c) +{ + ngx_http_v3_session_t *h3c; +#if (NGX_HTTP_V3_HQ) + ngx_http_connection_t *hc; + ngx_http_v3_srv_conf_t *h3scf; +#endif + ngx_http_core_loc_conf_t *clcf; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 init"); + + if (ngx_terminate || ngx_exiting) { + return NGX_ERROR; + } + + ngx_reusable_connection(c, 1); + + h3c = ngx_http_v3_get_session(c); + clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module); + ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout); + +#if (NGX_HTTP_V3_HQ) + + hc = c->data; + h3scf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_v3_module); + + if (h3scf->hq) { + return NGX_OK; + } + +#endif + + return ngx_http_v3_send_settings(c); +} + + void ngx_http_v3_shutdown(ngx_connection_t *c) { From mdounin at mdounin.ru Thu Jun 23 20:46:50 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 23 Jun 2022 23:46:50 +0300 Subject: what's the proper way to get module specific configuration in init_process In-Reply-To: References: <4e653e995eda3fbe41ac592d465b6709@asiainfo.com> Message-ID: Hello! On Thu, Jun 23, 2022 at 04:26:39PM +0800, lei shan wrote: > That means we need move some configuration commands form server to main > section. We will review the logic An option might be to maintain, for example, an array of servers where your modules are activated in the main configuration. Or some global things common for all servers, such as interpreter in the embedded Perl module. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Thu Jun 23 21:04:49 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 24 Jun 2022 00:04:49 +0300 Subject: nginx abort connection during reload when proxy_cache enabled In-Reply-To: References: Message-ID: Hello! On Thu, Jun 23, 2022 at 07:47:09PM +0800, Wenjun wrote: > We use nginx as the cluster ingress provider, and hit the case > when: > 1. enabled proxy_cache, and the specific cache is expired > 2. nginx is reloading config > then for incoming request, the backend servers can still receive > the request and send back response to nginx, while client will > receive "curl: (52) Empty reply from server". > > > It seems nginx reload may cause the connection break? > > > attached debug log below: >From the debug log it looks like an issue with "aio_write", as documented here: https://trac.nginx.org/nginx/ticket/2162 Switching off "aio_write" should fix this if this issue manifests itself in your particular setup. Note that you are writing to the mailing list dedicated to nginx development. Unless you are willing to submit patches, please avoid further user-level posts here. If you have additional questions, please use the nginx@ mailing list instead. Thank you. Note well that posting logs as images might not be a good idea. Posting plain text logs is generally a much better idea. Thank you again. -- Maxim Dounin http://mdounin.ru/ From P.Pautov at F5.com Sat Jun 25 01:02:21 2022 From: P.Pautov at F5.com (Pavel Pautov) Date: Sat, 25 Jun 2022 01:02:21 +0000 Subject: SSL contexts reuse across locations In-Reply-To: References: Message-ID: Hi, > -----Original Message----- > From: Maxim Dounin > Sent: Thursday, June 16, 2022 18:51 > > Hello! > > On Thu, Jun 16, 2022 at 08:26:48AM +0000, Pavel Pautov via nginx-devel wrote: > > > Looks like, we've made a full circle then... I've replied to > > that suggestion already and in last e-mail (with patch) I note > > that moving additional logic into the ngx_http_proxy_set_ssl() > > has its own drawbacks, but I can certainly move more stuff into > > it. > > > > So do you envision something like "ngx_http_proxy_set_ssl(cf, > > conf, prev, reuse_ssl)"? As previously we've established that > > directives merging stays out of ngx_http_proxy_set_ssl (and > > reuse_ssl calculation has to happen before it). > > I don't think further attempts to explain how to write readable > code worth the effort. > Too bad. > Please see the patch below. Review and testing appreciated. > The patch seems to contradict some previously discussed points. For example, it can actually increase memory usage in certain configurations (say, when proxy_ssl_* directives in location override server level directives or when proxy_pass "https://..." is absent). Or you don't consider this an issue anymore? More importantly, it doesn't really solve the use case from #1234, i.e. http level proxy_ssl_* directives with many servers (as by default http level values are remain unset and thus are not equal to server level values). Also, you effectively compare some directives by value (instead of checking the presence), so it might be surprising to the user that repeating some directives on inner level increases memory consumption and repeating others doesn't. My last patch already addresses all of above... Also, it would be nice to avoid all this copy-paste and have the same optimization in the stream module. > # HG changeset patch > # User Maxim Dounin > # Date 1655429915 -10800 > # Fri Jun 17 04:38:35 2022 +0300 > # Node ID e4a0eeb3edba037f0d090023a2242bda2f8dcb03 > # Parent e23a385cd0ec866a3eb1d8c9c956991e1ed50d78 > Upstream: optimized use of SSL contexts (ticket #1234). [...] > > #if (NGX_HTTP_SSL) > - u->ssl = (glcf->upstream.ssl != NULL); > + u->ssl = glcf->ssl; I like this change, with it additional 'shared_ssl' pointer can be removed from my patch. From mdounin at mdounin.ru Sun Jun 26 05:48:08 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 26 Jun 2022 08:48:08 +0300 Subject: SSL contexts reuse across locations In-Reply-To: References: Message-ID: Hello! On Sat, Jun 25, 2022 at 01:02:21AM +0000, Pavel Pautov via nginx-devel wrote: > > -----Original Message----- > > From: Maxim Dounin > > Sent: Thursday, June 16, 2022 18:51 > > > > Hello! > > > > On Thu, Jun 16, 2022 at 08:26:48AM +0000, Pavel Pautov via nginx-devel wrote: > > > > > Looks like, we've made a full circle then... I've replied to > > > that suggestion already and in last e-mail (with patch) I note > > > that moving additional logic into the ngx_http_proxy_set_ssl() > > > has its own drawbacks, but I can certainly move more stuff into > > > it. > > > > > > So do you envision something like "ngx_http_proxy_set_ssl(cf, > > > conf, prev, reuse_ssl)"? As previously we've established that > > > directives merging stays out of ngx_http_proxy_set_ssl (and > > > reuse_ssl calculation has to happen before it). > > > > I don't think further attempts to explain how to write readable > > code worth the effort. > > Too bad. Yes, that's unfortunate. You may want to consider asking more experienced developers in your company for mentorship. > > Please see the patch below. Review and testing appreciated. > > The patch seems to contradict some previously discussed points. > For example, it can actually increase memory usage in certain > configurations (say, when proxy_ssl_* directives in location > override server level directives or when proxy_pass > "https://..." is absent). Or you don't consider this an issue > anymore? Quoting the very first review: : Also, it should be a good idea to avoid creating SSL contexts if : there is no SSL proxying configured. Or, at very least, make sure : only one context at the http level is used in such cases, so : configurations with many servers won't suddenly blow up. The patch tries to be in line with this, and create at most one proxy SSL context as long as no SSL proxying is configured (it fails to do so though, see below). If proxy SSL settings are redefined at some level, this redefinition adds an SSL context, and this behaviour is something one might expect from the explicit proxy_ssl_* directives in the configuration. > More importantly, it doesn't really solve the use case from > #1234, i.e. http level proxy_ssl_* directives with many servers > (as by default http level values are remain unset and thus are > not equal to server level values). My bad, mostly focused on the location level and missed that it is not possible to properly compare http and server level settings after the merge. Updated patch below, with additional function called just before the merge of particular directives. The resulting codes tries to be close to ngx_http_proxy_set_ssl(), where the relevant settings are used to create SSL contexts, to be self-explanatory. Additionally, the patch now avoids creating SSL contexts if there are no SSL proxying configured. This seems to complicate things insignificantly given the new code layout, though ensures that configurations with "proxy_ssl_verify on;" at http or stream level without proxy_ssl_trusted_certificate set at the same level, as seen in stream_proxy_ssl_verify.t, won't blow up. > Also, you effectively compare some directives by value (instead > of checking the presence), so it might be surprising to the user > that repeating some directives on inner level increases memory > consumption and repeating others doesn't. I don't think this is an issue. SSL contexts can be inherited from the previous level if proxy_ssl_* directives are not redefined; if any of them are redefined, this can result in an additional SSL context. While there are settings which does not create additional SSL contexts even if set to a different value (such as proxy_ssl_server_name), the only guarantee we are willing to provide is that inherited settings will result in optimal memory usage. Either way, this is irrelevant with the updated patch. > My last patch already addresses all of above... Except you've failed to make it readable. > Also, it would be nice to avoid all this copy-paste As already explained, upstream SSL settings are protocol-specific in the current design, as well as upstream SSL context creation. While introducing some shared part is possible, it does not seem to worth the effort, especially given the existing protocol-specific difference. > and have the same optimization in the stream module. Given there are no locations in the stream module, I don't think this is an issue there, or at least no more than server-side SSL contexts. On the other hand, it is trivial to add the same optimization to the stream module, and good from the code consistency point of view. Added. > > # HG changeset patch > > # User Maxim Dounin > > # Date 1655429915 -10800 > > # Fri Jun 17 04:38:35 2022 +0300 > > # Node ID e4a0eeb3edba037f0d090023a2242bda2f8dcb03 > > # Parent e23a385cd0ec866a3eb1d8c9c956991e1ed50d78 > > Upstream: optimized use of SSL contexts (ticket #1234). > [...] > > > > #if (NGX_HTTP_SSL) > > - u->ssl = (glcf->upstream.ssl != NULL); > > + u->ssl = glcf->ssl; > > I like this change, with it additional 'shared_ssl' pointer can > be removed from my patch. Well, "shared_ssl" shouldn't have been added in the first place. Updated patch below. Review and testing appreciated. # HG changeset patch # User Maxim Dounin # Date 1656218606 -10800 # Sun Jun 26 07:43:26 2022 +0300 # Node ID f45142380cee37e71dab76d77ead279cf9b2f4e9 # Parent fecd73db563fb64108f7669eca419badb2aba633 Upstream: optimized use of SSL contexts (ticket #1234). To ensure optimal use of memory, SSL contexts for proxying are now inherited from previous levels as long as relevant proxy_ssl_* directives are not redefined. Further, when no proxy_ssl_* directives are redefined in a server block, we now preserve plcf->upstream.ssl in the "http" section configuration to inherit it to all servers. Similar changes made in uwsgi, grpc, and stream proxy. diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c --- a/src/http/modules/ngx_http_grpc_module.c +++ b/src/http/modules/ngx_http_grpc_module.c @@ -209,6 +209,8 @@ static char *ngx_http_grpc_ssl_password_ ngx_command_t *cmd, void *conf); static char *ngx_http_grpc_ssl_conf_command_check(ngx_conf_t *cf, void *post, void *data); +static ngx_int_t ngx_http_grpc_merge_ssl(ngx_conf_t *cf, + ngx_http_grpc_loc_conf_t *conf, ngx_http_grpc_loc_conf_t *prev); static ngx_int_t ngx_http_grpc_set_ssl(ngx_conf_t *cf, ngx_http_grpc_loc_conf_t *glcf); #endif @@ -562,7 +564,7 @@ ngx_http_grpc_handler(ngx_http_request_t ctx->host = glcf->host; #if (NGX_HTTP_SSL) - u->ssl = (glcf->upstream.ssl != NULL); + u->ssl = glcf->ssl; if (u->ssl) { ngx_str_set(&u->schema, "grpcs://"); @@ -4463,6 +4465,10 @@ ngx_http_grpc_merge_loc_conf(ngx_conf_t #if (NGX_HTTP_SSL) + if (ngx_http_grpc_merge_ssl(cf, conf, prev) != NGX_OK) { + return NGX_CONF_ERROR; + } + ngx_conf_merge_value(conf->upstream.ssl_session_reuse, prev->upstream.ssl_session_reuse, 1); @@ -4524,7 +4530,7 @@ ngx_http_grpc_merge_loc_conf(ngx_conf_t conf->grpc_values = prev->grpc_values; #if (NGX_HTTP_SSL) - conf->upstream.ssl = prev->upstream.ssl; + conf->ssl = prev->ssl; #endif } @@ -4874,17 +4880,63 @@ ngx_http_grpc_ssl_conf_command_check(ngx static ngx_int_t +ngx_http_grpc_merge_ssl(ngx_conf_t *cf, ngx_http_grpc_loc_conf_t *conf, + ngx_http_grpc_loc_conf_t *prev) +{ + ngx_uint_t preserve; + + if (conf->ssl_protocols == 0 + && conf->ssl_ciphers.data == NULL + && conf->upstream.ssl_certificate == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_certificate_key == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_passwords == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_verify == NGX_CONF_UNSET + && conf->ssl_verify_depth == NGX_CONF_UNSET_UINT + && conf->ssl_trusted_certificate.data == NULL + && conf->ssl_crl.data == NULL + && conf->upstream.ssl_session_reuse == NGX_CONF_UNSET + && conf->ssl_conf_commands == NGX_CONF_UNSET_PTR) + { + if (prev->upstream.ssl) { + conf->upstream.ssl = prev->upstream.ssl; + return NGX_OK; + } + + preserve = 1; + + } else { + preserve = 0; + } + + conf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); + if (conf->upstream.ssl == NULL) { + return NGX_ERROR; + } + + conf->upstream.ssl->log = cf->log; + + /* + * special handling to preserve conf->upstream.ssl + * in the "http" section to inherit it to all servers + */ + + if (preserve) { + prev->upstream.ssl = conf->upstream.ssl; + } + + return NGX_OK; +} + + +static ngx_int_t ngx_http_grpc_set_ssl(ngx_conf_t *cf, ngx_http_grpc_loc_conf_t *glcf) { ngx_pool_cleanup_t *cln; - glcf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); - if (glcf->upstream.ssl == NULL) { - return NGX_ERROR; + if (glcf->upstream.ssl->ctx) { + return NGX_OK; } - glcf->upstream.ssl->log = cf->log; - if (ngx_ssl_create(glcf->upstream.ssl, glcf->ssl_protocols, NULL) != NGX_OK) { diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c +++ b/src/http/modules/ngx_http_proxy_module.c @@ -236,6 +236,8 @@ static ngx_int_t ngx_http_proxy_rewrite_ ngx_http_proxy_rewrite_t *pr, ngx_str_t *regex, ngx_uint_t caseless); #if (NGX_HTTP_SSL) +static ngx_int_t ngx_http_proxy_merge_ssl(ngx_conf_t *cf, + ngx_http_proxy_loc_conf_t *conf, ngx_http_proxy_loc_conf_t *prev); static ngx_int_t ngx_http_proxy_set_ssl(ngx_conf_t *cf, ngx_http_proxy_loc_conf_t *plcf); #endif @@ -959,7 +961,7 @@ ngx_http_proxy_handler(ngx_http_request_ ctx->vars = plcf->vars; u->schema = plcf->vars.schema; #if (NGX_HTTP_SSL) - u->ssl = (plcf->upstream.ssl != NULL); + u->ssl = plcf->ssl; #endif } else { @@ -3724,6 +3726,10 @@ ngx_http_proxy_merge_loc_conf(ngx_conf_t #if (NGX_HTTP_SSL) + if (ngx_http_proxy_merge_ssl(cf, conf, prev) != NGX_OK) { + return NGX_CONF_ERROR; + } + ngx_conf_merge_value(conf->upstream.ssl_session_reuse, prev->upstream.ssl_session_reuse, 1); @@ -3857,7 +3863,7 @@ ngx_http_proxy_merge_loc_conf(ngx_conf_t conf->proxy_values = prev->proxy_values; #if (NGX_HTTP_SSL) - conf->upstream.ssl = prev->upstream.ssl; + conf->ssl = prev->ssl; #endif } @@ -4923,16 +4929,62 @@ ngx_http_proxy_ssl_conf_command_check(ng static ngx_int_t +ngx_http_proxy_merge_ssl(ngx_conf_t *cf, ngx_http_proxy_loc_conf_t *conf, + ngx_http_proxy_loc_conf_t *prev) +{ + ngx_uint_t preserve; + + if (conf->ssl_protocols == 0 + && conf->ssl_ciphers.data == NULL + && conf->upstream.ssl_certificate == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_certificate_key == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_passwords == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_verify == NGX_CONF_UNSET + && conf->ssl_verify_depth == NGX_CONF_UNSET_UINT + && conf->ssl_trusted_certificate.data == NULL + && conf->ssl_crl.data == NULL + && conf->upstream.ssl_session_reuse == NGX_CONF_UNSET + && conf->ssl_conf_commands == NGX_CONF_UNSET_PTR) + { + if (prev->upstream.ssl) { + conf->upstream.ssl = prev->upstream.ssl; + return NGX_OK; + } + + preserve = 1; + + } else { + preserve = 0; + } + + conf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); + if (conf->upstream.ssl == NULL) { + return NGX_ERROR; + } + + conf->upstream.ssl->log = cf->log; + + /* + * special handling to preserve conf->upstream.ssl + * in the "http" section to inherit it to all servers + */ + + if (preserve) { + prev->upstream.ssl = conf->upstream.ssl; + } + + return NGX_OK; +} + + +static ngx_int_t ngx_http_proxy_set_ssl(ngx_conf_t *cf, ngx_http_proxy_loc_conf_t *plcf) { ngx_pool_cleanup_t *cln; - plcf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); - if (plcf->upstream.ssl == NULL) { - return NGX_ERROR; - } - - plcf->upstream.ssl->log = cf->log; + if (plcf->upstream.ssl->ctx) { + return NGX_OK; + } if (ngx_ssl_create(plcf->upstream.ssl, plcf->ssl_protocols, NULL) != NGX_OK) diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c +++ b/src/http/modules/ngx_http_uwsgi_module.c @@ -96,6 +96,8 @@ static char *ngx_http_uwsgi_ssl_password ngx_command_t *cmd, void *conf); static char *ngx_http_uwsgi_ssl_conf_command_check(ngx_conf_t *cf, void *post, void *data); +static ngx_int_t ngx_http_uwsgi_merge_ssl(ngx_conf_t *cf, + ngx_http_uwsgi_loc_conf_t *conf, ngx_http_uwsgi_loc_conf_t *prev); static ngx_int_t ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, ngx_http_uwsgi_loc_conf_t *uwcf); #endif @@ -668,7 +670,7 @@ ngx_http_uwsgi_handler(ngx_http_request_ if (uwcf->uwsgi_lengths == NULL) { #if (NGX_HTTP_SSL) - u->ssl = (uwcf->upstream.ssl != NULL); + u->ssl = uwcf->ssl; if (u->ssl) { ngx_str_set(&u->schema, "suwsgi://"); @@ -1865,6 +1867,10 @@ ngx_http_uwsgi_merge_loc_conf(ngx_conf_t #if (NGX_HTTP_SSL) + if (ngx_http_uwsgi_merge_ssl(cf, conf, prev) != NGX_OK) { + return NGX_CONF_ERROR; + } + ngx_conf_merge_value(conf->upstream.ssl_session_reuse, prev->upstream.ssl_session_reuse, 1); @@ -1927,7 +1933,7 @@ ngx_http_uwsgi_merge_loc_conf(ngx_conf_t conf->uwsgi_values = prev->uwsgi_values; #if (NGX_HTTP_SSL) - conf->upstream.ssl = prev->upstream.ssl; + conf->ssl = prev->ssl; #endif } @@ -2455,17 +2461,63 @@ ngx_http_uwsgi_ssl_conf_command_check(ng static ngx_int_t +ngx_http_uwsgi_merge_ssl(ngx_conf_t *cf, ngx_http_uwsgi_loc_conf_t *conf, + ngx_http_uwsgi_loc_conf_t *prev) +{ + ngx_uint_t preserve; + + if (conf->ssl_protocols == 0 + && conf->ssl_ciphers.data == NULL + && conf->upstream.ssl_certificate == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_certificate_key == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_passwords == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_verify == NGX_CONF_UNSET + && conf->ssl_verify_depth == NGX_CONF_UNSET_UINT + && conf->ssl_trusted_certificate.data == NULL + && conf->ssl_crl.data == NULL + && conf->upstream.ssl_session_reuse == NGX_CONF_UNSET + && conf->ssl_conf_commands == NGX_CONF_UNSET_PTR) + { + if (prev->upstream.ssl) { + conf->upstream.ssl = prev->upstream.ssl; + return NGX_OK; + } + + preserve = 1; + + } else { + preserve = 0; + } + + conf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); + if (conf->upstream.ssl == NULL) { + return NGX_ERROR; + } + + conf->upstream.ssl->log = cf->log; + + /* + * special handling to preserve conf->upstream.ssl + * in the "http" section to inherit it to all servers + */ + + if (preserve) { + prev->upstream.ssl = conf->upstream.ssl; + } + + return NGX_OK; +} + + +static ngx_int_t ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, ngx_http_uwsgi_loc_conf_t *uwcf) { ngx_pool_cleanup_t *cln; - uwcf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); - if (uwcf->upstream.ssl == NULL) { - return NGX_ERROR; + if (uwcf->upstream.ssl->ctx) { + return NGX_OK; } - uwcf->upstream.ssl->log = cf->log; - if (ngx_ssl_create(uwcf->upstream.ssl, uwcf->ssl_protocols, NULL) != NGX_OK) { diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c --- a/src/stream/ngx_stream_proxy_module.c +++ b/src/stream/ngx_stream_proxy_module.c @@ -103,6 +103,8 @@ static void ngx_stream_proxy_ssl_handsha static void ngx_stream_proxy_ssl_save_session(ngx_connection_t *c); static ngx_int_t ngx_stream_proxy_ssl_name(ngx_stream_session_t *s); static ngx_int_t ngx_stream_proxy_ssl_certificate(ngx_stream_session_t *s); +static ngx_int_t ngx_stream_proxy_merge_ssl(ngx_conf_t *cf, + ngx_stream_proxy_srv_conf_t *conf, ngx_stream_proxy_srv_conf_t *prev); static ngx_int_t ngx_stream_proxy_set_ssl(ngx_conf_t *cf, ngx_stream_proxy_srv_conf_t *pscf); @@ -801,7 +803,7 @@ ngx_stream_proxy_init_upstream(ngx_strea #if (NGX_STREAM_SSL) - if (pc->type == SOCK_STREAM && pscf->ssl) { + if (pc->type == SOCK_STREAM && pscf->ssl_enable) { if (u->proxy_protocol) { if (ngx_stream_proxy_send_proxy_protocol(s) != NGX_OK) { @@ -2150,6 +2152,10 @@ ngx_stream_proxy_merge_srv_conf(ngx_conf #if (NGX_STREAM_SSL) + if (ngx_stream_proxy_merge_ssl(cf, conf, prev) != NGX_OK) { + return NGX_CONF_ERROR; + } + ngx_conf_merge_value(conf->ssl_enable, prev->ssl_enable, 0); ngx_conf_merge_value(conf->ssl_session_reuse, @@ -2199,17 +2205,63 @@ ngx_stream_proxy_merge_srv_conf(ngx_conf #if (NGX_STREAM_SSL) static ngx_int_t +ngx_stream_proxy_merge_ssl(ngx_conf_t *cf, ngx_stream_proxy_srv_conf_t *conf, + ngx_stream_proxy_srv_conf_t *prev) +{ + ngx_uint_t preserve; + + if (conf->ssl_protocols == 0 + && conf->ssl_ciphers.data == NULL + && conf->ssl_certificate == NGX_CONF_UNSET_PTR + && conf->ssl_certificate_key == NGX_CONF_UNSET_PTR + && conf->ssl_passwords == NGX_CONF_UNSET_PTR + && conf->ssl_verify == NGX_CONF_UNSET + && conf->ssl_verify_depth == NGX_CONF_UNSET_UINT + && conf->ssl_trusted_certificate.data == NULL + && conf->ssl_crl.data == NULL + && conf->ssl_session_reuse == NGX_CONF_UNSET + && conf->ssl_conf_commands == NGX_CONF_UNSET_PTR) + { + if (prev->ssl) { + conf->ssl = prev->ssl; + return NGX_OK; + } + + preserve = 1; + + } else { + preserve = 0; + } + + conf->ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); + if (conf->ssl == NULL) { + return NGX_ERROR; + } + + conf->ssl->log = cf->log; + + /* + * special handling to preserve conf->ssl + * in the "stream" section to inherit it to all servers + */ + + if (preserve) { + prev->ssl = conf->ssl; + } + + return NGX_OK; +} + + +static ngx_int_t ngx_stream_proxy_set_ssl(ngx_conf_t *cf, ngx_stream_proxy_srv_conf_t *pscf) { ngx_pool_cleanup_t *cln; - pscf->ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); - if (pscf->ssl == NULL) { - return NGX_ERROR; + if (pscf->ssl->ctx) { + return NGX_OK; } - pscf->ssl->log = cf->log; - if (ngx_ssl_create(pscf->ssl, pscf->ssl_protocols, NULL) != NGX_OK) { return NGX_ERROR; } -- Maxim Dounin http://mdounin.ru/ From P.Pautov at F5.com Tue Jun 28 05:26:38 2022 From: P.Pautov at F5.com (Pavel Pautov) Date: Tue, 28 Jun 2022 05:26:38 +0000 Subject: SSL contexts reuse across locations In-Reply-To: References: Message-ID: Hi, The patch seems fine and is somewhat similar to what I've posted before. I guess, the copy-paste can be addressed some time later by someone else. > -----Original Message----- > From: Maxim Dounin > Sent: Saturday, June 25, 2022 22:48 > To: Pavel Pautov via nginx-devel > Subject: Re: SSL contexts reuse across locations > > EXTERNAL MAIL: nginx-devel-bounces at nginx.org > > Hello! > > On Sat, Jun 25, 2022 at 01:02:21AM +0000, Pavel Pautov via nginx-devel wrote: > > > > -----Original Message----- > > > From: Maxim Dounin > > > Sent: Thursday, June 16, 2022 18:51 > > > > > > Hello! > > > > > > On Thu, Jun 16, 2022 at 08:26:48AM +0000, Pavel Pautov via nginx-devel > wrote: > > > > > > > Looks like, we've made a full circle then... I've replied to > > > > that suggestion already and in last e-mail (with patch) I note > > > > that moving additional logic into the ngx_http_proxy_set_ssl() > > > > has its own drawbacks, but I can certainly move more stuff into > > > > it. > > > > > > > > So do you envision something like "ngx_http_proxy_set_ssl(cf, > > > > conf, prev, reuse_ssl)"? As previously we've established that > > > > directives merging stays out of ngx_http_proxy_set_ssl (and > > > > reuse_ssl calculation has to happen before it). > > > > > > I don't think further attempts to explain how to write readable > > > code worth the effort. > > > > Too bad. > > Yes, that's unfortunate. You may want to consider asking more > experienced developers in your company for mentorship. > > > > Please see the patch below. Review and testing appreciated. > > > > The patch seems to contradict some previously discussed points. > > For example, it can actually increase memory usage in certain > > configurations (say, when proxy_ssl_* directives in location > > override server level directives or when proxy_pass > > "https://..." is absent). Or you don't consider this an issue > > anymore? > > Quoting the very first review: > > : Also, it should be a good idea to avoid creating SSL contexts if > : there is no SSL proxying configured. Or, at very least, make sure > : only one context at the http level is used in such cases, so > : configurations with many servers won't suddenly blow up. > > The patch tries to be in line with this, and create at most one > proxy SSL context as long as no SSL proxying is configured (it > fails to do so though, see below). If proxy SSL settings are > redefined at some level, this redefinition adds an SSL context, > and this behaviour is something one might expect from the explicit > proxy_ssl_* directives in the configuration. > > > More importantly, it doesn't really solve the use case from > > #1234, i.e. http level proxy_ssl_* directives with many servers > > (as by default http level values are remain unset and thus are > > not equal to server level values). > > My bad, mostly focused on the location level and missed that it is > not possible to properly compare http and server level settings > after the merge. > > Updated patch below, with additional function called just before > the merge of particular directives. The resulting codes tries to > be close to ngx_http_proxy_set_ssl(), where the relevant settings > are used to create SSL contexts, to be self-explanatory. > > Additionally, the patch now avoids creating SSL contexts if there > are no SSL proxying configured. This seems to complicate things > insignificantly given the new code layout, though ensures that > configurations with "proxy_ssl_verify on;" at http or stream level > without proxy_ssl_trusted_certificate set at the same level, as > seen in stream_proxy_ssl_verify.t, won't blow up. > > > Also, you effectively compare some directives by value (instead > > of checking the presence), so it might be surprising to the user > > that repeating some directives on inner level increases memory > > consumption and repeating others doesn't. > > I don't think this is an issue. SSL contexts can be inherited > from the previous level if proxy_ssl_* directives are not > redefined; if any of them are redefined, this can result in an > additional SSL context. While there are settings which does not > create additional SSL contexts even if set to a different value > (such as proxy_ssl_server_name), the only guarantee we are willing > to provide is that inherited settings will result in optimal > memory usage. > > Either way, this is irrelevant with the updated patch. > > > My last patch already addresses all of above... > > Except you've failed to make it readable. > > > Also, it would be nice to avoid all this copy-paste > > As already explained, upstream SSL settings are protocol-specific > in the current design, as well as upstream SSL context creation. > While introducing some shared part is possible, it does not seem > to worth the effort, especially given the existing > protocol-specific difference. > > > and have the same optimization in the stream module. > > Given there are no locations in the stream module, I don't think > this is an issue there, or at least no more than server-side SSL > contexts. On the other hand, it is trivial to add the same > optimization to the stream module, and good from the code > consistency point of view. Added. > > > > # HG changeset patch > > > # User Maxim Dounin > > > # Date 1655429915 -10800 > > > # Fri Jun 17 04:38:35 2022 +0300 > > > # Node ID e4a0eeb3edba037f0d090023a2242bda2f8dcb03 > > > # Parent e23a385cd0ec866a3eb1d8c9c956991e1ed50d78 > > > Upstream: optimized use of SSL contexts (ticket #1234). > > [...] > > > > > > #if (NGX_HTTP_SSL) > > > - u->ssl = (glcf->upstream.ssl != NULL); > > > + u->ssl = glcf->ssl; > > > > I like this change, with it additional 'shared_ssl' pointer can > > be removed from my patch. > > Well, "shared_ssl" shouldn't have been added in the first place. > > Updated patch below. Review and testing appreciated. > > # HG changeset patch > # User Maxim Dounin > # Date 1656218606 -10800 > # Sun Jun 26 07:43:26 2022 +0300 > # Node ID f45142380cee37e71dab76d77ead279cf9b2f4e9 > # Parent fecd73db563fb64108f7669eca419badb2aba633 > Upstream: optimized use of SSL contexts (ticket #1234). > > To ensure optimal use of memory, SSL contexts for proxying are now > inherited from previous levels as long as relevant proxy_ssl_* directives > are not redefined. > > Further, when no proxy_ssl_* directives are redefined in a server block, > we now preserve plcf->upstream.ssl in the "http" section configuration > to inherit it to all servers. > > Similar changes made in uwsgi, grpc, and stream proxy. > > diff --git a/src/http/modules/ngx_http_grpc_module.c > b/src/http/modules/ngx_http_grpc_module.c > --- a/src/http/modules/ngx_http_grpc_module.c > +++ b/src/http/modules/ngx_http_grpc_module.c > @@ -209,6 +209,8 @@ static char *ngx_http_grpc_ssl_password_ > ngx_command_t *cmd, void *conf); > static char *ngx_http_grpc_ssl_conf_command_check(ngx_conf_t *cf, void > *post, > void *data); > +static ngx_int_t ngx_http_grpc_merge_ssl(ngx_conf_t *cf, > + ngx_http_grpc_loc_conf_t *conf, ngx_http_grpc_loc_conf_t *prev); > static ngx_int_t ngx_http_grpc_set_ssl(ngx_conf_t *cf, > ngx_http_grpc_loc_conf_t *glcf); > #endif > @@ -562,7 +564,7 @@ ngx_http_grpc_handler(ngx_http_request_t > ctx->host = glcf->host; > > #if (NGX_HTTP_SSL) > - u->ssl = (glcf->upstream.ssl != NULL); > + u->ssl = glcf->ssl; > > if (u->ssl) { > ngx_str_set(&u->schema, "grpcs://"); > @@ -4463,6 +4465,10 @@ ngx_http_grpc_merge_loc_conf(ngx_conf_t > > #if (NGX_HTTP_SSL) > > + if (ngx_http_grpc_merge_ssl(cf, conf, prev) != NGX_OK) { > + return NGX_CONF_ERROR; > + } > + > ngx_conf_merge_value(conf->upstream.ssl_session_reuse, > prev->upstream.ssl_session_reuse, 1); > > @@ -4524,7 +4530,7 @@ ngx_http_grpc_merge_loc_conf(ngx_conf_t > conf->grpc_values = prev->grpc_values; > > #if (NGX_HTTP_SSL) > - conf->upstream.ssl = prev->upstream.ssl; > + conf->ssl = prev->ssl; > #endif > } > > @@ -4874,17 +4880,63 @@ ngx_http_grpc_ssl_conf_command_check(ngx > > > static ngx_int_t > +ngx_http_grpc_merge_ssl(ngx_conf_t *cf, ngx_http_grpc_loc_conf_t *conf, > + ngx_http_grpc_loc_conf_t *prev) > +{ > + ngx_uint_t preserve; > + > + if (conf->ssl_protocols == 0 > + && conf->ssl_ciphers.data == NULL > + && conf->upstream.ssl_certificate == NGX_CONF_UNSET_PTR > + && conf->upstream.ssl_certificate_key == NGX_CONF_UNSET_PTR > + && conf->upstream.ssl_passwords == NGX_CONF_UNSET_PTR > + && conf->upstream.ssl_verify == NGX_CONF_UNSET > + && conf->ssl_verify_depth == NGX_CONF_UNSET_UINT > + && conf->ssl_trusted_certificate.data == NULL > + && conf->ssl_crl.data == NULL > + && conf->upstream.ssl_session_reuse == NGX_CONF_UNSET > + && conf->ssl_conf_commands == NGX_CONF_UNSET_PTR) > + { > + if (prev->upstream.ssl) { > + conf->upstream.ssl = prev->upstream.ssl; > + return NGX_OK; > + } > + > + preserve = 1; > + > + } else { > + preserve = 0; > + } > + > + conf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); > + if (conf->upstream.ssl == NULL) { > + return NGX_ERROR; > + } > + > + conf->upstream.ssl->log = cf->log; > + > + /* > + * special handling to preserve conf->upstream.ssl > + * in the "http" section to inherit it to all servers > + */ > + > + if (preserve) { > + prev->upstream.ssl = conf->upstream.ssl; > + } > + > + return NGX_OK; > +} > + > + > +static ngx_int_t > ngx_http_grpc_set_ssl(ngx_conf_t *cf, ngx_http_grpc_loc_conf_t *glcf) > { > ngx_pool_cleanup_t *cln; > > - glcf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); > - if (glcf->upstream.ssl == NULL) { > - return NGX_ERROR; > + if (glcf->upstream.ssl->ctx) { > + return NGX_OK; > } > > - glcf->upstream.ssl->log = cf->log; > - > if (ngx_ssl_create(glcf->upstream.ssl, glcf->ssl_protocols, NULL) > != NGX_OK) > { [...] From arut at nginx.com Tue Jun 28 12:10:11 2022 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 28 Jun 2022 16:10:11 +0400 Subject: [PATCH] Add ipv4=off option in resolver like ipv6=off (ticket #1330) In-Reply-To: References: <20220216123055.GA18027@lo0.su> <20220223040934.GD18027@lo0.su> Message-ID: <20220628121011.qin4s4poblz3qdym@N00W24XTQX> On Wed, Jun 15, 2022 at 02:27:27PM +0400, Sergey Kandaurov wrote: > > > On 23 Feb 2022, at 08:09, Ruslan Ermilov wrote: > > > > On Wed, Feb 16, 2022 at 03:30:55PM +0300, Ruslan Ermilov wrote: > >> Hi Lukas, > >> > >> On Wed, Jan 19, 2022 at 07:47:44PM +0100, Lukas Lihotzki via nginx-devel wrote: > >>> # HG changeset patch > >>> # User Lukas Lihotzki > >>> # Date 1642618053 -3600 > >>> # Wed Jan 19 19:47:33 2022 +0100 > >>> # Node ID e9f06dc2d6a4a1aa61c15009b84ceedcaf5983b2 > >>> # Parent aeab41dfd2606dd36cabbf01f1472726e27e8aea > >>> Add ipv4=off option in resolver like ipv6=off (ticket #1330). > >>> > >>> IPv6-only hosts (ticket #1330) and upstreams with IPv6 bind address > >>> (ticket #1535) need to disable resolving to IPv4 addresses. > >>> > >>> Ticket #1330 mentions ipv4=off is the proper fix. > >> > >> There's a number of problems in your patch. Please try this > >> one instead: > >> > >> # HG changeset patch > >> # User Ruslan Ermilov > >> # Date 1644873563 -10800 > >> # Tue Feb 15 00:19:23 2022 +0300 > >> # Node ID 5d2cb60a78dd32a10a0010ccff39974fd7605867 > >> # Parent 1add55d236522616ce34ffaa4dc697a76d3d41a4 > >> The "ipv4=" parameter of the "resolver" directive (ticket #2196). > >> > >> When set to "off", only IPv6 addresses will be resolved, and no > >> A queries are ever sent. > >> > >> diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c > >> --- a/src/core/ngx_resolver.c > >> +++ b/src/core/ngx_resolver.c > >> @@ -157,6 +157,8 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > >> cln->handler = ngx_resolver_cleanup; > >> cln->data = r; > >> > >> + r->ipv4 = 1; > >> + > >> ngx_rbtree_init(&r->name_rbtree, &r->name_sentinel, > >> ngx_resolver_rbtree_insert_value); > >> > >> @@ -225,6 +227,23 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > >> } > >> > >> #if (NGX_HAVE_INET6) > >> + if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) { > >> + > >> + if (ngx_strcmp(&names[i].data[5], "on") == 0) { > >> + r->ipv4 = 1; > >> + > >> + } else if (ngx_strcmp(&names[i].data[5], "off") == 0) { > >> + r->ipv4 = 0; > >> + > >> + } else { > >> + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, > >> + "invalid parameter: %V", &names[i]); > >> + return NULL; > >> + } > >> + > >> + continue; > >> + } > >> + > >> if (ngx_strncmp(names[i].data, "ipv6=", 5) == 0) { > >> > >> if (ngx_strcmp(&names[i].data[5], "on") == 0) { > > > > Addon to the patch: > > > > diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c > > --- a/src/core/ngx_resolver.c > > +++ b/src/core/ngx_resolver.c > > @@ -229,10 +229,12 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > > #if (NGX_HAVE_INET6) > > if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) { > > > > - if (ngx_strcmp(&names[i].data[5], "on") == 0) { > > + if (ngx_strcasecmp(&names[i].data[5], (u_char *) "on") == 0) { > > r->ipv4 = 1; > > > > - } else if (ngx_strcmp(&names[i].data[5], "off") == 0) { > > + } else if (ngx_strcasecmp(&names[i].data[5], (u_char *) "off") > > + == 0) > > + { > > r->ipv4 = 0; > > > > } else { > > @@ -246,10 +248,12 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > > > > if (ngx_strncmp(names[i].data, "ipv6=", 5) == 0) { > > > > - if (ngx_strcmp(&names[i].data[5], "on") == 0) { > > + if (ngx_strcasecmp(&names[i].data[5], (u_char *) "on") == 0) { > > r->ipv6 = 1; > > > > - } else if (ngx_strcmp(&names[i].data[5], "off") == 0) { > > + } else if (ngx_strcasecmp(&names[i].data[5], (u_char *) "off") > > + == 0) > > + { > > r->ipv6 = 0; > > > > } else { > > > > I don't see the reasons to make the parameter value case-insensitive. > Existing parameters in ngx_http_core_module.c take it case-sensitive. > Otherwise, looks good to me. I agree. Although ngx_conf_set_flag_slot() treats its argument as case-insensitive, "listen" directive handler provides a number of case-sensitive examples. -- Roman Arutyunyan From arut at nginx.com Tue Jun 28 12:24:58 2022 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 28 Jun 2022 16:24:58 +0400 Subject: [PATCH] The "sort=" parameter of the "resolver" directive In-Reply-To: References: <20220223040934.GD18027@lo0.su> <8db4bbd67840e8bebb23.1645589485@1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa> Message-ID: <20220628122458.ufx7ou3zpvquu4q7@N00W24XTQX> On Thu, Jun 16, 2022 at 07:17:13PM +0400, Sergey Kandaurov wrote: > > > On 23 Feb 2022, at 08:11, Ruslan Ermilov wrote: > > > > src/core/ngx_resolver.c | 38 +++++++++++++++++++++++++++++++++++++- > > src/core/ngx_resolver.h | 5 +++++ > > 2 files changed, 42 insertions(+), 1 deletions(-) > > > > > > # HG changeset patch > > # User Ruslan Ermilov > > # Date 1645589387 -10800 > > # Wed Feb 23 07:09:47 2022 +0300 > > # Node ID 8db4bbd67840e8bebb23f9c6d10c0f633552e616 > > # Parent 1c19779448db2309d607c74e2628ff98f84569ff > > The "sort=" parameter of the "resolver" directive. > > > > IMHO, the name isn't enough self-documenting in this context. > It can be misinterpreted as sorting addresses, not families. > I'd prefer the parameter name "prefer" here. Jftr, we had a discussion about this. I like the new name. There's no sorting here. > > diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c > > --- a/src/core/ngx_resolver.c > > +++ b/src/core/ngx_resolver.c > > @@ -266,6 +266,27 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > > } > > #endif > > > > + if (ngx_strncmp(names[i].data, "sort=", 5) == 0) { > > + > > + if (ngx_strcasecmp(&names[i].data[5], (u_char *) "ipv4") == 0) { > > The same question raises about case-sensitivity. > I prefer to make it case-sensitive, like other similar directive parameters. > > > + r->sort = NGX_RESOLVE_A_FIRST; > > + > > +#if (NGX_HAVE_INET6) > > + } else if (ngx_strcasecmp(&names[i].data[5], (u_char *) "ipv6") > > + == 0) > > + { > > + r->sort = NGX_RESOLVE_AAAA_FIRST; > > +#endif > > + > > + } else { > > + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, > > + "invalid parameter: %V", &names[i]); > > + return NULL; > > + } > > + > > + continue; > > + } > > + > > What about to put the whole block under NGX_HAVE_INET6? > If built without ipv6 support it makes same little sense > as for "ipv6" and "ipv4" parameters that already there. > > > ngx_memzero(&u, sizeof(ngx_url_t)); > > > > u.url = names[i]; > > @@ -4253,7 +4274,22 @@ ngx_resolver_export(ngx_resolver_t *r, n > > } > > > > i = 0; > > - d = rotate ? ngx_random() % n : 0; > > + > > + if (r->sort == NGX_RESOLVE_A_FIRST) { > > + d = 0; > > + > > +#if (NGX_HAVE_INET6) > > + } else if (r->sort == NGX_RESOLVE_AAAA_FIRST) { > > + d = rn->naddrs6; > > + > > + if (d == n) { > > + d = 0; > > + } > > +#endif > > + > > + } else { > > + d = rotate ? ngx_random() % n : 0; > > + } > > > > if (rn->naddrs) { > > j = rotate ? ngx_random() % rn->naddrs : 0; > > diff --git a/src/core/ngx_resolver.h b/src/core/ngx_resolver.h > > --- a/src/core/ngx_resolver.h > > +++ b/src/core/ngx_resolver.h > > @@ -36,6 +36,9 @@ > > > > #define NGX_RESOLVER_MAX_RECURSION 50 > > > > +#define NGX_RESOLVE_A_FIRST 1 > > +#define NGX_RESOLVE_AAAA_FIRST 2 > > + > > I'd adjust name for better sorting. > > > > > typedef struct ngx_resolver_s ngx_resolver_t; > > > > @@ -185,6 +188,8 @@ struct ngx_resolver_s { > > ngx_queue_t addr6_expire_queue; > > #endif > > > > + ngx_uint_t sort; > > + > > Since the previous patch introduces ipv4/ipv6 bit fields, > it makes sense to make this one a bit field as well. > > > time_t resend_timeout; > > time_t tcp_timeout; > > time_t expire; > > > > diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c > --- a/src/core/ngx_resolver.c > +++ b/src/core/ngx_resolver.c > @@ -227,6 +227,7 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > } > > #if (NGX_HAVE_INET6) > + > if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) { > > if (ngx_strcmp(&names[i].data[5], "on") == 0) { > @@ -260,6 +261,24 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > > continue; > } > + > + if (ngx_strncmp(names[i].data, "prefer=", 7) == 0) { > + > + if (ngx_strcmp(&names[i].data[7], "ipv4") == 0) { > + r->prefer = NGX_RESOLVE_PREFER_A; > + > + } else if (ngx_strcmp(&names[i].data[7], "ipv6") == 0) { > + r->prefer = NGX_RESOLVE_PREFER_AAAA; > + > + } else { > + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, > + "invalid parameter: %V", &names[i]); > + return NULL; > + } > + > + continue; > + } > + > #endif > > ngx_memzero(&u, sizeof(ngx_url_t)); > @@ -4250,7 +4269,22 @@ ngx_resolver_export(ngx_resolver_t *r, n > } > > i = 0; > - d = rotate ? ngx_random() % n : 0; > + > + if (r->prefer == NGX_RESOLVE_PREFER_A) { > + d = 0; > + If we put "prefer=" handler under NGX_HAVE_INET6, the block above should also go under NGX_HAVE_INET6. All 3 blocks could even be substituted with a switch for simplicity. > +#if (NGX_HAVE_INET6) > + } else if (r->prefer == NGX_RESOLVE_PREFER_AAAA) { > + d = rn->naddrs6; > + > + if (d == n) { > + d = 0; > + } > +#endif > + > + } else { > + d = rotate ? ngx_random() % n : 0; > + } > > if (rn->naddrs) { > j = rotate ? ngx_random() % rn->naddrs : 0; > diff --git a/src/core/ngx_resolver.h b/src/core/ngx_resolver.h > --- a/src/core/ngx_resolver.h > +++ b/src/core/ngx_resolver.h > @@ -36,6 +36,9 @@ > > #define NGX_RESOLVER_MAX_RECURSION 50 > > +#define NGX_RESOLVE_PREFER_A 1 > +#define NGX_RESOLVE_PREFER_AAAA 2 > + > > typedef struct ngx_resolver_s ngx_resolver_t; > > @@ -175,6 +178,8 @@ struct ngx_resolver_s { > ngx_queue_t srv_expire_queue; > ngx_queue_t addr_expire_queue; > > + unsigned prefer:2; > + > unsigned ipv4:1; > > #if (NGX_HAVE_INET6) > > > -- > Sergey Kandaurov > > _______________________________________________ > nginx-devel mailing list -- nginx-devel at nginx.org > To unsubscribe send an email to nginx-devel-leave at nginx.org -- Roman Arutyunyan From pluknet at nginx.com Tue Jun 28 14:13:31 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 28 Jun 2022 18:13:31 +0400 Subject: SSL contexts reuse across locations In-Reply-To: References: Message-ID: <207365B4-BE73-4BEC-9162-749B84E63CFF@nginx.com> > On 28 Jun 2022, at 09:26, Pavel Pautov via nginx-devel wrote: > > Hi, > > The patch seems fine and is somewhat similar to what I've posted before. > > I guess, the copy-paste can be addressed some time later by someone else. I agree, the patch looks good to me, tested in various configurations (including if() block, etc.) > >> -----Original Message----- >> From: Maxim Dounin >> Sent: Saturday, June 25, 2022 22:48 >> To: Pavel Pautov via nginx-devel >> Subject: Re: SSL contexts reuse across locations >> >> EXTERNAL MAIL: nginx-devel-bounces at nginx.org >> >> Hello! >> >> On Sat, Jun 25, 2022 at 01:02:21AM +0000, Pavel Pautov via nginx-devel wrote: >> >>>> -----Original Message----- >>>> From: Maxim Dounin >>>> Sent: Thursday, June 16, 2022 18:51 >>>> >>>> Hello! >>>> >>>> On Thu, Jun 16, 2022 at 08:26:48AM +0000, Pavel Pautov via nginx-devel >> wrote: >>>> >>>>> Looks like, we've made a full circle then... I've replied to >>>>> that suggestion already and in last e-mail (with patch) I note >>>>> that moving additional logic into the ngx_http_proxy_set_ssl() >>>>> has its own drawbacks, but I can certainly move more stuff into >>>>> it. >>>>> >>>>> So do you envision something like "ngx_http_proxy_set_ssl(cf, >>>>> conf, prev, reuse_ssl)"? As previously we've established that >>>>> directives merging stays out of ngx_http_proxy_set_ssl (and >>>>> reuse_ssl calculation has to happen before it). >>>> >>>> I don't think further attempts to explain how to write readable >>>> code worth the effort. >>> >>> Too bad. >> >> Yes, that's unfortunate. You may want to consider asking more >> experienced developers in your company for mentorship. >> >>>> Please see the patch below. Review and testing appreciated. >>> >>> The patch seems to contradict some previously discussed points. >>> For example, it can actually increase memory usage in certain >>> configurations (say, when proxy_ssl_* directives in location >>> override server level directives or when proxy_pass >>> "https://..." is absent). Or you don't consider this an issue >>> anymore? >> >> Quoting the very first review: >> >> : Also, it should be a good idea to avoid creating SSL contexts if >> : there is no SSL proxying configured. Or, at very least, make sure >> : only one context at the http level is used in such cases, so >> : configurations with many servers won't suddenly blow up. >> >> The patch tries to be in line with this, and create at most one >> proxy SSL context as long as no SSL proxying is configured (it >> fails to do so though, see below). If proxy SSL settings are >> redefined at some level, this redefinition adds an SSL context, >> and this behaviour is something one might expect from the explicit >> proxy_ssl_* directives in the configuration. >> >>> More importantly, it doesn't really solve the use case from >>> #1234, i.e. http level proxy_ssl_* directives with many servers >>> (as by default http level values are remain unset and thus are >>> not equal to server level values). >> >> My bad, mostly focused on the location level and missed that it is >> not possible to properly compare http and server level settings >> after the merge. >> >> Updated patch below, with additional function called just before >> the merge of particular directives. The resulting codes tries to >> be close to ngx_http_proxy_set_ssl(), where the relevant settings >> are used to create SSL contexts, to be self-explanatory. >> >> Additionally, the patch now avoids creating SSL contexts if there >> are no SSL proxying configured. This seems to complicate things >> insignificantly given the new code layout, though ensures that >> configurations with "proxy_ssl_verify on;" at http or stream level >> without proxy_ssl_trusted_certificate set at the same level, as >> seen in stream_proxy_ssl_verify.t, won't blow up. >> >>> Also, you effectively compare some directives by value (instead >>> of checking the presence), so it might be surprising to the user >>> that repeating some directives on inner level increases memory >>> consumption and repeating others doesn't. >> >> I don't think this is an issue. SSL contexts can be inherited >> from the previous level if proxy_ssl_* directives are not >> redefined; if any of them are redefined, this can result in an >> additional SSL context. While there are settings which does not >> create additional SSL contexts even if set to a different value >> (such as proxy_ssl_server_name), the only guarantee we are willing >> to provide is that inherited settings will result in optimal >> memory usage. Well, redefining proxy_ssl_* directives by itself doesn't result in additional SSL context, it can be created in locations with proxy_pass. In my tests with basic configuration, without proxy_ssl_certificate and proxy_ssl_trusted_certificate and all that, on OpenSSL 3.0.x, creating SSL context results in allocation of by order of 10k memory (tested using FreeBSD ktrace + utrace(2) malloc(3) records.) In configuration levels with just redefined proxy_ssl_* directives, it's an additional sizeof(ngx_ssl_t) allocation, currently of 24 bytes. So, for example, if proxy_ssl_* directives are defined at http level, and then redefined at server level only, this results in one context per server (given that it has proxy_pass in locations). Just to make it clear. >> >> Either way, this is irrelevant with the updated patch. >> >>> My last patch already addresses all of above... >> >> Except you've failed to make it readable. >> >>> Also, it would be nice to avoid all this copy-paste >> >> As already explained, upstream SSL settings are protocol-specific >> in the current design, as well as upstream SSL context creation. >> While introducing some shared part is possible, it does not seem >> to worth the effort, especially given the existing >> protocol-specific difference. >> >>> and have the same optimization in the stream module. >> >> Given there are no locations in the stream module, I don't think >> this is an issue there, or at least no more than server-side SSL >> contexts. On the other hand, it is trivial to add the same >> optimization to the stream module, and good from the code >> consistency point of view. Added. >> >>>> # HG changeset patch >>>> # User Maxim Dounin >>>> # Date 1655429915 -10800 >>>> # Fri Jun 17 04:38:35 2022 +0300 >>>> # Node ID e4a0eeb3edba037f0d090023a2242bda2f8dcb03 >>>> # Parent e23a385cd0ec866a3eb1d8c9c956991e1ed50d78 >>>> Upstream: optimized use of SSL contexts (ticket #1234). >>> [...] >>>> >>>> #if (NGX_HTTP_SSL) >>>> - u->ssl = (glcf->upstream.ssl != NULL); >>>> + u->ssl = glcf->ssl; >>> >>> I like this change, with it additional 'shared_ssl' pointer can >>> be removed from my patch. >> >> Well, "shared_ssl" shouldn't have been added in the first place. >> >> Updated patch below. Review and testing appreciated. >> >> # HG changeset patch >> # User Maxim Dounin >> # Date 1656218606 -10800 >> # Sun Jun 26 07:43:26 2022 +0300 >> # Node ID f45142380cee37e71dab76d77ead279cf9b2f4e9 >> # Parent fecd73db563fb64108f7669eca419badb2aba633 >> Upstream: optimized use of SSL contexts (ticket #1234). >> >> To ensure optimal use of memory, SSL contexts for proxying are now >> inherited from previous levels as long as relevant proxy_ssl_* directives >> are not redefined. >> >> Further, when no proxy_ssl_* directives are redefined in a server block, >> we now preserve plcf->upstream.ssl in the "http" section configuration >> to inherit it to all servers. >> >> Similar changes made in uwsgi, grpc, and stream proxy. >> >> diff --git a/src/http/modules/ngx_http_grpc_module.c >> b/src/http/modules/ngx_http_grpc_module.c >> --- a/src/http/modules/ngx_http_grpc_module.c >> +++ b/src/http/modules/ngx_http_grpc_module.c >> @@ -209,6 +209,8 @@ static char *ngx_http_grpc_ssl_password_ >> ngx_command_t *cmd, void *conf); >> static char *ngx_http_grpc_ssl_conf_command_check(ngx_conf_t *cf, void >> *post, >> void *data); >> +static ngx_int_t ngx_http_grpc_merge_ssl(ngx_conf_t *cf, >> + ngx_http_grpc_loc_conf_t *conf, ngx_http_grpc_loc_conf_t *prev); >> static ngx_int_t ngx_http_grpc_set_ssl(ngx_conf_t *cf, >> ngx_http_grpc_loc_conf_t *glcf); >> #endif >> @@ -562,7 +564,7 @@ ngx_http_grpc_handler(ngx_http_request_t >> ctx->host = glcf->host; >> >> #if (NGX_HTTP_SSL) >> - u->ssl = (glcf->upstream.ssl != NULL); >> + u->ssl = glcf->ssl; >> >> if (u->ssl) { >> ngx_str_set(&u->schema, "grpcs://"); >> @@ -4463,6 +4465,10 @@ ngx_http_grpc_merge_loc_conf(ngx_conf_t >> >> #if (NGX_HTTP_SSL) >> >> + if (ngx_http_grpc_merge_ssl(cf, conf, prev) != NGX_OK) { >> + return NGX_CONF_ERROR; >> + } >> + >> ngx_conf_merge_value(conf->upstream.ssl_session_reuse, >> prev->upstream.ssl_session_reuse, 1); >> >> @@ -4524,7 +4530,7 @@ ngx_http_grpc_merge_loc_conf(ngx_conf_t >> conf->grpc_values = prev->grpc_values; >> >> #if (NGX_HTTP_SSL) >> - conf->upstream.ssl = prev->upstream.ssl; >> + conf->ssl = prev->ssl; >> #endif >> } >> >> @@ -4874,17 +4880,63 @@ ngx_http_grpc_ssl_conf_command_check(ngx >> >> >> static ngx_int_t >> +ngx_http_grpc_merge_ssl(ngx_conf_t *cf, ngx_http_grpc_loc_conf_t *conf, >> + ngx_http_grpc_loc_conf_t *prev) >> +{ >> + ngx_uint_t preserve; >> + >> + if (conf->ssl_protocols == 0 >> + && conf->ssl_ciphers.data == NULL >> + && conf->upstream.ssl_certificate == NGX_CONF_UNSET_PTR >> + && conf->upstream.ssl_certificate_key == NGX_CONF_UNSET_PTR >> + && conf->upstream.ssl_passwords == NGX_CONF_UNSET_PTR >> + && conf->upstream.ssl_verify == NGX_CONF_UNSET >> + && conf->ssl_verify_depth == NGX_CONF_UNSET_UINT >> + && conf->ssl_trusted_certificate.data == NULL >> + && conf->ssl_crl.data == NULL >> + && conf->upstream.ssl_session_reuse == NGX_CONF_UNSET >> + && conf->ssl_conf_commands == NGX_CONF_UNSET_PTR) >> + { >> + if (prev->upstream.ssl) { >> + conf->upstream.ssl = prev->upstream.ssl; >> + return NGX_OK; >> + } >> + >> + preserve = 1; >> + >> + } else { >> + preserve = 0; >> + } >> + >> + conf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); >> + if (conf->upstream.ssl == NULL) { >> + return NGX_ERROR; >> + } >> + >> + conf->upstream.ssl->log = cf->log; >> + >> + /* >> + * special handling to preserve conf->upstream.ssl >> + * in the "http" section to inherit it to all servers >> + */ >> + >> + if (preserve) { >> + prev->upstream.ssl = conf->upstream.ssl; >> + } >> + >> + return NGX_OK; >> +} >> + >> + >> +static ngx_int_t >> ngx_http_grpc_set_ssl(ngx_conf_t *cf, ngx_http_grpc_loc_conf_t *glcf) >> { >> ngx_pool_cleanup_t *cln; >> >> - glcf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); >> - if (glcf->upstream.ssl == NULL) { >> - return NGX_ERROR; >> + if (glcf->upstream.ssl->ctx) { >> + return NGX_OK; >> } >> >> - glcf->upstream.ssl->log = cf->log; >> - >> if (ngx_ssl_create(glcf->upstream.ssl, glcf->ssl_protocols, NULL) >> != NGX_OK) >> { > [...] > > _______________________________________________ > nginx-devel mailing list -- nginx-devel at nginx.org > To unsubscribe send an email to nginx-devel-leave at nginx.org -- Sergey Kandaurov From pluknet at nginx.com Tue Jun 28 16:00:05 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 28 Jun 2022 20:00:05 +0400 Subject: [PATCH] The "sort=" parameter of the "resolver" directive In-Reply-To: <20220628122458.ufx7ou3zpvquu4q7@N00W24XTQX> References: <20220223040934.GD18027@lo0.su> <8db4bbd67840e8bebb23.1645589485@1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa> <20220628122458.ufx7ou3zpvquu4q7@N00W24XTQX> Message-ID: <20220628160005.5nzafeou32amene5@Y9MQ9X2QVV> On Tue, Jun 28, 2022 at 04:24:58PM +0400, Roman Arutyunyan wrote: > On Thu, Jun 16, 2022 at 07:17:13PM +0400, Sergey Kandaurov wrote: > > > > > On 23 Feb 2022, at 08:11, Ruslan Ermilov wrote: > > > > > > src/core/ngx_resolver.c | 38 +++++++++++++++++++++++++++++++++++++- > > > src/core/ngx_resolver.h | 5 +++++ > > > 2 files changed, 42 insertions(+), 1 deletions(-) > > > > > > > > > # HG changeset patch > > > # User Ruslan Ermilov > > > # Date 1645589387 -10800 > > > # Wed Feb 23 07:09:47 2022 +0300 > > > # Node ID 8db4bbd67840e8bebb23f9c6d10c0f633552e616 > > > # Parent 1c19779448db2309d607c74e2628ff98f84569ff > > > The "sort=" parameter of the "resolver" directive. > > > > > > > IMHO, the name isn't enough self-documenting in this context. > > It can be misinterpreted as sorting addresses, not families. > > I'd prefer the parameter name "prefer" here. > > Jftr, we had a discussion about this. I like the new name. There's no > sorting here. > > > > diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c > > > --- a/src/core/ngx_resolver.c > > > +++ b/src/core/ngx_resolver.c > > > @@ -266,6 +266,27 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > > > } > > > #endif > > > > > > + if (ngx_strncmp(names[i].data, "sort=", 5) == 0) { > > > + > > > + if (ngx_strcasecmp(&names[i].data[5], (u_char *) "ipv4") == 0) { > > > > The same question raises about case-sensitivity. > > I prefer to make it case-sensitive, like other similar directive parameters. > > > > > + r->sort = NGX_RESOLVE_A_FIRST; > > > + > > > +#if (NGX_HAVE_INET6) > > > + } else if (ngx_strcasecmp(&names[i].data[5], (u_char *) "ipv6") > > > + == 0) > > > + { > > > + r->sort = NGX_RESOLVE_AAAA_FIRST; > > > +#endif > > > + > > > + } else { > > > + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, > > > + "invalid parameter: %V", &names[i]); > > > + return NULL; > > > + } > > > + > > > + continue; > > > + } > > > + > > > > What about to put the whole block under NGX_HAVE_INET6? > > If built without ipv6 support it makes same little sense > > as for "ipv6" and "ipv4" parameters that already there. > > > > > ngx_memzero(&u, sizeof(ngx_url_t)); > > > > > > u.url = names[i]; > > > @@ -4253,7 +4274,22 @@ ngx_resolver_export(ngx_resolver_t *r, n > > > } > > > > > > i = 0; > > > - d = rotate ? ngx_random() % n : 0; > > > + > > > + if (r->sort == NGX_RESOLVE_A_FIRST) { > > > + d = 0; > > > + > > > +#if (NGX_HAVE_INET6) > > > + } else if (r->sort == NGX_RESOLVE_AAAA_FIRST) { > > > + d = rn->naddrs6; > > > + > > > + if (d == n) { > > > + d = 0; > > > + } > > > +#endif > > > + > > > + } else { > > > + d = rotate ? ngx_random() % n : 0; > > > + } > > > > > > if (rn->naddrs) { > > > j = rotate ? ngx_random() % rn->naddrs : 0; > > > diff --git a/src/core/ngx_resolver.h b/src/core/ngx_resolver.h > > > --- a/src/core/ngx_resolver.h > > > +++ b/src/core/ngx_resolver.h > > > @@ -36,6 +36,9 @@ > > > > > > #define NGX_RESOLVER_MAX_RECURSION 50 > > > > > > +#define NGX_RESOLVE_A_FIRST 1 > > > +#define NGX_RESOLVE_AAAA_FIRST 2 > > > + > > > > I'd adjust name for better sorting. > > > > > > > > typedef struct ngx_resolver_s ngx_resolver_t; > > > > > > @@ -185,6 +188,8 @@ struct ngx_resolver_s { > > > ngx_queue_t addr6_expire_queue; > > > #endif > > > > > > + ngx_uint_t sort; > > > + > > > > Since the previous patch introduces ipv4/ipv6 bit fields, > > it makes sense to make this one a bit field as well. > > > > > time_t resend_timeout; > > > time_t tcp_timeout; > > > time_t expire; > > > > > > > diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c > > --- a/src/core/ngx_resolver.c > > +++ b/src/core/ngx_resolver.c > > @@ -227,6 +227,7 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > > } > > > > #if (NGX_HAVE_INET6) > > + > > if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) { > > > > if (ngx_strcmp(&names[i].data[5], "on") == 0) { > > @@ -260,6 +261,24 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > > > > continue; > > } > > + > > + if (ngx_strncmp(names[i].data, "prefer=", 7) == 0) { > > + > > + if (ngx_strcmp(&names[i].data[7], "ipv4") == 0) { > > + r->prefer = NGX_RESOLVE_PREFER_A; > > + > > + } else if (ngx_strcmp(&names[i].data[7], "ipv6") == 0) { > > + r->prefer = NGX_RESOLVE_PREFER_AAAA; > > + > > + } else { > > + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, > > + "invalid parameter: %V", &names[i]); > > + return NULL; > > + } > > + > > + continue; > > + } > > + > > #endif > > > > ngx_memzero(&u, sizeof(ngx_url_t)); > > @@ -4250,7 +4269,22 @@ ngx_resolver_export(ngx_resolver_t *r, n > > } > > > > i = 0; > > - d = rotate ? ngx_random() % n : 0; > > + > > + if (r->prefer == NGX_RESOLVE_PREFER_A) { > > + d = 0; > > + > > If we put "prefer=" handler under NGX_HAVE_INET6, the block above should > also go under NGX_HAVE_INET6. All 3 blocks could even be substituted with > a switch for simplicity. I agree it will never match in the no-inet6 case. Please apply on top off: diff -r 5210f97325ed src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c Tue Jun 28 19:36:18 2022 +0400 +++ b/src/core/ngx_resolver.c Tue Jun 28 19:58:26 2022 +0400 @@ -4270,19 +4270,24 @@ ngx_resolver_export(ngx_resolver_t *r, n i = 0; - if (r->prefer == NGX_RESOLVE_PREFER_A) { - d = 0; + switch (r->prefer) { #if (NGX_HAVE_INET6) - } else if (r->prefer == NGX_RESOLVE_PREFER_AAAA) { + case NGX_RESOLVE_PREFER_A: + d = 0; + break; + + case NGX_RESOLVE_PREFER_AAAA: d = rn->naddrs6; if (d == n) { d = 0; } + + break; #endif - } else { + default: d = rotate ? ngx_random() % n : 0; } > > > +#if (NGX_HAVE_INET6) > > + } else if (r->prefer == NGX_RESOLVE_PREFER_AAAA) { > > + d = rn->naddrs6; > > + > > + if (d == n) { > > + d = 0; > > + } > > +#endif > > + > > + } else { > > + d = rotate ? ngx_random() % n : 0; > > + } > > > > if (rn->naddrs) { > > j = rotate ? ngx_random() % rn->naddrs : 0; > > diff --git a/src/core/ngx_resolver.h b/src/core/ngx_resolver.h > > --- a/src/core/ngx_resolver.h > > +++ b/src/core/ngx_resolver.h > > @@ -36,6 +36,9 @@ > > > > #define NGX_RESOLVER_MAX_RECURSION 50 > > > > +#define NGX_RESOLVE_PREFER_A 1 > > +#define NGX_RESOLVE_PREFER_AAAA 2 > > + > > > > typedef struct ngx_resolver_s ngx_resolver_t; > > > > @@ -175,6 +178,8 @@ struct ngx_resolver_s { > > ngx_queue_t srv_expire_queue; > > ngx_queue_t addr_expire_queue; > > > > + unsigned prefer:2; > > + > > unsigned ipv4:1; > > > > #if (NGX_HAVE_INET6) > > From pluknet at nginx.com Tue Jun 28 16:25:35 2022 From: pluknet at nginx.com (=?iso-8859-1?q?Sergey_Kandaurov?=) Date: Tue, 28 Jun 2022 20:25:35 +0400 Subject: [PATCH 1 of 2] The "ipv4=" parameter of the "resolver" directive Message-ID: <04e314eb6b4d20a48c5d.1656433535@enoparse.local> # HG changeset patch # User Ruslan Ermilov # Date 1645589317 -10800 # Wed Feb 23 07:08:37 2022 +0300 # Node ID 04e314eb6b4d20a48c5d7bab0609e1b03b51b406 # Parent fecd73db563fb64108f7669eca419badb2aba633 The "ipv4=" parameter of the "resolver" directive. When set to "off", only IPv6 addresses will be resolved, and no A queries are ever sent (ticket #2196). diff -r fecd73db563f -r 04e314eb6b4d src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c Tue Jun 21 17:25:37 2022 +0300 +++ b/src/core/ngx_resolver.c Wed Feb 23 07:08:37 2022 +0300 @@ -157,6 +157,8 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ cln->handler = ngx_resolver_cleanup; cln->data = r; + r->ipv4 = 1; + ngx_rbtree_init(&r->name_rbtree, &r->name_sentinel, ngx_resolver_rbtree_insert_value); @@ -225,6 +227,23 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ } #if (NGX_HAVE_INET6) + if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) { + + if (ngx_strcmp(&names[i].data[5], "on") == 0) { + r->ipv4 = 1; + + } else if (ngx_strcmp(&names[i].data[5], "off") == 0) { + r->ipv4 = 0; + + } else { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid parameter: %V", &names[i]); + return NULL; + } + + continue; + } + if (ngx_strncmp(names[i].data, "ipv6=", 5) == 0) { if (ngx_strcmp(&names[i].data[5], "on") == 0) { @@ -273,6 +292,14 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ } } +#if (NGX_HAVE_INET6) + if (r->ipv4 + r->ipv6 == 0) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "\"ipv4\" and \"ipv6\" cannot both be \"off\""); + return NULL; + } +#endif + if (n && r->connections.nelts == 0) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "no name servers defined"); return NULL; @@ -836,7 +863,7 @@ ngx_resolve_name_locked(ngx_resolver_t * r->last_connection = 0; } - rn->naddrs = (u_short) -1; + rn->naddrs = r->ipv4 ? (u_short) -1 : 0; rn->tcp = 0; #if (NGX_HAVE_INET6) rn->naddrs6 = r->ipv6 ? (u_short) -1 : 0; @@ -1263,7 +1290,7 @@ ngx_resolver_send_query(ngx_resolver_t * rec->log.action = "resolving"; } - if (rn->naddrs == (u_short) -1) { + if (rn->query && rn->naddrs == (u_short) -1) { rc = rn->tcp ? ngx_resolver_send_tcp_query(r, rec, rn->query, rn->qlen) : ngx_resolver_send_udp_query(r, rec, rn->query, rn->qlen); @@ -1765,10 +1792,13 @@ ngx_resolver_process_response(ngx_resolv q = ngx_queue_next(q)) { rn = ngx_queue_data(q, ngx_resolver_node_t, queue); - qident = (rn->query[0] << 8) + rn->query[1]; - - if (qident == ident) { - goto dns_error_name; + + if (rn->query) { + qident = (rn->query[0] << 8) + rn->query[1]; + + if (qident == ident) { + goto dns_error_name; + } } #if (NGX_HAVE_INET6) @@ -3645,7 +3675,7 @@ ngx_resolver_create_name_query(ngx_resol len = sizeof(ngx_resolver_hdr_t) + nlen + sizeof(ngx_resolver_qs_t); #if (NGX_HAVE_INET6) - p = ngx_resolver_alloc(r, r->ipv6 ? len * 2 : len); + p = ngx_resolver_alloc(r, len * (r->ipv4 + r->ipv6)); #else p = ngx_resolver_alloc(r, len); #endif @@ -3654,23 +3684,28 @@ ngx_resolver_create_name_query(ngx_resol } rn->qlen = (u_short) len; - rn->query = p; + + if (r->ipv4) { + rn->query = p; + } #if (NGX_HAVE_INET6) if (r->ipv6) { - rn->query6 = p + len; + rn->query6 = r->ipv4 ? (p + len) : p; } #endif query = (ngx_resolver_hdr_t *) p; - ident = ngx_random(); - - ngx_log_debug2(NGX_LOG_DEBUG_CORE, r->log, 0, - "resolve: \"%V\" A %i", name, ident & 0xffff); - - query->ident_hi = (u_char) ((ident >> 8) & 0xff); - query->ident_lo = (u_char) (ident & 0xff); + if (r->ipv4) { + ident = ngx_random(); + + ngx_log_debug2(NGX_LOG_DEBUG_CORE, r->log, 0, + "resolve: \"%V\" A %i", name, ident & 0xffff); + + query->ident_hi = (u_char) ((ident >> 8) & 0xff); + query->ident_lo = (u_char) (ident & 0xff); + } /* recursion query */ query->flags_hi = 1; query->flags_lo = 0; @@ -3731,7 +3766,9 @@ ngx_resolver_create_name_query(ngx_resol p = rn->query6; - ngx_memcpy(p, rn->query, rn->qlen); + if (r->ipv4) { + ngx_memcpy(p, rn->query, rn->qlen); + } query = (ngx_resolver_hdr_t *) p; diff -r fecd73db563f -r 04e314eb6b4d src/core/ngx_resolver.h --- a/src/core/ngx_resolver.h Tue Jun 21 17:25:37 2022 +0300 +++ b/src/core/ngx_resolver.h Wed Feb 23 07:08:37 2022 +0300 @@ -175,8 +175,10 @@ struct ngx_resolver_s { ngx_queue_t srv_expire_queue; ngx_queue_t addr_expire_queue; + unsigned ipv4:1; + #if (NGX_HAVE_INET6) - ngx_uint_t ipv6; /* unsigned ipv6:1; */ + unsigned ipv6:1; ngx_rbtree_t addr6_rbtree; ngx_rbtree_node_t addr6_sentinel; ngx_queue_t addr6_resend_queue; From pluknet at nginx.com Tue Jun 28 16:25:36 2022 From: pluknet at nginx.com (=?iso-8859-1?q?Sergey_Kandaurov?=) Date: Tue, 28 Jun 2022 20:25:36 +0400 Subject: [PATCH 2 of 2] The "sort=" parameter of the "resolver" directive In-Reply-To: <04e314eb6b4d20a48c5d.1656433535@enoparse.local> References: <04e314eb6b4d20a48c5d.1656433535@enoparse.local> Message-ID: # HG changeset patch # User Ruslan Ermilov # Date 1645589387 -10800 # Wed Feb 23 07:09:47 2022 +0300 # Node ID e80adbf788f6796c6bdf415938abb19b7aa43e3e # Parent 04e314eb6b4d20a48c5d7bab0609e1b03b51b406 The "sort=" parameter of the "resolver" directive. diff -r 04e314eb6b4d -r e80adbf788f6 src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c Wed Feb 23 07:08:37 2022 +0300 +++ b/src/core/ngx_resolver.c Wed Feb 23 07:09:47 2022 +0300 @@ -227,6 +227,7 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ } #if (NGX_HAVE_INET6) + if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) { if (ngx_strcmp(&names[i].data[5], "on") == 0) { @@ -260,6 +261,24 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ continue; } + + if (ngx_strncmp(names[i].data, "prefer=", 7) == 0) { + + if (ngx_strcmp(&names[i].data[7], "ipv4") == 0) { + r->prefer = NGX_RESOLVE_PREFER_A; + + } else if (ngx_strcmp(&names[i].data[7], "ipv6") == 0) { + r->prefer = NGX_RESOLVE_PREFER_AAAA; + + } else { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid parameter: %V", &names[i]); + return NULL; + } + + continue; + } + #endif ngx_memzero(&u, sizeof(ngx_url_t)); @@ -4250,7 +4269,27 @@ ngx_resolver_export(ngx_resolver_t *r, n } i = 0; - d = rotate ? ngx_random() % n : 0; + + switch (r->prefer) { + +#if (NGX_HAVE_INET6) + case NGX_RESOLVE_PREFER_A: + d = 0; + break; + + case NGX_RESOLVE_PREFER_AAAA: + d = rn->naddrs6; + + if (d == n) { + d = 0; + } + + break; +#endif + + default: + d = rotate ? ngx_random() % n : 0; + } if (rn->naddrs) { j = rotate ? ngx_random() % rn->naddrs : 0; diff -r 04e314eb6b4d -r e80adbf788f6 src/core/ngx_resolver.h --- a/src/core/ngx_resolver.h Wed Feb 23 07:08:37 2022 +0300 +++ b/src/core/ngx_resolver.h Wed Feb 23 07:09:47 2022 +0300 @@ -36,6 +36,9 @@ #define NGX_RESOLVER_MAX_RECURSION 50 +#define NGX_RESOLVE_PREFER_A 1 +#define NGX_RESOLVE_PREFER_AAAA 2 + typedef struct ngx_resolver_s ngx_resolver_t; @@ -175,6 +178,8 @@ struct ngx_resolver_s { ngx_queue_t srv_expire_queue; ngx_queue_t addr_expire_queue; + unsigned prefer:2; + unsigned ipv4:1; #if (NGX_HAVE_INET6) From antoine.bonavita at gmail.com Tue Jun 28 19:48:20 2022 From: antoine.bonavita at gmail.com (Antoine Bonavita) Date: Tue, 28 Jun 2022 21:48:20 +0200 Subject: [PATCH 2 of 2] The "sort=" parameter of the "resolver" directive In-Reply-To: References: <04e314eb6b4d20a48c5d.1656433535@enoparse.local> Message-ID: Hello, Maybe the description/title of the patch should reflect latest code changes: "sort=" => "prefer=". A. On Tue, Jun 28, 2022 at 6:25 PM Sergey Kandaurov wrote: > # HG changeset patch > # User Ruslan Ermilov > # Date 1645589387 -10800 > # Wed Feb 23 07:09:47 2022 +0300 > # Node ID e80adbf788f6796c6bdf415938abb19b7aa43e3e > # Parent 04e314eb6b4d20a48c5d7bab0609e1b03b51b406 > The "sort=" parameter of the "resolver" directive. > > diff -r 04e314eb6b4d -r e80adbf788f6 src/core/ngx_resolver.c > --- a/src/core/ngx_resolver.c Wed Feb 23 07:08:37 2022 +0300 > +++ b/src/core/ngx_resolver.c Wed Feb 23 07:09:47 2022 +0300 > @@ -227,6 +227,7 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > } > > #if (NGX_HAVE_INET6) > + > if (ngx_strncmp(names[i].data, "ipv4=", 5) == 0) { > > if (ngx_strcmp(&names[i].data[5], "on") == 0) { > @@ -260,6 +261,24 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_ > > continue; > } > + > + if (ngx_strncmp(names[i].data, "prefer=", 7) == 0) { > + > + if (ngx_strcmp(&names[i].data[7], "ipv4") == 0) { > + r->prefer = NGX_RESOLVE_PREFER_A; > + > + } else if (ngx_strcmp(&names[i].data[7], "ipv6") == 0) { > + r->prefer = NGX_RESOLVE_PREFER_AAAA; > + > + } else { > + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, > + "invalid parameter: %V", &names[i]); > + return NULL; > + } > + > + continue; > + } > + > #endif > > ngx_memzero(&u, sizeof(ngx_url_t)); > @@ -4250,7 +4269,27 @@ ngx_resolver_export(ngx_resolver_t *r, n > } > > i = 0; > - d = rotate ? ngx_random() % n : 0; > + > + switch (r->prefer) { > + > +#if (NGX_HAVE_INET6) > + case NGX_RESOLVE_PREFER_A: > + d = 0; > + break; > + > + case NGX_RESOLVE_PREFER_AAAA: > + d = rn->naddrs6; > + > + if (d == n) { > + d = 0; > + } > + > + break; > +#endif > + > + default: > + d = rotate ? ngx_random() % n : 0; > + } > > if (rn->naddrs) { > j = rotate ? ngx_random() % rn->naddrs : 0; > diff -r 04e314eb6b4d -r e80adbf788f6 src/core/ngx_resolver.h > --- a/src/core/ngx_resolver.h Wed Feb 23 07:08:37 2022 +0300 > +++ b/src/core/ngx_resolver.h Wed Feb 23 07:09:47 2022 +0300 > @@ -36,6 +36,9 @@ > > #define NGX_RESOLVER_MAX_RECURSION 50 > > +#define NGX_RESOLVE_PREFER_A 1 > +#define NGX_RESOLVE_PREFER_AAAA 2 > + > > typedef struct ngx_resolver_s ngx_resolver_t; > > @@ -175,6 +178,8 @@ struct ngx_resolver_s { > ngx_queue_t srv_expire_queue; > ngx_queue_t addr_expire_queue; > > + unsigned prefer:2; > + > unsigned ipv4:1; > > #if (NGX_HAVE_INET6) > > _______________________________________________ > nginx-devel mailing list -- nginx-devel at nginx.org > To unsubscribe send an email to nginx-devel-leave at nginx.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Tue Jun 28 23:52:43 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 29 Jun 2022 02:52:43 +0300 Subject: SSL contexts reuse across locations In-Reply-To: <207365B4-BE73-4BEC-9162-749B84E63CFF@nginx.com> References: <207365B4-BE73-4BEC-9162-749B84E63CFF@nginx.com> Message-ID: Hello! On Tue, Jun 28, 2022 at 06:13:31PM +0400, Sergey Kandaurov wrote: > I agree, the patch looks good to me, > tested in various configurations (including if() block, etc.) Pushed to http://mdounin.ru/hg/nginx. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Wed Jun 29 00:51:44 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 29 Jun 2022 03:51:44 +0300 Subject: [PATCH] The "sort=" parameter of the "resolver" directive In-Reply-To: <20220628122458.ufx7ou3zpvquu4q7@N00W24XTQX> References: <20220223040934.GD18027@lo0.su> <8db4bbd67840e8bebb23.1645589485@1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa> <20220628122458.ufx7ou3zpvquu4q7@N00W24XTQX> Message-ID: Hello! On Tue, Jun 28, 2022 at 04:24:58PM +0400, Roman Arutyunyan wrote: > On Thu, Jun 16, 2022 at 07:17:13PM +0400, Sergey Kandaurov wrote: > > > > > On 23 Feb 2022, at 08:11, Ruslan Ermilov wrote: > > > > > > src/core/ngx_resolver.c | 38 +++++++++++++++++++++++++++++++++++++- > > > src/core/ngx_resolver.h | 5 +++++ > > > 2 files changed, 42 insertions(+), 1 deletions(-) > > > > > > > > > # HG changeset patch > > > # User Ruslan Ermilov > > > # Date 1645589387 -10800 > > > # Wed Feb 23 07:09:47 2022 +0300 > > > # Node ID 8db4bbd67840e8bebb23f9c6d10c0f633552e616 > > > # Parent 1c19779448db2309d607c74e2628ff98f84569ff > > > The "sort=" parameter of the "resolver" directive. > > > > > > > IMHO, the name isn't enough self-documenting in this context. > > It can be misinterpreted as sorting addresses, not families. > > I'd prefer the parameter name "prefer" here. > > Jftr, we had a discussion about this. I like the new name. There's no > sorting here. IIRC, the "sort=" option was expected to be a simplified version of the sortlist option of resolv.conf. It was intentionally made extendable, so potentially we can support all the sortlist features if needed. Ruslan might remember details, make sure to discuss this with him. [...] -- Maxim Dounin http://mdounin.ru/ From xeioex at nginx.com Wed Jun 29 06:05:38 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 29 Jun 2022 06:05:38 +0000 Subject: [njs] Added native function symbolizer for function tracing in debug. Message-ID: details: https://hg.nginx.org/njs/rev/8fe7d9723477 branches: changeset: 1899:8fe7d9723477 user: Dmitry Volyntsev date: Tue Jun 28 22:36:30 2022 -0700 description: Added native function symbolizer for function tracing in debug. diffstat: auto/cc | 4 + auto/help | 4 + auto/libbfd | 35 ++++++ auto/link | 31 +++++ auto/openssl | 1 + auto/options | 9 + auto/pcre | 3 + auto/sources | 4 + configure | 15 +- src/njs.h | 3 + src/njs_addr2line.c | 265 +++++++++++++++++++++++++++++++++++++++++++++++ src/njs_addr2line.h | 22 +++ src/njs_function.c | 25 +++- src/njs_main.h | 1 + src/njs_shell.c | 13 ++ src/njs_sprintf.c | 6 + src/njs_vmcode.c | 6 +- src/njs_vmcode.h | 4 +- src/test/njs_unit_test.c | 43 +++++++- 19 files changed, 474 insertions(+), 20 deletions(-) diffs (736 lines): diff -r 20ee5213e3c4 -r 8fe7d9723477 auto/cc --- a/auto/cc Wed Jun 22 23:37:27 2022 -0700 +++ b/auto/cc Tue Jun 28 22:36:30 2022 -0700 @@ -177,6 +177,10 @@ if [ "$NJS_DEBUG_MEMORY" = "YES" ]; then njs_define=NJS_DEBUG_MEMORY . auto/define fi +if [ "$NJS_DEBUG_OPCODE" = "YES" ]; then + njs_define=NJS_DEBUG_OPCODE . auto/define +fi + if [ "$NJS_TEST262" = "YES" ]; then njs_define=NJS_TEST262 . auto/define fi diff -r 20ee5213e3c4 -r 8fe7d9723477 auto/help --- a/auto/help Wed Jun 22 23:37:27 2022 -0700 +++ b/auto/help Tue Jun 28 22:36:30 2022 -0700 @@ -33,10 +33,14 @@ default: "$NJS_LD_OPT" --address-sanitizer=YES enables build with address sanitizer, \ default: "$NJS_ADDRESS_SANITIZER" + --addr2line=YES enables native function symbolization, \ +default: "$NJS_ADDR2LINE" --debug=YES enables additional runtime checks, \ default: "$NJS_DEBUG" --debug-memory=YES enables memory alloc debug, \ default: "$NJS_DEBUG_MEMORY" + --debug-opcode=YES enables runtime function tracing, \ +default: "$NJS_DEBUG_OPCODE" --test262=YES enables test262 extentions, \ default: "$NJS_TEST262" END diff -r 20ee5213e3c4 -r 8fe7d9723477 auto/libbfd --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/auto/libbfd Tue Jun 28 22:36:30 2022 -0700 @@ -0,0 +1,35 @@ +# Copyright (C) Dmitry Volyntsev +# Copyright (C) NGINX, Inc. + + +NJS_HAVE_LIBBFD=NO + +if [ $NJS_ADDR2LINE = YES ]; then + njs_found=no + + njs_feature="BFD library" + njs_feature_name=NJS_HAVE_LIBBFD + njs_feature_run=yes + njs_feature_incs= + njs_feature_libs="-lbfd" + njs_feature_test="#include + + int main() { + bfd_init(); + return 0; + }" + . auto/feature + + if [ $njs_found = no ]; then + njs_feature="OpenSSL library -lcrypto" + njs_feature_libs="-lcrypto" + + . auto/feature + fi + + + if [ $njs_found = yes ]; then + NJS_HAVE_LIBBFD=YES + NJS_LIB_AUX_LIBS="$NJS_LIB_AUX_LIBS $njs_feature_libs" + fi +fi diff -r 20ee5213e3c4 -r 8fe7d9723477 auto/link --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/auto/link Tue Jun 28 22:36:30 2022 -0700 @@ -0,0 +1,31 @@ + +# Copyright (C) Dmitry Volyntsev +# Copyright (C) NGINX, Inc. + + +NJS_HAVE_DL_ITERATE_PHDR=NO + +if [ $NJS_ADDR2LINE = YES ]; then + njs_feature="dl_iterate_phdr()" + njs_feature_name=NJS_HAVE_DL_ITERATE_PHDR + njs_feature_run=yes + njs_feature_incs= + njs_feature_libs= + njs_feature_test="#define _GNU_SOURCE + #include + + static int + cb(struct dl_phdr_info *info, size_t size, void *data) { + return 0; + } + + int main() { + dl_iterate_phdr(cb, 0); + return 0; + }" + . auto/feature + + if [ $njs_found = yes ]; then + NJS_HAVE_DL_ITERATE_PHDR=YES + fi +fi diff -r 20ee5213e3c4 -r 8fe7d9723477 auto/openssl --- a/auto/openssl Wed Jun 22 23:37:27 2022 -0700 +++ b/auto/openssl Tue Jun 28 22:36:30 2022 -0700 @@ -46,6 +46,7 @@ if [ $NJS_OPENSSL = YES ]; then NJS_HAVE_OPENSSL=YES NJS_OPENSSL_LIB="$njs_feature_libs" + NJS_LIB_AUX_LIBS="$NJS_LIB_AUX_LIBS $njs_feature_libs" fi fi diff -r 20ee5213e3c4 -r 8fe7d9723477 auto/options --- a/auto/options Wed Jun 22 23:37:27 2022 -0700 +++ b/auto/options Tue Jun 28 22:36:30 2022 -0700 @@ -8,7 +8,10 @@ NJS_LD_OPT=${NJS_CC_OPT:--O} NJS_DEBUG=NO NJS_DEBUG_MEMORY=NO +NJS_DEBUG_OPCODE=NO + NJS_ADDRESS_SANITIZER=NO +NJS_ADDR2LINE=NO NJS_TEST262=YES NJS_OPENSSL=YES @@ -34,8 +37,10 @@ do --build-dir=*) NJS_BUILD_DIR="$value" ;; --address-sanitizer=*) NJS_ADDRESS_SANITIZER="$value" ;; + --addr2line=*) NJS_ADDR2LINE="$value" ;; --debug=*) NJS_DEBUG="$value" ;; --debug-memory=*) NJS_DEBUG_MEMORY="$value" ;; + --debug-opcode=*) NJS_DEBUG_OPCODE="$value" ;; --test262=*) NJS_TEST262="$value" ;; --no-openssl) NJS_OPENSSL=NO ;; @@ -66,3 +71,7 @@ done if [ "$NJS_DEBUG_MEMORY" = "YES" ]; then NJS_DEBUG=YES fi + +if [ "$NJS_DEBUG_OPCODE" = "YES" ]; then + NJS_ADDR2LINE=YES +fi diff -r 20ee5213e3c4 -r 8fe7d9723477 auto/pcre --- a/auto/pcre Wed Jun 22 23:37:27 2022 -0700 +++ b/auto/pcre Tue Jun 28 22:36:30 2022 -0700 @@ -134,3 +134,6 @@ if [ $NJS_PCRE = YES ]; then fi fi + +NJS_LIB_AUX_CFLAGS="$NJS_LIB_AUX_CFLAGS $NJS_PCRE_CFLAGS" +NJS_LIB_AUX_LIBS="$NJS_LIB_AUX_LIBS $NJS_PCRE_LIB" diff -r 20ee5213e3c4 -r 8fe7d9723477 auto/sources --- a/auto/sources Wed Jun 22 23:37:27 2022 -0700 +++ b/auto/sources Tue Jun 28 22:36:30 2022 -0700 @@ -75,6 +75,10 @@ if [ "$NJS_PCRE" = "YES" ]; then NJS_LIB_SRCS="$NJS_LIB_SRCS external/njs_regex.c" fi +if [ "$NJS_HAVE_LIBBFD" = "YES" -a "$NJS_HAVE_DL_ITERATE_PHDR" = "YES" ]; then + NJS_LIB_SRCS="$NJS_LIB_SRCS src/njs_addr2line.c" +fi + NJS_TS_SRCS=$(find ts/ -name "*.d.ts" -o -name "*.json") NJS_TEST_TS_SRCS=$(find test/ts/ -name "*.ts" -o -name "*.json") diff -r 20ee5213e3c4 -r 8fe7d9723477 configure --- a/configure Wed Jun 22 23:37:27 2022 -0700 +++ b/configure Tue Jun 28 22:36:30 2022 -0700 @@ -33,6 +33,10 @@ cat << END > $NJS_AUTO_CONFIG_H END +NJS_LIBS="$NJS_LIBRT" +NJS_LIB_AUX_CFLAGS= +NJS_LIB_AUX_LIBS= + . auto/os . auto/cc . auto/types @@ -46,16 +50,11 @@ END . auto/pcre . auto/readline . auto/openssl -. auto/sources - -NJS_LIB_AUX_CFLAGS="$NJS_PCRE_CFLAGS" +. auto/libbfd +. auto/link -NJS_LIBS="$NJS_LIBRT" -NJS_LIB_AUX_LIBS="$NJS_PCRE_LIB $NJS_OPENSSL_LIB" - +. auto/sources . auto/modules . auto/make - . auto/expect - . auto/summary diff -r 20ee5213e3c4 -r 8fe7d9723477 src/njs.h --- a/src/njs.h Wed Jun 22 23:37:27 2022 -0700 +++ b/src/njs.h Tue Jun 28 22:36:30 2022 -0700 @@ -250,6 +250,9 @@ typedef struct { uint8_t unsafe; /* 1 bit */ uint8_t module; /* 1 bit */ uint8_t ast; /* 1 bit */ +#ifdef NJS_DEBUG_OPCODE + uint8_t opcode_debug; /* 1 bit */ +#endif uint8_t unhandled_rejection; } njs_vm_opt_t; diff -r 20ee5213e3c4 -r 8fe7d9723477 src/njs_addr2line.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/njs_addr2line.c Tue Jun 28 22:36:30 2022 -0700 @@ -0,0 +1,265 @@ + +/* + * Copyright (C) Dmitry Volyntsev + * Copyright (C) NGINX, Inc. + * + * addr2line impementaton based upon the work by Jeff Muizelaar. + * + * A hacky replacement for backtrace_symbols in glibc + * + * backtrace_symbols in glibc looks up symbols using dladdr which is limited in + * the symbols that it sees. libbacktracesymbols opens the executable and + * shared libraries using libbfd and will look up backtrace information using + * the symbol table and the dwarf line information. + * + * Derived from addr2line.c from GNU Binutils by Jeff Muizelaar + * + * Copyright 2007 Jeff Muizelaar + * + * addr2line.c -- convert addresses to line number and function name + * Copyright 1997, 1998, 1999, 2000, 2001, 2002 Free Software Foundation, Inc. + * Contributed by Ulrich Lauther + * + * This file was part of GNU Binutils. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2, or (at your option) + * any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + */ + +#define _GNU_SOURCE +#include +#include + +#include +#include + + +typedef struct { + const char *file; + ElfW(Addr) address; + ElfW(Addr) base; + void *hdr; +} njs_file_match_t; + + +typedef struct { + bfd_vma pc; + const char *filename; + const char *functionname; + unsigned int line; + njs_bool_t found; + asymbol **syms; +} njs_translate_address_t; + + +static u_char *njs_process_file(u_char *buf, u_char *end, bfd_vma *addr, + const char *file_name); +static long njs_read_symtab(bfd *abfd, asymbol ***syms); +static u_char *njs_translate_address(u_char *buf, u_char *end, bfd_vma *addr, + bfd *abfd, asymbol **syms); +static void njs_find_address_in_section(bfd *abfd, asection *section, + void *data); +static int njs_find_matching_file(struct dl_phdr_info *info, size_t size, + void *data); + + +u_char * +_njs_addr2line(u_char *buf, u_char *end, void *address) +{ + bfd_vma addr; + const char *fname; + + njs_file_match_t match = { .address = (ElfW(Addr)) address }; + + bfd_init(); + + dl_iterate_phdr(njs_find_matching_file, &match); + + fname = "/proc/self/exe"; + if (match.file != NULL && njs_strlen(match.file)) { + fname = match.file; + } + + addr = (ElfW(Addr)) address - match.base; + + return njs_process_file(buf, end, &addr, fname); +} + + +static u_char * +njs_process_file(u_char *buf, u_char *end, bfd_vma *addr, const char *file_name) +{ + bfd *abfd; + char **matching; + u_char *p; + asymbol **syms; + + abfd = bfd_openr(file_name, NULL); + if (abfd == NULL) { + njs_stderror("%s: failed to open while looking for addr2line", + file_name); + return NULL; + } + + if (bfd_check_format(abfd, bfd_archive)) { + njs_stderror("%s: can not get addresses from archive", file_name); + return NULL; + } + + if (!bfd_check_format_matches(abfd, bfd_object, &matching)) { + njs_stderror("%s: bfd_check_format_matches() failed", + bfd_get_filename(abfd)); + return NULL; + } + + if (njs_read_symtab(abfd, &syms) <= 0) { + njs_stderror("%s: njs_read_symtab() failed", + bfd_get_filename(abfd)); + return NULL; + } + + p = njs_translate_address(buf, end, addr, abfd, syms); + + if (syms != NULL) { + free(syms); + syms = NULL; + } + + bfd_close(abfd); + + return p; +} + + +static long +njs_read_symtab(bfd *abfd, asymbol ***syms) +{ + long symcount; + unsigned size; + + if ((bfd_get_file_flags(abfd) & HAS_SYMS) == 0) { + return 0; + } + + symcount = bfd_read_minisymbols(abfd, 0, (PTR) syms, &size); + if (symcount == 0) { + symcount = bfd_read_minisymbols(abfd, 1 /* dynamic */, + (PTR) syms, &size); + } + + return symcount; +} + + +static u_char * +njs_translate_address(u_char *buf, u_char *end, bfd_vma *addr, bfd *abfd, + asymbol **syms) +{ + char *h; + const char *name; + njs_translate_address_t ctx; + + ctx.pc = *addr; + ctx.found = 0; + ctx.syms = syms; + + bfd_map_over_sections(abfd, njs_find_address_in_section, &ctx); + + if (!ctx.found) { + return njs_sprintf(buf, end, "\?\? \t\?\?:0 [0x%p]", addr); + } + + name = ctx.functionname; + + if (name == NULL || *name == '\0') { + name = "??"; + } + + if (ctx.filename != NULL) { + h = strrchr(ctx.filename, '/'); + if (h != NULL) { + ctx.filename = h + 1; + } + } + + return njs_sprintf(buf, end, "%s() %s:%ud [0x%p]", name, + ctx.filename ? ctx.filename : "??", ctx.line, addr); +} + + +static void +njs_find_address_in_section(bfd *abfd, asection *section, void *data) +{ + bfd_vma vma; + bfd_size_type size; + njs_translate_address_t *ctx; + + ctx = data; + + if (ctx->found) { + return; + } + + if ((bfd_section_flags(section) & SEC_ALLOC) == 0) { + return; + } + + vma = bfd_section_vma(section); + if (ctx->pc < vma) { + return; + } + + size = bfd_section_size(section); + if (ctx->pc >= vma + size) { + return; + } + + ctx->found = bfd_find_nearest_line(abfd, section, ctx->syms, ctx->pc - vma, + &ctx->filename, &ctx->functionname, + &ctx->line); +} + + +static int +njs_find_matching_file(struct dl_phdr_info *info, size_t size, void *data) +{ + long n; + const ElfW(Phdr) *phdr; + + ElfW(Addr) load_base = info->dlpi_addr; + njs_file_match_t *match = data; + + /* + * This code is modeled from Gfind_proc_info-lsb.c:callback() + * from libunwind. + */ + + phdr = info->dlpi_phdr; + + for (n = info->dlpi_phnum; --n >= 0; phdr++) { + if (phdr->p_type == PT_LOAD) { + ElfW(Addr) vaddr = phdr->p_vaddr + load_base; + + if (match->address >= vaddr + && match->address < vaddr + phdr->p_memsz) + { + /* we found a match */ + match->file = info->dlpi_name; + match->base = info->dlpi_addr; + } + } + } + + return 0; +} diff -r 20ee5213e3c4 -r 8fe7d9723477 src/njs_addr2line.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/njs_addr2line.h Tue Jun 28 22:36:30 2022 -0700 @@ -0,0 +1,22 @@ + +/* + * Copyright (C) Dmitry Volyntsev + * Copyright (C) Nginx, Inc. + */ + +#ifndef _NJS_ADDR2LINE_H_INCLUDED_ +#define _NJS_ADDR2LINE_H_INCLUDED_ + + + u_char *_njs_addr2line(u_char *buf, u_char *end, void *address); + + +#if defined(NJS_HAVE_LIBBFD) && defined(NJS_HAVE_DL_ITERATE_PHDR) +#define NJS_HAVE_ADDR2LINE 1 +#define njs_addr2line(buf, end, addr) _njs_addr2line(buf, end, addr) +#else +#define njs_addr2line(buf, end, addr) \ + njs_sprintf(buf, end, "\?\?() \?\?:0 [0x%p]", addr) +#endif + +#endif /* _NJS_ADDR2LINE_H_INCLUDED_ */ diff -r 20ee5213e3c4 -r 8fe7d9723477 src/njs_function.c --- a/src/njs_function.c Wed Jun 22 23:37:27 2022 -0700 +++ b/src/njs_function.c Tue Jun 28 22:36:30 2022 -0700 @@ -711,15 +711,18 @@ njs_function_native_call(njs_vm_t *vm) native = vm->top_frame; function = native->function; -#ifdef NJS_OPCODE_DEBUG - njs_str_t name; +#ifdef NJS_DEBUG_OPCODE + njs_str_t name; + + if (vm->options.opcode_debug) { - ret = njs_builtin_match_native_function(vm, function, &name); - if (ret != NJS_OK) { - name = njs_entry_unknown; - } + ret = njs_builtin_match_native_function(vm, function, &name); + if (ret != NJS_OK) { + name = njs_str_value("unmapped"); + } - njs_printf("CALL NATIVE %V\n", &name); + njs_printf("CALL NATIVE %V %P\n", &name, function->u.native); + } #endif if (njs_fast_path(function->bound == NULL)) { @@ -737,6 +740,14 @@ njs_function_native_call(njs_vm_t *vm) } ret = call(vm, native->arguments, native->nargs, function->magic8); + +#ifdef NJS_DEBUG_OPCODE + if (vm->options.opcode_debug) { + njs_printf("CALL NATIVE RETCODE: %i %V %P\n", ret, &name, + function->u.native); + } +#endif + if (njs_slow_path(ret == NJS_ERROR)) { return ret; } diff -r 20ee5213e3c4 -r 8fe7d9723477 src/njs_main.h --- a/src/njs_main.h Wed Jun 22 23:37:27 2022 -0700 +++ b/src/njs_main.h Tue Jun 28 22:36:30 2022 -0700 @@ -37,6 +37,7 @@ #include #include #include +#include #include diff -r 20ee5213e3c4 -r 8fe7d9723477 src/njs_shell.c --- a/src/njs_shell.c Wed Jun 22 23:37:27 2022 -0700 +++ b/src/njs_shell.c Tue Jun 28 22:36:30 2022 -0700 @@ -36,6 +36,7 @@ typedef struct { uint8_t version; uint8_t ast; uint8_t unhandled_rejection; + uint8_t opcode_debug; int exit_code; char *file; @@ -272,6 +273,9 @@ main(int argc, char **argv) vm_options.sandbox = opts.sandbox; vm_options.unsafe = !opts.safe; vm_options.module = opts.module; +#ifdef NJS_DEBUG_OPCODE + vm_options.opcode_debug = opts.opcode_debug; +#endif vm_options.ops = &njs_console_ops; vm_options.addons = njs_console_addon_modules; @@ -334,6 +338,9 @@ njs_options_parse(njs_opts_t *opts, int " -d print disassembled code.\n" " -e set failure exit code.\n" " -f disabled denormals mode.\n" +#ifdef NJS_DEBUG_OPCODE + " -o enable opcode debug.\n" +#endif " -p set path prefix for modules.\n" " -q disable interactive introduction prompt.\n" " -r ignore unhandled promise rejection.\n" @@ -410,6 +417,12 @@ njs_options_parse(njs_opts_t *opts, int opts->denormals = 0; break; +#ifdef NJS_DEBUG_OPCODE + case 'o': + opts->opcode_debug = 1; + break; +#endif + case 'p': if (++i < argc) { opts->n_paths++; diff -r 20ee5213e3c4 -r 8fe7d9723477 src/njs_sprintf.c --- a/src/njs_sprintf.c Wed Jun 22 23:37:27 2022 -0700 +++ b/src/njs_sprintf.c Tue Jun 28 22:36:30 2022 -0700 @@ -27,6 +27,7 @@ * %*s length and string * * %p void * + * %P symbolized function address * %b njs_bool_t * %V njs_str_t * * %Z '\0' @@ -372,6 +373,11 @@ njs_vsprintf(u_char *buf, u_char *end, c */ goto number; + case 'P': + buf = njs_addr2line(buf, end, va_arg(args, void *)); + fmt++; + continue; + case 'c': d = va_arg(args, int); *buf++ = (u_char) (d & 0xFF); diff -r 20ee5213e3c4 -r 8fe7d9723477 src/njs_vmcode.c --- a/src/njs_vmcode.c Wed Jun 22 23:37:27 2022 -0700 +++ b/src/njs_vmcode.c Tue Jun 28 22:36:30 2022 -0700 @@ -165,8 +165,10 @@ next: * as a single unsigned comparision. */ -#ifdef NJS_OPCODE_DEBUG - njs_disassemble(pc, NULL, 1, NULL); +#ifdef NJS_DEBUG_OPCODE + if (vm->options.opcode_debug) { + njs_disassemble(pc, NULL, 1, NULL); + } #endif if (op > NJS_VMCODE_NORET) { diff -r 20ee5213e3c4 -r 8fe7d9723477 src/njs_vmcode.h --- a/src/njs_vmcode.h Wed Jun 22 23:37:27 2022 -0700 +++ b/src/njs_vmcode.h Tue Jun 28 22:36:30 2022 -0700 @@ -450,9 +450,9 @@ njs_int_t njs_vmcode_interpreter(njs_vm_ njs_object_t *njs_function_new_object(njs_vm_t *vm, njs_value_t *constructor); -#ifdef NJS_OPCODE_DEBUG +#ifdef NJS_DEBUG_OPCODE #define njs_vmcode_debug(vm, pc, prefix) { \ - do { \ + if (vm->options.opcode_debug) do { \ njs_vm_code_t *code; \ \ code = njs_lookup_code(vm, pc); \ diff -r 20ee5213e3c4 -r 8fe7d9723477 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Wed Jun 22 23:37:27 2022 -0700 +++ b/src/test/njs_unit_test.c Tue Jun 28 22:36:30 2022 -0700 @@ -13269,7 +13269,7 @@ static njs_unit_test_t njs_test[] = njs_str("5") }, { njs_str("var a = (new Function('return [' + ','.repeat(2**16) + ']'))();" - "njs.dump(a)"), + "njs.dump(a)"), njs_str("[<65536 empty items>]") }, { njs_str("(new Function('var a = 7; return a' + '= a'.repeat(2**13)))()"), @@ -23415,6 +23415,43 @@ njs_to_int32_test(njs_vm_t *vm, njs_opts } +#ifdef NJS_HAVE_ADDR2LINE +static njs_int_t +njs_addr2line_test(njs_vm_t *vm, njs_opts_t *opts, njs_stat_t *stat) +{ + njs_str_t v; + njs_uint_t i; + u_char buf[512]; + + static const struct { + void *fp; + const char *name; + } tests[] = { + { njs_addr2line_test, njs_stringify(njs_addr2line_test) }, + { njs_to_int32_test, njs_stringify(njs_to_int32_test) }, + }; + + for (i = 0; i < njs_nitems(tests); i++) { + v.start = buf; + v.length = njs_sprintf(buf, &buf[512], "%P", tests[i].fp) - buf; + + if (memcmp(buf, tests[i].name, njs_strlen(tests[i].name))) { + njs_printf("njs_addr2line_test(%p):\n" + "expected: %s\n got: %V\n", + tests[i].fp, tests[i].name, &v); + + stat->failed++; + continue; + } + + stat->passed++; + } + + return NJS_OK; +} +#endif + + static njs_int_t njs_vm_internal_api_test(njs_unit_test_t unused[], size_t num, njs_str_t *name, njs_opts_t *opts, njs_stat_t *stat) @@ -23443,6 +23480,10 @@ njs_vm_internal_api_test(njs_unit_test_t njs_str("njs_string_to_index_test") }, { njs_to_int32_test, njs_str("njs_to_int32_test") }, +#ifdef NJS_HAVE_ADDR2LINE + { njs_addr2line_test, + njs_str("njs_addr2line_test") }, +#endif }; vm = NULL; From xeioex at nginx.com Wed Jun 29 06:05:40 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 29 Jun 2022 06:05:40 +0000 Subject: [njs] Added generator debug. Message-ID: details: https://hg.nginx.org/njs/rev/0cdbc3d35a2a branches: changeset: 1900:0cdbc3d35a2a user: Dmitry Volyntsev date: Tue Jun 28 22:36:37 2022 -0700 description: Added generator debug. diffstat: auto/cc | 4 + auto/help | 2 + auto/options | 2 + src/njs.h | 3 + src/njs_generator.c | 152 ++++++++++++++++++++++++++++++++++++++++----------- src/njs_shell.c | 13 ++++ 6 files changed, 143 insertions(+), 33 deletions(-) diffs (512 lines): diff -r 8fe7d9723477 -r 0cdbc3d35a2a auto/cc --- a/auto/cc Tue Jun 28 22:36:30 2022 -0700 +++ b/auto/cc Tue Jun 28 22:36:37 2022 -0700 @@ -181,6 +181,10 @@ if [ "$NJS_DEBUG_OPCODE" = "YES" ]; then njs_define=NJS_DEBUG_OPCODE . auto/define fi +if [ "$NJS_DEBUG_GENERATOR" = "YES" ]; then + njs_define=NJS_DEBUG_GENERATOR . auto/define +fi + if [ "$NJS_TEST262" = "YES" ]; then njs_define=NJS_TEST262 . auto/define fi diff -r 8fe7d9723477 -r 0cdbc3d35a2a auto/help --- a/auto/help Tue Jun 28 22:36:30 2022 -0700 +++ b/auto/help Tue Jun 28 22:36:37 2022 -0700 @@ -41,6 +41,8 @@ default: "$NJS_DEBUG" default: "$NJS_DEBUG_MEMORY" --debug-opcode=YES enables runtime function tracing, \ default: "$NJS_DEBUG_OPCODE" + --debug-generator=YES enables generator debug, \ +default: "$NJS_DEBUG_GENERATOR" --test262=YES enables test262 extentions, \ default: "$NJS_TEST262" END diff -r 8fe7d9723477 -r 0cdbc3d35a2a auto/options --- a/auto/options Tue Jun 28 22:36:30 2022 -0700 +++ b/auto/options Tue Jun 28 22:36:37 2022 -0700 @@ -9,6 +9,7 @@ NJS_LD_OPT=${NJS_CC_OPT:--O} NJS_DEBUG=NO NJS_DEBUG_MEMORY=NO NJS_DEBUG_OPCODE=NO +NJS_DEBUG_GENERATOR=NO NJS_ADDRESS_SANITIZER=NO NJS_ADDR2LINE=NO @@ -41,6 +42,7 @@ do --debug=*) NJS_DEBUG="$value" ;; --debug-memory=*) NJS_DEBUG_MEMORY="$value" ;; --debug-opcode=*) NJS_DEBUG_OPCODE="$value" ;; + --debug-generator=*) NJS_DEBUG_GENERATOR="$value" ;; --test262=*) NJS_TEST262="$value" ;; --no-openssl) NJS_OPENSSL=NO ;; diff -r 8fe7d9723477 -r 0cdbc3d35a2a src/njs.h --- a/src/njs.h Tue Jun 28 22:36:30 2022 -0700 +++ b/src/njs.h Tue Jun 28 22:36:37 2022 -0700 @@ -253,6 +253,9 @@ typedef struct { #ifdef NJS_DEBUG_OPCODE uint8_t opcode_debug; /* 1 bit */ #endif +#ifdef NJS_DEBUG_GENERATOR + uint8_t generator_debug; /* 1 bit */ +#endif uint8_t unhandled_rejection; } njs_vm_opt_t; diff -r 8fe7d9723477 -r 0cdbc3d35a2a src/njs_generator.c --- a/src/njs_generator.c Tue Jun 28 22:36:30 2022 -0700 +++ b/src/njs_generator.c Tue Jun 28 22:36:37 2022 -0700 @@ -24,6 +24,12 @@ typedef enum { } njs_generator_block_type_t; +typedef enum { + NJS_GENERATOR_CONTINUATION = 1, + NJS_GENERATOR_EXIT = 2, +} njs_generator_patch_type_t; + + struct njs_generator_patch_s { /* * The jump_offset field points to jump offset field which contains a small @@ -179,10 +185,10 @@ static njs_int_t njs_generate_start_bloc const njs_str_t *label); static njs_generator_block_t *njs_generate_lookup_block( njs_generator_block_t *block, uint32_t mask, const njs_str_t *label); -static njs_generator_block_t *njs_generate_find_block( +static njs_generator_block_t *njs_generate_find_block(njs_vm_t *vm, njs_generator_block_t *block, uint32_t mask, const njs_str_t *label); static void njs_generate_patch_block(njs_vm_t *vm, njs_generator_t *generator, - njs_generator_patch_t *list); + njs_generator_block_t *block, unsigned type); static njs_generator_patch_t *njs_generate_make_continuation_patch(njs_vm_t *vm, njs_generator_block_t *block, const njs_str_t *label, njs_jump_off_t offset); @@ -427,13 +433,16 @@ static njs_int_t njs_generate_index_rele ##__VA_ARGS__) -#ifdef NJS_GENERATOR_DEBUG -#define njs_generator_debug(msg, ...) njs_printf(msg "\n", ##__VA_ARGS__) -#define njs_generator_debug_code(code) \ - njs_disassemble((u_char *) code, NULL, 1, NULL) +#ifdef NJS_DEBUG_GENERATOR +#define njs_debug_generator(vm, msg, ...) \ + if (vm->options.generator_debug) \ + njs_printf("GENERATOR " msg "\n", ##__VA_ARGS__) +#define njs_debug_generator_code(code) \ + if (vm->options.generator_debug) \ + njs_disassemble((u_char *) code, NULL, 1, NULL) #else -#define njs_generator_debug(msg, ...) -#define njs_generator_debug_code(code) +#define njs_debug_generator(vm, msg, ...) +#define njs_debug_generator_code(code) #endif @@ -1590,7 +1599,8 @@ njs_generate_while_condition(njs_vm_t *v ctx = generator->context; - njs_generate_patch_block(vm, generator, generator->block->continuation); + njs_generate_patch_block(vm, generator, generator->block, + NJS_GENERATOR_CONTINUATION); njs_code_set_jump_offset(generator, njs_vmcode_jump_t, ctx->jump_offset); @@ -1657,7 +1667,8 @@ static njs_int_t njs_generate_do_while_condition(njs_vm_t *vm, njs_generator_t *generator, njs_parser_node_t *node) { - njs_generate_patch_block(vm, generator, generator->block->continuation); + njs_generate_patch_block(vm, generator, generator->block, + NJS_GENERATOR_CONTINUATION); njs_generator_next(generator, njs_generate, node->right); @@ -1795,7 +1806,8 @@ njs_generate_for_body(njs_vm_t *vm, njs_ return ret; } - njs_generate_patch_block(vm, generator, generator->block->continuation); + njs_generate_patch_block(vm, generator, generator->block, + NJS_GENERATOR_CONTINUATION); njs_generator_next(generator, njs_generate, update); @@ -2062,7 +2074,8 @@ njs_generate_for_in_body(njs_vm_t *vm, n } } - njs_generate_patch_block(vm, generator, generator->block->continuation); + njs_generate_patch_block(vm, generator, generator->block, + NJS_GENERATOR_CONTINUATION); njs_code_set_jump_offset(generator, njs_vmcode_prop_foreach_t, ctx->jump_offset); @@ -2095,6 +2108,24 @@ njs_generate_for_in_body(njs_vm_t *vm, n } +#ifdef NJS_DEBUG_GENERATOR +njs_inline const char* +njs_block_type(njs_generator_block_type_t type) +{ + switch (type) { + case NJS_GENERATOR_LOOP: + return "LOOP "; + case NJS_GENERATOR_SWITCH: + return "SWITCH"; + case NJS_GENERATOR_BLOCK: + return "BLOCK "; + default: + return "TRY "; + } +} +#endif + + static njs_int_t njs_generate_start_block(njs_vm_t *vm, njs_generator_t *generator, njs_generator_block_type_t type, const njs_str_t *label) @@ -2114,6 +2145,8 @@ njs_generate_start_block(njs_vm_t *vm, n block->index = 0; + njs_debug_generator(vm, "START %s %p", njs_block_type(type), block); + return NJS_OK; } @@ -2145,8 +2178,8 @@ njs_generate_lookup_block(njs_generator_ static njs_generator_block_t * -njs_generate_find_block(njs_generator_block_t *block, uint32_t mask, - const njs_str_t *label) +njs_generate_find_block(njs_vm_t *vm, njs_generator_block_t *block, + uint32_t mask, const njs_str_t *label) { njs_generator_block_t *dest_block; @@ -2172,17 +2205,24 @@ njs_generate_find_block(njs_generator_bl while (block != NULL) { if (block->type & NJS_GENERATOR_TRY) { + njs_debug_generator(vm, "FIND %s %p", + njs_block_type(block->type), block); return block; } if (block == dest_block) { - return block; + break; } block = block->next; } } + njs_debug_generator(vm, "FIND %s %p", + dest_block != NULL ? njs_block_type(dest_block->type) + : "NONE ", + dest_block); + return dest_block; } @@ -2206,18 +2246,26 @@ njs_generate_make_continuation_patch(njs patch->label = *label; + + njs_debug_generator(vm, "MAKE CONT %p %L %V", patch, patch->jump_offset, + &patch->label); + return patch; } static void -njs_generate_patch_block(njs_vm_t *vm, njs_generator_t *generator, +njs_generate_patch(njs_vm_t *vm, njs_generator_t *generator, njs_generator_patch_t *list) { njs_generator_patch_t *patch, *next; for (patch = list; patch != NULL; patch = next) { njs_code_update_offset(generator, patch); + njs_debug_generator(vm, "PATCH %p at %L to %L %V", patch, + patch->jump_offset, + *(njs_code_jump_ptr(generator, patch->jump_offset)), + &patch->label); next = patch->next; njs_mp_free(vm->mem_pool, patch); @@ -2225,6 +2273,22 @@ njs_generate_patch_block(njs_vm_t *vm, n } +static void +njs_generate_patch_block(njs_vm_t *vm, njs_generator_t *generator, + njs_generator_block_t *block, unsigned type) +{ + if (type & NJS_GENERATOR_CONTINUATION) { + njs_debug_generator(vm, "PATCH CONT %p", block); + njs_generate_patch(vm, generator, block->continuation); + } + + if (type & NJS_GENERATOR_EXIT) { + njs_debug_generator(vm, "PATCH EXIT %p", block); + njs_generate_patch(vm, generator, block->exit); + } +} + + static njs_generator_patch_t * njs_generate_make_exit_patch(njs_vm_t *vm, njs_generator_block_t *block, const njs_str_t *label, njs_jump_off_t offset) @@ -2244,6 +2308,9 @@ njs_generate_make_exit_patch(njs_vm_t *v patch->label = *label; + njs_debug_generator(vm, "MAKE EXIT %p %L %V", patch, patch->jump_offset, + &patch->label); + return patch; } @@ -2256,7 +2323,9 @@ njs_generate_patch_block_exit(njs_vm_t * block = generator->block; generator->block = block->next; - njs_generate_patch_block(vm, generator, block->exit); + njs_generate_patch_block(vm, generator, block, NJS_GENERATOR_EXIT); + + njs_debug_generator(vm, "EXIT %s %p", njs_block_type(block->type), block); njs_mp_free(vm->mem_pool, block); } @@ -2307,7 +2376,7 @@ njs_generate_continue_statement(njs_vm_t label = &node->name; - block = njs_generate_find_block(generator->block, NJS_GENERATOR_LOOP, + block = njs_generate_find_block(vm, generator->block, NJS_GENERATOR_LOOP, label); if (njs_slow_path(block == NULL)) { @@ -2356,7 +2425,8 @@ njs_generate_break_statement(njs_vm_t *v label = &node->name; - block = njs_generate_find_block(generator->block, NJS_GENERATOR_ALL, label); + block = njs_generate_find_block(vm, generator->block, NJS_GENERATOR_ALL, + label); if (njs_slow_path(block == NULL)) { goto syntax_error; } @@ -3377,7 +3447,7 @@ njs_generate_3addr_operation_end(njs_vm_ code->dst = node->index; - njs_generator_debug_code(code); + njs_debug_generator_code(code); return njs_generator_stack_pop(vm, generator, generator->context); } @@ -3412,7 +3482,7 @@ njs_generate_2addr_operation_end(njs_vm_ code->dst = node->index; - njs_generator_debug_code(code); + njs_debug_generator_code(code); return njs_generator_stack_pop(vm, generator, NULL); } @@ -3461,7 +3531,7 @@ njs_generate_typeof_operation_end(njs_vm code->dst = node->index; - njs_generator_debug_code(code); + njs_debug_generator_code(code); return njs_generator_stack_pop(vm, generator, NULL); } @@ -3884,6 +3954,8 @@ njs_generate_return_statement_end(njs_vm immediate = njs_generate_lookup_block(generator->block, NJS_GENERATOR_TRY, &no_label); + njs_debug_generator(vm, "LOOKUP TRY %p", immediate); + if (njs_fast_path(immediate == NULL)) { njs_generate_code(generator, njs_vmcode_return_t, code, NJS_VMCODE_RETURN, 1, node); @@ -4267,7 +4339,11 @@ njs_generate_try_left(njs_vm_t *vm, njs_ if (try_block->exit != NULL) { ctx->try_exit_label = try_block->exit->label; - njs_generate_patch_block(vm, generator, try_block->exit); + njs_debug_generator(vm, "TRY CTX %p EXIT LABEL %V", ctx, + &ctx->try_exit_label); + + njs_generate_patch_block(vm, generator, try_block, + NJS_GENERATOR_EXIT); njs_generate_code(generator, njs_vmcode_try_trampoline_t, try_break, NJS_VMCODE_TRY_BREAK, 1, NULL); @@ -4282,7 +4358,8 @@ njs_generate_try_left(njs_vm_t *vm, njs_ if (try_block->continuation != NULL) { ctx->try_cont_label = try_block->continuation->label; - njs_generate_patch_block(vm, generator, try_block->continuation); + njs_generate_patch_block(vm, generator, try_block, + NJS_GENERATOR_CONTINUATION); njs_generate_code(generator, njs_vmcode_try_trampoline_t, try_continue, NJS_VMCODE_TRY_CONTINUE, 1, NULL); @@ -4295,6 +4372,10 @@ njs_generate_try_left(njs_vm_t *vm, njs_ } } + njs_debug_generator(vm, "EXIT %s %p", + njs_block_type(generator->block->type), + generator->block); + generator->block = try_block->next; njs_code_set_jump_offset(generator, njs_vmcode_try_start_t, @@ -4396,7 +4477,7 @@ njs_generate_try_catch(njs_vm_t *vm, njs * block != NULL is checked * by njs_generate_continue_statement() */ - block = njs_generate_find_block(generator->block, + block = njs_generate_find_block(vm, generator->block, NJS_GENERATOR_LOOP, &ctx->try_cont_label); @@ -4410,7 +4491,7 @@ njs_generate_try_catch(njs_vm_t *vm, njs } if (try_block->exit != NULL) { - block = njs_generate_find_block(generator->block, + block = njs_generate_find_block(vm, generator->block, NJS_GENERATOR_ALL, &ctx->try_exit_label); @@ -4462,7 +4543,8 @@ njs_generate_try_finally(njs_vm_t *vm, n if (catch_block->exit != NULL) { ctx->catch_exit_label = catch_block->exit->label; - njs_generate_patch_block(vm, generator, catch_block->exit); + njs_generate_patch_block(vm, generator, catch_block, + NJS_GENERATOR_EXIT); njs_generate_code(generator, njs_vmcode_try_trampoline_t, try_break, NJS_VMCODE_TRY_BREAK, 1, NULL); @@ -4478,8 +4560,8 @@ njs_generate_try_finally(njs_vm_t *vm, n if (catch_block->continuation != NULL) { ctx->catch_cont_label = catch_block->continuation->label; - njs_generate_patch_block(vm, generator, - catch_block->continuation); + njs_generate_patch_block(vm, generator, catch_block, + NJS_GENERATOR_CONTINUATION); njs_generate_code(generator, njs_vmcode_try_trampoline_t, try_continue, NJS_VMCODE_TRY_CONTINUE, 1, @@ -4494,6 +4576,10 @@ njs_generate_try_finally(njs_vm_t *vm, n } } + njs_debug_generator(vm, "EXIT %s %p", + njs_block_type(generator->block->type), + generator->block); + generator->block = catch_block->next; njs_code_set_jump_offset(generator, njs_vmcode_catch_t, @@ -4554,7 +4640,7 @@ njs_generate_try_end(njs_vm_t *vm, njs_g * block != NULL is checked * by njs_generate_continue_statement() */ - block = njs_generate_find_block(generator->block, + block = njs_generate_find_block(vm, generator->block, NJS_GENERATOR_LOOP, dest_label); patch = njs_generate_make_continuation_patch(vm, block, dest_label, @@ -4582,7 +4668,7 @@ njs_generate_try_end(njs_vm_t *vm, njs_g * block can be NULL for "return" instruction in * outermost try-catch block. */ - block = njs_generate_find_block(generator->block, + block = njs_generate_find_block(vm, generator->block, NJS_GENERATOR_ALL | NJS_GENERATOR_TRY, dest_label); if (block != NULL) { @@ -4911,7 +4997,7 @@ njs_generate_temp_index_get(njs_vm_t *vm if (cache != NULL && cache->items != 0) { last = njs_arr_remove_last(cache); - njs_generator_debug("INDEX REUSE %04Xz", (size_t) *last); + njs_debug_generator(vm, "INDEX REUSE %04Xz", (size_t) *last); return *last; } @@ -4977,7 +5063,7 @@ njs_generate_index_release(njs_vm_t *vm, njs_arr_t *cache; njs_index_t *last; - njs_generator_debug("INDEX RELEASE %04Xz", (size_t) index); + njs_debug_generator(vm, "INDEX RELEASE %04Xz", (size_t) index); cache = generator->index_cache; diff -r 8fe7d9723477 -r 0cdbc3d35a2a src/njs_shell.c --- a/src/njs_shell.c Tue Jun 28 22:36:30 2022 -0700 +++ b/src/njs_shell.c Tue Jun 28 22:36:37 2022 -0700 @@ -37,6 +37,7 @@ typedef struct { uint8_t ast; uint8_t unhandled_rejection; uint8_t opcode_debug; + uint8_t generator_debug; int exit_code; char *file; @@ -273,6 +274,9 @@ main(int argc, char **argv) vm_options.sandbox = opts.sandbox; vm_options.unsafe = !opts.safe; vm_options.module = opts.module; +#ifdef NJS_DEBUG_GENERATOR + vm_options.generator_debug = opts.generator_debug; +#endif #ifdef NJS_DEBUG_OPCODE vm_options.opcode_debug = opts.opcode_debug; #endif @@ -338,6 +342,9 @@ njs_options_parse(njs_opts_t *opts, int " -d print disassembled code.\n" " -e set failure exit code.\n" " -f disabled denormals mode.\n" +#ifdef NJS_DEBUG_GENERATOR + " -g enable generator debug.\n" +#endif #ifdef NJS_DEBUG_OPCODE " -o enable opcode debug.\n" #endif @@ -417,6 +424,12 @@ njs_options_parse(njs_opts_t *opts, int opts->denormals = 0; break; +#ifdef NJS_DEBUG_GENERATOR + case 'g': + opts->generator_debug = 1; + break; +#endif + #ifdef NJS_DEBUG_OPCODE case 'o': opts->opcode_debug = 1; From xeioex at nginx.com Wed Jun 29 06:05:42 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 29 Jun 2022 06:05:42 +0000 Subject: [njs] Catching in runtime invalid JUMP offsets at FINALLY instruction. Message-ID: details: https://hg.nginx.org/njs/rev/116b09a57817 branches: changeset: 1901:116b09a57817 user: Dmitry Volyntsev date: Tue Jun 28 22:36:38 2022 -0700 description: Catching in runtime invalid JUMP offsets at FINALLY instruction. diffstat: src/njs_vmcode.c | 17 ++++++++++++++--- 1 files changed, 14 insertions(+), 3 deletions(-) diffs (34 lines): diff -r 0cdbc3d35a2a -r 116b09a57817 src/njs_vmcode.c --- a/src/njs_vmcode.c Tue Jun 28 22:36:37 2022 -0700 +++ b/src/njs_vmcode.c Tue Jun 28 22:36:38 2022 -0700 @@ -2122,6 +2122,7 @@ njs_vmcode_finally(njs_vm_t *vm, njs_val u_char *pc) { njs_value_t *exception_value, *exit_value; + njs_jump_off_t offset; njs_vmcode_finally_t *finally; exception_value = njs_scope_value(vm, (njs_index_t) retval); @@ -2148,9 +2149,19 @@ njs_vmcode_finally(njs_vm_t *vm, njs_val return njs_vmcode_return(vm, NULL, exit_value); } else if (njs_number(exit_value) != 0) { - return (njs_jump_off_t) (njs_number(exit_value) > 0) - ? finally->break_offset - : finally->continue_offset; + offset = (njs_number(exit_value) > 0) ? finally->break_offset + : finally->continue_offset; + + if (njs_slow_path(offset + < (njs_jump_off_t) sizeof(njs_vmcode_finally_t))) + { + njs_internal_error(vm, "unset %s offset for FINALLY block", + (njs_number(exit_value) > 0) ? "exit" + : "continuaion"); + return NJS_ERROR; + } + + return offset; } return sizeof(njs_vmcode_finally_t); From xeioex at nginx.com Wed Jun 29 06:05:44 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 29 Jun 2022 06:05:44 +0000 Subject: [njs] Fixed break instruction in a try-catch block. Message-ID: details: https://hg.nginx.org/njs/rev/b7c4e0f714a9 branches: changeset: 1902:b7c4e0f714a9 user: Dmitry Volyntsev date: Tue Jun 28 23:04:00 2022 -0700 description: Fixed break instruction in a try-catch block. Previously, JUMP offset for a break instruction inside a try-catch block was not set to a correct offset during code generation when a return instruction was present in inner try-catch block. The fix is to update the JUMP offset appropriately. This closes #553 issue on Github. diffstat: src/njs_generator.c | 41 +++++++++++++++++++++++++++++++++- src/test/njs_unit_test.c | 56 +++++++++++++++++++++++++++++++++++++---------- 2 files changed, 83 insertions(+), 14 deletions(-) diffs (140 lines): diff -r 116b09a57817 -r b7c4e0f714a9 src/njs_generator.c --- a/src/njs_generator.c Tue Jun 28 22:36:38 2022 -0700 +++ b/src/njs_generator.c Tue Jun 28 23:04:00 2022 -0700 @@ -4495,6 +4495,11 @@ njs_generate_try_catch(njs_vm_t *vm, njs NJS_GENERATOR_ALL, &ctx->try_exit_label); + /* + * block can be NULL when &ctx->try_exit_label is "@return" + * for outermost try-catch block. + */ + if (block != NULL) { patch = njs_generate_make_exit_patch(vm, block, &ctx->try_exit_label, @@ -4503,6 +4508,26 @@ njs_generate_try_catch(njs_vm_t *vm, njs if (njs_slow_path(patch == NULL)) { return NJS_ERROR; } + + } else { + + /* + * when block == NULL, we still want to patch the "finally" + * instruction break_offset. + */ + + block = njs_generate_find_block(vm, generator->block, + NJS_GENERATOR_ALL, + &no_label); + + if (block != NULL) { + patch = njs_generate_make_exit_patch(vm, block, &no_label, + njs_code_offset(generator, finally) + + offsetof(njs_vmcode_finally_t, break_offset)); + if (njs_slow_path(patch == NULL)) { + return NJS_ERROR; + } + } } } } @@ -4669,8 +4694,7 @@ njs_generate_try_end(njs_vm_t *vm, njs_g * outermost try-catch block. */ block = njs_generate_find_block(vm, generator->block, - NJS_GENERATOR_ALL - | NJS_GENERATOR_TRY, dest_label); + NJS_GENERATOR_ALL, dest_label); if (block != NULL) { patch = njs_generate_make_exit_patch(vm, block, dest_label, njs_code_offset(generator, finally) @@ -4678,6 +4702,19 @@ njs_generate_try_end(njs_vm_t *vm, njs_g if (njs_slow_path(patch == NULL)) { return NJS_ERROR; } + + } else { + + block = njs_generate_find_block(vm, generator->block, + NJS_GENERATOR_ALL, &no_label); + if (block != NULL) { + patch = njs_generate_make_exit_patch(vm, block, &no_label, + njs_code_offset(generator, finally) + + offsetof(njs_vmcode_finally_t, break_offset)); + if (njs_slow_path(patch == NULL)) { + return NJS_ERROR; + } + } } } diff -r 116b09a57817 -r b7c4e0f714a9 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Tue Jun 28 22:36:38 2022 -0700 +++ b/src/test/njs_unit_test.c Tue Jun 28 23:04:00 2022 -0700 @@ -3411,20 +3411,52 @@ static njs_unit_test_t njs_test[] = njs_str("a,2") }, { njs_str("function f(n) { " - " var r1 = 0, r2 = 0, r3 = 0;" - " a:{ try { try { " - " if (n == 0) { break a; } " - " if (n == 1) { throw 'a'; } " - " } " - " catch (e) { break a; } finally { r1++; } } " - " catch (e) {} " - " finally { r2++; } " - " r3++; " - " }; " - "return [r1, r2, r3]" - "}; njs.dump([f(0), f(1), f(3)])"), + " var r1 = 0, r2 = 0, r3 = 0;" + " a:{ try { try { " + " if (n == 0) { break a; } " + " if (n == 1) { throw 'a'; } " + " } " + " catch (e) { break a; } finally { r1++; } } " + " catch (e) {} " + " finally { r2++; } " + " r3++; " + " }; " + "return [r1, r2, r3]" + "}; njs.dump([f(0), f(1), f(3)])"), njs_str("[[1,1,0],[1,1,0],[1,1,1]]") }, + + { njs_str("function f(n) {" + " while (1)" + " try {" + " if (n == 0) { break; }" + " if (n == 1) { throw 'a'; }" + "" + " try { return 42; }" + " catch (a) {}" + "" + " } catch (b) { return b; }" + "};" + "njs.dump([f(0), f(1), f(2)])"), + njs_str("[undefined,'a',42]") }, + + { njs_str("function f(n, r) {" + " while (1)" + " try {" + " if (n == 0) { break; }" + " if (n == 1) { throw 'a'; }" + "" + " try { return 42; }" + " catch (a) {}" + " finally { r.push('in');}" + "" + " } catch (b) { return b; }" + " finally { r.push('out'); }" + "};" + "function g(n) { var r = []; return [f(n, r), r]}" + "njs.dump([g(0), g(1), g(2)])"), + njs_str("[[undefined,['out']],['a',['out']],[42,['in','out']]]") }, + /**/ { njs_str("function f() { Object.prototype.toString = 1; };" From pluknet at nginx.com Wed Jun 29 12:05:23 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Wed, 29 Jun 2022 12:05:23 +0000 Subject: [nginx] Version bump. Message-ID: details: https://hg.nginx.org/nginx/rev/e210c8942a54 branches: changeset: 8052:e210c8942a54 user: Maxim Dounin date: Wed Jun 29 02:47:38 2022 +0300 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r fecd73db563f -r e210c8942a54 src/core/nginx.h --- a/src/core/nginx.h Tue Jun 21 17:25:37 2022 +0300 +++ b/src/core/nginx.h Wed Jun 29 02:47:38 2022 +0300 @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1023000 -#define NGINX_VERSION "1.23.0" +#define nginx_version 1023001 +#define NGINX_VERSION "1.23.1" #define NGINX_VER "nginx/" NGINX_VERSION #ifdef NGX_BUILD From pluknet at nginx.com Wed Jun 29 12:05:26 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Wed, 29 Jun 2022 12:05:26 +0000 Subject: [nginx] Upstream: optimized use of SSL contexts (ticket #1234). Message-ID: details: https://hg.nginx.org/nginx/rev/9d98d524bd02 branches: changeset: 8053:9d98d524bd02 user: Maxim Dounin date: Wed Jun 29 02:47:45 2022 +0300 description: Upstream: optimized use of SSL contexts (ticket #1234). To ensure optimal use of memory, SSL contexts for proxying are now inherited from previous levels as long as relevant proxy_ssl_* directives are not redefined. Further, when no proxy_ssl_* directives are redefined in a server block, we now preserve plcf->upstream.ssl in the "http" section configuration to inherit it to all servers. Similar changes made in uwsgi, grpc, and stream proxy. diffstat: src/http/modules/ngx_http_grpc_module.c | 66 +++++++++++++++++++++++++++--- src/http/modules/ngx_http_proxy_module.c | 68 ++++++++++++++++++++++++++++--- src/http/modules/ngx_http_uwsgi_module.c | 66 +++++++++++++++++++++++++++--- src/stream/ngx_stream_proxy_module.c | 64 +++++++++++++++++++++++++++-- 4 files changed, 236 insertions(+), 28 deletions(-) diffs (431 lines): diff -r e210c8942a54 -r 9d98d524bd02 src/http/modules/ngx_http_grpc_module.c --- a/src/http/modules/ngx_http_grpc_module.c Wed Jun 29 02:47:38 2022 +0300 +++ b/src/http/modules/ngx_http_grpc_module.c Wed Jun 29 02:47:45 2022 +0300 @@ -209,6 +209,8 @@ static char *ngx_http_grpc_ssl_password_ ngx_command_t *cmd, void *conf); static char *ngx_http_grpc_ssl_conf_command_check(ngx_conf_t *cf, void *post, void *data); +static ngx_int_t ngx_http_grpc_merge_ssl(ngx_conf_t *cf, + ngx_http_grpc_loc_conf_t *conf, ngx_http_grpc_loc_conf_t *prev); static ngx_int_t ngx_http_grpc_set_ssl(ngx_conf_t *cf, ngx_http_grpc_loc_conf_t *glcf); #endif @@ -562,7 +564,7 @@ ngx_http_grpc_handler(ngx_http_request_t ctx->host = glcf->host; #if (NGX_HTTP_SSL) - u->ssl = (glcf->upstream.ssl != NULL); + u->ssl = glcf->ssl; if (u->ssl) { ngx_str_set(&u->schema, "grpcs://"); @@ -4463,6 +4465,10 @@ ngx_http_grpc_merge_loc_conf(ngx_conf_t #if (NGX_HTTP_SSL) + if (ngx_http_grpc_merge_ssl(cf, conf, prev) != NGX_OK) { + return NGX_CONF_ERROR; + } + ngx_conf_merge_value(conf->upstream.ssl_session_reuse, prev->upstream.ssl_session_reuse, 1); @@ -4524,7 +4530,7 @@ ngx_http_grpc_merge_loc_conf(ngx_conf_t conf->grpc_values = prev->grpc_values; #if (NGX_HTTP_SSL) - conf->upstream.ssl = prev->upstream.ssl; + conf->ssl = prev->ssl; #endif } @@ -4874,17 +4880,63 @@ ngx_http_grpc_ssl_conf_command_check(ngx static ngx_int_t +ngx_http_grpc_merge_ssl(ngx_conf_t *cf, ngx_http_grpc_loc_conf_t *conf, + ngx_http_grpc_loc_conf_t *prev) +{ + ngx_uint_t preserve; + + if (conf->ssl_protocols == 0 + && conf->ssl_ciphers.data == NULL + && conf->upstream.ssl_certificate == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_certificate_key == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_passwords == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_verify == NGX_CONF_UNSET + && conf->ssl_verify_depth == NGX_CONF_UNSET_UINT + && conf->ssl_trusted_certificate.data == NULL + && conf->ssl_crl.data == NULL + && conf->upstream.ssl_session_reuse == NGX_CONF_UNSET + && conf->ssl_conf_commands == NGX_CONF_UNSET_PTR) + { + if (prev->upstream.ssl) { + conf->upstream.ssl = prev->upstream.ssl; + return NGX_OK; + } + + preserve = 1; + + } else { + preserve = 0; + } + + conf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); + if (conf->upstream.ssl == NULL) { + return NGX_ERROR; + } + + conf->upstream.ssl->log = cf->log; + + /* + * special handling to preserve conf->upstream.ssl + * in the "http" section to inherit it to all servers + */ + + if (preserve) { + prev->upstream.ssl = conf->upstream.ssl; + } + + return NGX_OK; +} + + +static ngx_int_t ngx_http_grpc_set_ssl(ngx_conf_t *cf, ngx_http_grpc_loc_conf_t *glcf) { ngx_pool_cleanup_t *cln; - glcf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); - if (glcf->upstream.ssl == NULL) { - return NGX_ERROR; + if (glcf->upstream.ssl->ctx) { + return NGX_OK; } - glcf->upstream.ssl->log = cf->log; - if (ngx_ssl_create(glcf->upstream.ssl, glcf->ssl_protocols, NULL) != NGX_OK) { diff -r e210c8942a54 -r 9d98d524bd02 src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c Wed Jun 29 02:47:38 2022 +0300 +++ b/src/http/modules/ngx_http_proxy_module.c Wed Jun 29 02:47:45 2022 +0300 @@ -236,6 +236,8 @@ static ngx_int_t ngx_http_proxy_rewrite_ ngx_http_proxy_rewrite_t *pr, ngx_str_t *regex, ngx_uint_t caseless); #if (NGX_HTTP_SSL) +static ngx_int_t ngx_http_proxy_merge_ssl(ngx_conf_t *cf, + ngx_http_proxy_loc_conf_t *conf, ngx_http_proxy_loc_conf_t *prev); static ngx_int_t ngx_http_proxy_set_ssl(ngx_conf_t *cf, ngx_http_proxy_loc_conf_t *plcf); #endif @@ -959,7 +961,7 @@ ngx_http_proxy_handler(ngx_http_request_ ctx->vars = plcf->vars; u->schema = plcf->vars.schema; #if (NGX_HTTP_SSL) - u->ssl = (plcf->upstream.ssl != NULL); + u->ssl = plcf->ssl; #endif } else { @@ -3724,6 +3726,10 @@ ngx_http_proxy_merge_loc_conf(ngx_conf_t #if (NGX_HTTP_SSL) + if (ngx_http_proxy_merge_ssl(cf, conf, prev) != NGX_OK) { + return NGX_CONF_ERROR; + } + ngx_conf_merge_value(conf->upstream.ssl_session_reuse, prev->upstream.ssl_session_reuse, 1); @@ -3857,7 +3863,7 @@ ngx_http_proxy_merge_loc_conf(ngx_conf_t conf->proxy_values = prev->proxy_values; #if (NGX_HTTP_SSL) - conf->upstream.ssl = prev->upstream.ssl; + conf->ssl = prev->ssl; #endif } @@ -4923,16 +4929,62 @@ ngx_http_proxy_ssl_conf_command_check(ng static ngx_int_t +ngx_http_proxy_merge_ssl(ngx_conf_t *cf, ngx_http_proxy_loc_conf_t *conf, + ngx_http_proxy_loc_conf_t *prev) +{ + ngx_uint_t preserve; + + if (conf->ssl_protocols == 0 + && conf->ssl_ciphers.data == NULL + && conf->upstream.ssl_certificate == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_certificate_key == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_passwords == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_verify == NGX_CONF_UNSET + && conf->ssl_verify_depth == NGX_CONF_UNSET_UINT + && conf->ssl_trusted_certificate.data == NULL + && conf->ssl_crl.data == NULL + && conf->upstream.ssl_session_reuse == NGX_CONF_UNSET + && conf->ssl_conf_commands == NGX_CONF_UNSET_PTR) + { + if (prev->upstream.ssl) { + conf->upstream.ssl = prev->upstream.ssl; + return NGX_OK; + } + + preserve = 1; + + } else { + preserve = 0; + } + + conf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); + if (conf->upstream.ssl == NULL) { + return NGX_ERROR; + } + + conf->upstream.ssl->log = cf->log; + + /* + * special handling to preserve conf->upstream.ssl + * in the "http" section to inherit it to all servers + */ + + if (preserve) { + prev->upstream.ssl = conf->upstream.ssl; + } + + return NGX_OK; +} + + +static ngx_int_t ngx_http_proxy_set_ssl(ngx_conf_t *cf, ngx_http_proxy_loc_conf_t *plcf) { ngx_pool_cleanup_t *cln; - plcf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); - if (plcf->upstream.ssl == NULL) { - return NGX_ERROR; - } - - plcf->upstream.ssl->log = cf->log; + if (plcf->upstream.ssl->ctx) { + return NGX_OK; + } if (ngx_ssl_create(plcf->upstream.ssl, plcf->ssl_protocols, NULL) != NGX_OK) diff -r e210c8942a54 -r 9d98d524bd02 src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c Wed Jun 29 02:47:38 2022 +0300 +++ b/src/http/modules/ngx_http_uwsgi_module.c Wed Jun 29 02:47:45 2022 +0300 @@ -96,6 +96,8 @@ static char *ngx_http_uwsgi_ssl_password ngx_command_t *cmd, void *conf); static char *ngx_http_uwsgi_ssl_conf_command_check(ngx_conf_t *cf, void *post, void *data); +static ngx_int_t ngx_http_uwsgi_merge_ssl(ngx_conf_t *cf, + ngx_http_uwsgi_loc_conf_t *conf, ngx_http_uwsgi_loc_conf_t *prev); static ngx_int_t ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, ngx_http_uwsgi_loc_conf_t *uwcf); #endif @@ -668,7 +670,7 @@ ngx_http_uwsgi_handler(ngx_http_request_ if (uwcf->uwsgi_lengths == NULL) { #if (NGX_HTTP_SSL) - u->ssl = (uwcf->upstream.ssl != NULL); + u->ssl = uwcf->ssl; if (u->ssl) { ngx_str_set(&u->schema, "suwsgi://"); @@ -1865,6 +1867,10 @@ ngx_http_uwsgi_merge_loc_conf(ngx_conf_t #if (NGX_HTTP_SSL) + if (ngx_http_uwsgi_merge_ssl(cf, conf, prev) != NGX_OK) { + return NGX_CONF_ERROR; + } + ngx_conf_merge_value(conf->upstream.ssl_session_reuse, prev->upstream.ssl_session_reuse, 1); @@ -1927,7 +1933,7 @@ ngx_http_uwsgi_merge_loc_conf(ngx_conf_t conf->uwsgi_values = prev->uwsgi_values; #if (NGX_HTTP_SSL) - conf->upstream.ssl = prev->upstream.ssl; + conf->ssl = prev->ssl; #endif } @@ -2455,17 +2461,63 @@ ngx_http_uwsgi_ssl_conf_command_check(ng static ngx_int_t +ngx_http_uwsgi_merge_ssl(ngx_conf_t *cf, ngx_http_uwsgi_loc_conf_t *conf, + ngx_http_uwsgi_loc_conf_t *prev) +{ + ngx_uint_t preserve; + + if (conf->ssl_protocols == 0 + && conf->ssl_ciphers.data == NULL + && conf->upstream.ssl_certificate == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_certificate_key == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_passwords == NGX_CONF_UNSET_PTR + && conf->upstream.ssl_verify == NGX_CONF_UNSET + && conf->ssl_verify_depth == NGX_CONF_UNSET_UINT + && conf->ssl_trusted_certificate.data == NULL + && conf->ssl_crl.data == NULL + && conf->upstream.ssl_session_reuse == NGX_CONF_UNSET + && conf->ssl_conf_commands == NGX_CONF_UNSET_PTR) + { + if (prev->upstream.ssl) { + conf->upstream.ssl = prev->upstream.ssl; + return NGX_OK; + } + + preserve = 1; + + } else { + preserve = 0; + } + + conf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); + if (conf->upstream.ssl == NULL) { + return NGX_ERROR; + } + + conf->upstream.ssl->log = cf->log; + + /* + * special handling to preserve conf->upstream.ssl + * in the "http" section to inherit it to all servers + */ + + if (preserve) { + prev->upstream.ssl = conf->upstream.ssl; + } + + return NGX_OK; +} + + +static ngx_int_t ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, ngx_http_uwsgi_loc_conf_t *uwcf) { ngx_pool_cleanup_t *cln; - uwcf->upstream.ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); - if (uwcf->upstream.ssl == NULL) { - return NGX_ERROR; + if (uwcf->upstream.ssl->ctx) { + return NGX_OK; } - uwcf->upstream.ssl->log = cf->log; - if (ngx_ssl_create(uwcf->upstream.ssl, uwcf->ssl_protocols, NULL) != NGX_OK) { diff -r e210c8942a54 -r 9d98d524bd02 src/stream/ngx_stream_proxy_module.c --- a/src/stream/ngx_stream_proxy_module.c Wed Jun 29 02:47:38 2022 +0300 +++ b/src/stream/ngx_stream_proxy_module.c Wed Jun 29 02:47:45 2022 +0300 @@ -103,6 +103,8 @@ static void ngx_stream_proxy_ssl_handsha static void ngx_stream_proxy_ssl_save_session(ngx_connection_t *c); static ngx_int_t ngx_stream_proxy_ssl_name(ngx_stream_session_t *s); static ngx_int_t ngx_stream_proxy_ssl_certificate(ngx_stream_session_t *s); +static ngx_int_t ngx_stream_proxy_merge_ssl(ngx_conf_t *cf, + ngx_stream_proxy_srv_conf_t *conf, ngx_stream_proxy_srv_conf_t *prev); static ngx_int_t ngx_stream_proxy_set_ssl(ngx_conf_t *cf, ngx_stream_proxy_srv_conf_t *pscf); @@ -801,7 +803,7 @@ ngx_stream_proxy_init_upstream(ngx_strea #if (NGX_STREAM_SSL) - if (pc->type == SOCK_STREAM && pscf->ssl) { + if (pc->type == SOCK_STREAM && pscf->ssl_enable) { if (u->proxy_protocol) { if (ngx_stream_proxy_send_proxy_protocol(s) != NGX_OK) { @@ -2150,6 +2152,10 @@ ngx_stream_proxy_merge_srv_conf(ngx_conf #if (NGX_STREAM_SSL) + if (ngx_stream_proxy_merge_ssl(cf, conf, prev) != NGX_OK) { + return NGX_CONF_ERROR; + } + ngx_conf_merge_value(conf->ssl_enable, prev->ssl_enable, 0); ngx_conf_merge_value(conf->ssl_session_reuse, @@ -2199,17 +2205,63 @@ ngx_stream_proxy_merge_srv_conf(ngx_conf #if (NGX_STREAM_SSL) static ngx_int_t +ngx_stream_proxy_merge_ssl(ngx_conf_t *cf, ngx_stream_proxy_srv_conf_t *conf, + ngx_stream_proxy_srv_conf_t *prev) +{ + ngx_uint_t preserve; + + if (conf->ssl_protocols == 0 + && conf->ssl_ciphers.data == NULL + && conf->ssl_certificate == NGX_CONF_UNSET_PTR + && conf->ssl_certificate_key == NGX_CONF_UNSET_PTR + && conf->ssl_passwords == NGX_CONF_UNSET_PTR + && conf->ssl_verify == NGX_CONF_UNSET + && conf->ssl_verify_depth == NGX_CONF_UNSET_UINT + && conf->ssl_trusted_certificate.data == NULL + && conf->ssl_crl.data == NULL + && conf->ssl_session_reuse == NGX_CONF_UNSET + && conf->ssl_conf_commands == NGX_CONF_UNSET_PTR) + { + if (prev->ssl) { + conf->ssl = prev->ssl; + return NGX_OK; + } + + preserve = 1; + + } else { + preserve = 0; + } + + conf->ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); + if (conf->ssl == NULL) { + return NGX_ERROR; + } + + conf->ssl->log = cf->log; + + /* + * special handling to preserve conf->ssl + * in the "stream" section to inherit it to all servers + */ + + if (preserve) { + prev->ssl = conf->ssl; + } + + return NGX_OK; +} + + +static ngx_int_t ngx_stream_proxy_set_ssl(ngx_conf_t *cf, ngx_stream_proxy_srv_conf_t *pscf) { ngx_pool_cleanup_t *cln; - pscf->ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); - if (pscf->ssl == NULL) { - return NGX_ERROR; + if (pscf->ssl->ctx) { + return NGX_OK; } - pscf->ssl->log = cf->log; - if (ngx_ssl_create(pscf->ssl, pscf->ssl_protocols, NULL) != NGX_OK) { return NGX_ERROR; } From pluknet at nginx.com Thu Jun 30 16:13:55 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Thu, 30 Jun 2022 20:13:55 +0400 Subject: [PATCH] The "sort=" parameter of the "resolver" directive In-Reply-To: References: <20220223040934.GD18027@lo0.su> <8db4bbd67840e8bebb23.1645589485@1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa> <20220628122458.ufx7ou3zpvquu4q7@N00W24XTQX> Message-ID: <4D2F36E6-06EF-4535-A667-220BDDDA7950@nginx.com> > On 29 Jun 2022, at 04:51, Maxim Dounin wrote: > > Hello! > > On Tue, Jun 28, 2022 at 04:24:58PM +0400, Roman Arutyunyan wrote: > >> On Thu, Jun 16, 2022 at 07:17:13PM +0400, Sergey Kandaurov wrote: >>> >>>> On 23 Feb 2022, at 08:11, Ruslan Ermilov wrote: >>>> >>>> src/core/ngx_resolver.c | 38 +++++++++++++++++++++++++++++++++++++- >>>> src/core/ngx_resolver.h | 5 +++++ >>>> 2 files changed, 42 insertions(+), 1 deletions(-) >>>> >>>> >>>> # HG changeset patch >>>> # User Ruslan Ermilov >>>> # Date 1645589387 -10800 >>>> # Wed Feb 23 07:09:47 2022 +0300 >>>> # Node ID 8db4bbd67840e8bebb23f9c6d10c0f633552e616 >>>> # Parent 1c19779448db2309d607c74e2628ff98f84569ff >>>> The "sort=" parameter of the "resolver" directive. >>>> >>> >>> IMHO, the name isn't enough self-documenting in this context. >>> It can be misinterpreted as sorting addresses, not families. >>> I'd prefer the parameter name "prefer" here. >> >> Jftr, we had a discussion about this. I like the new name. There's no >> sorting here. > > IIRC, the "sort=" option was expected to be a simplified version > of the sortlist option of resolv.conf. It was intentionally made > extendable, so potentially we can support all the sortlist > features if needed. Ruslan might remember details, make sure to > discuss this with him. As discussed with Ruslan, and if I got him right, the "sort" name has a weak relation to sortlist, the intension is to sort address families, including future ones (which is rather theoretical). In principle, the syntax can be extended to arbitrary address prefixes. As additionally discussed, sortlist is substituted with precedence based lookup table in RFC 3484 (now RFC 6724) for default address selection. In gethostbyname() sortlist is used for query type A only, and in getaddrinfo() it is the same limitation "for backward compatibility against gethostbyname(3)". On the other hand, address selection policy based reordering is applied for all address families in getaddrinfo(). (This is based on my research of KAME/BSD sources.) The "prefer" name is actually inspired by RFC 3484 and related terminology. -- Sergey Kandaurov