+ rev="77">

@@ -88,11 +88,6 @@ versions: -21.10 “impish” -x86_64, aarch64/arm64 - - - 22.04 “jammy” x86_64, aarch64/arm64, s390x diff -r e06cf66a9f63 -r ca4adc1068f0 xml/ru/linux_packages.xml --- a/xml/ru/linux_packages.xml Tue Jul 19 14:10:02 2022 +0100 +++ b/xml/ru/linux_packages.xml Tue Jul 19 19:28:08 2022 +0400 @@ -7,7 +7,7 @@

+ rev="77">

@@ -88,11 +88,6 @@ -21.10 “impish” -x86_64, aarch64/arm64 - - - 22.04 “jammy” x86_64, aarch64/arm64, s390x From pluknet at nginx.com Tue Jul 19 15:45:57 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 19 Jul 2022 19:45:57 +0400 Subject: [PATCH] Linux packages: removed Ubuntu 21.10 'impish' due to EOL In-Reply-To: References: Message-ID: <20220719154557.ymhissljbnaqlxs3@Y9MQ9X2QVV> On Tue, Jul 19, 2022 at 07:28:25PM +0400, Konstantin Pavlov wrote: > # HG changeset patch > # User Konstantin Pavlov > # Date 1658244488 -14400 > # Tue Jul 19 19:28:08 2022 +0400 > # Node ID ca4adc1068f0ba18c477f9816ce2b798f675fbe0 > # Parent e06cf66a9f630d376699be0fd78b9fc64ef6256e > Linux packages: removed Ubuntu 21.10 'impish' due to EOL. Looks good. From pluknet at nginx.com Wed Jul 20 13:58:26 2022 From: pluknet at nginx.com (=?iso-8859-1?q?Sergey_Kandaurov?=) Date: Wed, 20 Jul 2022 17:58:26 +0400 Subject: [PATCH] SSL: consistent certificate verification depth Message-ID: <5a9cc2a846c9ea4c0af0.1658325506@enoparse.local> # HG changeset patch # User Sergey Kandaurov # Date 1658325446 -14400 # Wed Jul 20 17:57:26 2022 +0400 # Node ID 5a9cc2a846c9ea4c0af03109ab186af1ac28e222 # Parent 069a4813e8d6d7ec662d282a10f5f7062ebd817f SSL: consistent certificate verification depth. Originally, certificate verification depth was used to limit the number of signatures to validate, that is, to limit chains with intermediate certificates one less. The semantics was changed in OpenSSL 1.1.0, and instead it limits now the number of intermediate certificates allowed. This makes it not possible to limit certificate checking to self-signed certificates with verify depth 0 in OpenSSL 1.1.0+, and is inconsistent compared with former behaviour in BoringSSL and older OpenSSL versions. This change restores former verification logic when using OpenSSL 1.1.0+. The verify callback is adjusted to emit the "certificate chain too long" error when the certificate chain exceeds the verification depth. It has no effect to other SSL libraries, where this is limited by other means. Also, this fixes verification checks when using LibreSSL 3.4.0+, where a chain depth is not limited except by X509_VERIFY_MAX_CHAIN_CERTS (32). diff -r 069a4813e8d6 -r 5a9cc2a846c9 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Tue Jul 19 17:05:27 2022 +0300 +++ b/src/event/ngx_event_openssl.c Wed Jul 20 17:57:26 2022 +0400 @@ -997,16 +997,26 @@ ngx_ssl_crl(ngx_conf_t *cf, ngx_ssl_t *s static int ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) { + int depth, verify_depth; + ngx_ssl_conn_t *ssl_conn; + + ssl_conn = X509_STORE_CTX_get_ex_data(x509_store, + SSL_get_ex_data_X509_STORE_CTX_idx()); + + depth = X509_STORE_CTX_get_error_depth(x509_store); + verify_depth = SSL_CTX_get_verify_depth(SSL_get_SSL_CTX(ssl_conn)); + + if (depth > verify_depth) { + X509_STORE_CTX_set_error(x509_store, X509_V_ERR_CERT_CHAIN_TOO_LONG); + } + #if (NGX_DEBUG) + + int err; char *subject, *issuer; - int err, depth; X509 *cert; X509_NAME *sname, *iname; ngx_connection_t *c; - ngx_ssl_conn_t *ssl_conn; - - ssl_conn = X509_STORE_CTX_get_ex_data(x509_store, - SSL_get_ex_data_X509_STORE_CTX_idx()); c = ngx_ssl_get_connection(ssl_conn); @@ -1016,7 +1026,6 @@ ngx_ssl_verify_callback(int ok, X509_STO cert = X509_STORE_CTX_get_current_cert(x509_store); err = X509_STORE_CTX_get_error(x509_store); - depth = X509_STORE_CTX_get_error_depth(x509_store); sname = X509_get_subject_name(cert); @@ -1058,6 +1067,7 @@ ngx_ssl_verify_callback(int ok, X509_STO if (issuer) { OPENSSL_free(issuer); } + #endif return 1; From mdounin at mdounin.ru Wed Jul 20 18:04:52 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 20 Jul 2022 21:04:52 +0300 Subject: [PATCH] SSL: consistent certificate verification depth In-Reply-To: <5a9cc2a846c9ea4c0af0.1658325506@enoparse.local> References: <5a9cc2a846c9ea4c0af0.1658325506@enoparse.local> Message-ID: Hello! On Wed, Jul 20, 2022 at 05:58:26PM +0400, Sergey Kandaurov wrote: > # HG changeset patch > # User Sergey Kandaurov > # Date 1658325446 -14400 > # Wed Jul 20 17:57:26 2022 +0400 > # Node ID 5a9cc2a846c9ea4c0af03109ab186af1ac28e222 > # Parent 069a4813e8d6d7ec662d282a10f5f7062ebd817f > SSL: consistent certificate verification depth. > > Originally, certificate verification depth was used to limit the number > of signatures to validate, that is, to limit chains with intermediate > certificates one less. The semantics was changed in OpenSSL 1.1.0, and > instead it limits now the number of intermediate certificates allowed. > This makes it not possible to limit certificate checking to self-signed > certificates with verify depth 0 in OpenSSL 1.1.0+, and is inconsistent > compared with former behaviour in BoringSSL and older OpenSSL versions. > > This change restores former verification logic when using OpenSSL 1.1.0+. > The verify callback is adjusted to emit the "certificate chain too long" > error when the certificate chain exceeds the verification depth. It has > no effect to other SSL libraries, where this is limited by other means. > Also, this fixes verification checks when using LibreSSL 3.4.0+, where > a chain depth is not limited except by X509_VERIFY_MAX_CHAIN_CERTS (32). This (highly questionable) OpenSSL behaviour seems to be status quo for a while, also recorded in tests (see ssl_verify_depth.t). Any specific reasons for the nginx-side workaround? > > diff -r 069a4813e8d6 -r 5a9cc2a846c9 src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c Tue Jul 19 17:05:27 2022 +0300 > +++ b/src/event/ngx_event_openssl.c Wed Jul 20 17:57:26 2022 +0400 > @@ -997,16 +997,26 @@ ngx_ssl_crl(ngx_conf_t *cf, ngx_ssl_t *s > static int > ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) > { > + int depth, verify_depth; > + ngx_ssl_conn_t *ssl_conn; > + > + ssl_conn = X509_STORE_CTX_get_ex_data(x509_store, > + SSL_get_ex_data_X509_STORE_CTX_idx()); > + > + depth = X509_STORE_CTX_get_error_depth(x509_store); > + verify_depth = SSL_CTX_get_verify_depth(SSL_get_SSL_CTX(ssl_conn)); s/verify_depth/limit/? Also, using SSL_get_verify_depth() instead of going through SSL context might be easier. > + > + if (depth > verify_depth) { > + X509_STORE_CTX_set_error(x509_store, X509_V_ERR_CERT_CHAIN_TOO_LONG); This is going to overwrite earlier errors, if any. Does not look like a good idea. > + } > + > #if (NGX_DEBUG) > + > + int err; This is not going to work, variables have to be defined at the start of a block unless you assume C99. At least MSVC 2010 won't be able to compile this. > char *subject, *issuer; > - int err, depth; > X509 *cert; > X509_NAME *sname, *iname; > ngx_connection_t *c; > - ngx_ssl_conn_t *ssl_conn; > - > - ssl_conn = X509_STORE_CTX_get_ex_data(x509_store, > - SSL_get_ex_data_X509_STORE_CTX_idx()); > > c = ngx_ssl_get_connection(ssl_conn); > > @@ -1016,7 +1026,6 @@ ngx_ssl_verify_callback(int ok, X509_STO > > cert = X509_STORE_CTX_get_current_cert(x509_store); > err = X509_STORE_CTX_get_error(x509_store); > - depth = X509_STORE_CTX_get_error_depth(x509_store); > > sname = X509_get_subject_name(cert); > > @@ -1058,6 +1067,7 @@ ngx_ssl_verify_callback(int ok, X509_STO > if (issuer) { > OPENSSL_free(issuer); > } > + > #endif > > return 1; > -- Maxim Dounin http://mdounin.ru/ From V.Kokshenev at F5.com Wed Jul 20 19:08:40 2022 From: V.Kokshenev at F5.com (Vladimir Kokshenev) Date: Wed, 20 Jul 2022 19:08:40 +0000 Subject: Open-sourcing periodic upstream server resolution and implementing a dedicated service worker. Message-ID: <9E5B3F41-2CAD-47ED-A2EC-EE807DC54A18@f5.com> Hello! This is the two-part proposal to open-source periodic upstream server resolution and implement a dedicated service worker for nginx. The purpose of this e-mail is to describe the WHY and solicit feedback. Nginx supports domain names in the upstream server configuration. Currently, domain names are resolved at configuration time only, and there are no subsequent name resolutions. There are plans to open-source re-resolvable upstream servers. This will allow applying DNS updates to upstream configurations in runtime. So, there is a need to support periodic asynchronous operations. And a dedicated service worker is a possible architectural way to address this. The master process reads and parses configuration and creates the service worker when needed (in a similar way to cache-related processes). The service worker manages periodic name resolutions and updates corresponding upstream configurations. The name resolution relies on the existing nginx resolver and upstream zone functionality. The service worker will be responsible solely for periodic background tasks and wouldn't accept client connections. The service worker should be the last worker process to shut down to maintain the actual state of upstreams when there are active workers. Alternative architecture considered was about choosing one of the regular workers (e.g., worker zero) to take care of periodic upstream server resolution, but it creates asymmetry in responsibilities and load for this dedicated worker. -- Vladimir Kokshenev From mdounin at mdounin.ru Wed Jul 20 20:36:03 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 20 Jul 2022 23:36:03 +0300 Subject: Open-sourcing periodic upstream server resolution and implementing a dedicated service worker. In-Reply-To: <9E5B3F41-2CAD-47ED-A2EC-EE807DC54A18@f5.com> References: <9E5B3F41-2CAD-47ED-A2EC-EE807DC54A18@f5.com> Message-ID: Hello! On Wed, Jul 20, 2022 at 07:08:40PM +0000, Vladimir Kokshenev via nginx-devel wrote: > Hello! > > This is the two-part proposal to open-source periodic upstream server resolution > and implement a dedicated service worker for nginx. The purpose of this e-mail > is to describe the WHY and solicit feedback. > > Nginx supports domain names in the upstream server configuration. > Currently, domain names are resolved at configuration time only, > and there are no subsequent name resolutions. > > There are plans to open-source re-resolvable upstream servers. > This will allow applying DNS updates to upstream configurations in runtime. > So, there is a need to support periodic asynchronous operations. > And a dedicated service worker is a possible architectural way to address this. > > The master process reads and parses configuration and creates the service worker > when needed (in a similar way to cache-related processes). > > The service worker manages periodic name resolutions and updates corresponding > upstream configurations. The name resolution relies on the existing nginx > resolver and upstream zone functionality. > > The service worker will be responsible solely for periodic background tasks > and wouldn't accept client connections. > > The service worker should be the last worker process to shut down > to maintain the actual state of upstreams when there are active workers. > > Alternative architecture considered was about choosing one of the regular > workers (e.g., worker zero) to take care of periodic upstream server resolution, > but it creates asymmetry in responsibilities and load for this dedicated worker. Both alternatives look bad to me. We already have dedicated processes to load and manage caches, and I tend to think that the only thing which somehow justifies these being dedicated is that cache management implies disk-intensive blocking operations. Mixing these with normal request processing will cause latency issues, not to mention will be non-trivial to implement. On the other hand, we are already seeing issues with dedicated process being used: in some configurations just one cache manager process simply isn't enough to remove all the files being add by many worker processes. As such, I generally tend to think that dedicated processes is a wrong way to go. With any special requirements like "last to shut down" the whole idea becomes even worse. And in this particular case all operations are perfectly asynchronous, so there is no justification like in the cache manager case. Similarly, "choosing one of the regular workers (e.g., worker zero)" looks wrong (and I've already provided this feedback previously). All workers are expected to be equal, and doing something only in a particular worker is expected to cause issues. E.g., consider a worker is stopped (due to a bug or intentionally to debug an issue) - this shouldn't disrupt operations of other workers. Rather, I would recommend focusing on doing all periodic tasks in a way which doesn't depend on being run in the particular worker. The simplest approach would be to run tasks in all worker processes, with some minimal checks to avoid duplicate work. -- Maxim Dounin http://mdounin.ru/ From xeioex at nginx.com Thu Jul 21 02:58:35 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 21 Jul 2022 02:58:35 +0000 Subject: [njs] Version bump. Message-ID: details: https://hg.nginx.org/njs/rev/d7a0a16d22e4 branches: changeset: 1914:d7a0a16d22e4 user: Dmitry Volyntsev date: Wed Jul 20 19:56:42 2022 -0700 description: Version bump. diffstat: src/njs.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r 61d5b54b2026 -r d7a0a16d22e4 src/njs.h --- a/src/njs.h Mon Jul 18 18:42:28 2022 -0700 +++ b/src/njs.h Wed Jul 20 19:56:42 2022 -0700 @@ -11,8 +11,8 @@ #include -#define NJS_VERSION "0.7.6" -#define NJS_VERSION_NUMBER 0x000706 +#define NJS_VERSION "0.7.7" +#define NJS_VERSION_NUMBER 0x000707 #include /* STDOUT_FILENO, STDERR_FILENO */ From xeioex at nginx.com Thu Jul 21 02:58:37 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 21 Jul 2022 02:58:37 +0000 Subject: [njs] Modules: fixed reading response body in fetch API. Message-ID: details: https://hg.nginx.org/njs/rev/c597ee3450ab branches: changeset: 1915:c597ee3450ab user: Dmitry Volyntsev date: Wed Jul 20 17:49:00 2022 -0700 description: Modules: fixed reading response body in fetch API. Previously, the response body was ignored if the Content-Length was missing. This closes #557 issue on Github. diffstat: nginx/ngx_js_fetch.c | 49 ++++++++++++++++++++++++++++++++----------------- 1 files changed, 32 insertions(+), 17 deletions(-) diffs (90 lines): diff -r d7a0a16d22e4 -r c597ee3450ab nginx/ngx_js_fetch.c --- a/nginx/ngx_js_fetch.c Wed Jul 20 19:56:42 2022 -0700 +++ b/nginx/ngx_js_fetch.c Wed Jul 20 17:49:00 2022 -0700 @@ -623,6 +623,8 @@ ngx_js_http_alloc(njs_vm_t *vm, ngx_pool http->timeout = 10000; + http->http_parse.content_length_n = -1; + ret = njs_vm_promise_create(vm, njs_value_arg(&http->promise), njs_value_arg(&http->promise_callbacks)); if (ret != NJS_OK) { @@ -1388,7 +1390,7 @@ ngx_js_http_process_headers(ngx_js_http_ static ngx_int_t ngx_js_http_process_body(ngx_js_http_t *http) { - ssize_t size, need; + ssize_t size, chsize, need; ngx_int_t rc; njs_int_t ret; ngx_buf_t *b; @@ -1403,7 +1405,16 @@ ngx_js_http_process_body(ngx_js_http_t * return NGX_ERROR; } - if (size == http->http_parse.content_length_n) { + if (http->http_parse.chunked + && http->http_parse.content_length_n == -1) + { + ngx_js_http_error(http, 0, "invalid fetch chunked response"); + return NGX_ERROR; + } + + if (http->http_parse.content_length_n == -1 + || size == http->http_parse.content_length_n) + { ret = njs_vm_external_create(http->vm, njs_value_arg(&http->reply), ngx_http_js_fetch_proto_id, http, 0); if (ret != NJS_OK) { @@ -1415,13 +1426,6 @@ ngx_js_http_process_body(ngx_js_http_t * return NGX_DONE; } - if (http->http_parse.chunked - && http->http_parse.content_length_n == 0) - { - ngx_js_http_error(http, 0, "invalid fetch chunked response"); - return NGX_ERROR; - } - if (size < http->http_parse.content_length_n) { return NGX_AGAIN; } @@ -1454,17 +1458,28 @@ ngx_js_http_process_body(ngx_js_http_t * b->pos = http->http_chunk_parse.pos; } else { - need = http->http_parse.content_length_n - njs_chb_size(&http->chain); - size = ngx_min(need, b->last - b->pos); - - if (size > 0) { - njs_chb_append(&http->chain, b->pos, size); - b->pos += size; - rc = NGX_AGAIN; + size = njs_chb_size(&http->chain); + + if (http->http_parse.content_length_n == -1) { + need = http->max_response_body_size - size; } else { - rc = NGX_DONE; + need = http->http_parse.content_length_n - size; } + + chsize = ngx_min(need, b->last - b->pos); + + if (size + chsize > http->max_response_body_size) { + ngx_js_http_error(http, 0, "fetch response body is too large"); + return NGX_ERROR; + } + + if (chsize > 0) { + njs_chb_append(&http->chain, b->pos, chsize); + b->pos += chsize; + } + + rc = (need > chsize) ? NGX_AGAIN : NGX_DONE; } if (b->pos == b->end) { From Neil.Craig at bbc.co.uk Thu Jul 21 13:01:24 2022 From: Neil.Craig at bbc.co.uk (Neil Craig) Date: Thu, 21 Jul 2022 13:01:24 +0000 Subject: Open-sourcing periodic upstream server resolution and implementing a dedicated service worker. In-Reply-To: <9E5B3F41-2CAD-47ED-A2EC-EE807DC54A18@f5.com> References: <9E5B3F41-2CAD-47ED-A2EC-EE807DC54A18@f5.com> Message-ID: <0D24ED91-EED7-471C-941B-4F29B37D351D@bbc.co.uk> Hi Vladimir I hope you don't mind me chipping in here. We currently have 2 services which use Nginx with upstreams whose DNS can change frequently (e.g. AWS S3, ELB/ALB origins) so we're familiar with this problem. Initially we did a config reload every minute to solve the issue but since then, we've created custom Lua (using the OpenResty Lua module) to achieve the same thing in a more efficient way. What you're proposing sounds great to me. I wanted to suggest a few feature ideas for your consideration: * DNS TTL-based refresh (rather than refreshing every N seconds) * DNS prefresh (update DNS when the elapsed time since the last refresh is perhaps 90% of the TTL) * IPv4/6 ignore (useful for example when there's a v6 IP available but the system/network doesn't support v6) Cheers Neil Craig (He/Him) Lead Architect, BBC Digital Distribution https://www.bbc.co.uk/blogs/internet/authors/1633673f-c77c-4bc6-b100-0664db0db613 https://twitter.com/tdp_org On 20/07/2022, 20:09, "Vladimir Kokshenev via nginx-devel" wrote: Hello! This is the two-part proposal to open-source periodic upstream server resolution and implement a dedicated service worker for nginx. The purpose of this e-mail is to describe the WHY and solicit feedback. Nginx supports domain names in the upstream server configuration. Currently, domain names are resolved at configuration time only, and there are no subsequent name resolutions. There are plans to open-source re-resolvable upstream servers. This will allow applying DNS updates to upstream configurations in runtime. So, there is a need to support periodic asynchronous operations. And a dedicated service worker is a possible architectural way to address this. The master process reads and parses configuration and creates the service worker when needed (in a similar way to cache-related processes). The service worker manages periodic name resolutions and updates corresponding upstream configurations. The name resolution relies on the existing nginx resolver and upstream zone functionality. The service worker will be responsible solely for periodic background tasks and wouldn't accept client connections. The service worker should be the last worker process to shut down to maintain the actual state of upstreams when there are active workers. Alternative architecture considered was about choosing one of the regular workers (e.g., worker zero) to take care of periodic upstream server resolution, but it creates asymmetry in responsibilities and load for this dedicated worker. -- Vladimir Kokshenev _______________________________________________ nginx-devel mailing list -- nginx-devel at nginx.org To unsubscribe send an email to nginx-devel-leave at nginx.org From pluknet at nginx.com Thu Jul 21 14:18:24 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Thu, 21 Jul 2022 18:18:24 +0400 Subject: [PATCH] SSL: consistent certificate verification depth In-Reply-To: References: <5a9cc2a846c9ea4c0af0.1658325506@enoparse.local> Message-ID: > On 20 Jul 2022, at 22:04, Maxim Dounin wrote: > > Hello! > > On Wed, Jul 20, 2022 at 05:58:26PM +0400, Sergey Kandaurov wrote: > >> # HG changeset patch >> # User Sergey Kandaurov >> # Date 1658325446 -14400 >> # Wed Jul 20 17:57:26 2022 +0400 >> # Node ID 5a9cc2a846c9ea4c0af03109ab186af1ac28e222 >> # Parent 069a4813e8d6d7ec662d282a10f5f7062ebd817f >> SSL: consistent certificate verification depth. >> >> Originally, certificate verification depth was used to limit the number >> of signatures to validate, that is, to limit chains with intermediate >> certificates one less. The semantics was changed in OpenSSL 1.1.0, and >> instead it limits now the number of intermediate certificates allowed. >> This makes it not possible to limit certificate checking to self-signed >> certificates with verify depth 0 in OpenSSL 1.1.0+, and is inconsistent >> compared with former behaviour in BoringSSL and older OpenSSL versions. >> >> This change restores former verification logic when using OpenSSL 1.1.0+. >> The verify callback is adjusted to emit the "certificate chain too long" >> error when the certificate chain exceeds the verification depth. It has >> no effect to other SSL libraries, where this is limited by other means. >> Also, this fixes verification checks when using LibreSSL 3.4.0+, where >> a chain depth is not limited except by X509_VERIFY_MAX_CHAIN_CERTS (32). > > This (highly questionable) OpenSSL behaviour seems to be status > quo for a while, also recorded in tests (see ssl_verify_depth.t). > Any specific reasons for the nginx-side workaround? As explained in the commit message, main motivation is to eliminate annoying difference in behaviour among various SSL libraries (aside from working around the arguably broken LibreSSL verifier). Nothing specific behind that. Disambiguating ssl_verify_depth.t is a good demo of net result. The downside is that this can potentially break previously working configurations when using with modern OpenSSL versions. So the patch is to seek feedback on whether this makes sense. > >> >> diff -r 069a4813e8d6 -r 5a9cc2a846c9 src/event/ngx_event_openssl.c >> --- a/src/event/ngx_event_openssl.c Tue Jul 19 17:05:27 2022 +0300 >> +++ b/src/event/ngx_event_openssl.c Wed Jul 20 17:57:26 2022 +0400 >> @@ -997,16 +997,26 @@ ngx_ssl_crl(ngx_conf_t *cf, ngx_ssl_t *s >> static int >> ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) >> { >> + int depth, verify_depth; >> + ngx_ssl_conn_t *ssl_conn; >> + >> + ssl_conn = X509_STORE_CTX_get_ex_data(x509_store, >> + SSL_get_ex_data_X509_STORE_CTX_idx()); >> + >> + depth = X509_STORE_CTX_get_error_depth(x509_store); >> + verify_depth = SSL_CTX_get_verify_depth(SSL_get_SSL_CTX(ssl_conn)); > > s/verify_depth/limit/? > > Also, using SSL_get_verify_depth() instead of going through SSL > context might be easier. Fixed, thanks. Some remnants left after r&d. > >> + >> + if (depth > verify_depth) { >> + X509_STORE_CTX_set_error(x509_store, X509_V_ERR_CERT_CHAIN_TOO_LONG); > > This is going to overwrite earlier errors, if any. Does not look like a good > idea. Agree, it can actually - not what I want to do within this change. Also, refined commit message. > >> + } >> + >> #if (NGX_DEBUG) >> + >> + int err; > > This is not going to work, variables have to be defined at the > start of a block unless you assume C99. At least MSVC 2010 won't > be able to compile this. Fixed, thanks. # HG changeset patch # User Sergey Kandaurov # Date 1658412531 -14400 # Thu Jul 21 18:08:51 2022 +0400 # Node ID d064df4a9eb7f74f328d01337e7743adb6468dd5 # Parent 069a4813e8d6d7ec662d282a10f5f7062ebd817f SSL: consistent certificate verification depth. Originally, certificate verification depth was used to limit the number of signatures to validate, that is, to limit chains with intermediate certificates one less. The semantics was changed in OpenSSL 1.1.0, and instead it limits now the number of intermediate certificates allowed. This makes it not possible to limit certificate checking to self-signed certificates with verify depth 0 in OpenSSL 1.1.0+, and is inconsistent compared with BoringSSL and former behaviour in older OpenSSL versions. This change restores former verification logic when using OpenSSL 1.1.0+. The verify callback is adjusted to set the "certificate chain too long" error if the certificate chain exceeds the verification depth and the error wasn't previously set by other means. Also, this fixes verification checks when using LibreSSL 3.4.0+, where a chain depth is not limited except by X509_VERIFY_MAX_CHAIN_CERTS (32). diff -r 069a4813e8d6 -r d064df4a9eb7 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Tue Jul 19 17:05:27 2022 +0300 +++ b/src/event/ngx_event_openssl.c Thu Jul 21 18:08:51 2022 +0400 @@ -997,16 +997,28 @@ ngx_ssl_crl(ngx_conf_t *cf, ngx_ssl_t *s static int ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) { + int err, depth, limit; + ngx_ssl_conn_t *ssl_conn; + + ssl_conn = X509_STORE_CTX_get_ex_data(x509_store, + SSL_get_ex_data_X509_STORE_CTX_idx()); + + err = X509_STORE_CTX_get_error(x509_store); + depth = X509_STORE_CTX_get_error_depth(x509_store); + + limit = SSL_get_verify_depth(ssl_conn); + + if (depth > limit && err == X509_V_OK) { + err = X509_V_ERR_CERT_CHAIN_TOO_LONG; + X509_STORE_CTX_set_error(x509_store, err); + } + #if (NGX_DEBUG) + { char *subject, *issuer; - int err, depth; X509 *cert; X509_NAME *sname, *iname; ngx_connection_t *c; - ngx_ssl_conn_t *ssl_conn; - - ssl_conn = X509_STORE_CTX_get_ex_data(x509_store, - SSL_get_ex_data_X509_STORE_CTX_idx()); c = ngx_ssl_get_connection(ssl_conn); @@ -1015,8 +1027,6 @@ ngx_ssl_verify_callback(int ok, X509_STO } cert = X509_STORE_CTX_get_current_cert(x509_store); - err = X509_STORE_CTX_get_error(x509_store); - depth = X509_STORE_CTX_get_error_depth(x509_store); sname = X509_get_subject_name(cert); @@ -1058,6 +1068,7 @@ ngx_ssl_verify_callback(int ok, X509_STO if (issuer) { OPENSSL_free(issuer); } + } #endif return 1; -- Sergey Kandaurov From V.Kokshenev at F5.com Fri Jul 22 00:42:15 2022 From: V.Kokshenev at F5.com (Vladimir Kokshenev) Date: Fri, 22 Jul 2022 00:42:15 +0000 Subject: Open-sourcing periodic upstream server resolution and implementing a dedicated service worker. In-Reply-To: <0D24ED91-EED7-471C-941B-4F29B37D351D@bbc.co.uk> References: <9E5B3F41-2CAD-47ED-A2EC-EE807DC54A18@f5.com> <0D24ED91-EED7-471C-941B-4F29B37D351D@bbc.co.uk> Message-ID: Hi Neil Thank you for sharing your perspectives. TTL-based DNS re-resolving is planned for this feature. Also, nginx resolver (which we plan to use) supports "ipv6=off" and (since 1.23.1) "ipv4=off" parameters for cases when IPv6 or IPv4 addresses are not desired. We know about solutions based on Lua. However, the motivation is to provide a solution that works out of the box. -- Vladimir Kokshenev On 7/21/22, 6:02 AM, "Neil Craig" wrote: EXTERNAL MAIL: Neil.Craig at bbc.co.uk Hi Vladimir I hope you don't mind me chipping in here. We currently have 2 services which use Nginx with upstreams whose DNS can change frequently (e.g. AWS S3, ELB/ALB origins) so we're familiar with this problem. Initially we did a config reload every minute to solve the issue but since then, we've created custom Lua (using the OpenResty Lua module) to achieve the same thing in a more efficient way. What you're proposing sounds great to me. I wanted to suggest a few feature ideas for your consideration: * DNS TTL-based refresh (rather than refreshing every N seconds) * DNS prefresh (update DNS when the elapsed time since the last refresh is perhaps 90% of the TTL) * IPv4/6 ignore (useful for example when there's a v6 IP available but the system/network doesn't support v6) Cheers Neil Craig (He/Him) Lead Architect, BBC Digital Distribution On 20/07/2022, 20:09, "Vladimir Kokshenev via nginx-devel" wrote: Hello! This is the two-part proposal to open-source periodic upstream server resolution and implement a dedicated service worker for nginx. The purpose of this e-mail is to describe the WHY and solicit feedback. Nginx supports domain names in the upstream server configuration. Currently, domain names are resolved at configuration time only, and there are no subsequent name resolutions. There are plans to open-source re-resolvable upstream servers. This will allow applying DNS updates to upstream configurations in runtime. So, there is a need to support periodic asynchronous operations. And a dedicated service worker is a possible architectural way to address this. The master process reads and parses configuration and creates the service worker when needed (in a similar way to cache-related processes). The service worker manages periodic name resolutions and updates corresponding upstream configurations. The name resolution relies on the existing nginx resolver and upstream zone functionality. The service worker will be responsible solely for periodic background tasks and wouldn't accept client connections. The service worker should be the last worker process to shut down to maintain the actual state of upstreams when there are active workers. Alternative architecture considered was about choosing one of the regular workers (e.g., worker zero) to take care of periodic upstream server resolution, but it creates asymmetry in responsibilities and load for this dedicated worker. -- Vladimir Kokshenev _______________________________________________ nginx-devel mailing list -- nginx-devel at nginx.org To unsubscribe send an email to nginx-devel-leave at nginx.org From v.zhestikov at f5.com Fri Jul 22 01:40:12 2022 From: v.zhestikov at f5.com (Vadim Zhestikov) Date: Fri, 22 Jul 2022 01:40:12 +0000 Subject: [njs] Fixed assignment to global property by name only. Message-ID: details: https://hg.nginx.org/njs/rev/4b8d8237f598 branches: changeset: 1916:4b8d8237f598 user: Vadim Zhestikov date: Thu Jul 21 18:33:20 2022 -0700 description: Fixed assignment to global property by name only. This closes #145 issue on Github. diffstat: src/njs_generator.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++-- src/test/njs_unit_test.c | 22 ++++++++++++++ 2 files changed, 91 insertions(+), 3 deletions(-) diffs (170 lines): diff -r c597ee3450ab -r 4b8d8237f598 src/njs_generator.c --- a/src/njs_generator.c Wed Jul 20 17:49:00 2022 -0700 +++ b/src/njs_generator.c Thu Jul 21 18:33:20 2022 -0700 @@ -222,6 +222,9 @@ static njs_int_t njs_generate_comma_expr njs_generator_t *generator, njs_parser_node_t *node); static njs_int_t njs_generate_comma_expression_end(njs_vm_t *vm, njs_generator_t *generator, njs_parser_node_t *node); +static njs_int_t njs_generate_global_property_set(njs_vm_t *vm, + njs_generator_t *generator, njs_parser_node_t *node_dst, + njs_parser_node_t *node_src); static njs_int_t njs_generate_assignment(njs_vm_t *vm, njs_generator_t *generator, njs_parser_node_t *node); static njs_int_t njs_generate_assignment_name(njs_vm_t *vm, @@ -2659,6 +2662,53 @@ njs_generate_comma_expression_end(njs_vm static njs_int_t +njs_generate_global_property_set(njs_vm_t *vm, njs_generator_t *generator, + njs_parser_node_t *node_dst, njs_parser_node_t *node_src) +{ + ssize_t length; + njs_int_t ret; + njs_value_t property; + njs_variable_t *var; + njs_vmcode_prop_set_t *prop_set; + const njs_lexer_entry_t *lex_entry; + + var = njs_variable_reference(vm, node_dst); + if (var == NULL) { + njs_generate_code(generator, njs_vmcode_prop_set_t, prop_set, + NJS_VMCODE_PROPERTY_SET, 3, node_src); + + prop_set->value = node_dst->index; + prop_set->object = njs_scope_global_this_index(); + + lex_entry = njs_lexer_entry(node_dst->u.reference.unique_id); + if (njs_slow_path(lex_entry == NULL)) { + return NJS_ERROR; + } + + length = njs_utf8_length(lex_entry->name.start, lex_entry->name.length); + if (njs_slow_path(length < 0)) { + return NJS_ERROR; + } + + ret = njs_string_new(vm, &property, lex_entry->name.start, + lex_entry->name.length, length); + if (njs_slow_path(ret != NJS_OK)) { + return NJS_ERROR; + } + + prop_set->property = njs_scope_global_index(vm, &property, + generator->runtime); + if (njs_slow_path(prop_set->property == NJS_INDEX_ERROR)) { + return NJS_ERROR; + } + + } + + return NJS_OK; +} + + +static njs_int_t njs_generate_assignment(njs_vm_t *vm, njs_generator_t *generator, njs_parser_node_t *node) { @@ -2673,7 +2723,7 @@ njs_generate_assignment(njs_vm_t *vm, nj if (lvalue->token_type == NJS_TOKEN_NAME) { - ret = njs_generate_variable(vm, generator, lvalue, NJS_DECLARATION, + ret = njs_generate_variable(vm, generator, lvalue, NJS_REFERENCE, &var); if (njs_slow_path(ret != NJS_OK)) { return ret; @@ -2721,6 +2771,7 @@ static njs_int_t njs_generate_assignment_name(njs_vm_t *vm, njs_generator_t *generator, njs_parser_node_t *node) { + njs_int_t ret; njs_parser_node_t *lvalue, *expr; njs_vmcode_move_t *move; @@ -2739,6 +2790,11 @@ njs_generate_assignment_name(njs_vm_t *v node->index = expr->index; node->temporary = expr->temporary; + ret = njs_generate_global_property_set(vm, generator, node->left, expr); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + return njs_generator_stack_pop(vm, generator, NULL); } @@ -2855,7 +2911,7 @@ njs_generate_operation_assignment(njs_vm if (lvalue->token_type == NJS_TOKEN_NAME) { - ret = njs_generate_variable(vm, generator, lvalue, NJS_DECLARATION, + ret = njs_generate_variable(vm, generator, lvalue, NJS_REFERENCE, &var); if (njs_slow_path(ret != NJS_OK)) { return ret; @@ -2938,6 +2994,11 @@ njs_generate_operation_assignment_name(n node->index = lvalue->index; + ret = njs_generate_global_property_set(vm, generator, node->left, expr); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + if (lvalue->index != index) { ret = njs_generate_index_release(vm, generator, index); if (njs_slow_path(ret != NJS_OK)) { @@ -3553,7 +3614,7 @@ njs_generate_inc_dec_operation(njs_vm_t if (lvalue->token_type == NJS_TOKEN_NAME) { - ret = njs_generate_variable(vm, generator, lvalue, NJS_DECLARATION, + ret = njs_generate_variable(vm, generator, lvalue, NJS_REFERENCE, &var); if (njs_slow_path(ret != NJS_OK)) { return ret; @@ -3580,6 +3641,11 @@ njs_generate_inc_dec_operation(njs_vm_t code->src1 = lvalue->index; code->src2 = lvalue->index; + ret = njs_generate_global_property_set(vm, generator, lvalue, lvalue); + if (njs_slow_path(ret) != NJS_OK) { + return ret; + } + return njs_generator_stack_pop(vm, generator, NULL); } diff -r c597ee3450ab -r 4b8d8237f598 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Wed Jul 20 17:49:00 2022 -0700 +++ b/src/test/njs_unit_test.c Thu Jul 21 18:33:20 2022 -0700 @@ -12077,6 +12077,28 @@ static njs_unit_test_t njs_test[] = { njs_str("this.a = 1; a"), njs_str("1") }, + { njs_str("this.a = 1; a = 3; this.a"), + njs_str("3") }, + + { njs_str("this.a = 1; ++a; this.a"), + njs_str("2") }, + + { njs_str("this.a = 1; a += 3; this.a"), + njs_str("4") }, + + { njs_str("var b=11;" + "var t = function () {b += 5; return b};" + "t() === 16 && b === 16 && this.b === 16" ), + njs_str("true") }, + + { njs_str("this.c=15;" + "var t = function () {c += 5; return c};" + "t() === 20 && c === 20 && this.c === 20" ), + njs_str("true") }, + + { njs_str("--undefined"), + njs_str("TypeError: Cannot assign to read-only property \"undefined\" of object") }, + { njs_str("this.a = 2; this.b = 3; a * b - a"), njs_str("4") }, From mdounin at mdounin.ru Sat Jul 23 21:53:26 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 24 Jul 2022 00:53:26 +0300 Subject: [PATCH] SSL: consistent certificate verification depth In-Reply-To: References: <5a9cc2a846c9ea4c0af0.1658325506@enoparse.local>

Message-ID: Hello! On Thu, Jul 21, 2022 at 06:18:24PM +0400, Sergey Kandaurov wrote: > > On 20 Jul 2022, at 22:04, Maxim Dounin wrote: > > > > On Wed, Jul 20, 2022 at 05:58:26PM +0400, Sergey Kandaurov wrote: > > > >> # HG changeset patch > >> # User Sergey Kandaurov > >> # Date 1658325446 -14400 > >> # Wed Jul 20 17:57:26 2022 +0400 > >> # Node ID 5a9cc2a846c9ea4c0af03109ab186af1ac28e222 > >> # Parent 069a4813e8d6d7ec662d282a10f5f7062ebd817f > >> SSL: consistent certificate verification depth. > >> > >> Originally, certificate verification depth was used to limit the number > >> of signatures to validate, that is, to limit chains with intermediate > >> certificates one less. The semantics was changed in OpenSSL 1.1.0, and > >> instead it limits now the number of intermediate certificates allowed. > >> This makes it not possible to limit certificate checking to self-signed > >> certificates with verify depth 0 in OpenSSL 1.1.0+, and is inconsistent > >> compared with former behaviour in BoringSSL and older OpenSSL versions. > >> > >> This change restores former verification logic when using OpenSSL 1.1.0+. > >> The verify callback is adjusted to emit the "certificate chain too long" > >> error when the certificate chain exceeds the verification depth. It has > >> no effect to other SSL libraries, where this is limited by other means. > >> Also, this fixes verification checks when using LibreSSL 3.4.0+, where > >> a chain depth is not limited except by X509_VERIFY_MAX_CHAIN_CERTS (32). > > > > This (highly questionable) OpenSSL behaviour seems to be status > > quo for a while, also recorded in tests (see ssl_verify_depth.t). > > Any specific reasons for the nginx-side workaround? > > As explained in the commit message, main motivation is to eliminate > annoying difference in behaviour among various SSL libraries > (aside from working around the arguably broken LibreSSL verifier). > Nothing specific behind that. > Disambiguating ssl_verify_depth.t is a good demo of net result. > The downside is that this can potentially break previously working > configurations when using with modern OpenSSL versions. > So the patch is to seek feedback on whether this makes sense. Apart from potentially breaking some existing configurations (especially with LibreSSL), I have several basic concerns here: 1. As of now, it's OpenSSL responsibility to apply verify depth limit, and "openssl verify -verify_depth " can be used to test the particular verification. Switching to our implementation of the depth limit means that it will be our responsibility to apply and document the limit. 2. The verify callback behaviour is complex enough, and I expect portability issues. 3. Further, the main reason why the depth limit exists is to limit maximum resource consumption of the whole certificate verification process. Raising an error within the verify callback without returning failure won't stop OpenSSL from checking additional certificates, and hence is not going to limit the resource usage. That is, such a limit would be misleading: it will prevent certificates from being accepted, but won't stop DoS attacks. I don't think that there is a way to implement the limit via verify callback in a way which will prevent excessive signature checks yet preserve nginx existing behaviour with not closing the connection in case of verification failures. Summing the above, I tend to think that it's not worth the effort. [...] -- Maxim Dounin http://mdounin.ru/ From pluknet at nginx.com Mon Jul 25 22:36:37 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 26 Jul 2022 02:36:37 +0400 Subject: [PATCH 1 of 4] QUIC: fixed-length buffers for secrets In-Reply-To: References:

Message-ID: <762C0281-9179-4352-9BD0-ACB70F98D2BF@nginx.com> > On 31 May 2022, at 11:06, Roman Arutyunyan wrote: > > # HG changeset patch > # User Vladimir Homutov > # Date 1645524401 -10800 > # Tue Feb 22 13:06:41 2022 +0300 > # Branch quic > # Node ID a881ff28070262f3810517d5d3cb4ff67a4b7121 > # Parent 5b1011b5702b5c5db2ba3d392a4da25596183cc2 > QUIC: fixed-length buffers for secrets. > > diff --git a/src/event/quic/ngx_event_quic_protection.c b/src/event/quic/ngx_event_quic_protection.c > --- a/src/event/quic/ngx_event_quic_protection.c > +++ b/src/event/quic/ngx_event_quic_protection.c > @@ -17,6 +17,9 @@ > > #define NGX_QUIC_AES_128_KEY_LEN 16 > > +/* largest hash used in TLS is SHA-384 */ > +#define NGX_QUIC_MAX_MD_SIZE 48 > + > #define NGX_AES_128_GCM_SHA256 0x1301 > #define NGX_AES_256_GCM_SHA384 0x1302 > #define NGX_CHACHA20_POLY1305_SHA256 0x1303 > @@ -30,6 +33,18 @@ > > > typedef struct { > + size_t len; > + u_char data[NGX_QUIC_MAX_MD_SIZE]; > +} ngx_quic_md_t; > + > + > +typedef struct { > + size_t len; > + u_char data[NGX_QUIC_IV_LEN]; > +} ngx_quic_iv_t; > + > + > +typedef struct { > const ngx_quic_cipher_t *c; > const EVP_CIPHER *hp; > const EVP_MD *d; > @@ -37,10 +52,10 @@ typedef struct { > > > typedef struct ngx_quic_secret_s { > - ngx_str_t secret; > - ngx_str_t key; > - ngx_str_t iv; > - ngx_str_t hp; > + ngx_quic_md_t secret; > + ngx_quic_md_t key; > + ngx_quic_iv_t iv; > + ngx_quic_md_t hp; > } ngx_quic_secret_t; > > > @@ -57,6 +72,25 @@ struct ngx_quic_keys_s { > }; > > > +typedef struct { > + size_t out_len; > + u_char *out; > + > + size_t prk_len; > + const uint8_t *prk; > + > + size_t label_len; > + const u_char *label; > +} ngx_quic_hkdf_t; > + > +#define ngx_quic_hkdf_set(label, out, prk) \ > + { \ > + (out)->len, (out)->data, \ > + (prk)->len, (prk)->data, \ > + (sizeof(label) - 1), (u_char *)(label), \ > + } > + > + > static ngx_int_t ngx_hkdf_expand(u_char *out_key, size_t out_len, > const EVP_MD *digest, const u_char *prk, size_t prk_len, > const u_char *info, size_t info_len); > @@ -78,8 +112,8 @@ static ngx_int_t ngx_quic_tls_seal(const > ngx_str_t *ad, ngx_log_t *log); > static ngx_int_t ngx_quic_tls_hp(ngx_log_t *log, const EVP_CIPHER *cipher, > ngx_quic_secret_t *s, u_char *out, u_char *in); > -static ngx_int_t ngx_quic_hkdf_expand(ngx_pool_t *pool, const EVP_MD *digest, > - ngx_str_t *out, ngx_str_t *label, const uint8_t *prk, size_t prk_len); > +static ngx_int_t ngx_quic_hkdf_expand(ngx_quic_hkdf_t *hkdf, > + const EVP_MD *digest, ngx_pool_t *pool); > > static ngx_int_t ngx_quic_create_packet(ngx_quic_header_t *pkt, > ngx_str_t *res); > @@ -204,28 +238,20 @@ ngx_quic_keys_set_initial_secret(ngx_poo > client->iv.len = NGX_QUIC_IV_LEN; > server->iv.len = NGX_QUIC_IV_LEN; > > - struct { > - ngx_str_t label; > - ngx_str_t *key; > - ngx_str_t *prk; > - } seq[] = { > + ngx_quic_hkdf_t seq[] = { > /* labels per RFC 9001, 5.1. Packet Protection Keys */ > - { ngx_string("tls13 client in"), &client->secret, &iss }, > - { ngx_string("tls13 quic key"), &client->key, &client->secret }, > - { ngx_string("tls13 quic iv"), &client->iv, &client->secret }, > - { ngx_string("tls13 quic hp"), &client->hp, &client->secret }, > - { ngx_string("tls13 server in"), &server->secret, &iss }, > - { ngx_string("tls13 quic key"), &server->key, &server->secret }, > - { ngx_string("tls13 quic iv"), &server->iv, &server->secret }, > - { ngx_string("tls13 quic hp"), &server->hp, &server->secret }, > + ngx_quic_hkdf_set("tls13 client in", &client->secret, &iss), > + ngx_quic_hkdf_set("tls13 quic key", &client->key, &client->secret), > + ngx_quic_hkdf_set("tls13 quic iv", &client->iv, &client->secret), > + ngx_quic_hkdf_set("tls13 quic hp", &client->hp, &client->secret), > + ngx_quic_hkdf_set("tls13 server in", &server->secret, &iss), > + ngx_quic_hkdf_set("tls13 quic key", &server->key, &server->secret), > + ngx_quic_hkdf_set("tls13 quic iv", &server->iv, &server->secret), > + ngx_quic_hkdf_set("tls13 quic hp", &server->hp, &server->secret), > }; > > for (i = 0; i < (sizeof(seq) / sizeof(seq[0])); i++) { > - > - if (ngx_quic_hkdf_expand(pool, digest, seq[i].key, &seq[i].label, > - seq[i].prk->data, seq[i].prk->len) > - != NGX_OK) > - { > + if (ngx_quic_hkdf_expand(&seq[i], digest, pool) != NGX_OK) { > return NGX_ERROR; > } > } > @@ -235,40 +261,41 @@ ngx_quic_keys_set_initial_secret(ngx_poo > > > static ngx_int_t > -ngx_quic_hkdf_expand(ngx_pool_t *pool, const EVP_MD *digest, ngx_str_t *out, > - ngx_str_t *label, const uint8_t *prk, size_t prk_len) > +ngx_quic_hkdf_expand(ngx_quic_hkdf_t *h, const EVP_MD *digest, ngx_pool_t *pool) > { > size_t info_len; > uint8_t *p; > uint8_t info[20]; > > - if (out->data == NULL) { > - out->data = ngx_pnalloc(pool, out->len); > - if (out->data == NULL) { > + if (h->out == NULL) { > + h->out = ngx_pnalloc(pool, h->out_len); > + if (h->out == NULL) { > return NGX_ERROR; > } > } The condition can be removed now. I see no reason to postpone this to the next change. > > - info_len = 2 + 1 + label->len + 1; > + info_len = 2 + 1 + h->label_len + 1; > > info[0] = 0; > - info[1] = out->len; > - info[2] = label->len; > - p = ngx_cpymem(&info[3], label->data, label->len); > + info[1] = h->out_len; > + info[2] = h->label_len; > + > + p = ngx_cpymem(&info[3], h->label, h->label_len); > *p = '\0'; > > - if (ngx_hkdf_expand(out->data, out->len, digest, > - prk, prk_len, info, info_len) > + if (ngx_hkdf_expand(h->out, h->out_len, digest, > + h->prk, h->prk_len, info, info_len) > != NGX_OK) > { > ngx_ssl_error(NGX_LOG_INFO, pool->log, 0, > - "ngx_hkdf_expand(%V) failed", label); > + "ngx_hkdf_expand(%*s) failed", h->label_len, h->label); > return NGX_ERROR; > } > > #ifdef NGX_QUIC_DEBUG_CRYPTO > - ngx_log_debug3(NGX_LOG_DEBUG_EVENT, pool->log, 0, > - "quic expand %V key len:%uz %xV", label, out->len, out); > + ngx_log_debug5(NGX_LOG_DEBUG_EVENT, pool->log, 0, > + "quic expand \"%*s\" key len:%uz %*xs", > + h->label_len, h->label, h->out_len, h->out_len, h->out); > #endif While here, I'd also drop "key" from the message (an 64a484fd40a9 leftover). [..] -- Sergey Kandaurov From pluknet at nginx.com Mon Jul 25 23:00:37 2022 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 26 Jul 2022 03:00:37 +0400 Subject: [PATCH 3 of 4] QUIC: removed ngx_quic_keys_new() In-Reply-To: <7929cae8d65fd1f41d07.1653980793@arut-laptop> References: <7929cae8d65fd1f41d07.1653980793@arut-laptop> Message-ID: <7363BF1F-F37B-4A6A-B527-E56D53629BAC@nginx.com> > On 31 May 2022, at 11:06, Roman Arutyunyan wrote: > > # HG changeset patch > # User Vladimir Homutov > # Date 1653652352 -14400 > # Fri May 27 15:52:32 2022 +0400 > # Branch quic > # Node ID 7929cae8d65fd1f41d07365cae93970b29f2d03d > # Parent 41f47332273e0350157258cc40dd0ede4ee86c69 > QUIC: removed ngx_quic_keys_new(). > > The ngx_quic_keys_t structure is now exposed. IMHO, this line suites as the log summary. > This allows to use it in contexts where no pool/connection is available, > i.e. early packet processing. > > diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c > --- a/src/event/quic/ngx_event_quic.c > +++ b/src/event/quic/ngx_event_quic.c > @@ -238,7 +238,7 @@ ngx_quic_new_connection(ngx_connection_t > return NULL; > } > > - qc->keys = ngx_quic_keys_new(c->pool); > + qc->keys = ngx_pcalloc(c->pool, sizeof(ngx_quic_keys_t)); > if (qc->keys == NULL) { > return NULL; > } > diff --git a/src/event/quic/ngx_event_quic_output.c b/src/event/quic/ngx_event_quic_output.c > --- a/src/event/quic/ngx_event_quic_output.c > +++ b/src/event/quic/ngx_event_quic_output.c > @@ -928,6 +928,7 @@ ngx_quic_send_early_cc(ngx_connection_t > { > ssize_t len; > ngx_str_t res; > + ngx_quic_keys_t keys; > ngx_quic_frame_t frame; > ngx_quic_header_t pkt; > > @@ -956,10 +957,9 @@ ngx_quic_send_early_cc(ngx_connection_t > return NGX_ERROR; > } > > - pkt.keys = ngx_quic_keys_new(c->pool); > - if (pkt.keys == NULL) { > - return NGX_ERROR; > - } > + ngx_memzero(&keys, sizeof(ngx_quic_keys_t)); > + > + pkt.keys = &keys; > > if (ngx_quic_keys_set_initial_secret(pkt.keys, &inpkt->dcid, c->log) > != NGX_OK) > diff --git a/src/event/quic/ngx_event_quic_protection.c b/src/event/quic/ngx_event_quic_protection.c > --- a/src/event/quic/ngx_event_quic_protection.c > +++ b/src/event/quic/ngx_event_quic_protection.c > @@ -10,16 +10,11 @@ > #include > > > -/* RFC 5116, 5.1 and RFC 8439, 2.3 for all supported ciphers */ > -#define NGX_QUIC_IV_LEN 12 > /* RFC 9001, 5.4.1. Header Protection Application: 5-byte mask */ > #define NGX_QUIC_HP_LEN 5 > > #define NGX_QUIC_AES_128_KEY_LEN 16 > > -/* largest hash used in TLS is SHA-384 */ > -#define NGX_QUIC_MAX_MD_SIZE 48 > - > #define NGX_AES_128_GCM_SHA256 0x1301 > #define NGX_AES_256_GCM_SHA384 0x1302 > #define NGX_CHACHA20_POLY1305_SHA256 0x1303 > @@ -33,45 +28,12 @@ > > > typedef struct { > - size_t len; > - u_char data[NGX_QUIC_MAX_MD_SIZE]; > -} ngx_quic_md_t; > - > - > -typedef struct { > - size_t len; > - u_char data[NGX_QUIC_IV_LEN]; > -} ngx_quic_iv_t; > - > - > -typedef struct { > const ngx_quic_cipher_t *c; > const EVP_CIPHER *hp; > const EVP_MD *d; > } ngx_quic_ciphers_t; > > > -typedef struct ngx_quic_secret_s { > - ngx_quic_md_t secret; > - ngx_quic_md_t key; > - ngx_quic_iv_t iv; > - ngx_quic_md_t hp; > -} ngx_quic_secret_t; > - > - > -typedef struct { > - ngx_quic_secret_t client; > - ngx_quic_secret_t server; > -} ngx_quic_secrets_t; > - > - > -struct ngx_quic_keys_s { > - ngx_quic_secrets_t secrets[NGX_QUIC_ENCRYPTION_LAST]; > - ngx_quic_secrets_t next_key; > - ngx_uint_t cipher; > -}; > - > - > typedef struct { > size_t out_len; > u_char *out; > @@ -721,13 +683,6 @@ ngx_quic_keys_set_encryption_secret(ngx_ > } > > > -ngx_quic_keys_t * > -ngx_quic_keys_new(ngx_pool_t *pool) > -{ > - return ngx_pcalloc(pool, sizeof(ngx_quic_keys_t)); > -} > - > - > ngx_uint_t > ngx_quic_keys_available(ngx_quic_keys_t *keys, > enum ssl_encryption_level_t level) > diff --git a/src/event/quic/ngx_event_quic_protection.h b/src/event/quic/ngx_event_quic_protection.h > --- a/src/event/quic/ngx_event_quic_protection.h > +++ b/src/event/quic/ngx_event_quic_protection.h > @@ -16,8 +16,46 @@ > > #define NGX_QUIC_ENCRYPTION_LAST ((ssl_encryption_application) + 1) > > +/* RFC 5116, 5.1 and RFC 8439, 2.3 for all supported ciphers */ > +#define NGX_QUIC_IV_LEN 12 > > -ngx_quic_keys_t *ngx_quic_keys_new(ngx_pool_t *pool); > +/* largest hash used in TLS is SHA-384 */ > +#define NGX_QUIC_MAX_MD_SIZE 48 > + > + > +typedef struct { > + size_t len; > + u_char data[NGX_QUIC_MAX_MD_SIZE]; > +} ngx_quic_md_t; > + > + > +typedef struct { > + size_t len; > + u_char data[NGX_QUIC_IV_LEN]; > +} ngx_quic_iv_t; > + > + > +typedef struct ngx_quic_secret_s { The "ngx_quic_secret_s" part can be dropped, unused since 9c3be23ddbe7. > + ngx_quic_md_t secret; > + ngx_quic_md_t key; > + ngx_quic_iv_t iv; > + ngx_quic_md_t hp; > +} ngx_quic_secret_t; > + > + > +typedef struct { > + ngx_quic_secret_t client; > + ngx_quic_secret_t server; > +} ngx_quic_secrets_t; > + > + > +struct ngx_quic_keys_s { > + ngx_quic_secrets_t secrets[NGX_QUIC_ENCRYPTION_LAST]; > + ngx_quic_secrets_t next_key; > + ngx_uint_t cipher; > +}; > + > + > ngx_int_t ngx_quic_keys_set_initial_secret(ngx_quic_keys_t *keys, > ngx_str_t *secret, ngx_log_t *log); > ngx_int_t ngx_quic_keys_set_encryption_secret(ngx_log_t *log, > [ The remaining patches look good. ] -- Sergey Kandaurov From xeioex at nginx.com Tue Jul 26 01:41:26 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 26 Jul 2022 01:41:26 +0000 Subject: [njs] Added generic logger callback. Message-ID: details: https://hg.nginx.org/njs/rev/14426cb84197 branches: changeset: 1917:14426cb84197 user: Dmitry Volyntsev date: Mon Jul 25 18:40:24 2022 -0700 description: Added generic logger callback. This allows for a host environment to control when and how internal NJS messages are logged. diffstat: nginx/ngx_http_js_module.c | 1 + nginx/ngx_js.c | 15 +++++++++++++++ nginx/ngx_js.h | 2 ++ nginx/ngx_stream_js_module.c | 1 + src/njs.h | 19 +++++++++++++++++++ src/njs_shell.c | 38 +++++++++++++++++++++++++++----------- src/njs_vm.c | 26 ++++++++++++++++++++++++++ 7 files changed, 91 insertions(+), 11 deletions(-) diffs (274 lines): diff -r 4b8d8237f598 -r 14426cb84197 nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Thu Jul 21 18:33:20 2022 -0700 +++ b/nginx/ngx_http_js_module.c Mon Jul 25 18:40:24 2022 -0700 @@ -783,6 +783,7 @@ static njs_vm_ops_t ngx_http_js_ops = { ngx_http_js_set_timer, ngx_http_js_clear_timer, NULL, + ngx_js_logger, }; diff -r 4b8d8237f598 -r 14426cb84197 nginx/ngx_js.c --- a/nginx/ngx_js.c Thu Jul 21 18:33:20 2022 -0700 +++ b/nginx/ngx_js.c Mon Jul 25 18:40:24 2022 -0700 @@ -318,3 +318,18 @@ ngx_js_ext_log(njs_vm_t *vm, njs_value_t } +void +ngx_js_logger(njs_vm_t *vm, njs_external_ptr_t external, njs_log_level_t level, + const u_char *start, size_t length) +{ + ngx_connection_t *c; + ngx_log_handler_pt handler; + + c = ngx_external_connection(vm, external); + handler = c->log->handler; + c->log->handler = NULL; + + ngx_log_error((ngx_uint_t) level, c->log, 0, "js: %*s", length, start); + + c->log->handler = handler; +} diff -r 4b8d8237f598 -r 14426cb84197 nginx/ngx_js.h --- a/nginx/ngx_js.h Thu Jul 21 18:33:20 2022 -0700 +++ b/nginx/ngx_js.h Mon Jul 25 18:40:24 2022 -0700 @@ -68,6 +68,8 @@ ngx_int_t ngx_js_retval(njs_vm_t *vm, nj njs_int_t ngx_js_ext_log(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t level); +void ngx_js_logger(njs_vm_t *vm, njs_external_ptr_t external, + njs_log_level_t level, const u_char *start, size_t length); njs_int_t ngx_js_ext_string(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval); diff -r 4b8d8237f598 -r 14426cb84197 nginx/ngx_stream_js_module.c --- a/nginx/ngx_stream_js_module.c Thu Jul 21 18:33:20 2022 -0700 +++ b/nginx/ngx_stream_js_module.c Mon Jul 25 18:40:24 2022 -0700 @@ -499,6 +499,7 @@ static njs_vm_ops_t ngx_stream_js_ops = ngx_stream_js_set_timer, ngx_stream_js_clear_timer, NULL, + ngx_js_logger, }; diff -r 4b8d8237f598 -r 14426cb84197 src/njs.h --- a/src/njs.h Thu Jul 21 18:33:20 2022 -0700 +++ b/src/njs.h Mon Jul 25 18:40:24 2022 -0700 @@ -45,6 +45,11 @@ typedef struct { uint64_t filler[2]; } njs_opaque_value_t; +typedef enum { + NJS_LOG_LEVEL_ERROR = 4, + NJS_LOG_LEVEL_WARN = 5, + NJS_LOG_LEVEL_INFO = 7, +} njs_log_level_t; /* sizeof(njs_value_t) is 16 bytes. */ #define njs_argument(args, n) \ @@ -69,6 +74,12 @@ extern const njs_value_t njs_ #define njs_vm_error(vm, fmt, ...) \ njs_vm_value_error_set(vm, njs_vm_retval(vm), fmt, ##__VA_ARGS__) +#define njs_vm_log(vm, fmt, ...) njs_vm_logger(vm, NJS_LOG_LEVEL_INFO, fmt, \ + ##__VA_ARGS__) +#define njs_vm_warn(vm, fmt, ...) njs_vm_logger(vm, NJS_LOG_LEVEL_WARN, fmt, \ + ##__VA_ARGS__) +#define njs_vm_err(vm, fmt, ...) njs_vm_logger(vm, NJS_LOG_LEVEL_ERROR, fmt, \ + ##__VA_ARGS__) /* * njs_prop_handler_t operates as a property getter/setter or delete handler. @@ -187,12 +198,15 @@ typedef void (*njs_event_destructor_t)(n njs_host_event_t event); typedef njs_mod_t *(*njs_module_loader_t)(njs_vm_t *vm, njs_external_ptr_t external, njs_str_t *name); +typedef void (*njs_logger_t)(njs_vm_t *vm, njs_external_ptr_t external, + njs_log_level_t level, const u_char *start, size_t length); typedef struct { njs_set_timer_t set_timer; njs_event_destructor_t clear_timer; njs_module_loader_t module_loader; + njs_logger_t logger; } njs_vm_ops_t; @@ -221,6 +235,8 @@ typedef struct { char **argv; njs_uint_t argc; + njs_log_level_t log_level; + #define NJS_VM_OPT_UNHANDLED_REJECTION_IGNORE 0 #define NJS_VM_OPT_UNHANDLED_REJECTION_THROW 1 @@ -401,6 +417,9 @@ NJS_EXPORT void njs_vm_value_error_set(n const char *fmt, ...); NJS_EXPORT void njs_vm_memory_error(njs_vm_t *vm); +NJS_EXPORT void njs_vm_logger(njs_vm_t *vm, njs_log_level_t level, + const char *fmt, ...); + NJS_EXPORT void njs_value_undefined_set(njs_value_t *value); NJS_EXPORT void njs_value_null_set(njs_value_t *value); NJS_EXPORT void njs_value_invalid_set(njs_value_t *value); diff -r 4b8d8237f598 -r 14426cb84197 src/njs_shell.c --- a/src/njs_shell.c Thu Jul 21 18:33:20 2022 -0700 +++ b/src/njs_shell.c Mon Jul 25 18:40:24 2022 -0700 @@ -121,6 +121,8 @@ static njs_host_event_t njs_console_set_ static void njs_console_clear_timer(njs_external_ptr_t external, njs_host_event_t event); +static void njs_console_log(njs_vm_t *vm, njs_external_ptr_t external, + njs_log_level_t level, const u_char *start, size_t length); static njs_int_t njs_timelabel_hash_test(njs_lvlhsh_query_t *lhq, void *data); @@ -207,6 +209,7 @@ static njs_vm_ops_t njs_console_ops = { njs_console_set_timer, njs_console_clear_timer, NULL, + njs_console_log, }; @@ -1204,14 +1207,14 @@ njs_ext_console_log(njs_vm_t *vm, njs_va return NJS_ERROR; } - njs_printf("%s", (n != 1) ? " " : ""); - njs_print(msg.start, msg.length); + njs_vm_log(vm, "%s", (n != 1) ? " " : ""); + njs_vm_log(vm, "%*s", msg.length, msg.start); n++; } if (nargs > 1) { - njs_printf("\n"); + njs_vm_log(vm, "\n"); } njs_set_undefined(&vm->retval); @@ -1284,7 +1287,7 @@ njs_ext_console_time(njs_vm_t *vm, njs_v return NJS_ERROR; } - njs_printf("Timer \"%V\" already exists.\n", &name); + njs_vm_log(vm, "Timer \"%V\" already exists.\n", &name); label = lhq.value; } @@ -1349,7 +1352,7 @@ njs_ext_console_time_end(njs_vm_t *vm, n ms = ns / 1000000; ns = ns % 1000000; - njs_printf("%V: %uL.%06uLms\n", &name, ms, ns); + njs_vm_log(vm, "%V: %uL.%06uLms\n", &name, ms, ns); /* GC: release. */ njs_mp_free(vm->mem_pool, label); @@ -1361,7 +1364,7 @@ njs_ext_console_time_end(njs_vm_t *vm, n return NJS_ERROR; } - njs_printf("Timer \"%V\" doesn’t exist.\n", &name); + njs_vm_log(vm, "Timer \"%V\" doesn’t exist.\n", &name); } njs_set_undefined(&vm->retval); @@ -1380,14 +1383,14 @@ njs_console_set_timer(njs_external_ptr_t njs_console_t *console; njs_lvlhsh_query_t lhq; + console = external; + vm = console->vm; + if (delay != 0) { - njs_stderror("njs_console_set_timer(): async timers unsupported\n"); + njs_vm_err(vm, "njs_console_set_timer(): async timers unsupported\n"); return NULL; } - console = external; - vm = console->vm; - ev = njs_mp_alloc(vm->mem_pool, sizeof(njs_ev_t)); if (njs_slow_path(ev == NULL)) { return NULL; @@ -1441,13 +1444,26 @@ njs_console_clear_timer(njs_external_ptr ret = njs_lvlhsh_delete(&console->events, &lhq); if (ret != NJS_OK) { - njs_stderror("njs_lvlhsh_delete() failed\n"); + njs_vm_err(vm, "njs_lvlhsh_delete() failed\n"); } njs_mp_free(vm->mem_pool, ev); } +static void +njs_console_log(njs_vm_t *vm, njs_external_ptr_t external, + njs_log_level_t level, const u_char *start, size_t length) +{ + if (level == NJS_LOG_LEVEL_ERROR) { + njs_stderror("%*s", length, start); + + } else { + njs_printf("%*s", length, start); + } +} + + static njs_int_t njs_timelabel_hash_test(njs_lvlhsh_query_t *lhq, void *data) { diff -r 4b8d8237f598 -r 14426cb84197 src/njs_vm.c --- a/src/njs_vm.c Thu Jul 21 18:33:20 2022 -0700 +++ b/src/njs_vm.c Mon Jul 25 18:40:24 2022 -0700 @@ -23,6 +23,8 @@ void njs_vm_opt_init(njs_vm_opt_t *options) { njs_memzero(options, sizeof(njs_vm_opt_t)); + + options->log_level = NJS_LOG_LEVEL_INFO; } @@ -881,6 +883,30 @@ njs_vm_memory_error(njs_vm_t *vm) } +njs_noinline void +njs_vm_logger(njs_vm_t *vm, njs_log_level_t level, const char *fmt, ...) +{ + u_char *p; + va_list args; + njs_logger_t logger; + u_char buf[NJS_MAX_ERROR_STR]; + + if (vm->options.ops == NULL) { + return; + } + + logger = vm->options.ops->logger; + + if (logger != NULL && vm->options.log_level >= level) { + va_start(args, fmt); + p = njs_vsprintf(buf, buf + sizeof(buf), fmt, args); + va_end(args); + + logger(vm, vm->external, level, buf, p - buf); + } +} + + njs_int_t njs_vm_value_string(njs_vm_t *vm, njs_str_t *dst, njs_value_t *src) { From xeioex at nginx.com Tue Jul 26 01:41:28 2022 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 26 Jul 2022 01:41:28 +0000 Subject: [njs] Added soft deprecation warning for deprecated methods and properties. Message-ID: details: https://hg.nginx.org/njs/rev/beaff2c39864 branches: changeset: 1918:beaff2c39864 user: Dmitry Volyntsev date: Mon Jul 25 18:40:24 2022 -0700 description: Added soft deprecation warning for deprecated methods and properties. diffstat: nginx/ngx_http_js_module.c | 4 ++++ src/njs.h | 11 +++++++++++ src/njs_string.c | 10 ++++++++++ 3 files changed, 25 insertions(+), 0 deletions(-) diffs (90 lines): diff -r 14426cb84197 -r beaff2c39864 nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Mon Jul 25 18:40:24 2022 -0700 +++ b/nginx/ngx_http_js_module.c Mon Jul 25 18:40:24 2022 -0700 @@ -2572,6 +2572,8 @@ ngx_http_js_ext_get_request_body(njs_vm_ ngx_http_js_ctx_t *ctx; ngx_http_request_t *r; + njs_deprecated(vm, "r.requestBody"); + r = njs_vm_external(vm, ngx_http_js_request_proto_id, value); if (r == NULL) { njs_value_undefined_set(retval); @@ -3416,6 +3418,8 @@ ngx_http_js_ext_get_response_body(njs_vm ngx_http_js_ctx_t *ctx; ngx_http_request_t *r; + njs_deprecated(vm, "r.responseBody"); + r = njs_vm_external(vm, ngx_http_js_request_proto_id, value); if (r == NULL) { njs_value_undefined_set(retval); diff -r 14426cb84197 -r beaff2c39864 src/njs.h --- a/src/njs.h Mon Jul 25 18:40:24 2022 -0700 +++ b/src/njs.h Mon Jul 25 18:40:24 2022 -0700 @@ -81,6 +81,17 @@ extern const njs_value_t njs_ #define njs_vm_err(vm, fmt, ...) njs_vm_logger(vm, NJS_LOG_LEVEL_ERROR, fmt, \ ##__VA_ARGS__) +#define njs_deprecated(vm, text) \ + do { \ + static njs_bool_t reported; \ + \ + if (!reported) { \ + njs_vm_warn(vm, text " is deprecated " \ + "and will be removed in the future"); \ + reported = 1; \ + } \ + } while(0) + /* * njs_prop_handler_t operates as a property getter/setter or delete handler. * - retval != NULL && setval == NULL - GET context. diff -r 14426cb84197 -r beaff2c39864 src/njs_string.c --- a/src/njs_string.c Mon Jul 25 18:40:24 2022 -0700 +++ b/src/njs_string.c Mon Jul 25 18:40:24 2022 -0700 @@ -982,6 +982,8 @@ njs_string_prototype_from_utf8(njs_vm_t njs_slice_prop_t slice; njs_string_prop_t string; + njs_deprecated(vm, "String.prototype.fromUTF8()"); + ret = njs_string_object_validate(vm, njs_argument(args, 0)); if (njs_slow_path(ret != NJS_OK)) { return ret; @@ -1025,6 +1027,8 @@ njs_string_prototype_to_utf8(njs_vm_t *v njs_slice_prop_t slice; njs_string_prop_t string; + njs_deprecated(vm, "String.prototype.toUTF8()"); + ret = njs_string_object_validate(vm, njs_argument(args, 0)); if (njs_slow_path(ret != NJS_OK)) { return ret; @@ -1059,6 +1063,8 @@ njs_string_prototype_from_bytes(njs_vm_t njs_slice_prop_t slice; njs_string_prop_t string; + njs_deprecated(vm, "String.prototype.fromBytes()"); + ret = njs_string_object_validate(vm, njs_argument(args, 0)); if (njs_slow_path(ret != NJS_OK)) { return ret; @@ -1125,6 +1131,8 @@ njs_string_prototype_to_bytes(njs_vm_t * njs_string_prop_t string; njs_unicode_decode_t ctx; + njs_deprecated(vm, "String.prototype.toBytes()"); + ret = njs_string_object_validate(vm, njs_argument(args, 0)); if (njs_slow_path(ret != NJS_OK)) { return ret; @@ -1617,6 +1625,8 @@ njs_string_bytes_from(njs_vm_t *vm, njs_ { njs_value_t *value; + njs_deprecated(vm, "String.bytesFrom()"); + value = njs_arg(args, nargs, 1); if (njs_is_string(value)) { From Eckart.Haufler at rohde-schwarz.com Tue Jul 26 07:39:29 2022 From: Eckart.Haufler at rohde-schwarz.com (Eckart Haufler) Date: Tue, 26 Jul 2022 07:39:29 +0000 Subject: ngx_http_dav_module disable_symlink question Message-ID: Hi, We want to use the ngx_http_dav_module with the nginx server (1.21) on a linux machine. For security reasons, we would like to forbid to follow symbol links (e.g. for the case of accidental symbol links to directories like root / ). The nginx directive “disable_symlinks“ looked promising. It suppresses the download of files, but “MOVE” or “DELETE” seems not to be blocked. Also the documentation says “ngx_http_autoindex_module, ngx_http_random_index_module, and ngx_http_dav_module modules currently ignore this directive.” Is this planned or resolved on some newer release branches – or are there other settings to achieve better protection? Thanks for any hints! Eckart -------------- next part -------------- An HTML attachment was scrubbed... URL: From dnj0496 at gmail.com Wed Jul 27 02:21:29 2022 From: dnj0496 at gmail.com (Dk Jack) Date: Tue, 26 Jul 2022 19:21:29 -0700 Subject: nginx crash Message-ID: Hi, I am noticing a crash in my nginx module. The crash is happening after an internal redirect. It's not always happening but for certain requests. Besides the trace log I do not have much info about the request. In my module, I am restoring the my module context in a similar fashion as the ` ngx_http_realip_get_module_ctx`. What I noticed is `r->pool == 0`. Why would the r->pool be ever zero'ed? The crash is happening in my get_module_ctx function which was called immediately after returning from the ngx_http_internal_redirect call. Any ideas on how to go about resolving this is greatly appreciated. Thanks. Regards, Dk. static ngx_http_realip_ctx_t * ngx_http_realip_get_module_ctx(ngx_http_request_t *r) { ngx_pool_cleanup_t *cln; ngx_http_realip_ctx_t *ctx; ctx = ngx_http_get_module_ctx(r, ngx_http_realip_module); if (ctx == NULL && (r->internal || r->filter_finalize)) { /* * if module context was reset, the original address * can still be found in the cleanup handler */ for (cln = r->pool->cleanup; cln; cln = cln->next) { if (cln->handler == ngx_http_realip_cleanup) { ctx = cln->data; break; } } } return ctx; } -------------- next part -------------- An HTML attachment was scrubbed... URL: From dnj0496 at gmail.com Wed Jul 27 02:22:06 2022 From: dnj0496 at gmail.com (Dk Jack) Date: Tue, 26 Jul 2022 19:22:06 -0700 Subject: nginx crash In-Reply-To: References: Message-ID: Forgot to add, this is with nginx version: 1.20.1 On Tue, Jul 26, 2022 at 7:21 PM Dk Jack wrote: > Hi, > I am noticing a crash in my nginx module. The crash is happening after an > internal redirect. It's not always happening but for certain requests. > Besides the trace log I do not have much info about the request. In my > module, I am restoring the my module context in a similar fashion as the ` > ngx_http_realip_get_module_ctx`. What I noticed is `r->pool == 0`. Why > would the r->pool be ever zero'ed? > > The crash is happening in my get_module_ctx function which was called > immediately after returning from the ngx_http_internal_redirect call. Any > ideas on how to go about resolving this is greatly appreciated. Thanks. > > Regards, > Dk. > > static ngx_http_realip_ctx_t * > ngx_http_realip_get_module_ctx(ngx_http_request_t *r) > { > ngx_pool_cleanup_t *cln; > ngx_http_realip_ctx_t *ctx; > > ctx = ngx_http_get_module_ctx(r, ngx_http_realip_module); > if (ctx == NULL && (r->internal || r->filter_finalize)) { > /* > * if module context was reset, the original address > * can still be found in the cleanup handler > */ > for (cln = r->pool->cleanup; cln; cln = cln->next) { > if (cln->handler == ngx_http_realip_cleanup) { > ctx = cln->data; > break; > } > } > } > > return ctx; > } > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Wed Jul 27 13:34:39 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 27 Jul 2022 16:34:39 +0300 Subject: nginx crash In-Reply-To: References: Message-ID: Hello! On Tue, Jul 26, 2022 at 07:21:29PM -0700, Dk Jack wrote: > Hi, > I am noticing a crash in my nginx module. The crash is happening after an > internal redirect. It's not always happening but for certain requests. > Besides the trace log I do not have much info about the request. In my > module, I am restoring the my module context in a similar fashion as the ` > ngx_http_realip_get_module_ctx`. What I noticed is `r->pool == 0`. Why > would the r->pool be ever zero'ed? > > The crash is happening in my get_module_ctx function which was called > immediately after returning from the ngx_http_internal_redirect call. Any > ideas on how to go about resolving this is greatly appreciated. Thanks. It looks like there are issues with request reference counting somewhere in your code, and the request is freed in the ngx_http_internal_redirect() call. Review your code logic and corresponding request reference counting, it should help. -- Maxim Dounin http://mdounin.ru/ From dnj0496 at gmail.com Wed Jul 27 15:40:41 2022 From: dnj0496 at gmail.com (Dk Jack) Date: Wed, 27 Jul 2022 08:40:41 -0700 Subject: nginx crash In-Reply-To: References: Message-ID: <568013AB-8211-4189-B916-2FF84412219B@gmail.com> In my code I am not modifying the reference count. Could you let me if there any function calls if invoked would indirectly update the reference count? Dk. > On Jul 27, 2022, at 6:35 AM, Maxim Dounin wrote: > > Hello! > >> On Tue, Jul 26, 2022 at 07:21:29PM -0700, Dk Jack wrote: >> >> Hi, >> I am noticing a crash in my nginx module. The crash is happening after an >> internal redirect. It's not always happening but for certain requests. >> Besides the trace log I do not have much info about the request. In my >> module, I am restoring the my module context in a similar fashion as the ` >> ngx_http_realip_get_module_ctx`. What I noticed is `r->pool == 0`. Why >> would the r->pool be ever zero'ed? >> >> The crash is happening in my get_module_ctx function which was called >> immediately after returning from the ngx_http_internal_redirect call. Any >> ideas on how to go about resolving this is greatly appreciated. Thanks. > > It looks like there are issues with request reference counting > somewhere in your code, and the request is freed in the > ngx_http_internal_redirect() call. Review your code logic and > corresponding request reference counting, it should help. > > -- > Maxim Dounin > http://mdounin.ru/ > _______________________________________________ > nginx-devel mailing list -- nginx-devel at nginx.org > To unsubscribe send an email to nginx-devel-leave at nginx.org From mdounin at mdounin.ru Thu Jul 28 03:05:33 2022 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 28 Jul 2022 06:05:33 +0300 Subject: nginx crash In-Reply-To: <568013AB-8211-4189-B916-2FF84412219B@gmail.com> References: <568013AB-8211-4189-B916-2FF84412219B@gmail.com> Message-ID: Hello! On Wed, Jul 27, 2022 at 08:40:41AM -0700, Dk Jack wrote: > In my code I am not modifying the reference count. Could you let > me if there any function calls if invoked would indirectly > update the reference count? There are quite a few functions and/or return codes which might indirectly modify request reference counter, including the ngx_http_internal_redirect() function (which increments reference count and assumes it is decremented by further processing), request body reading code (which increments reference counter and assumes it is decremented by the body handler) and ngx_http_finalize_request() (which is normally used to decrement reference counter). Further, lack of explicit reference counting might be the issue if you are trying to do something non-trivial, and, for example, calling functions which might indirectly finalize the request. That's why I've suggested to review the code logic in the first place. -- Maxim Dounin http://mdounin.ru/ From dnj0496 at gmail.com Thu Jul 28 03:16:55 2022 From: dnj0496 at gmail.com (Dk Jack) Date: Wed, 27 Jul 2022 20:16:55 -0700 Subject: nginx crash In-Reply-To: References: Message-ID: Thanks Dounin, I will look through my code. As I said, I try not to touch nginx data structure or variables in my module. Is there a guideline document that details when a module should increment or decrement reference count value. When invoking some of these functions? Thanks, Dk. > On Jul 27, 2022, at 8:05 PM, Maxim Dounin wrote: > > Hello! > >> On Wed, Jul 27, 2022 at 08:40:41AM -0700, Dk Jack wrote: >> >> In my code I am not modifying the reference count. Could you let >> me if there any function calls if invoked would indirectly >> update the reference count? > > There are quite a few functions and/or return codes which might > indirectly modify request reference counter, including the > ngx_http_internal_redirect() function (which increments reference > count and assumes it is decremented by further processing), > request body reading code (which increments reference counter and > assumes it is decremented by the body handler) and > ngx_http_finalize_request() (which is normally used to decrement > reference counter). Further, lack of explicit reference counting > might be the issue if you are trying to do something non-trivial, > and, for example, calling functions which might indirectly > finalize the request. That's why I've suggested to review the > code logic in the first place. > > -- > Maxim Dounin > http://mdounin.ru/ > _______________________________________________ > nginx-devel mailing list -- nginx-devel at nginx.org > To unsubscribe send an email to nginx-devel-leave at nginx.org