From hunter at nocservice.biz  Mon Aug  1 06:40:24 2022
From: hunter at nocservice.biz (=?iso-8859-1?q?Serhii_Smitiienko?=)
Date: Mon, 01 Aug 2022 09:40:24 +0300
Subject: [PATCH] Allow external auth server to return http status code 429
 (too many
 requests)
Message-ID: <2baa3e46385ac3c671d2.1659336024@Nexus>

# HG changeset patch
# User Serhii Smitiienko <hunter at nocservice.biz>
# Date 1659334979 -10800
#      Mon Aug 01 09:22:59 2022 +0300
# Node ID 2baa3e46385ac3c671d257dfe715bff54c896855
# Parent  069a4813e8d6d7ec662d282a10f5f7062ebd817f
Allow external auth server to return http status code 429 (too many requests)

diff -r 069a4813e8d6 -r 2baa3e46385a src/http/modules/ngx_http_auth_request_module.c
--- a/src/http/modules/ngx_http_auth_request_module.c	Tue Jul 19 17:05:27 2022 +0300
+++ b/src/http/modules/ngx_http_auth_request_module.c	Mon Aug 01 09:22:59 2022 +0300
@@ -134,7 +134,7 @@
 
         /* return appropriate status */
 
-        if (ctx->status == NGX_HTTP_FORBIDDEN) {
+        if (ctx->status == NGX_HTTP_FORBIDDEN || ctx->status == NGX_HTTP_TOO_MANY_REQUESTS) {
             return ctx->status;
         }
 


From pluknet at nginx.com  Tue Aug  2 13:48:56 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Tue, 2 Aug 2022 17:48:56 +0400
Subject: [PATCH 1 of 8] QUIC: treat qc->error == -1 as a missing error
In-Reply-To: <951d7116f37dc39d9eba.1655999919@arut-laptop>
References: <patchbomb.1655999918@arut-laptop>
 <951d7116f37dc39d9eba.1655999919@arut-laptop>
Message-ID: <20220802134856.y5uju2ko33y45opf@Y9MQ9X2QVV>

On Thu, Jun 23, 2022 at 07:58:39PM +0400, Roman Arutyunyan wrote:
> # HG changeset patch
> # User Roman Arutyunyan <arut at nginx.com>
> # Date 1655889315 -14400
> #      Wed Jun 22 13:15:15 2022 +0400
> # Branch quic
> # Node ID 951d7116f37dc39d9eba20ceae49434592ce4677
> # Parent  5b1011b5702b5c5db2ba3d392a4da25596183cc2
> QUIC: treat qc->error == -1 as a missing error.
> 
> Previously, zero was used for this purpose.  However, NGX_QUIC_ERR_NO_ERROR is
> zero too.  As a result, NGX_QUIC_ERR_NO_ERROR was changed to
> NGX_QUIC_ERR_INTERNAL_ERROR when closing a QUIC connection.
> 
> diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c
> --- a/src/event/quic/ngx_event_quic.c
> +++ b/src/event/quic/ngx_event_quic.c
> @@ -317,6 +317,8 @@ ngx_quic_new_connection(ngx_connection_t
>      qc->congestion.ssthresh = (size_t) -1;
>      qc->congestion.recovery_start = ngx_current_msec;
>  
> +    qc->error = (ngx_uint_t) -1;
> +
>      if (pkt->validated && pkt->retried) {
>          qc->tp.retry_scid.len = pkt->dcid.len;
>          qc->tp.retry_scid.data = ngx_pstrdup(c->pool, &pkt->dcid);
> @@ -523,7 +525,7 @@ ngx_quic_close_connection(ngx_connection
>                  qc->error = NGX_QUIC_ERR_NO_ERROR;
>  
>              } else {
> -                if (qc->error == 0 && !qc->error_app) {
> +                if (qc->error == (ngx_uint_t) -1 && !qc->error_app) {
>                      qc->error = NGX_QUIC_ERR_INTERNAL_ERROR;
>                  }
>  

A former initialization is in ngx_quic_handle_payload().
Leaving it there makes connection close on the error path, for example,
on ngx_quic_set_path() failure, with a wrong NO_ERROR code.

I'd just leave (a new value of) initialization in place:
ngx_quic_new_connection() gives an established quic connection or nothing,
it cannot fail with qc initialized. ngx_quic_handle_payload() is called
right after, and is a point of packet decryption, which implies qc->error
reset.  This makes it a suitable place.


From pluknet at nginx.com  Tue Aug  2 15:44:11 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Tue, 2 Aug 2022 19:44:11 +0400
Subject: [PATCH 4 of 8] QUIC: removed ngx_quic_shutdown_connection()
In-Reply-To: <1912c09e0e4d746ec0a4.1655999922@arut-laptop>
References: <patchbomb.1655999918@arut-laptop>
 <1912c09e0e4d746ec0a4.1655999922@arut-laptop>
Message-ID: <20220802154411.l5hfvk5wnc2pmklb@Y9MQ9X2QVV>

On Thu, Jun 23, 2022 at 07:58:42PM +0400, Roman Arutyunyan wrote:
> # HG changeset patch
> # User Roman Arutyunyan <arut at nginx.com>
> # Date 1655904279 -14400
> #      Wed Jun 22 17:24:39 2022 +0400
> # Branch quic
> # Node ID 1912c09e0e4d746ec0a4c2140b6d2046b283b647
> # Parent  ea555f7caec372e93e1b9a11ed822002e97ed58b
> QUIC: removed ngx_quic_shutdown_connection().
> 
> HTTP/3 shutdown is now controlled by HTTP/3 layer.  QUIC connection is
> finalized after reaching requests limit when all request streams have finished.

Finalizing QUIC connection from HTTP/3 connection cleanup handler
prevents the last stream cleanup handler from regular run.  Normally
it is called next, but after QUIC shutdown initiated, that's too late,
and stream FIN won't be sent.


From xeioex at nginx.com  Wed Aug  3 05:16:00 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 03 Aug 2022 05:16:00 +0000
Subject: [njs] Fixed deprecation warnings introduced in beaff2c39864.
Message-ID: <hg.bb7a3a0017b2.1659503760.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/bb7a3a0017b2
branches:  
changeset: 1919:bb7a3a0017b2
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Aug 02 20:40:20 2022 -0700
description:
Fixed deprecation warnings introduced in beaff2c39864.

Previously, deprecated and non-deprecated properties shared
a common handler.

diffstat:

 nginx/ngx_http_js_module.c |  24 ++++++++++++++++--------
 nginx/ngx_js.h             |   9 ++++++---
 2 files changed, 22 insertions(+), 11 deletions(-)

diffs (119 lines):

diff -r beaff2c39864 -r bb7a3a0017b2 nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c	Mon Jul 25 18:40:24 2022 -0700
+++ b/nginx/ngx_http_js_module.c	Tue Aug 02 20:40:20 2022 -0700
@@ -501,7 +501,7 @@ static njs_external_t  ngx_http_js_ext_r
         .name.string = njs_str("requestBody"),
         .u.property = {
             .handler = ngx_http_js_ext_get_request_body,
-            .magic32 = NGX_JS_STRING,
+            .magic32 = NGX_JS_STRING | NGX_JS_DEPRECATED,
         }
     },
 
@@ -537,7 +537,7 @@ static njs_external_t  ngx_http_js_ext_r
         .name.string = njs_str("responseBody"),
         .u.property = {
             .handler = ngx_http_js_ext_get_response_body,
-            .magic32 = NGX_JS_STRING,
+            .magic32 = NGX_JS_STRING | NGX_JS_DEPRECATED,
         }
     },
 
@@ -2565,6 +2565,7 @@ ngx_http_js_ext_get_request_body(njs_vm_
 {
     u_char              *p, *body;
     size_t               len;
+    uint32_t             buffer_type;
     ngx_buf_t           *buf;
     njs_int_t            ret;
     njs_value_t         *request_body;
@@ -2572,7 +2573,9 @@ ngx_http_js_ext_get_request_body(njs_vm_
     ngx_http_js_ctx_t   *ctx;
     ngx_http_request_t  *r;
 
-    njs_deprecated(vm, "r.requestBody");
+    if (njs_vm_prop_magic32(prop) & NGX_JS_DEPRECATED) {
+        njs_deprecated(vm, "r.requestBody");
+    }
 
     r = njs_vm_external(vm, ngx_http_js_request_proto_id, value);
     if (r == NULL) {
@@ -2582,9 +2585,10 @@ ngx_http_js_ext_get_request_body(njs_vm_
 
     ctx = ngx_http_get_module_ctx(r, ngx_http_js_module);
     request_body = (njs_value_t *) &ctx->request_body;
+    buffer_type = ngx_js_buffer_type(njs_vm_prop_magic32(prop));
 
     if (!njs_value_is_null(request_body)) {
-        if ((njs_vm_prop_magic32(prop) == NGX_JS_BUFFER)
+        if ((buffer_type == NGX_JS_BUFFER)
             == (uint32_t) njs_value_is_buffer(request_body))
         {
             njs_value_assign(retval, request_body);
@@ -2636,7 +2640,7 @@ ngx_http_js_ext_get_request_body(njs_vm_
 
 done:
 
-    ret = ngx_js_prop(vm, njs_vm_prop_magic32(prop), request_body, body, len);
+    ret = ngx_js_prop(vm, buffer_type, request_body, body, len);
     if (ret != NJS_OK) {
         return NJS_ERROR;
     }
@@ -3412,13 +3416,16 @@ ngx_http_js_ext_get_response_body(njs_vm
 {
     size_t               len;
     u_char              *p;
+    uint32_t             buffer_type;
     njs_int_t            ret;
     ngx_buf_t           *b;
     njs_value_t         *response_body;
     ngx_http_js_ctx_t   *ctx;
     ngx_http_request_t  *r;
 
-    njs_deprecated(vm, "r.responseBody");
+    if (njs_vm_prop_magic32(prop) & NGX_JS_DEPRECATED) {
+        njs_deprecated(vm, "r.responseBody");
+    }
 
     r = njs_vm_external(vm, ngx_http_js_request_proto_id, value);
     if (r == NULL) {
@@ -3428,9 +3435,10 @@ ngx_http_js_ext_get_response_body(njs_vm
 
     ctx = ngx_http_get_module_ctx(r, ngx_http_js_module);
     response_body = (njs_value_t *) &ctx->response_body;
+    buffer_type = ngx_js_buffer_type(njs_vm_prop_magic32(prop));
 
     if (!njs_value_is_null(response_body)) {
-        if ((njs_vm_prop_magic32(prop) == NGX_JS_BUFFER)
+        if ((buffer_type == NGX_JS_BUFFER)
             == (uint32_t) njs_value_is_buffer(response_body))
         {
             njs_value_assign(retval, response_body);
@@ -3457,7 +3465,7 @@ ngx_http_js_ext_get_response_body(njs_vm
         ngx_memcpy(p, b->pos, len);
     }
 
-    ret = ngx_js_prop(vm, njs_vm_prop_magic32(prop), response_body, p, len);
+    ret = ngx_js_prop(vm, buffer_type, response_body, p, len);
     if (ret != NJS_OK) {
         return NJS_ERROR;
     }
diff -r beaff2c39864 -r bb7a3a0017b2 nginx/ngx_js.h
--- a/nginx/ngx_js.h	Mon Jul 25 18:40:24 2022 -0700
+++ b/nginx/ngx_js.h	Tue Aug 02 20:40:20 2022 -0700
@@ -15,9 +15,12 @@
 #include <njs.h>
 
 
-#define NGX_JS_UNSET   0
-#define NGX_JS_STRING  1
-#define NGX_JS_BUFFER  2
+#define NGX_JS_UNSET        0
+#define NGX_JS_DEPRECATED   1
+#define NGX_JS_STRING       2
+#define NGX_JS_BUFFER       4
+
+#define ngx_js_buffer_type(btype) ((btype) & ~NGX_JS_DEPRECATED)
 
 
 typedef ngx_pool_t *(*ngx_external_pool_pt)(njs_vm_t *vm, njs_external_ptr_t e);


From xeioex at nginx.com  Wed Aug  3 05:16:02 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 03 Aug 2022 05:16:02 +0000
Subject: [njs] Stream: fixed "js_fetch_timeout" introduced in b20de7bcee61
 (0.7.4).
Message-ID: <hg.076cdf8a1ced.1659503762.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/076cdf8a1ced
branches:  
changeset: 1920:076cdf8a1ced
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Aug 02 20:42:04 2022 -0700
description:
Stream: fixed "js_fetch_timeout" introduced in b20de7bcee61 (0.7.4).

Previously, the value specified in js_fetch_timeout directive was
ignored for stream module.

diffstat:

 nginx/ngx_stream_js_module.c |  10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

diffs (20 lines):

diff -r bb7a3a0017b2 -r 076cdf8a1ced nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c	Tue Aug 02 20:40:20 2022 -0700
+++ b/nginx/ngx_stream_js_module.c	Tue Aug 02 20:42:04 2022 -0700
@@ -1510,11 +1510,11 @@ ngx_stream_js_resolver_timeout(njs_vm_t 
 static ngx_msec_t
 ngx_stream_js_fetch_timeout(njs_vm_t *vm, ngx_stream_session_t *s)
 {
-    ngx_stream_core_srv_conf_t  *cscf;
-
-    cscf = ngx_stream_get_module_srv_conf(s, ngx_stream_js_module);
-
-    return cscf->resolver_timeout;
+    ngx_stream_js_srv_conf_t  *jscf;
+
+    jscf = ngx_stream_get_module_srv_conf(s, ngx_stream_js_module);
+
+    return jscf->timeout;
 }
 
 


From xeioex at nginx.com  Wed Aug  3 05:16:04 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 03 Aug 2022 05:16:04 +0000
Subject: [njs] Modules: extending allowed context for js directives.
Message-ID: <hg.648a8092c05a.1659503764.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/648a8092c05a
branches:  
changeset: 1921:648a8092c05a
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Aug 02 20:45:17 2022 -0700
description:
Modules: extending allowed context for js directives.

HTTP: js_import, js_path, js_set and js_var are allowed in server and
location contexts. js_content, js_body_filter and js_header_filter
are allowed in 'if' context.
Stream: js_import, js_path, js_set and js_var are allowed in server context.

This closes #566 issue on Github.

diffstat:

 nginx/ngx_http_js_module.c   |  293 +++++++++++++++++++++++++++---------------
 nginx/ngx_stream_js_module.c |  281 ++++++++++++++++++++++++++--------------
 2 files changed, 372 insertions(+), 202 deletions(-)

diffs (truncated from 1008 to 1000 lines):

diff -r 076cdf8a1ced -r 648a8092c05a nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c	Tue Aug 02 20:42:04 2022 -0700
+++ b/nginx/ngx_http_js_module.c	Tue Aug 02 20:45:17 2022 -0700
@@ -21,10 +21,7 @@ typedef struct {
     njs_vm_t              *vm;
     ngx_array_t           *imports;
     ngx_array_t           *paths;
-} ngx_http_js_main_conf_t;
-
-
-typedef struct {
+
     ngx_str_t              content;
     ngx_str_t              header_filter;
     ngx_str_t              body_filter;
@@ -271,8 +268,10 @@ static char *ngx_http_js_content(ngx_con
     void *conf);
 static char *ngx_http_js_body_filter_set(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf);
-static void *ngx_http_js_create_main_conf(ngx_conf_t *cf);
-static char *ngx_http_js_init_main_conf(ngx_conf_t *cf, void *conf);
+static ngx_int_t ngx_http_js_merge_vm(ngx_conf_t *cf,
+    ngx_http_js_loc_conf_t *conf, ngx_http_js_loc_conf_t *prev);
+static ngx_int_t ngx_http_js_init_conf_vm(ngx_conf_t *cf,
+    ngx_http_js_loc_conf_t *conf);
 static void *ngx_http_js_create_loc_conf(ngx_conf_t *cf);
 static char *ngx_http_js_merge_loc_conf(ngx_conf_t *cf, void *parent,
     void *child);
@@ -298,49 +297,49 @@ static ngx_conf_bitmask_t  ngx_http_js_s
 static ngx_command_t  ngx_http_js_commands[] = {
 
     { ngx_string("js_import"),
-      NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE13,
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE13,
       ngx_http_js_import,
-      NGX_HTTP_MAIN_CONF_OFFSET,
+      NGX_HTTP_LOC_CONF_OFFSET,
       0,
       NULL },
 
     { ngx_string("js_path"),
-      NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1,
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
       ngx_conf_set_str_array_slot,
-      NGX_HTTP_MAIN_CONF_OFFSET,
-      offsetof(ngx_http_js_main_conf_t, paths),
+      NGX_HTTP_LOC_CONF_OFFSET,
+      offsetof(ngx_http_js_loc_conf_t, paths),
       NULL },
 
     { ngx_string("js_set"),
-      NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE2,
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
       ngx_http_js_set,
       0,
       0,
       NULL },
 
     { ngx_string("js_var"),
-      NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE12,
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE12,
       ngx_http_js_var,
       0,
       0,
       NULL },
 
     { ngx_string("js_content"),
-      NGX_HTTP_LOC_CONF|NGX_HTTP_LMT_CONF|NGX_CONF_TAKE1,
+      NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF|NGX_HTTP_LMT_CONF|NGX_CONF_TAKE1,
       ngx_http_js_content,
       NGX_HTTP_LOC_CONF_OFFSET,
       0,
       NULL },
 
     { ngx_string("js_header_filter"),
-      NGX_HTTP_LOC_CONF|NGX_HTTP_LMT_CONF|NGX_CONF_TAKE1,
+      NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF|NGX_HTTP_LMT_CONF|NGX_CONF_TAKE1,
       ngx_conf_set_str_slot,
       NGX_HTTP_LOC_CONF_OFFSET,
       offsetof(ngx_http_js_loc_conf_t, header_filter),
       NULL },
 
     { ngx_string("js_body_filter"),
-      NGX_HTTP_LOC_CONF|NGX_HTTP_LMT_CONF|NGX_CONF_TAKE12,
+      NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF|NGX_HTTP_LMT_CONF|NGX_CONF_TAKE12,
       ngx_http_js_body_filter_set,
       NGX_HTTP_LOC_CONF_OFFSET,
       0,
@@ -414,8 +413,8 @@ static ngx_http_module_t  ngx_http_js_mo
     NULL,                          /* preconfiguration */
     ngx_http_js_init,              /* postconfiguration */
 
-    ngx_http_js_create_main_conf,  /* create main configuration */
-    ngx_http_js_init_main_conf,    /* init main configuration */
+    NULL,                          /* create main configuration */
+    NULL,                          /* init main configuration */
 
     NULL,                          /* create server configuration */
     NULL,                          /* merge server configuration */
@@ -1212,14 +1211,14 @@ ngx_http_js_variable_var(ngx_http_reques
 static ngx_int_t
 ngx_http_js_init_vm(ngx_http_request_t *r)
 {
-    njs_int_t                 rc;
-    ngx_str_t                 exception;
-    ngx_http_js_ctx_t        *ctx;
-    ngx_pool_cleanup_t       *cln;
-    ngx_http_js_main_conf_t  *jmcf;
-
-    jmcf = ngx_http_get_module_main_conf(r, ngx_http_js_module);
-    if (jmcf->vm == NULL) {
+    njs_int_t                rc;
+    ngx_str_t                exception;
+    ngx_http_js_ctx_t       *ctx;
+    ngx_pool_cleanup_t      *cln;
+    ngx_http_js_loc_conf_t  *jlcf;
+
+    jlcf = ngx_http_get_module_loc_conf(r, ngx_http_js_module);
+    if (jlcf->vm == NULL) {
         return NGX_DECLINED;
     }
 
@@ -1240,7 +1239,7 @@ ngx_http_js_init_vm(ngx_http_request_t *
         return NGX_OK;
     }
 
-    ctx->vm = njs_vm_clone(jmcf->vm, r);
+    ctx->vm = njs_vm_clone(jlcf->vm, r);
     if (ctx->vm == NULL) {
         return NGX_ERROR;
     }
@@ -4150,11 +4149,116 @@ ngx_http_js_handle_event(ngx_http_reques
 }
 
 
-static char *
-ngx_http_js_init_main_conf(ngx_conf_t *cf, void *conf)
+static ngx_int_t
+ngx_http_js_merge_vm(ngx_conf_t *cf, ngx_http_js_loc_conf_t *conf,
+    ngx_http_js_loc_conf_t *prev)
 {
-    ngx_http_js_main_conf_t *jmcf = conf;
-
+    ngx_str_t             *path, *s;
+    ngx_uint_t             i;
+    ngx_array_t           *imports, *paths;
+    ngx_http_js_import_t  *import, *pi;
+
+    if (prev->imports != NGX_CONF_UNSET_PTR && prev->vm == NULL) {
+        if (ngx_http_js_init_conf_vm(cf, prev) != NGX_OK) {
+            return NGX_ERROR;
+        }
+    }
+
+    if (conf->imports == NGX_CONF_UNSET_PTR
+        && conf->paths == NGX_CONF_UNSET_PTR)
+    {
+        if (prev->vm != NULL) {
+            conf->imports = prev->imports;
+            conf->paths = prev->paths;
+            conf->vm = prev->vm;
+            return NGX_OK;
+        }
+    }
+
+    if (prev->imports != NGX_CONF_UNSET_PTR) {
+        if (conf->imports == NGX_CONF_UNSET_PTR) {
+            conf->imports = prev->imports;
+
+        } else {
+            imports = ngx_array_create(cf->pool, 4,
+                                       sizeof(ngx_http_js_import_t));
+            if (imports == NULL) {
+                return NGX_ERROR;
+            }
+
+            pi = prev->imports->elts;
+
+            for (i = 0; i < prev->imports->nelts; i++) {
+                import = ngx_array_push(imports);
+                if (import == NULL) {
+                    return NGX_ERROR;
+                }
+
+                *import = pi[i];
+            }
+
+            pi = conf->imports->elts;
+
+            for (i = 0; i < conf->imports->nelts; i++) {
+                import = ngx_array_push(imports);
+                if (import == NULL) {
+                    return NGX_ERROR;
+                }
+
+                *import = pi[i];
+            }
+
+            conf->imports = imports;
+        }
+    }
+
+    if (prev->paths != NGX_CONF_UNSET_PTR) {
+        if (conf->paths == NGX_CONF_UNSET_PTR) {
+            conf->paths = prev->paths;
+
+        } else {
+            paths = ngx_array_create(cf->pool, 4, sizeof(ngx_str_t));
+            if (paths == NULL) {
+                return NGX_ERROR;
+            }
+
+            s = prev->imports->elts;
+
+            for (i = 0; i < prev->paths->nelts; i++) {
+                path = ngx_array_push(paths);
+                if (path == NULL) {
+                    return NGX_ERROR;
+                }
+
+                *path = s[i];
+            }
+
+            s = conf->imports->elts;
+
+            for (i = 0; i < conf->paths->nelts; i++) {
+                path = ngx_array_push(paths);
+                if (path == NULL) {
+                    return NGX_ERROR;
+                }
+
+                *path = s[i];
+            }
+
+            conf->paths = paths;
+        }
+    }
+
+    if (conf->imports == NGX_CONF_UNSET_PTR) {
+        return NGX_OK;
+    }
+
+    return ngx_http_js_init_conf_vm(cf, conf);
+}
+
+
+static ngx_int_t
+ngx_http_js_init_conf_vm(ngx_conf_t *cf, ngx_http_js_loc_conf_t *conf)
+{
     size_t                 size;
     u_char                *start, *end, *p;
     ngx_str_t             *m, file;
@@ -4170,14 +4274,10 @@ ngx_http_js_init_main_conf(ngx_conf_t *c
     static const njs_str_t line_number_key = njs_str("lineNumber");
     static const njs_str_t file_name_key = njs_str("fileName");
 
-    if (jmcf->imports == NGX_CONF_UNSET_PTR) {
-        return NGX_CONF_OK;
-    }
-
     size = 0;
 
-    import = jmcf->imports->elts;
-    for (i = 0; i < jmcf->imports->nelts; i++) {
+    import = conf->imports->elts;
+    for (i = 0; i < conf->imports->nelts; i++) {
 
         /* import <name> from '<path>'; globalThis.<name> = <name>; */
 
@@ -4188,12 +4288,12 @@ ngx_http_js_init_main_conf(ngx_conf_t *c
 
     start = ngx_pnalloc(cf->pool, size);
     if (start == NULL) {
-        return NGX_CONF_ERROR;
+        return NGX_ERROR;
     }
 
     p = start;
-    import = jmcf->imports->elts;
-    for (i = 0; i < jmcf->imports->nelts; i++) {
+    import = conf->imports->elts;
+    for (i = 0; i < conf->imports->nelts; i++) {
 
         /* import <name> from '<path>'; globalThis.<name> = <name>; */
 
@@ -4223,103 +4323,103 @@ ngx_http_js_init_main_conf(ngx_conf_t *c
     options.file.start = file.data;
     options.file.length = file.len;
 
-    jmcf->vm = njs_vm_create(&options);
-    if (jmcf->vm == NULL) {
+    conf->vm = njs_vm_create(&options);
+    if (conf->vm == NULL) {
         ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "failed to create js VM");
-        return NGX_CONF_ERROR;
+        return NGX_ERROR;
     }
 
     cln = ngx_pool_cleanup_add(cf->pool, 0);
     if (cln == NULL) {
-        return NGX_CONF_ERROR;
+        return NGX_ERROR;
     }
 
     cln->handler = ngx_http_js_cleanup_vm;
-    cln->data = jmcf->vm;
+    cln->data = conf->vm;
 
     path.start = ngx_cycle->conf_prefix.data;
     path.length = ngx_cycle->conf_prefix.len;
 
-    rc = njs_vm_add_path(jmcf->vm, &path);
+    rc = njs_vm_add_path(conf->vm, &path);
     if (rc != NJS_OK) {
         ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "failed to add \"js_path\"");
-        return NGX_CONF_ERROR;
-    }
-
-    if (jmcf->paths != NGX_CONF_UNSET_PTR) {
-        m = jmcf->paths->elts;
-
-        for (i = 0; i < jmcf->paths->nelts; i++) {
+        return NGX_ERROR;
+    }
+
+    if (conf->paths != NGX_CONF_UNSET_PTR) {
+        m = conf->paths->elts;
+
+        for (i = 0; i < conf->paths->nelts; i++) {
             if (ngx_conf_full_name(cf->cycle, &m[i], 1) != NGX_OK) {
-                return NGX_CONF_ERROR;
+                return NGX_ERROR;
             }
 
             path.start = m[i].data;
             path.length = m[i].len;
 
-            rc = njs_vm_add_path(jmcf->vm, &path);
+            rc = njs_vm_add_path(conf->vm, &path);
             if (rc != NJS_OK) {
                 ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                               "failed to add \"js_path\"");
-                return NGX_CONF_ERROR;
+                return NGX_ERROR;
             }
         }
     }
 
-    ngx_http_js_request_proto_id = njs_vm_external_prototype(jmcf->vm,
+    ngx_http_js_request_proto_id = njs_vm_external_prototype(conf->vm,
                                            ngx_http_js_ext_request,
                                            njs_nitems(ngx_http_js_ext_request));
     if (ngx_http_js_request_proto_id < 0) {
         ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                       "failed to add js request proto");
-        return NGX_CONF_ERROR;
-    }
-
-    rc = ngx_js_core_init(jmcf->vm, cf->log);
+        return NGX_ERROR;
+    }
+
+    rc = ngx_js_core_init(conf->vm, cf->log);
     if (njs_slow_path(rc != NJS_OK)) {
-        return NGX_CONF_ERROR;
+        return NGX_ERROR;
     }
 
     end = start + size;
 
-    rc = njs_vm_compile(jmcf->vm, &start, end);
+    rc = njs_vm_compile(conf->vm, &start, end);
 
     if (rc != NJS_OK) {
-        njs_value_assign(&exception, njs_vm_retval(jmcf->vm));
-        njs_vm_retval_string(jmcf->vm, &text);
-
-        value = njs_vm_object_prop(jmcf->vm, njs_value_arg(&exception),
+        njs_value_assign(&exception, njs_vm_retval(conf->vm));
+        njs_vm_retval_string(conf->vm, &text);
+
+        value = njs_vm_object_prop(conf->vm, njs_value_arg(&exception),
                                    &file_name_key, &lvalue);
         if (value == NULL) {
-            value = njs_vm_object_prop(jmcf->vm, njs_value_arg(&exception),
+            value = njs_vm_object_prop(conf->vm, njs_value_arg(&exception),
                                        &line_number_key, &lvalue);
 
             if (value != NULL) {
                 i = njs_value_number(value) - 1;
 
-                if (i < jmcf->imports->nelts) {
-                    import = jmcf->imports->elts;
+                if (i < conf->imports->nelts) {
+                    import = conf->imports->elts;
                     ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                                   "%*s, included in %s:%ui", text.length,
                                   text.start, import[i].file, import[i].line);
-                    return NGX_CONF_ERROR;
+                    return NGX_ERROR;
                 }
             }
         }
 
         ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "%*s", text.length,
                       text.start);
-        return NGX_CONF_ERROR;
+        return NGX_ERROR;
     }
 
     if (start != end) {
         ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                       "extra characters in js script: \"%*s\"",
                       end - start, start);
-        return NGX_CONF_ERROR;
-    }
-
-    return NGX_CONF_OK;
+        return NGX_ERROR;
+    }
+
+    return NGX_OK;
 }
 
 
@@ -4339,7 +4439,7 @@ ngx_http_js_init(ngx_conf_t *cf)
 static char *
 ngx_http_js_import(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
 {
-    ngx_http_js_main_conf_t *jmcf = conf;
+    ngx_http_js_loc_conf_t *jlcf = conf;
 
     u_char                *p, *end, c;
     ngx_int_t              from;
@@ -4420,15 +4520,15 @@ ngx_http_js_import(ngx_conf_t *cf, ngx_c
         return NGX_CONF_ERROR;
     }
 
-    if (jmcf->imports == NGX_CONF_UNSET_PTR) {
-        jmcf->imports = ngx_array_create(cf->pool, 4,
+    if (jlcf->imports == NGX_CONF_UNSET_PTR) {
+        jlcf->imports = ngx_array_create(cf->pool, 4,
                                          sizeof(ngx_http_js_import_t));
-        if (jmcf->imports == NULL) {
+        if (jlcf->imports == NULL) {
             return NGX_CONF_ERROR;
         }
     }
 
-    import = ngx_array_push(jmcf->imports);
+    import = ngx_array_push(jlcf->imports);
     if (import == NULL) {
         return NGX_CONF_ERROR;
     }
@@ -4595,29 +4695,6 @@ ngx_http_js_body_filter_set(ngx_conf_t *
 
 
 static void *
-ngx_http_js_create_main_conf(ngx_conf_t *cf)
-{
-    ngx_http_js_main_conf_t  *conf;
-
-    conf = ngx_pcalloc(cf->pool, sizeof(ngx_http_js_main_conf_t));
-    if (conf == NULL) {
-        return NULL;
-    }
-
-    /*
-     * set by ngx_pcalloc():
-     *
-     *     conf->vm = NULL;
-     */
-
-    conf->paths = NGX_CONF_UNSET_PTR;
-    conf->imports = NGX_CONF_UNSET_PTR;
-
-    return conf;
-}
-
-
-static void *
 ngx_http_js_create_loc_conf(ngx_conf_t *cf)
 {
     ngx_http_js_loc_conf_t  *conf;
@@ -4630,6 +4707,7 @@ ngx_http_js_create_loc_conf(ngx_conf_t *
     /*
      * set by ngx_pcalloc():
      *
+     *     conf->vm = NULL;
      *     conf->content = { 0, NULL };
      *     conf->header_filter = { 0, NULL };
      *     conf->body_filter = { 0, NULL };
@@ -4639,6 +4717,9 @@ ngx_http_js_create_loc_conf(ngx_conf_t *
      *     conf->ssl_trusted_certificate = { 0, NULL };
      */
 
+    conf->paths = NGX_CONF_UNSET_PTR;
+    conf->imports = NGX_CONF_UNSET_PTR;
+
     conf->buffer_size = NGX_CONF_UNSET_SIZE;
     conf->max_response_body_size = NGX_CONF_UNSET_SIZE;
     conf->timeout = NGX_CONF_UNSET_MSEC;
@@ -4669,6 +4750,10 @@ ngx_http_js_merge_loc_conf(ngx_conf_t *c
     ngx_conf_merge_size_value(conf->max_response_body_size,
                               prev->max_response_body_size, 1048576);
 
+    if (ngx_http_js_merge_vm(cf, conf, prev) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
+
 #if (NGX_HTTP_SSL)
     ngx_conf_merge_str_value(conf->ssl_ciphers, prev->ssl_ciphers, "DEFAULT");
 
diff -r 076cdf8a1ced -r 648a8092c05a nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c	Tue Aug 02 20:42:04 2022 -0700
+++ b/nginx/ngx_stream_js_module.c	Tue Aug 02 20:45:17 2022 -0700
@@ -13,13 +13,6 @@
 
 
 typedef struct {
-    njs_vm_t              *vm;
-    ngx_array_t           *imports;
-    ngx_array_t           *paths;
-} ngx_stream_js_main_conf_t;
-
-
-typedef struct {
     ngx_str_t              name;
     ngx_str_t              path;
     u_char                *file;
@@ -28,6 +21,10 @@ typedef struct {
 
 
 typedef struct {
+    njs_vm_t              *vm;
+    ngx_array_t           *imports;
+    ngx_array_t           *paths;
+
     ngx_str_t              access;
     ngx_str_t              preread;
     ngx_str_t              filter;
@@ -144,8 +141,10 @@ static char *ngx_stream_js_set(ngx_conf_
     void *conf);
 static char *ngx_stream_js_var(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf);
-static void *ngx_stream_js_create_main_conf(ngx_conf_t *cf);
-static char *ngx_stream_js_init_main_conf(ngx_conf_t *cf, void *conf);
+static ngx_int_t ngx_stream_js_merge_vm(ngx_conf_t *cf,
+    ngx_stream_js_srv_conf_t *conf, ngx_stream_js_srv_conf_t *prev);
+static ngx_int_t ngx_stream_js_init_conf_vm(ngx_conf_t *cf,
+    ngx_stream_js_srv_conf_t *conf);
 static void *ngx_stream_js_create_srv_conf(ngx_conf_t *cf);
 static char *ngx_stream_js_merge_srv_conf(ngx_conf_t *cf, void *parent,
     void *child);
@@ -174,28 +173,28 @@ static ngx_conf_bitmask_t  ngx_stream_js
 static ngx_command_t  ngx_stream_js_commands[] = {
 
     { ngx_string("js_import"),
-      NGX_STREAM_MAIN_CONF|NGX_CONF_TAKE13,
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_TAKE13,
       ngx_stream_js_import,
-      NGX_STREAM_MAIN_CONF_OFFSET,
+      NGX_STREAM_SRV_CONF_OFFSET,
       0,
       NULL },
 
     { ngx_string("js_path"),
-      NGX_STREAM_MAIN_CONF|NGX_CONF_TAKE1,
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_TAKE1,
       ngx_conf_set_str_array_slot,
-      NGX_STREAM_MAIN_CONF_OFFSET,
-      offsetof(ngx_stream_js_main_conf_t, paths),
+      NGX_STREAM_SRV_CONF_OFFSET,
+      offsetof(ngx_stream_js_srv_conf_t, paths),
       NULL },
 
     { ngx_string("js_set"),
-      NGX_STREAM_MAIN_CONF|NGX_CONF_TAKE2,
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_TAKE2,
       ngx_stream_js_set,
       0,
       0,
       NULL },
 
     { ngx_string("js_var"),
-      NGX_STREAM_MAIN_CONF|NGX_CONF_TAKE12,
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_TAKE12,
       ngx_stream_js_var,
       0,
       0,
@@ -290,8 +289,8 @@ static ngx_stream_module_t  ngx_stream_j
     NULL,                           /* preconfiguration */
     ngx_stream_js_init,             /* postconfiguration */
 
-    ngx_stream_js_create_main_conf, /* create main configuration */
-    ngx_stream_js_init_main_conf,   /* init main configuration */
+    NULL,                           /* create main configuration */
+    NULL,                           /* init main configuration */
 
     ngx_stream_js_create_srv_conf,  /* create server configuration */
     ngx_stream_js_merge_srv_conf,   /* merge server configuration */
@@ -820,14 +819,14 @@ ngx_stream_js_variable_var(ngx_stream_se
 static ngx_int_t
 ngx_stream_js_init_vm(ngx_stream_session_t *s)
 {
-    njs_int_t                   rc;
-    ngx_str_t                   exception;
-    ngx_pool_cleanup_t         *cln;
-    ngx_stream_js_ctx_t        *ctx;
-    ngx_stream_js_main_conf_t  *jmcf;
-
-    jmcf = ngx_stream_get_module_main_conf(s, ngx_stream_js_module);
-    if (jmcf->vm == NULL) {
+    njs_int_t                  rc;
+    ngx_str_t                  exception;
+    ngx_pool_cleanup_t        *cln;
+    ngx_stream_js_ctx_t       *ctx;
+    ngx_stream_js_srv_conf_t  *jscf;
+
+    jscf = ngx_stream_get_module_srv_conf(s, ngx_stream_js_module);
+    if (jscf->vm == NULL) {
         return NGX_DECLINED;
     }
 
@@ -848,7 +847,7 @@ ngx_stream_js_init_vm(ngx_stream_session
         return NGX_OK;
     }
 
-    ctx->vm = njs_vm_clone(jmcf->vm, s);
+    ctx->vm = njs_vm_clone(jscf->vm, s);
     if (ctx->vm == NULL) {
         return NGX_ERROR;
     }
@@ -1569,11 +1568,116 @@ ngx_stream_js_handle_event(ngx_stream_se
 }
 
 
-static char *
-ngx_stream_js_init_main_conf(ngx_conf_t *cf, void *conf)
+static ngx_int_t
+ngx_stream_js_merge_vm(ngx_conf_t *cf, ngx_stream_js_srv_conf_t *conf,
+    ngx_stream_js_srv_conf_t *prev)
 {
-    ngx_stream_js_main_conf_t *jmcf = conf;
-
+    ngx_str_t               *path, *s;
+    ngx_uint_t               i;
+    ngx_array_t             *imports, *paths;
+    ngx_stream_js_import_t  *import, *pi;
+
+    if (prev->imports != NGX_CONF_UNSET_PTR && prev->vm == NULL) {
+        if (ngx_stream_js_init_conf_vm(cf, prev) != NGX_OK) {
+            return NGX_ERROR;
+        }
+    }
+
+    if (conf->imports == NGX_CONF_UNSET_PTR
+        && conf->paths == NGX_CONF_UNSET_PTR)
+    {
+        if (prev->vm != NULL) {
+            conf->imports = prev->imports;
+            conf->paths = prev->paths;
+            conf->vm = prev->vm;
+            return NGX_OK;
+        }
+    }
+
+    if (prev->imports != NGX_CONF_UNSET_PTR) {
+        if (conf->imports == NGX_CONF_UNSET_PTR) {
+            conf->imports = prev->imports;
+
+        } else {
+            imports = ngx_array_create(cf->pool, 4,
+                                       sizeof(ngx_stream_js_import_t));
+            if (imports == NULL) {
+                return NGX_ERROR;
+            }
+
+            pi = prev->imports->elts;
+
+            for (i = 0; i < prev->imports->nelts; i++) {
+                import = ngx_array_push(imports);
+                if (import == NULL) {
+                    return NGX_ERROR;
+                }
+
+                *import = pi[i];
+            }
+
+            pi = conf->imports->elts;
+
+            for (i = 0; i < conf->imports->nelts; i++) {
+                import = ngx_array_push(imports);
+                if (import == NULL) {
+                    return NGX_ERROR;
+                }
+
+                *import = pi[i];
+            }
+
+            conf->imports = imports;
+        }
+    }
+
+    if (prev->paths != NGX_CONF_UNSET_PTR) {
+        if (conf->paths == NGX_CONF_UNSET_PTR) {
+            conf->paths = prev->paths;
+
+        } else {
+            paths = ngx_array_create(cf->pool, 4, sizeof(ngx_str_t));
+            if (paths == NULL) {
+                return NGX_ERROR;
+            }
+
+            s = prev->imports->elts;
+
+            for (i = 0; i < prev->paths->nelts; i++) {
+                path = ngx_array_push(paths);
+                if (path == NULL) {
+                    return NGX_ERROR;
+                }
+
+                *path = s[i];
+            }
+
+            s = conf->imports->elts;
+
+            for (i = 0; i < conf->paths->nelts; i++) {
+                path = ngx_array_push(paths);
+                if (path == NULL) {
+                    return NGX_ERROR;
+                }
+
+                *path = s[i];
+            }
+
+            conf->paths = paths;
+        }
+    }
+
+    if (conf->imports == NGX_CONF_UNSET_PTR) {
+        return NGX_OK;
+    }
+
+    return ngx_stream_js_init_conf_vm(cf, conf);
+}
+
+
+static ngx_int_t
+ngx_stream_js_init_conf_vm(ngx_conf_t *cf, ngx_stream_js_srv_conf_t *conf)
+{
     size_t                   size;
     u_char                  *start, *end, *p;
     ngx_str_t               *m, file;
@@ -1589,14 +1693,10 @@ ngx_stream_js_init_main_conf(ngx_conf_t 
     static const njs_str_t line_number_key = njs_str("lineNumber");
     static const njs_str_t file_name_key = njs_str("fileName");
 
-    if (jmcf->imports == NGX_CONF_UNSET_PTR) {
-        return NGX_CONF_OK;
-    }
-
     size = 0;
 
-    import = jmcf->imports->elts;
-    for (i = 0; i < jmcf->imports->nelts; i++) {
+    import = conf->imports->elts;
+    for (i = 0; i < conf->imports->nelts; i++) {
         /* import <name> from '<path>'; globalThis.<name> = <name>; */
 
         size += sizeof("import  from '';") - 1 + import[i].name.len * 3
@@ -1606,12 +1706,12 @@ ngx_stream_js_init_main_conf(ngx_conf_t 
 
     start = ngx_pnalloc(cf->pool, size);
     if (start == NULL) {
-        return NGX_CONF_ERROR;
+        return NGX_ERROR;
     }
 
     p = start;
-    import = jmcf->imports->elts;
-    for (i = 0; i < jmcf->imports->nelts; i++) {
+    import = conf->imports->elts;
+    for (i = 0; i < conf->imports->nelts; i++) {
 
         /* import <name> from '<path>'; globalThis.<name> = <name>; */
 
@@ -1641,110 +1741,110 @@ ngx_stream_js_init_main_conf(ngx_conf_t 
     options.file.start = file.data;
     options.file.length = file.len;
 
-    jmcf->vm = njs_vm_create(&options);
-    if (jmcf->vm == NULL) {
+    conf->vm = njs_vm_create(&options);
+    if (conf->vm == NULL) {
         ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "failed to create js VM");
-        return NGX_CONF_ERROR;
+        return NGX_ERROR;
     }
 
     cln = ngx_pool_cleanup_add(cf->pool, 0);
     if (cln == NULL) {
-        return NGX_CONF_ERROR;
+        return NGX_ERROR;
     }
 
     cln->handler = ngx_stream_js_cleanup_vm;
-    cln->data = jmcf->vm;
+    cln->data = conf->vm;
 
     path.start = ngx_cycle->conf_prefix.data;
     path.length = ngx_cycle->conf_prefix.len;
 
-    rc = njs_vm_add_path(jmcf->vm, &path);
+    rc = njs_vm_add_path(conf->vm, &path);
     if (rc != NJS_OK) {
         ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "failed to add \"js_path\"");
-        return NGX_CONF_ERROR;
+        return NGX_ERROR;
     }
 
-    if (jmcf->paths != NGX_CONF_UNSET_PTR) {
-        m = jmcf->paths->elts;
-
-        for (i = 0; i < jmcf->paths->nelts; i++) {
+    if (conf->paths != NGX_CONF_UNSET_PTR) {
+        m = conf->paths->elts;
+
+        for (i = 0; i < conf->paths->nelts; i++) {
             if (ngx_conf_full_name(cf->cycle, &m[i], 1) != NGX_OK) {
-                return NGX_CONF_ERROR;
+                return NGX_ERROR;
             }
 
             path.start = m[i].data;
             path.length = m[i].len;
 
-            rc = njs_vm_add_path(jmcf->vm, &path);
+            rc = njs_vm_add_path(conf->vm, &path);
             if (rc != NJS_OK) {
                 ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                               "failed to add \"js_path\"");
-                return NGX_CONF_ERROR;
+                return NGX_ERROR;
             }
         }
     }
 
-    ngx_stream_js_session_proto_id = njs_vm_external_prototype(jmcf->vm,
+    ngx_stream_js_session_proto_id = njs_vm_external_prototype(conf->vm,
                                          ngx_stream_js_ext_session,
                                          njs_nitems(ngx_stream_js_ext_session));
     if (ngx_stream_js_session_proto_id < 0) {
         ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                       "failed to add js request proto");
-        return NGX_CONF_ERROR;
+        return NGX_ERROR;
     }
 
-    rc = ngx_js_core_init(jmcf->vm, cf->log);
+    rc = ngx_js_core_init(conf->vm, cf->log);
     if (njs_slow_path(rc != NJS_OK)) {
-        return NGX_CONF_ERROR;
+        return NGX_ERROR;
     }
 
     end = start + size;
 
-    rc = njs_vm_compile(jmcf->vm, &start, end);
+    rc = njs_vm_compile(conf->vm, &start, end);
 
     if (rc != NJS_OK) {
-        njs_value_assign(&exception, njs_vm_retval(jmcf->vm));
-        njs_vm_retval_string(jmcf->vm, &text);
-
-        value = njs_vm_object_prop(jmcf->vm, njs_value_arg(&exception),
+        njs_value_assign(&exception, njs_vm_retval(conf->vm));
+        njs_vm_retval_string(conf->vm, &text);
+
+        value = njs_vm_object_prop(conf->vm, njs_value_arg(&exception),
                                    &file_name_key, &lvalue);
         if (value == NULL) {
-            value = njs_vm_object_prop(jmcf->vm, njs_value_arg(&exception),
+            value = njs_vm_object_prop(conf->vm, njs_value_arg(&exception),
                                        &line_number_key, &lvalue);
 
             if (value != NULL) {
                 i = njs_value_number(value) - 1;
 
-                if (i < jmcf->imports->nelts) {
-                    import = jmcf->imports->elts;
+                if (i < conf->imports->nelts) {
+                    import = conf->imports->elts;
                     ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                                   "%*s, included in %s:%ui", text.length,
                                   text.start, import[i].file, import[i].line);
-                    return NGX_CONF_ERROR;
+                    return NGX_ERROR;
                 }
             }
         }
 
         ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "%*s", text.length,
                       text.start);
-        return NGX_CONF_ERROR;
+        return NGX_ERROR;
     }
 
     if (start != end) {
         ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                       "extra characters in js script: \"%*s\"",
                       end - start, start);
-        return NGX_CONF_ERROR;
+        return NGX_ERROR;
     }
 
-    return NGX_CONF_OK;
+    return NGX_OK;
 }
 
 
 static char *
 ngx_stream_js_import(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
 {
-    ngx_stream_js_main_conf_t *jmcf = conf;
+    ngx_stream_js_srv_conf_t *jscf = conf;
 
     u_char                  *p, *end, c;
     ngx_int_t               from;
@@ -1824,15 +1924,15 @@ ngx_stream_js_import(ngx_conf_t *cf, ngx
         return NGX_CONF_ERROR;
     }
 
-    if (jmcf->imports == NGX_CONF_UNSET_PTR) {
-        jmcf->imports = ngx_array_create(cf->pool, 4,
+    if (jscf->imports == NGX_CONF_UNSET_PTR) {
+        jscf->imports = ngx_array_create(cf->pool, 4,
                                          sizeof(ngx_stream_js_import_t));
-        if (jmcf->imports == NULL) {
+        if (jscf->imports == NULL) {
             return NGX_CONF_ERROR;
         }
     }
 
-    import = ngx_array_push(jmcf->imports);
+    import = ngx_array_push(jscf->imports);
     if (import == NULL) {
         return NGX_CONF_ERROR;
     }
@@ -1939,29 +2039,6 @@ ngx_stream_js_var(ngx_conf_t *cf, ngx_co
 
 
 static void *
-ngx_stream_js_create_main_conf(ngx_conf_t *cf)
-{
-    ngx_stream_js_main_conf_t  *conf;
-
-    conf = ngx_pcalloc(cf->pool, sizeof(ngx_stream_js_main_conf_t));
-    if (conf == NULL) {
-        return NULL;
-    }
-
-    /*
-     * set by ngx_pcalloc():
-     *
-     *     conf->vm = NULL;
-     */
-
-    conf->paths = NGX_CONF_UNSET_PTR;
-    conf->imports = NGX_CONF_UNSET_PTR;
-
-    return conf;
-}
-
-
-static void *
 ngx_stream_js_create_srv_conf(ngx_conf_t *cf)
 {
     ngx_stream_js_srv_conf_t  *conf;
@@ -1974,6 +2051,7 @@ ngx_stream_js_create_srv_conf(ngx_conf_t
     /*
      * set by ngx_pcalloc():
      *
+     *     conf->vm = NULL;
      *     conf->access = { 0, NULL };
      *     conf->preread = { 0, NULL };
      *     conf->filter = { 0, NULL };
@@ -1982,6 +2060,9 @@ ngx_stream_js_create_srv_conf(ngx_conf_t
      *     conf->ssl_trusted_certificate = { 0, NULL };
      */
 
+    conf->paths = NGX_CONF_UNSET_PTR;
+    conf->imports = NGX_CONF_UNSET_PTR;
+
     conf->buffer_size = NGX_CONF_UNSET_SIZE;
     conf->max_response_body_size = NGX_CONF_UNSET_SIZE;
     conf->timeout = NGX_CONF_UNSET_MSEC;
@@ -2009,6 +2090,10 @@ ngx_stream_js_merge_srv_conf(ngx_conf_t 
     ngx_conf_merge_size_value(conf->max_response_body_size,
                               prev->max_response_body_size, 1048576);


From xeioex at nginx.com  Thu Aug  4 03:51:01 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Thu, 04 Aug 2022 03:51:01 +0000
Subject: [njs] Types: updates ts types.
Message-ID: <hg.ffd0f3d3f6df.1659585061.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/ffd0f3d3f6df
branches:  
changeset: 1922:ffd0f3d3f6df
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Aug 03 20:26:24 2022 -0700
description:
Types: updates ts types.

diffstat:

 test/ts/test.ts            |   6 ++++++
 ts/ngx_http_js_module.d.ts |  10 ++++++++++
 ts/njs_core.d.ts           |  36 ++++++++++++++++++++++++++++++++++++
 ts/tsconfig.json           |   2 +-
 4 files changed, 53 insertions(+), 1 deletions(-)

diffs (136 lines):

diff -r 648a8092c05a -r ffd0f3d3f6df test/ts/test.ts
--- a/test/ts/test.ts	Tue Aug 02 20:45:17 2022 -0700
+++ b/test/ts/test.ts	Wed Aug 03 20:26:24 2022 -0700
@@ -174,9 +174,15 @@ function timers() {
     // Warning: clearTimeout(123);
 }
 
+function global_functions() {
+    const encodedData = btoa("text to encode");
+    const decodedData = atob(encodedData);
+}
+
 function njs_object() {
     njs.dump('asdf');
     njs.version != process.argv[1];
+    typeof njs.version_number == 'number';
     njs.on('exit', ()=> {});
 }
 
diff -r 648a8092c05a -r ffd0f3d3f6df ts/ngx_http_js_module.d.ts
--- a/ts/ngx_http_js_module.d.ts	Tue Aug 02 20:45:17 2022 -0700
+++ b/ts/ngx_http_js_module.d.ts	Wed Aug 03 20:26:24 2022 -0700
@@ -276,6 +276,15 @@ interface NginxHTTPSendBufferOptions {
 interface NginxHTTPRequest {
     /**
      * Request arguments object.
+     *
+     * Since 0.7.6, duplicate keys are returned as an array, keys are
+     * case-sensitive, both keys and values are percent-decoded.
+     * For example, the query string
+     *
+     * 'a=1&b=%32&A=3&b=4&B=two%20words'
+     * is converted to r.args as:
+     *
+     *   {a: "1", b: ["2", "4"], A: "3", B: "two words"}
      */
     readonly args: NginxHTTPArgs;
     /**
@@ -312,6 +321,7 @@ interface NginxHTTPRequest {
      * Performs an internal redirect to the specified uri.
      * If the uri starts with the “@” prefix, it is considered a named location.
      * The actual redirect happens after the handler execution is completed.
+     * Since 0.7.4, the method accepts escaped URIs.
      * @param uri Location to redirect to.
      */
     internalRedirect(uri: NjsStringOrBuffer): void;
diff -r 648a8092c05a -r ffd0f3d3f6df ts/njs_core.d.ts
--- a/ts/njs_core.d.ts	Tue Aug 02 20:45:17 2022 -0700
+++ b/ts/njs_core.d.ts	Wed Aug 03 20:26:24 2022 -0700
@@ -19,10 +19,14 @@ interface String {
     /**
      * Serializes a Unicode string with code points up to 255
      * into a byte string, otherwise, null is returned.
+     *
+     * @deprecated will be removed in the future.
      */
     toBytes(start?: number, end?: number): NjsByteString | null;
     /**
      * Serializes a Unicode string to a byte string using UTF8 encoding.
+     *
+     * @deprecated will be removed in the future.
      */
     toUTF8(start?: number, end?: number): NjsByteString;
 }
@@ -31,15 +35,21 @@ type NjsByteString = string & {
     /**
      * Returns a new Unicode string from a byte string where each byte is replaced
      * with a corresponding Unicode code point.
+     *
+     * @deprecated will be removed in the future.
      */
     fromBytes(start?: number, end?: number): string;
     /**
      * Converts a byte string containing a valid UTF8 string into a Unicode string,
      * otherwise null is returned.
+     *
+     * @deprecated will be removed in the future.
      */
     fromUTF8(start?: number, end?: number): string | null;
     /**
      * Encodes a byte string to hex, base64, or base64url.
+     *
+     * @deprecated will be removed in the future.
      */
     toString(encoding: Exclude<BufferEncoding, "utf8">): string;
 };
@@ -589,7 +599,17 @@ type NjsStringOrBuffer = NjsStringLike |
 // Global objects
 
 interface NjsGlobal {
+    /**
+     * Returns current njs version as a string.
+     * For example, '0.7.4'.
+     */
     readonly version: string;
+    /**
+     * Returns a number with the current version of njs.
+     * For example, “0.7.4” is returned as 0x000704.
+     * @since 0.7.4
+     */
+    readonly version_number: number;
     dump(value: any, indent?: number): string;
     /**
      * Registers a callback for the "exit" event. The callback is called before
@@ -654,3 +674,19 @@ declare function setTimeout<TArgs extend
  * @param handle A value returned by `setTimeout()`.
  */
 declare function clearTimeout(handle?: TimerHandle): void;
+
+/**
+ * Decodes a string of data which has been encoded using Base64 encoding.
+ *
+ * @param encodedData is a binary string that contains Base64-encoded data.
+ * @returns A string that contains decoded data from encodedData.
+ */
+declare function atob(encodedData: string): string;
+
+/**
+ * Creates a Base64-encoded ASCII string from a binary string.
+ *
+ * @param stringToEncode is a binary string to encode.
+ * @returns A string containing the Base64 representation of stringToEncode.
+ */
+declare function btoa(stringToEncode: string): string;
diff -r 648a8092c05a -r ffd0f3d3f6df ts/tsconfig.json
--- a/ts/tsconfig.json	Tue Aug 02 20:45:17 2022 -0700
+++ b/ts/tsconfig.json	Wed Aug 03 20:26:24 2022 -0700
@@ -1,7 +1,7 @@
 {
     "compilerOptions": {
         "target": "es5",
-        "module": "ES2017",
+        "module": "ES2015",
         "lib": [
             "ES2015",
             "ES2016.Array.Include",

From xeioex at nginx.com  Thu Aug  4 03:51:03 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Thu, 04 Aug 2022 03:51:03 +0000
Subject: [njs] Increased njs_printf() output limit to 16384 bytes.
Message-ID: <hg.82e97561e0b4.1659585063.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/82e97561e0b4
branches:  
changeset: 1923:82e97561e0b4
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Aug 03 20:47:41 2022 -0700
description:
Increased njs_printf() output limit to 16384 bytes.

diffstat:

 src/njs_sprintf.c |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff -r ffd0f3d3f6df -r 82e97561e0b4 src/njs_sprintf.c
--- a/src/njs_sprintf.c	Wed Aug 03 20:26:24 2022 -0700
+++ b/src/njs_sprintf.c	Wed Aug 03 20:47:41 2022 -0700
@@ -601,7 +601,7 @@ int
 njs_dprintf(int fd, const char *fmt, ...)
 {
     size_t   size;
-    u_char   text[2048], *p;
+    u_char   text[16384], *p;
     va_list  args;
 
     va_start(args, fmt);


From arut at nginx.com  Thu Aug  4 09:43:43 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Thu, 04 Aug 2022 13:43:43 +0400
Subject: [PATCH] HTTP/3: skip empty request body buffers (ticket #2374)
Message-ID: <999e6a73ff50a41bdbce.1659606223@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1659531591 -14400
#      Wed Aug 03 16:59:51 2022 +0400
# Branch quic
# Node ID 999e6a73ff50a41bdbce467e9572f4ad52bbf2cf
# Parent  f919f13cb11e3bc106e9c4bc2aeadd12b43e7e97
HTTP/3: skip empty request body buffers (ticket #2374).

When client DATA frame header and its content come in different QUIC packets,
it may happen that only the header is processed by the first
ngx_http_v3_request_body_filter() call.  In this case an empty request body
buffer is added to r->request_body->bufs, which is later reused in a
subsequent ngx_http_v3_request_body_filter() call without being removed from
the body chain.  As a result, rb->request_body->bufs ends up with two copies of
the same buffer.

The fix is to avoid adding empty request body buffers to r->request_body->bufs.

diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c
--- a/src/http/v3/ngx_http_v3_request.c
+++ b/src/http/v3/ngx_http_v3_request.c
@@ -1552,15 +1552,17 @@ ngx_http_v3_request_body_filter(ngx_http
                 }
 
                 /* rc == NGX_OK */
-            }
 
-            if (max != -1 && (uint64_t) (max - rb->received) < st->length) {
-                ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
-                              "client intended to send too large "
-                              "body: %O+%ui bytes",
-                              rb->received, st->length);
+                if (max != -1 && (uint64_t) (max - rb->received) < st->length) {
+                    ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                                  "client intended to send too large "
+                                  "body: %O+%ui bytes",
+                                  rb->received, st->length);
 
-                return NGX_HTTP_REQUEST_ENTITY_TOO_LARGE;
+                    return NGX_HTTP_REQUEST_ENTITY_TOO_LARGE;
+                }
+
+                continue;
             }
 
             if (b


From pluknet at nginx.com  Thu Aug  4 14:54:12 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Thu, 4 Aug 2022 18:54:12 +0400
Subject: [PATCH] HTTP/3: skip empty request body buffers (ticket #2374)
In-Reply-To: <999e6a73ff50a41bdbce.1659606223@arut-laptop>
References: <999e6a73ff50a41bdbce.1659606223@arut-laptop>
Message-ID: <D99962E2-DF68-43DC-B9B0-39BEC6C0C916@nginx.com>



> On 4 Aug 2022, at 13:43, Roman Arutyunyan <arut at nginx.com> wrote:
> 
> # HG changeset patch
> # User Roman Arutyunyan <arut at nginx.com>
> # Date 1659531591 -14400
> #      Wed Aug 03 16:59:51 2022 +0400
> # Branch quic
> # Node ID 999e6a73ff50a41bdbce467e9572f4ad52bbf2cf
> # Parent  f919f13cb11e3bc106e9c4bc2aeadd12b43e7e97
> HTTP/3: skip empty request body buffers (ticket #2374).
> 
> When client DATA frame header and its content come in different QUIC packets,
> it may happen that only the header is processed by the first
> ngx_http_v3_request_body_filter() call.  In this case an empty request body
> buffer is added to r->request_body->bufs, which is later reused in a
> subsequent ngx_http_v3_request_body_filter() call without being removed from
> the body chain.  As a result, rb->request_body->bufs ends up with two copies of
> the same buffer.

It can also happen with empty DATA frames ("\x00\x00") each in its own packet.
This results in buffer copies multiplied by the number of such empty frames.

> 
> The fix is to avoid adding empty request body buffers to r->request_body->bufs.
> 
> diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c
> --- a/src/http/v3/ngx_http_v3_request.c
> +++ b/src/http/v3/ngx_http_v3_request.c
> @@ -1552,15 +1552,17 @@ ngx_http_v3_request_body_filter(ngx_http
>                 }
> 
>                 /* rc == NGX_OK */
> -            }
> 
> -            if (max != -1 && (uint64_t) (max - rb->received) < st->length) {
> -                ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> -                              "client intended to send too large "
> -                              "body: %O+%ui bytes",
> -                              rb->received, st->length);
> +                if (max != -1 && (uint64_t) (max - rb->received) < st->length) {
> +                    ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> +                                  "client intended to send too large "
> +                                  "body: %O+%ui bytes",
> +                                  rb->received, st->length);
> 
> -                return NGX_HTTP_REQUEST_ENTITY_TOO_LARGE;
> +                    return NGX_HTTP_REQUEST_ENTITY_TOO_LARGE;
> +                }
> +
> +                continue;
>             }
> 
>             if (b
> 

Looks good.

-- 
Sergey Kandaurov


From dnj0496 at gmail.com  Fri Aug  5 04:42:07 2022
From: dnj0496 at gmail.com (Dk Jack)
Date: Thu, 4 Aug 2022 21:42:07 -0700
Subject: close request
In-Reply-To: <Yo0Sma0IlV14o3iG@mdounin.ru>
References: <CAGATen6HyzkqBbD5MvFyk_iPFOgs38SYQH=mUdYrw4ua-Cg0fg@mail.gmail.com>
 <CAGATen4rHby1inr5Wk9j-D8fLxG3hdq=uqi0OBoQVEPbtnwYGg@mail.gmail.com> <Yo0Sma0IlV14o3iG@mdounin.ru>
Message-ID: <CAGATen6ucr38RoGfenOctJZ3UUFN-t+BvAiTpPYwZqEkLkSXMA@mail.gmail.com>

Hello Maxim,
Interesting thing I noticed is that this alert only happens when the
received request is HTTPv2. If the received is HTTPv1.1 and with no changes
to my module, I don't see this log message. Does this suggest something?

On Tue, May 24, 2022 at 10:23 AM Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Tue, May 24, 2022 at 08:04:16AM -0700, Dk Jack wrote:
>
> > I am noticing this with our modules. Although, we do not do not message
> > with any nginx reference counting etc.
> > At most, we are sending 403-forbidden for some requests. Could you
> suggest
> > some ways about debugging this?
>
> Request reference counting automatically happens in various nginx
> functions, such as ngx_http_read_client_request_body() and
> ngx_http_finalize_request().  Incorrect reference counting
> suggests that your module does something wrong: for example, calls
> ngx_http_finalize_request() when it shouldn't.  Or returns an
> incorrect result from a handler, which causes
> ngx_http_finalize_request() to be called when it shouldn't.
>
> The most straightforward way to debug such problems is to enable
> debug logging and check all reference counting step-by-step for a
> request which results in the alert.
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx-devel mailing list -- nginx-devel at nginx.org
> To unsubscribe send an email to nginx-devel-leave at nginx.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220804/f7186e22/attachment.htm>

From murilo.b.andrade at gmail.com  Tue Aug  9 20:34:53 2022
From: murilo.b.andrade at gmail.com (Murilo Andrade)
Date: Tue, 9 Aug 2022 17:34:53 -0300
Subject: [PATCH] SSL: logging level of client issue during TLSv1.3
Message-ID: <2BCAB58C-861D-46FC-8750-AD8137116F27@gmail.com>

# HG changeset patch
# User Murilo Andrade <murilo.b.andrade at gmail.com>
# Date 1660076026 10800
#      Tue Aug 09 17:13:46 2022 -0300
# Node ID 8b57fd5e8fac9d04cd286e2ad8a18a4030819234
# Parent  069a4813e8d6d7ec662d282a10f5f7062ebd817f
SSL: logging level of client issue during TLSv1.3

Such fatal errors are reported by OpenSSL 1.1.1 during TLSv1.3,
caused by client issue. For example: when the handshake is
concluded, the client send a "change_cipher_spec(20)" followed
by an unknown Content-Type, for example: 26; the OpenSSL library
will fail with SSL_R_BAD_RECORD_TYPE ("bad record type"). This
failure now are logged at the "info" level.

diff -r 069a4813e8d6 -r 8b57fd5e8fac src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c	Tue Jul 19 17:05:27 2022 +0300
+++ b/src/event/ngx_event_openssl.c	Tue Aug 09 17:13:46 2022 -0300
@@ -3423,6 +3423,9 @@
#ifdef SSL_R_VERSION_TOO_LOW
            || n == SSL_R_VERSION_TOO_LOW                            /*  396 */
#endif
+#ifdef SSL_R_BAD_RECORD_TYPE
+            || n == SSL_R_BAD_RECORD_TYPE                            /*  443 */
+#endif
            || n == 1000 /* SSL_R_SSLV3_ALERT_CLOSE_NOTIFY */
#ifdef SSL_R_SSLV3_ALERT_UNEXPECTED_MESSAGE
            || n == SSL_R_SSLV3_ALERT_UNEXPECTED_MESSAGE             /* 1010 */



From xeioex at nginx.com  Tue Aug  9 23:43:43 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 09 Aug 2022 23:43:43 +0000
Subject: [njs] Modules: fixed socket leak with 0 fetch timeout.
Message-ID: <hg.5ee3ef8b969c.1660088623.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/5ee3ef8b969c
branches:  
changeset: 1924:5ee3ef8b969c
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Aug 08 22:41:34 2022 -0700
description:
Modules: fixed socket leak with 0 fetch timeout.

diffstat:

 nginx/ngx_js_fetch.c |  8 +++-----
 1 files changed, 3 insertions(+), 5 deletions(-)

diffs (25 lines):

diff -r 82e97561e0b4 -r 5ee3ef8b969c nginx/ngx_js_fetch.c
--- a/nginx/ngx_js_fetch.c	Wed Aug 03 20:47:41 2022 -0700
+++ b/nginx/ngx_js_fetch.c	Mon Aug 08 22:41:34 2022 -0700
@@ -907,10 +907,8 @@ ngx_js_http_connect(ngx_js_http_t *http)
 
     http->process = ngx_js_http_process_status_line;
 
-    if (http->timeout) {
-        ngx_add_timer(http->peer.connection->read, http->timeout);
-        ngx_add_timer(http->peer.connection->write, http->timeout);
-    }
+    ngx_add_timer(http->peer.connection->read, http->timeout);
+    ngx_add_timer(http->peer.connection->write, http->timeout);
 
 #if (NGX_SSL)
     if (http->ssl != NULL && http->peer.connection->ssl == NULL) {
@@ -1175,7 +1173,7 @@ ngx_js_http_write_handler(ngx_event_t *w
         }
     }
 
-    if (!wev->timer_set && http->timeout) {
+    if (!wev->timer_set) {
         ngx_add_timer(wev, http->timeout);
     }
 }


From xeioex at nginx.com  Tue Aug  9 23:43:45 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 09 Aug 2022 23:43:45 +0000
Subject: [njs] Fixed ts_test target on MacOS.
Message-ID: <hg.335ff5a4539d.1660088625.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/335ff5a4539d
branches:  
changeset: 1925:335ff5a4539d
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Aug 08 23:18:51 2022 -0700
description:
Fixed ts_test target on MacOS.

diffstat:

 auto/make       |  2 +-
 ts/package.json |  2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diffs (23 lines):

diff -r 5ee3ef8b969c -r 335ff5a4539d auto/make
--- a/auto/make	Mon Aug 08 22:41:34 2022 -0700
+++ b/auto/make	Mon Aug 08 23:18:51 2022 -0700
@@ -275,7 +275,7 @@ cat << END >> $NJS_MAKEFILE
 $NJS_BUILD_DIR/ts/package.json: $njs_ts_deps
 	cp -fr ts $NJS_BUILD_DIR/
 	cp LICENSE $NJS_BUILD_DIR/ts/
-	sed 's/\("version":\s*\)"\([^"]\+\)"/\1"\$(NJS_TYPES_VER)"/' \\
+	sed 's/__VERSION__/"\$(NJS_TYPES_VER)"/' \\
 		ts/package.json > $NJS_BUILD_DIR/ts/package.json
 
 $NJS_BUILD_DIR/ts/node_modules: $NJS_BUILD_DIR/ts/package.json
diff -r 5ee3ef8b969c -r 335ff5a4539d ts/package.json
--- a/ts/package.json	Mon Aug 08 22:41:34 2022 -0700
+++ b/ts/package.json	Mon Aug 08 23:18:51 2022 -0700
@@ -1,6 +1,6 @@
 {
   "name": "njs-types",
-  "version": "0.0.0-dev",
+  "version": __VERSION__,
   "description": "TypeScript definitions for njs scripting language and nginx js modules.",
   "scripts": {
     "lint": "tsc"


From mdounin at mdounin.ru  Wed Aug 10 00:31:27 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 10 Aug 2022 03:31:27 +0300
Subject: [PATCH] SSL: logging level of client issue during TLSv1.3
In-Reply-To: <2BCAB58C-861D-46FC-8750-AD8137116F27@gmail.com>
References: <2BCAB58C-861D-46FC-8750-AD8137116F27@gmail.com>
Message-ID: <YvL8Xz31hG+/aiLM@mdounin.ru>

Hello!

On Tue, Aug 09, 2022 at 05:34:53PM -0300, Murilo Andrade wrote:

> # HG changeset patch
> # User Murilo Andrade <murilo.b.andrade at gmail.com>
> # Date 1660076026 10800
> #      Tue Aug 09 17:13:46 2022 -0300
> # Node ID 8b57fd5e8fac9d04cd286e2ad8a18a4030819234
> # Parent  069a4813e8d6d7ec662d282a10f5f7062ebd817f
> SSL: logging level of client issue during TLSv1.3
> 
> Such fatal errors are reported by OpenSSL 1.1.1 during TLSv1.3,
> caused by client issue. For example: when the handshake is
> concluded, the client send a "change_cipher_spec(20)" followed
> by an unknown Content-Type, for example: 26; the OpenSSL library
> will fail with SSL_R_BAD_RECORD_TYPE ("bad record type"). This
> failure now are logged at the "info" level.
> 
> diff -r 069a4813e8d6 -r 8b57fd5e8fac src/event/ngx_event_openssl.c
> --- a/src/event/ngx_event_openssl.c	Tue Jul 19 17:05:27 2022 +0300
> +++ b/src/event/ngx_event_openssl.c	Tue Aug 09 17:13:46 2022 -0300
> @@ -3423,6 +3423,9 @@
> #ifdef SSL_R_VERSION_TOO_LOW
>             || n == SSL_R_VERSION_TOO_LOW                            /*  396 */
> #endif
> +#ifdef SSL_R_BAD_RECORD_TYPE
> +            || n == SSL_R_BAD_RECORD_TYPE                            /*  443 */
> +#endif
>             || n == 1000 /* SSL_R_SSLV3_ALERT_CLOSE_NOTIFY */
> #ifdef SSL_R_SSLV3_ALERT_UNEXPECTED_MESSAGE
>             || n == SSL_R_SSLV3_ALERT_UNEXPECTED_MESSAGE             /* 1010 */
> 
> 

Are you seeing these errors in practice?

-- 
Maxim Dounin
http://mdounin.ru/


From arut at nginx.com  Wed Aug 10 08:18:46 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Wed, 10 Aug 2022 12:18:46 +0400
Subject: [PATCH] QUIC: a socket for new connections in BPF mode
Message-ID: <7fc94f5288f492d2147b.1660119526@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1660119182 -14400
#      Wed Aug 10 12:13:02 2022 +0400
# Branch quic
# Node ID 7fc94f5288f492d2147bf22850744cd9888d2c8f
# Parent  f9d7930d0eedae28defd0803cb95dc8ab68e56b3
QUIC: a socket for new connections in BPF mode.

Previously, when a packet from a new connection was received, Linux reuseport
BPF program failed to select a socket for it and fell back to a default kernel
selection algorithm.  This worked well most of the time.  However, during nginx
reload, two sets of workers (and sockets) may coexist and only the existing
connections should be served by old sockets.  All new connections should be
routed to new sockets since workers in closing state drop new connections.

Now a socket for new connections is introduced.  This socket is added to the
reuseport group and registered in the socket map with the key = 0.  When a map
lookup fails with the key extracted from packet dcid, the packet is routed to
the new socket.  This socket is read by all nginx workers like a regular TCP
listen socket.

The approach has a drawback though.  For correct routing it is expected that
client will not send more than one datagram before receiving a packet from
server.  This requirement can be violated if large enough TLS early data is
supplied by client.  The size of early data can be limited with a properly
tuned initial_max_data transport parameter.  Currently its value is derived
from QUIC stream buffer size and number of concurrent streams.

diff --git a/src/core/ngx_connection.c b/src/core/ngx_connection.c
--- a/src/core/ngx_connection.c
+++ b/src/core/ngx_connection.c
@@ -1034,7 +1034,9 @@ ngx_close_listening_sockets(ngx_cycle_t 
     for (i = 0; i < cycle->listening.nelts; i++) {
 
 #if (NGX_QUIC)
-        if (ls[i].quic) {
+        if (ngx_process == NGX_PROCESS_WORKER
+            && ls[i].quic && ls[i].worker != (ngx_uint_t) -1)
+        {
             continue;
         }
 #endif
diff --git a/src/core/ngx_connection.h b/src/core/ngx_connection.h
--- a/src/core/ngx_connection.h
+++ b/src/core/ngx_connection.h
@@ -47,6 +47,7 @@ struct ngx_listening_s {
     size_t              post_accept_buffer_size;
 
     ngx_listening_t    *previous;
+    ngx_listening_t    *main;
     ngx_connection_t   *connection;
 
     ngx_rbtree_t        rbtree;
diff --git a/src/event/ngx_event.c b/src/event/ngx_event.c
--- a/src/event/ngx_event.c
+++ b/src/event/ngx_event.c
@@ -802,7 +802,10 @@ ngx_event_process_init(ngx_cycle_t *cycl
     for (i = 0; i < cycle->listening.nelts; i++) {
 
 #if (NGX_HAVE_REUSEPORT)
-        if (ls[i].reuseport && ls[i].worker != ngx_worker) {
+        if (ls[i].reuseport
+            && ls[i].worker != ngx_worker
+            && ls[i].worker != (ngx_uint_t) -1)
+        {
             continue;
         }
 #endif
@@ -899,7 +902,7 @@ ngx_event_process_init(ngx_cycle_t *cycl
 
 #if (NGX_HAVE_REUSEPORT)
 
-        if (ls[i].reuseport) {
+        if (ls[i].reuseport && ls[i].worker != (ngx_uint_t) -1) {
             if (ngx_add_event(rev, NGX_READ_EVENT, 0) == NGX_ERROR) {
                 return NGX_ERROR;
             }
diff --git a/src/event/ngx_event.h b/src/event/ngx_event.h
--- a/src/event/ngx_event.h
+++ b/src/event/ngx_event.h
@@ -496,6 +496,9 @@ extern ngx_module_t           ngx_event_
 void ngx_event_accept(ngx_event_t *ev);
 ngx_int_t ngx_trylock_accept_mutex(ngx_cycle_t *cycle);
 ngx_int_t ngx_enable_accept_events(ngx_cycle_t *cycle);
+#if (NGX_HAVE_EPOLLEXCLUSIVE)
+void ngx_reorder_accept_events(ngx_listening_t *ls);
+#endif
 u_char *ngx_accept_log_error(ngx_log_t *log, u_char *buf, size_t len);
 #if (NGX_DEBUG)
 void ngx_debug_accepted_connection(ngx_event_conf_t *ecf, ngx_connection_t *c);
diff --git a/src/event/ngx_event_accept.c b/src/event/ngx_event_accept.c
--- a/src/event/ngx_event_accept.c
+++ b/src/event/ngx_event_accept.c
@@ -11,9 +11,6 @@
 
 
 static ngx_int_t ngx_disable_accept_events(ngx_cycle_t *cycle, ngx_uint_t all);
-#if (NGX_HAVE_EPOLLEXCLUSIVE)
-static void ngx_reorder_accept_events(ngx_listening_t *ls);
-#endif
 static void ngx_close_accepted_connection(ngx_connection_t *c);
 
 
@@ -410,7 +407,7 @@ ngx_disable_accept_events(ngx_cycle_t *c
          * when disabling accept events due to accept mutex
          */
 
-        if (ls[i].reuseport && !all) {
+        if (ls[i].reuseport && ls[i].worker != (ngx_uint_t) -1 && !all) {
             continue;
         }
 
@@ -429,7 +426,7 @@ ngx_disable_accept_events(ngx_cycle_t *c
 
 #if (NGX_HAVE_EPOLLEXCLUSIVE)
 
-static void
+void
 ngx_reorder_accept_events(ngx_listening_t *ls)
 {
     ngx_connection_t  *c;
@@ -448,7 +445,7 @@ ngx_reorder_accept_events(ngx_listening_
 
 #if (NGX_HAVE_REUSEPORT)
 
-    if (ls->reuseport) {
+    if (ls->reuseport && ls->worker != (ngx_uint_t) -1) {
         return;
     }
 
diff --git a/src/event/quic/bpf/ngx_quic_reuseport_helper.c b/src/event/quic/bpf/ngx_quic_reuseport_helper.c
--- a/src/event/quic/bpf/ngx_quic_reuseport_helper.c
+++ b/src/event/quic/bpf/ngx_quic_reuseport_helper.c
@@ -48,7 +48,7 @@ char _license[] SEC("license") = LICENSE
     offset += nbytes;                                                         \
     if (start + offset > end) {                                               \
         debugmsg("cannot read %ld bytes at offset %ld", nbytes, offset);      \
-        goto failed;                                                          \
+        goto new;                                                             \
     }                                                                         \
     data = start + offset - 1;
 
@@ -93,8 +93,7 @@ int ngx_quic_select_socket_by_dcid(struc
         len = data[0];   /* read DCID length */
 
         if (len < 8) {
-            /* it's useless to search for key in such short DCID */
-            return SK_PASS;
+            goto new;
         }
 
     } else {
@@ -102,39 +101,39 @@ int ngx_quic_select_socket_by_dcid(struc
     }
 
     dcid = &data[1];
-    advance_data(len); /* we expect the packet to have full DCID */
+    advance_data(len);
 
-    /* make verifier happy */
-    if (dcid + sizeof(__u64) > end) {
-        goto failed;
+    if (dcid + 8 > end) {
+        goto new;
     }
 
+    /* search for existing connection */
+
     key = ngx_quic_parse_uint64(dcid);
 
     rc = bpf_sk_select_reuseport(ctx, &ngx_quic_sockmap, &key, 0);
 
-    switch (rc) {
-    case 0:
-        debugmsg("nginx quic socket selected by key 0x%llx", key);
+    if (rc == 0) {
+        debugmsg("nginx quic hit key:0x%llx", key);
         return SK_PASS;
-
-    /* kernel returns positive error numbers, errno.h defines positive */
-    case -ENOENT:
-        debugmsg("nginx quic default route for key 0x%llx", key);
-        /* let the default reuseport logic decide which socket to choose */
-        return SK_PASS;
-
-    default:
-        debugmsg("nginx quic bpf_sk_select_reuseport err: %d key 0x%llx",
-                 rc, key);
-        goto failed;
     }
 
-failed:
-    /*
-     * SK_DROP will generate ICMP, but we may want to process "invalid" packet
-     * in userspace quic to investigate further and finally react properly
-     * (maybe ignore, maybe send something in response or close connection)
-     */
-    return SK_PASS;
+    debugmsg("nginx quic miss err:%d key:0x%llx", rc, key);
+
+new:
+
+    /* new connection */
+
+    key = 0;
+
+    rc = bpf_sk_select_reuseport(ctx, &ngx_quic_sockmap, &key, 0);
+
+    if (rc == 0) {
+        debugmsg("nginx quic new hit");
+        return SK_PASS;
+    }
+
+    debugmsg("nginx quic new miss err:%d", rc);
+
+    return SK_DROP;
 }
diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c
--- a/src/event/quic/ngx_event_quic.c
+++ b/src/event/quic/ngx_event_quic.c
@@ -910,14 +910,6 @@ ngx_quic_handle_packet(ngx_connection_t 
         pkt->odcid = pkt->dcid;
     }
 
-    if (ngx_terminate || ngx_exiting) {
-        if (conf->retry) {
-            return ngx_quic_send_retry(c, conf, pkt);
-        }
-
-        return NGX_ERROR;
-    }
-
     c->log->action = "creating quic connection";
 
     qc = ngx_quic_new_connection(c, conf, pkt);
diff --git a/src/event/quic/ngx_event_quic_bpf.c b/src/event/quic/ngx_event_quic_bpf.c
--- a/src/event/quic/ngx_event_quic_bpf.c
+++ b/src/event/quic/ngx_event_quic_bpf.c
@@ -42,7 +42,9 @@ typedef struct {
 
 
 static void *ngx_quic_bpf_create_conf(ngx_cycle_t *cycle);
+static char *ngx_quic_bpf_init_conf(ngx_cycle_t *cycle, void *conf);
 static ngx_int_t ngx_quic_bpf_module_init(ngx_cycle_t *cycle);
+static ngx_int_t ngx_quic_bpf_process_init(ngx_cycle_t *cycle);
 
 static void ngx_quic_bpf_cleanup(void *data);
 static ngx_inline void ngx_quic_bpf_close(ngx_log_t *log, int fd,
@@ -82,7 +84,7 @@ static ngx_command_t  ngx_quic_bpf_comma
 static ngx_core_module_t  ngx_quic_bpf_module_ctx = {
     ngx_string("quic_bpf"),
     ngx_quic_bpf_create_conf,
-    NULL
+    ngx_quic_bpf_init_conf
 };
 
 
@@ -93,7 +95,7 @@ ngx_module_t  ngx_quic_bpf_module = {
     NGX_CORE_MODULE,                       /* module type */
     NULL,                                  /* init master */
     ngx_quic_bpf_module_init,              /* init module */
-    NULL,                                  /* init process */
+    ngx_quic_bpf_process_init,             /* init process */
     NULL,                                  /* init thread */
     NULL,                                  /* exit thread */
     NULL,                                  /* exit process */
@@ -121,30 +123,67 @@ ngx_quic_bpf_create_conf(ngx_cycle_t *cy
 }
 
 
+static char *
+ngx_quic_bpf_init_conf(ngx_cycle_t *cycle, void *conf)
+{
+    ngx_quic_bpf_conf_t *bcf = conf;
+
+    ngx_uint_t        i, nelts;
+    ngx_core_conf_t  *ccf;
+    ngx_listening_t  *ls, *nls;
+
+    ngx_conf_init_value(bcf->enabled, 0);
+
+    if (!bcf->enabled || ngx_test_config) {
+        return NGX_CONF_OK;
+    }
+
+    ccf = ngx_core_get_conf(cycle);
+    bcf->map_size = ccf->worker_processes * 4;
+
+    ls = cycle->listening.elts;
+    nelts = cycle->listening.nelts;
+
+    for (i = 0; i < nelts; i++) {
+
+        if (!ls[i].quic || !ls[i].reuseport || ls[i].worker != 0) {
+            continue;
+        }
+
+        nls = ngx_array_push(&cycle->listening);
+        if (nls == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        *nls = ls[i];
+
+        /* cloning may change cycle->listening.elts */
+
+        ls = cycle->listening.elts;
+
+        /* make new listening first to inherit fd */
+
+        ls[i].worker = (ngx_uint_t) -1;
+    }
+
+    return NGX_CONF_OK;
+}
+
+
 static ngx_int_t
 ngx_quic_bpf_module_init(ngx_cycle_t *cycle)
 {
     ngx_uint_t            i;
     ngx_listening_t      *ls;
-    ngx_core_conf_t      *ccf;
     ngx_pool_cleanup_t   *cln;
     ngx_quic_bpf_conf_t  *bcf;
 
-    if (ngx_test_config) {
-        /*
-         * during config test, SO_REUSEPORT socket option is
-         * not set, thus making further processing meaningless
-         */
+    bcf = ngx_quic_bpf_get_conf(cycle);
+
+    if (!bcf->enabled || ngx_test_config) {
         return NGX_OK;
     }
 
-    ccf = ngx_core_get_conf(cycle);
-    bcf = ngx_quic_bpf_get_conf(cycle);
-
-    ngx_conf_init_value(bcf->enabled, 0);
-
-    bcf->map_size = ccf->worker_processes * 4;
-
     cln = ngx_pool_cleanup_add(cycle->pool, 0);
     if (cln == NULL) {
         goto failed;
@@ -203,6 +242,46 @@ failed:
 }
 
 
+static ngx_int_t
+ngx_quic_bpf_process_init(ngx_cycle_t *cycle)
+{
+    ngx_uint_t            i, j;
+    ngx_listening_t      *ls;
+    ngx_quic_bpf_conf_t  *bcf;
+
+    bcf = ngx_quic_bpf_get_conf(cycle);
+
+    if (!bcf->enabled) {
+        return NGX_OK;
+    }
+
+    ls = cycle->listening.elts;
+    for (i = 0; i < cycle->listening.nelts; i++) {
+
+        if (!ls[i].quic || ls[i].worker != (ngx_uint_t) -1) {
+            continue;
+        }
+
+        for (j = i + 1; j < cycle->listening.nelts; j++) {
+
+            if (ls[j].quic
+                && ls[j].reuseport
+                && ls[j].worker == ngx_worker
+                && ls[j].type == ls[i].type
+                && ngx_cmp_sockaddr(ls[j].sockaddr, ls[j].socklen,
+                                    ls[i].sockaddr, ls[i].socklen, 1)
+                   == 0)
+            {
+                ls[i].main = &ls[j];
+                break;
+            }
+        }
+    }
+
+    return NGX_OK;
+}
+
+
 static void
 ngx_quic_bpf_cleanup(void *data)
 {
@@ -419,26 +498,32 @@ static ngx_int_t
 ngx_quic_bpf_group_add_socket(ngx_cycle_t *cycle,  ngx_listening_t *ls)
 {
     uint64_t                cookie;
-    ngx_quic_bpf_conf_t    *bcf;
     ngx_quic_sock_group_t  *grp;
 
-    bcf = ngx_quic_bpf_get_conf(cycle);
-
     grp = ngx_quic_bpf_get_group(cycle, ls);
-
     if (grp == NULL) {
-        if (!bcf->enabled) {
-            return NGX_OK;
-        }
-
         return NGX_ERROR;
     }
 
     grp->unused = 0;
 
-    cookie = ngx_quic_bpf_socket_key(ls->fd, cycle->log);
-    if (cookie == (uint64_t) NGX_ERROR) {
-        return NGX_ERROR;
+    if (ls->worker == (ngx_uint_t) -1) {
+        cookie = 0;
+
+    } else {
+        cookie = ngx_quic_bpf_socket_key(ls->fd, cycle->log);
+        if (cookie == (uint64_t) NGX_ERROR) {
+            return NGX_ERROR;
+        }
+
+        /* do not inherit this socket */
+        ls->ignore = 1;
+
+        if (fcntl(ls->fd, F_SETFD, FD_CLOEXEC) == -1) {
+            ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_errno,
+                          "fcntl(FD_CLOEXEC) fd:%d failed", ls->fd);
+            return NGX_ERROR;
+        }
     }
 
     /* map[cookie] = socket; for use in kernel helper */
@@ -452,9 +537,6 @@ ngx_quic_bpf_group_add_socket(ngx_cycle_
                  "quic bpf sockmap fd:%d add socket:%d cookie:0x%xL worker:%ui",
                  grp->map_fd, ls->fd, cookie, ls->worker);
 
-    /* do not inherit this socket */
-    ls->ignore = 1;
-
     return NGX_OK;
 }
 
diff --git a/src/event/quic/ngx_event_quic_bpf_code.c b/src/event/quic/ngx_event_quic_bpf_code.c
--- a/src/event/quic/ngx_event_quic_bpf_code.c
+++ b/src/event/quic/ngx_event_quic_bpf_code.c
@@ -7,71 +7,93 @@
 
 
 static ngx_bpf_reloc_t bpf_reloc_prog_ngx_quic_reuseport_helper[] = {
-    { "ngx_quic_sockmap", 55 },
+    { "ngx_quic_sockmap", 57 },
+    { "ngx_quic_sockmap", 71 },
 };
 
 static struct bpf_insn bpf_insn_prog_ngx_quic_reuseport_helper[] = {
     /* opcode dst          src         offset imm */
-    { 0x79,   BPF_REG_4,   BPF_REG_1, (int16_t)      0,        0x0 },
-    { 0x79,   BPF_REG_3,   BPF_REG_1, (int16_t)      8,        0x0 },
-    { 0xbf,   BPF_REG_2,   BPF_REG_4, (int16_t)      0,        0x0 },
-    {  0x7,   BPF_REG_2,   BPF_REG_0, (int16_t)      0,        0x8 },
-    { 0x2d,   BPF_REG_2,   BPF_REG_3, (int16_t)     54,        0x0 },
-    { 0xbf,   BPF_REG_5,   BPF_REG_4, (int16_t)      0,        0x0 },
-    {  0x7,   BPF_REG_5,   BPF_REG_0, (int16_t)      0,        0x9 },
-    { 0x2d,   BPF_REG_5,   BPF_REG_3, (int16_t)     51,        0x0 },
-    { 0xb7,   BPF_REG_5,   BPF_REG_0, (int16_t)      0,       0x14 },
-    { 0xb7,   BPF_REG_0,   BPF_REG_0, (int16_t)      0,        0x9 },
-    { 0x71,   BPF_REG_6,   BPF_REG_2, (int16_t)      0,        0x0 },
-    { 0x67,   BPF_REG_6,   BPF_REG_0, (int16_t)      0,       0x38 },
-    { 0xc7,   BPF_REG_6,   BPF_REG_0, (int16_t)      0,       0x38 },
-    { 0x65,   BPF_REG_6,   BPF_REG_0, (int16_t)     10, 0xffffffff },
-    { 0xbf,   BPF_REG_2,   BPF_REG_4, (int16_t)      0,        0x0 },
-    {  0x7,   BPF_REG_2,   BPF_REG_0, (int16_t)      0,        0xd },
-    { 0x2d,   BPF_REG_2,   BPF_REG_3, (int16_t)     42,        0x0 },
-    { 0xbf,   BPF_REG_5,   BPF_REG_4, (int16_t)      0,        0x0 },
-    {  0x7,   BPF_REG_5,   BPF_REG_0, (int16_t)      0,        0xe },
-    { 0x2d,   BPF_REG_5,   BPF_REG_3, (int16_t)     39,        0x0 },
-    { 0xb7,   BPF_REG_0,   BPF_REG_0, (int16_t)      0,        0xe },
-    { 0x71,   BPF_REG_5,   BPF_REG_2, (int16_t)      0,        0x0 },
-    { 0xb7,   BPF_REG_6,   BPF_REG_0, (int16_t)      0,        0x8 },
-    { 0x2d,   BPF_REG_6,   BPF_REG_5, (int16_t)     35,        0x0 },
-    {  0xf,   BPF_REG_5,   BPF_REG_0, (int16_t)      0,        0x0 },
+    { 0xbf,   BPF_REG_6,   BPF_REG_1, (int16_t)      0,        0x0 },
+    { 0x79,   BPF_REG_3,   BPF_REG_6, (int16_t)      0,        0x0 },
+    { 0x79,   BPF_REG_2,   BPF_REG_6, (int16_t)      8,        0x0 },
+    { 0xbf,   BPF_REG_1,   BPF_REG_3, (int16_t)      0,        0x0 },
+    {  0x7,   BPF_REG_1,   BPF_REG_0, (int16_t)      0,        0x8 },
+    { 0x2d,   BPF_REG_1,   BPF_REG_2, (int16_t)     60,        0x0 },
+    { 0xbf,   BPF_REG_4,   BPF_REG_3, (int16_t)      0,        0x0 },
+    {  0x7,   BPF_REG_4,   BPF_REG_0, (int16_t)      0,        0x9 },
+    { 0x2d,   BPF_REG_4,   BPF_REG_2, (int16_t)     57,        0x0 },
+    { 0xb7,   BPF_REG_4,   BPF_REG_0, (int16_t)      0,       0x14 },
+    { 0xb7,   BPF_REG_5,   BPF_REG_0, (int16_t)      0,        0x9 },
+    { 0x71,   BPF_REG_0,   BPF_REG_1, (int16_t)      0,        0x0 },
+    { 0x67,   BPF_REG_0,   BPF_REG_0, (int16_t)      0,       0x38 },
+    { 0xc7,   BPF_REG_0,   BPF_REG_0, (int16_t)      0,       0x38 },
+    { 0x65,   BPF_REG_0,   BPF_REG_0, (int16_t)     10, 0xffffffff },
+    { 0xbf,   BPF_REG_1,   BPF_REG_3, (int16_t)      0,        0x0 },
+    {  0x7,   BPF_REG_1,   BPF_REG_0, (int16_t)      0,        0xd },
+    { 0x2d,   BPF_REG_1,   BPF_REG_2, (int16_t)     48,        0x0 },
+    { 0xbf,   BPF_REG_4,   BPF_REG_3, (int16_t)      0,        0x0 },
+    {  0x7,   BPF_REG_4,   BPF_REG_0, (int16_t)      0,        0xe },
+    { 0x2d,   BPF_REG_4,   BPF_REG_2, (int16_t)     45,        0x0 },
+    { 0xb7,   BPF_REG_5,   BPF_REG_0, (int16_t)      0,        0xe },
+    { 0x71,   BPF_REG_4,   BPF_REG_1, (int16_t)      0,        0x0 },
+    { 0xb7,   BPF_REG_0,   BPF_REG_0, (int16_t)      0,        0x8 },
+    { 0x2d,   BPF_REG_0,   BPF_REG_4, (int16_t)     41,        0x0 },
     {  0xf,   BPF_REG_4,   BPF_REG_5, (int16_t)      0,        0x0 },
-    { 0x2d,   BPF_REG_4,   BPF_REG_3, (int16_t)     32,        0x0 },
-    { 0xbf,   BPF_REG_4,   BPF_REG_2, (int16_t)      0,        0x0 },
-    {  0x7,   BPF_REG_4,   BPF_REG_0, (int16_t)      0,        0x9 },
-    { 0x2d,   BPF_REG_4,   BPF_REG_3, (int16_t)     29,        0x0 },
-    { 0x71,   BPF_REG_4,   BPF_REG_2, (int16_t)      1,        0x0 },
-    { 0x67,   BPF_REG_4,   BPF_REG_0, (int16_t)      0,       0x38 },
-    { 0x71,   BPF_REG_3,   BPF_REG_2, (int16_t)      2,        0x0 },
-    { 0x67,   BPF_REG_3,   BPF_REG_0, (int16_t)      0,       0x30 },
-    { 0x4f,   BPF_REG_3,   BPF_REG_4, (int16_t)      0,        0x0 },
-    { 0x71,   BPF_REG_4,   BPF_REG_2, (int16_t)      3,        0x0 },
-    { 0x67,   BPF_REG_4,   BPF_REG_0, (int16_t)      0,       0x28 },
-    { 0x4f,   BPF_REG_3,   BPF_REG_4, (int16_t)      0,        0x0 },
-    { 0x71,   BPF_REG_4,   BPF_REG_2, (int16_t)      4,        0x0 },
-    { 0x67,   BPF_REG_4,   BPF_REG_0, (int16_t)      0,       0x20 },
-    { 0x4f,   BPF_REG_3,   BPF_REG_4, (int16_t)      0,        0x0 },
-    { 0x71,   BPF_REG_4,   BPF_REG_2, (int16_t)      5,        0x0 },
-    { 0x67,   BPF_REG_4,   BPF_REG_0, (int16_t)      0,       0x18 },
-    { 0x4f,   BPF_REG_3,   BPF_REG_4, (int16_t)      0,        0x0 },
-    { 0x71,   BPF_REG_4,   BPF_REG_2, (int16_t)      6,        0x0 },
-    { 0x67,   BPF_REG_4,   BPF_REG_0, (int16_t)      0,       0x10 },
-    { 0x4f,   BPF_REG_3,   BPF_REG_4, (int16_t)      0,        0x0 },
-    { 0x71,   BPF_REG_4,   BPF_REG_2, (int16_t)      7,        0x0 },
-    { 0x67,   BPF_REG_4,   BPF_REG_0, (int16_t)      0,        0x8 },
-    { 0x4f,   BPF_REG_3,   BPF_REG_4, (int16_t)      0,        0x0 },
-    { 0x71,   BPF_REG_2,   BPF_REG_2, (int16_t)      8,        0x0 },
-    { 0x4f,   BPF_REG_3,   BPF_REG_2, (int16_t)      0,        0x0 },
-    { 0x7b,  BPF_REG_10,   BPF_REG_3, (int16_t)  65528,        0x0 },
+    {  0xf,   BPF_REG_3,   BPF_REG_4, (int16_t)      0,        0x0 },
+    { 0x2d,   BPF_REG_3,   BPF_REG_2, (int16_t)     38,        0x0 },
+    { 0xbf,   BPF_REG_3,   BPF_REG_1, (int16_t)      0,        0x0 },
+    {  0x7,   BPF_REG_3,   BPF_REG_0, (int16_t)      0,        0x9 },
+    { 0x2d,   BPF_REG_3,   BPF_REG_2, (int16_t)     35,        0x0 },
+    { 0x71,   BPF_REG_3,   BPF_REG_1, (int16_t)      1,        0x0 },
+    { 0x67,   BPF_REG_3,   BPF_REG_0, (int16_t)      0,       0x38 },
+    { 0x71,   BPF_REG_2,   BPF_REG_1, (int16_t)      2,        0x0 },
+    { 0x67,   BPF_REG_2,   BPF_REG_0, (int16_t)      0,       0x30 },
+    { 0x4f,   BPF_REG_2,   BPF_REG_3, (int16_t)      0,        0x0 },
+    { 0x71,   BPF_REG_3,   BPF_REG_1, (int16_t)      3,        0x0 },
+    { 0x67,   BPF_REG_3,   BPF_REG_0, (int16_t)      0,       0x28 },
+    { 0x4f,   BPF_REG_2,   BPF_REG_3, (int16_t)      0,        0x0 },
+    { 0x71,   BPF_REG_3,   BPF_REG_1, (int16_t)      4,        0x0 },
+    { 0x67,   BPF_REG_3,   BPF_REG_0, (int16_t)      0,       0x20 },
+    { 0x4f,   BPF_REG_2,   BPF_REG_3, (int16_t)      0,        0x0 },
+    { 0x71,   BPF_REG_3,   BPF_REG_1, (int16_t)      5,        0x0 },
+    { 0x67,   BPF_REG_3,   BPF_REG_0, (int16_t)      0,       0x18 },
+    { 0x4f,   BPF_REG_2,   BPF_REG_3, (int16_t)      0,        0x0 },
+    { 0x71,   BPF_REG_3,   BPF_REG_1, (int16_t)      6,        0x0 },
+    { 0x67,   BPF_REG_3,   BPF_REG_0, (int16_t)      0,       0x10 },
+    { 0x4f,   BPF_REG_2,   BPF_REG_3, (int16_t)      0,        0x0 },
+    { 0x71,   BPF_REG_3,   BPF_REG_1, (int16_t)      7,        0x0 },
+    { 0x67,   BPF_REG_3,   BPF_REG_0, (int16_t)      0,        0x8 },
+    { 0x4f,   BPF_REG_2,   BPF_REG_3, (int16_t)      0,        0x0 },
+    { 0x71,   BPF_REG_1,   BPF_REG_1, (int16_t)      8,        0x0 },
+    { 0x4f,   BPF_REG_2,   BPF_REG_1, (int16_t)      0,        0x0 },
+    { 0x7b,  BPF_REG_10,   BPF_REG_2, (int16_t)  65528,        0x0 },
     { 0xbf,   BPF_REG_3,  BPF_REG_10, (int16_t)      0,        0x0 },
     {  0x7,   BPF_REG_3,   BPF_REG_0, (int16_t)      0, 0xfffffff8 },
+    { 0xbf,   BPF_REG_1,   BPF_REG_6, (int16_t)      0,        0x0 },
     { 0x18,   BPF_REG_2,   BPF_REG_0, (int16_t)      0,        0x0 },
     {  0x0,   BPF_REG_0,   BPF_REG_0, (int16_t)      0,        0x0 },
     { 0xb7,   BPF_REG_4,   BPF_REG_0, (int16_t)      0,        0x0 },
     { 0x85,   BPF_REG_0,   BPF_REG_0, (int16_t)      0,       0x52 },
+    { 0xbf,   BPF_REG_1,   BPF_REG_0, (int16_t)      0,        0x0 },
     { 0xb7,   BPF_REG_0,   BPF_REG_0, (int16_t)      0,        0x1 },
+    { 0x67,   BPF_REG_1,   BPF_REG_0, (int16_t)      0,       0x20 },
+    { 0x77,   BPF_REG_1,   BPF_REG_0, (int16_t)      0,       0x20 },
+    { 0x15,   BPF_REG_1,   BPF_REG_0, (int16_t)     15,        0x0 },
+    { 0xb7,   BPF_REG_1,   BPF_REG_0, (int16_t)      0,        0x0 },
+    { 0x7b,  BPF_REG_10,   BPF_REG_1, (int16_t)  65528,        0x0 },
+    { 0xbf,   BPF_REG_3,  BPF_REG_10, (int16_t)      0,        0x0 },
+    {  0x7,   BPF_REG_3,   BPF_REG_0, (int16_t)      0, 0xfffffff8 },
+    { 0xbf,   BPF_REG_1,   BPF_REG_6, (int16_t)      0,        0x0 },
+    { 0x18,   BPF_REG_2,   BPF_REG_0, (int16_t)      0,        0x0 },
+    {  0x0,   BPF_REG_0,   BPF_REG_0, (int16_t)      0,        0x0 },
+    { 0xb7,   BPF_REG_4,   BPF_REG_0, (int16_t)      0,        0x0 },
+    { 0x85,   BPF_REG_0,   BPF_REG_0, (int16_t)      0,       0x52 },
+    { 0xbf,   BPF_REG_1,   BPF_REG_0, (int16_t)      0,        0x0 },
+    { 0x67,   BPF_REG_1,   BPF_REG_0, (int16_t)      0,       0x20 },
+    { 0x77,   BPF_REG_1,   BPF_REG_0, (int16_t)      0,       0x20 },
+    { 0xb7,   BPF_REG_0,   BPF_REG_0, (int16_t)      0,        0x1 },
+    { 0x15,   BPF_REG_1,   BPF_REG_0, (int16_t)      1,        0x0 },
+    { 0xb7,   BPF_REG_0,   BPF_REG_0, (int16_t)      0,        0x0 },
     { 0x95,   BPF_REG_0,   BPF_REG_0, (int16_t)      0,        0x0 },
 };
 
@@ -86,3 +108,4 @@ ngx_bpf_program_t ngx_quic_reuseport_hel
     .license = "BSD",
     .type = BPF_PROG_TYPE_SK_REUSEPORT,
 };
+
diff --git a/src/event/quic/ngx_event_quic_udp.c b/src/event/quic/ngx_event_quic_udp.c
--- a/src/event/quic/ngx_event_quic_udp.c
+++ b/src/event/quic/ngx_event_quic_udp.c
@@ -30,7 +30,7 @@ ngx_quic_recvmsg(ngx_event_t *ev)
     struct msghdr       msg;
     ngx_sockaddr_t      sa, lsa;
     struct sockaddr    *sockaddr, *local_sockaddr;
-    ngx_listening_t    *ls;
+    ngx_listening_t    *ls, *mls;
     ngx_event_conf_t   *ecf;
     ngx_connection_t   *c, *lc;
     ngx_quic_socket_t  *qsock;
@@ -56,11 +56,12 @@ ngx_quic_recvmsg(ngx_event_t *ev)
 
     lc = ev->data;
     ls = lc->listening;
+    mls = ls->main ? ls->main : ls;
     ev->ready = 0;
 
-    ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0,
-                   "quic recvmsg on %V, ready: %d",
-                   &ls->addr_text, ev->available);
+    ngx_log_debug3(NGX_LOG_DEBUG_EVENT, ev->log, 0,
+                   "quic recvmsg on %V, fd:%d, ready: %d",
+                   &ls->addr_text, lc->fd, ev->available);
 
     do {
         ngx_memzero(&msg, sizeof(struct msghdr));
@@ -156,7 +157,8 @@ ngx_quic_recvmsg(ngx_event_t *ev)
             goto next;
         }
 
-        c = ngx_quic_lookup_connection(ls, &key, local_sockaddr, local_socklen);
+        c = ngx_quic_lookup_connection(mls, &key,
+                                       local_sockaddr, local_socklen);
 
         if (c) {
 
@@ -204,6 +206,12 @@ ngx_quic_recvmsg(ngx_event_t *ev)
             goto next;
         }
 
+        if ((ngx_terminate || ngx_exiting) && ls->worker != (ngx_uint_t) -1) {
+            ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0,
+                           "quic ignore new connections in a closing worker");
+            goto next;
+        }
+
 #if (NGX_STAT_STUB)
         (void) ngx_atomic_fetch_add(ngx_stat_accepted, 1);
 #endif
@@ -211,7 +219,7 @@ ngx_quic_recvmsg(ngx_event_t *ev)
         ngx_accept_disabled = ngx_cycle->connection_n / 8
                               - ngx_cycle->free_connection_n;
 
-        c = ngx_get_connection(lc->fd, ev->log);
+        c = ngx_get_connection(mls->connection->fd, ev->log);
         if (c == NULL) {
             return;
         }
@@ -248,7 +256,7 @@ ngx_quic_recvmsg(ngx_event_t *ev)
 
         c->log = log;
         c->pool->log = log;
-        c->listening = ls;
+        c->listening = mls;
 
         if (local_sockaddr == &lsa.sockaddr) {
             local_sockaddr = ngx_palloc(c->pool, local_socklen);
@@ -345,6 +353,10 @@ ngx_quic_recvmsg(ngx_event_t *ev)
         }
 
     } while (ev->available);
+
+#if (NGX_HAVE_EPOLLEXCLUSIVE)
+    ngx_reorder_accept_events(ls);
+#endif
 }
 
 


From murilo.b.andrade at gmail.com  Wed Aug 10 17:00:25 2022
From: murilo.b.andrade at gmail.com (Murilo Andrade)
Date: Wed, 10 Aug 2022 14:00:25 -0300
Subject: [PATCH] SSL: logging level of client issue during TLSv1.3
Message-ID: <B2590EA5-BBEB-4059-99C8-60645888ED9E@gmail.com>

Hello Maxim!

> Are you seeing these errors in practice?
Yes, it's happening 310 times an hour. We are seeing in log like this:
`2022/07/26 13:49:51 [crit] 1316#1316: *147702846 SSL_read() failed (SSL: error:1408F1BB:SSL routines:ssl3_get_record:bad record type) while waiting for request`


From murilo.b.andrade at gmail.com  Wed Aug 10 22:15:29 2022
From: murilo.b.andrade at gmail.com (Murilo Andrade)
Date: Wed, 10 Aug 2022 19:15:29 -0300
Subject: [PATCH] SSL: logging level of client issue during TLSv1.3
In-Reply-To: <2BCAB58C-861D-46FC-8750-AD8137116F27@gmail.com>
References: <2BCAB58C-861D-46FC-8750-AD8137116F27@gmail.com>
Message-ID: <9B8DF086-1BAA-4F3D-BE16-392DBC81E887@gmail.com>

Hello Maxim!

> Are you seeing these errors in practice?
Yes, it's happening 310 times an hour. We are seeing in log like this:
    2022/07/26 13:49:51 [crit] 1316#1316: *147702846 SSL_read() failed =
(SSL: error:1408F1BB:SSL routines:ssl3_get_record:bad record type) while =
waiting for request


> On 9 Aug 2022, at 17:34, Murilo Andrade <murilo.b.andrade at gmail.com> wrote:
> 
> # HG changeset patch
> # User Murilo Andrade <murilo.b.andrade at gmail.com>
> # Date 1660076026 10800
> #      Tue Aug 09 17:13:46 2022 -0300
> # Node ID 8b57fd5e8fac9d04cd286e2ad8a18a4030819234
> # Parent  069a4813e8d6d7ec662d282a10f5f7062ebd817f
> SSL: logging level of client issue during TLSv1.3
> 
> Such fatal errors are reported by OpenSSL 1.1.1 during TLSv1.3,
> caused by client issue. For example: when the handshake is
> concluded, the client send a "change_cipher_spec(20)" followed
> by an unknown Content-Type, for example: 26; the OpenSSL library
> will fail with SSL_R_BAD_RECORD_TYPE ("bad record type"). This
> failure now are logged at the "info" level.
> 
> diff -r 069a4813e8d6 -r 8b57fd5e8fac src/event/ngx_event_openssl.c
> --- a/src/event/ngx_event_openssl.c	Tue Jul 19 17:05:27 2022 +0300
> +++ b/src/event/ngx_event_openssl.c	Tue Aug 09 17:13:46 2022 -0300
> @@ -3423,6 +3423,9 @@
> #ifdef SSL_R_VERSION_TOO_LOW
>            || n == SSL_R_VERSION_TOO_LOW                            /*  396 */
> #endif
> +#ifdef SSL_R_BAD_RECORD_TYPE
> +            || n == SSL_R_BAD_RECORD_TYPE                            /*  443 */
> +#endif
>            || n == 1000 /* SSL_R_SSLV3_ALERT_CLOSE_NOTIFY */
> #ifdef SSL_R_SSLV3_ALERT_UNEXPECTED_MESSAGE
>            || n == SSL_R_SSLV3_ALERT_UNEXPECTED_MESSAGE             /* 1010 */
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220810/75255f4c/attachment.htm>

From xeioex at nginx.com  Thu Aug 11 03:07:33 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Thu, 11 Aug 2022 03:07:33 +0000
Subject: [njs] Stream: removed unused field ctx->from_upstream.
Message-ID: <hg.358e048185d8.1660187253.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/358e048185d8
branches:  
changeset: 1926:358e048185d8
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Aug 10 20:03:54 2022 -0700
description:
Stream: removed unused field ctx->from_upstream.

diffstat:

 nginx/ngx_stream_js_module.c |  2 --
 1 files changed, 0 insertions(+), 2 deletions(-)

diffs (19 lines):

diff -r 335ff5a4539d -r 358e048185d8 nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c	Mon Aug 08 23:18:51 2022 -0700
+++ b/nginx/ngx_stream_js_module.c	Wed Aug 10 20:03:54 2022 -0700
@@ -64,7 +64,6 @@ typedef struct {
 #define NGX_JS_EVENT_DOWNLOAD 1
 #define NGX_JS_EVENT_MAX      2
     ngx_stream_js_ev_t      events[2];
-    unsigned                from_upstream:1;
     unsigned                filter:1;
     unsigned                in_progress:1;
 } ngx_stream_js_ctx_t;
@@ -671,7 +670,6 @@ ngx_stream_js_body_filter(ngx_stream_ses
     }
 
     ctx->filter = 1;
-    ctx->from_upstream = from_upstream;
 
     ctx->last_out = &out;
 


From xeioex at nginx.com  Thu Aug 11 03:07:35 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Thu, 11 Aug 2022 03:07:35 +0000
Subject: [njs] Stream: improved flags argument for s.on() callback methods.
Message-ID: <hg.446a1cb64a6a.1660187255.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/446a1cb64a6a
branches:  
changeset: 1927:446a1cb64a6a
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Aug 10 20:04:40 2022 -0700
description:
Stream: improved flags argument for s.on() callback methods.

diffstat:

 nginx/ngx_js.c               |  25 +++++++++++++++++
 nginx/ngx_js.h               |   4 ++
 nginx/ngx_stream_js_module.c |  64 +++++++++++++++++++++++++++++++------------
 src/njs.h                    |   1 +
 src/njs_extern.c             |   2 +-
 5 files changed, 77 insertions(+), 19 deletions(-)

diffs (182 lines):

diff -r 358e048185d8 -r 446a1cb64a6a nginx/ngx_js.c
--- a/nginx/ngx_js.c	Wed Aug 10 20:03:54 2022 -0700
+++ b/nginx/ngx_js.c	Wed Aug 10 20:04:40 2022 -0700
@@ -263,6 +263,31 @@ ngx_js_ext_constant(njs_vm_t *vm, njs_ob
 
 
 njs_int_t
+ngx_js_ext_flags(njs_vm_t *vm, njs_object_prop_t *prop,
+    njs_value_t *value, njs_value_t *setval, njs_value_t *retval)
+{
+    uintptr_t  data;
+
+    data = (uintptr_t) njs_vm_external(vm, NJS_PROTO_ID_ANY, value);
+    if (data == 0) {
+        njs_value_undefined_set(retval);
+        return NJS_DECLINED;
+    }
+
+    data = data & (uintptr_t) njs_vm_prop_magic32(prop);
+
+    switch (njs_vm_prop_magic16(prop)) {
+    case NGX_JS_BOOLEAN:
+    default:
+        njs_value_boolean_set(retval, data);
+        break;
+    }
+
+    return NJS_OK;
+}
+
+
+njs_int_t
 ngx_js_ext_boolean(njs_vm_t *vm, njs_object_prop_t *prop,
     njs_value_t *value, njs_value_t *setval, njs_value_t *retval)
 {
diff -r 358e048185d8 -r 446a1cb64a6a nginx/ngx_js.h
--- a/nginx/ngx_js.h	Wed Aug 10 20:03:54 2022 -0700
+++ b/nginx/ngx_js.h	Wed Aug 10 20:04:40 2022 -0700
@@ -19,6 +19,8 @@
 #define NGX_JS_DEPRECATED   1
 #define NGX_JS_STRING       2
 #define NGX_JS_BUFFER       4
+#define NGX_JS_BOOLEAN      8
+#define NGX_JS_NUMBER       16
 
 #define ngx_js_buffer_type(btype) ((btype) & ~NGX_JS_DEPRECATED)
 
@@ -80,6 +82,8 @@ njs_int_t ngx_js_ext_uint(njs_vm_t *vm, 
     njs_value_t *value, njs_value_t *setval, njs_value_t *retval);
 njs_int_t ngx_js_ext_constant(njs_vm_t *vm, njs_object_prop_t *prop,
     njs_value_t *value, njs_value_t *setval, njs_value_t *retval);
+njs_int_t ngx_js_ext_flags(njs_vm_t *vm, njs_object_prop_t *prop,
+    njs_value_t *value, njs_value_t *setval, njs_value_t *retval);
 njs_int_t ngx_js_ext_boolean(njs_vm_t *vm, njs_object_prop_t *prop,
     njs_value_t *value, njs_value_t *setval, njs_value_t *retval);
 
diff -r 358e048185d8 -r 446a1cb64a6a nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c	Wed Aug 10 20:03:54 2022 -0700
+++ b/nginx/ngx_stream_js_module.c	Wed Aug 10 20:04:40 2022 -0700
@@ -493,6 +493,30 @@ static njs_external_t  ngx_stream_js_ext
 };
 
 
+static njs_external_t  ngx_stream_js_ext_session_flags[] = {
+
+    {
+        .flags = NJS_EXTERN_PROPERTY | NJS_EXTERN_SYMBOL,
+        .name.symbol = NJS_SYMBOL_TO_STRING_TAG,
+        .u.property = {
+            .value = "Stream Flags",
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("last"),
+        .enumerable = 1,
+        .u.property = {
+            .handler = ngx_js_ext_flags,
+            .magic16 = NGX_JS_BOOLEAN,
+            .magic32 = 0x00000001,
+        }
+    },
+
+};
+
+
 static njs_vm_ops_t ngx_stream_js_ops = {
     ngx_stream_js_set_timer,
     ngx_stream_js_clear_timer,
@@ -525,6 +549,7 @@ static ngx_stream_filter_pt  ngx_stream_
 
 
 static njs_int_t    ngx_stream_js_session_proto_id;
+static njs_int_t    ngx_stream_js_session_flags_proto_id;
 
 
 static ngx_int_t
@@ -923,14 +948,12 @@ static njs_int_t
 ngx_stream_js_run_event(ngx_stream_session_t *s, ngx_stream_js_ctx_t *ctx,
     ngx_stream_js_ev_t *event)
 {
-    size_t               len;
-    u_char              *p;
-    njs_int_t            ret;
-    ngx_buf_t           *b;
-    ngx_connection_t    *c;
-    njs_opaque_value_t   last_key, last;
-
-    static const njs_str_t last_str = njs_str("last");
+    size_t             len;
+    u_char            *p;
+    njs_int_t          ret;
+    ngx_buf_t         *b;
+    uintptr_t          flags;
+    ngx_connection_t  *c;
 
     if (event->ev == NULL) {
         return NJS_OK;
@@ -957,16 +980,12 @@ ngx_stream_js_run_event(ngx_stream_sessi
         return ret;
     }
 
-    njs_vm_value_string_set(ctx->vm, njs_value_arg(&last_key), last_str.start,
-                            last_str.length);
-
-    njs_value_boolean_set(njs_value_arg(&last), b && b->last_buf);
-
-    ret = njs_vm_object_alloc(ctx->vm, njs_value_arg(&ctx->args[2]),
-                               njs_value_arg(&last_key),
-                               njs_value_arg(&last), NULL);
+    flags = b && b->last_buf;
+
+    ret = njs_vm_external_create(ctx->vm, njs_value_arg(&ctx->args[2]),
+                       ngx_stream_js_session_flags_proto_id, (void *) flags, 0);
     if (ret != NJS_OK) {
-        return ret;
+        return NGX_ERROR;
     }
 
     njs_vm_post_event(ctx->vm, event->ev, njs_value_arg(&ctx->args[1]), 2);
@@ -1787,7 +1806,16 @@ ngx_stream_js_init_conf_vm(ngx_conf_t *c
                                          njs_nitems(ngx_stream_js_ext_session));
     if (ngx_stream_js_session_proto_id < 0) {
         ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
-                      "failed to add js request proto");
+                      "failed to add js session proto");
+        return NGX_ERROR;
+    }
+
+    ngx_stream_js_session_flags_proto_id = njs_vm_external_prototype(conf->vm,
+                                   ngx_stream_js_ext_session_flags,
+                                   njs_nitems(ngx_stream_js_ext_session_flags));
+    if (ngx_stream_js_session_flags_proto_id < 0) {
+        ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
+                      "failed to add js session flags proto");
         return NGX_ERROR;
     }
 
diff -r 358e048185d8 -r 446a1cb64a6a src/njs.h
--- a/src/njs.h	Wed Aug 10 20:03:54 2022 -0700
+++ b/src/njs.h	Wed Aug 10 20:04:40 2022 -0700
@@ -155,6 +155,7 @@ struct njs_external_s {
         struct {
             const char              value[15]; /* NJS_STRING_SHORT + 1. */
             njs_prop_handler_t      handler;
+            uint16_t                magic16;
             uint32_t                magic32;
         } property;
 
diff -r 358e048185d8 -r 446a1cb64a6a src/njs_extern.c
--- a/src/njs_extern.c	Wed Aug 10 20:03:54 2022 -0700
+++ b/src/njs_extern.c	Wed Aug 10 20:04:40 2022 -0700
@@ -92,7 +92,7 @@ njs_external_add(njs_vm_t *vm, njs_arr_t
                 prop->type = NJS_PROPERTY_HANDLER;
                 prop->value.type = NJS_INVALID;
                 prop->value.data.truth = 1;
-                prop->value.data.magic16 = 0;
+                prop->value.data.magic16 = external->u.property.magic16;
                 prop->value.data.magic32 = external->u.property.magic32;
                 prop->value.data.u.prop_handler = external->u.property.handler;
 


From gaoyan09 at baidu.com  Thu Aug 11 09:25:31 2022
From: gaoyan09 at baidu.com (=?utf-8?B?R2FvLFlhbijlqpLkvZPkupEp?=)
Date: Thu, 11 Aug 2022 09:25:31 +0000
Subject: [quic] Can nginx use sendmmsg directly instead of sendmsg?
Message-ID: <708DAF49-190F-4BEC-A8C8-6AD6886AEAA3@baidu.com>

Can nginx use sendmmsg directly instead of sendmsg?

Gao,Yan(ACG VCP)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220811/7a728441/attachment.htm>

From gaoyan09 at baidu.com  Thu Aug 11 09:41:04 2022
From: gaoyan09 at baidu.com (=?utf-8?B?R2FvLFlhbijlqpLkvZPkupEp?=)
Date: Thu, 11 Aug 2022 09:41:04 +0000
Subject: [quic] Do have exact new plans for implementing bbr 
Message-ID: <851859AE-B852-4C06-A421-C2E5FB7C549A@baidu.com>

Any update?

> Message: 2
> Date: Fri, 10 Dec 2021 10:43:40 +0300
> From: Vladimir Homutov <vl at nginx.com>
> To: nginx-devel at nginx.org
> Subject: Re: Congestion control questions
> Message-ID: <YbMFLKtAeCbVMGqj at vl.krasnogorsk.ru>
> Content-Type: text/plain; charset=utf-8
________________________________
> On Tue, Dec 07, 2021 at 06:05:48PM +0800, sun edward wrote:
> > Hi dev team,
> >        I have some questions about congestion control,   what's the current
> > congestion control algorithm in nginx quic,  is there any way or plan to
> > support CUBIC or BBR in nginx quic?
> >
> > thanks & regards

> Currently we have implemented minimalistic congestion control, as
> described in RFC 9002 [1].
> There are no exact plans for implementing more advanced schemes, but it
> is quite obvious that this area needs improvements, so we will have to
> do something about it in future.


Gao,Yan(ACG VCP)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220811/87ac919f/attachment.htm>

From murilo.b.andrade at gmail.com  Thu Aug 11 13:54:41 2022
From: murilo.b.andrade at gmail.com (Murilo Andrade)
Date: Thu, 11 Aug 2022 10:54:41 -0300
Subject: [PATCH] SSL: logging level of client issue during TLSv1.3
In-Reply-To: <B2590EA5-BBEB-4059-99C8-60645888ED9E@gmail.com>
References: <B2590EA5-BBEB-4059-99C8-60645888ED9E@gmail.com>
Message-ID: <89236170-62BD-4424-9563-4BF907F07778@gmail.com>

Sorry, this was my mistake.
We can delete this post please.


> On 10 Aug 2022, at 14:00, Murilo Andrade <murilo.b.andrade at gmail.com> wrote:
> 
> Hello Maxim!
> 
>> Are you seeing these errors in practice?
> Yes, it's happening 310 times an hour. We are seeing in log like this:
> `2022/07/26 13:49:51 [crit] 1316#1316: *147702846 SSL_read() failed (SSL: error:1408F1BB:SSL routines:ssl3_get_record:bad record type) while waiting for request`
> 


From lllkjok at gmail.com  Sat Aug 13 03:07:33 2022
From: lllkjok at gmail.com (Jabasukuriputo Wang)
Date: Sat, 13 Aug 2022 11:07:33 +0800
Subject: Nginx holding subrequests when using thread pool
In-Reply-To: <0229D633-BD72-425A-A256-921CA68D0F65@hxcore.ol>
References: <0229D633-BD72-425A-A256-921CA68D0F65@hxcore.ol>
Message-ID: <CA+mPnouD1AwK-3KL5SrhiBa=bJ32mq7cTsWxUH4m2F7PKJouQA@mail.gmail.com>

Hi,

I’m trying to modify the ModSecurity NGINX connector to use the thread
pool feature offered by NGINX. However when running the test suites
involving auth_request, as long as I did a ngx_thread_task_post in
either the rewrite phase or preaccess phase, the auth subrequest
always seems to somehow block until the main request canceled. What
happened in between?

Cheers,
W.

NGINX version: 1.21.4

Code of rewrite handler:

ngx_int_t ngx_http_modsecurity_rewrite_handler(ngx_http_request_t *r)
{
    ngx_http_modsecurity_rewrite_thread_ctx_t *ctx;
    ngx_http_modsecurity_ctx_t *m_ctx;
    ngx_thread_task_t *task;

    m_ctx = ngx_http_get_module_ctx(r, ngx_http_modsecurity_module);

    ngx_log_error(NGX_LOG_DEBUG, r->connection->log, 0, "recovering
ctx: %p", m_ctx);

    task = ngx_thread_task_alloc(r->pool,
sizeof(ngx_http_modsecurity_rewrite_thread_ctx_t));

    ctx = task->ctx;
    ctx->r = r;
    ctx->ctx = m_ctx;
    ctx->return_code = NGX_DECLINED;

    task->handler = ngx_http_modsecurity_rewrite_worker;
    task->event.handler = ngx_http_modsecurity_rewrite_finalizer;
    task->event.data = ctx;

    ngx_log_error(NGX_LOG_DEBUG, r->connection->log, 0, "[ModSecurity]
Using Thread Pool: %p", mcf->thread_pool);

    if (ngx_thread_task_post(mcf->thread_pool, task) != NGX_OK)
    {
        return NGX_ERROR;
    }

    return NGX_DONE;
}

The completion handler:

void ngx_http_modsecurity_rewrite_finalizer(ngx_event_t *ev)
{
    ngx_http_modsecurity_rewrite_thread_ctx_t *ctx = ev->data;
    ngx_http_core_main_conf_t *cmcf;

    ngx_log_error(NGX_LOG_DEBUG, ctx->r->connection->log, 0,
"[ModSecurity] Rewrite Job Finalized");

    cmcf = ngx_http_get_module_main_conf(ctx->r, ngx_http_core_module);

    switch (ctx->return_code)
    {
    case NGX_OK:
        ctx->r->phase_handler = cmcf->phase_engine.handlers->next;
        ngx_http_core_run_phases(ctx->r);
        break;
    case NGX_DECLINED:
        ctx->r->phase_handler++;
        ngx_http_core_run_phases(ctx->r);
        break;
    case NGX_AGAIN:
    case NGX_DONE:
        // ngx_http_core_run_phases(ctx->r);
        break;
    default:
        ngx_http_finalize_request(ctx->r, ctx->return_code);
    }
}

Note that even if I don’t do anything in
ngx_http_modsecurity_rewrite_worker, it still seems to block.

Relevant Config:

        location = /auth {
            return 200;
        }

        location = /useauth {
            modsecurity on;
            modsecurity_rules '
                SecRuleEngine On
                SecRequestBodyAccess On
            ';
            auth_request /auth;
            proxy_pass http://127.0.0.1:8081;
        }

Debug log (notice the gap, that’s when I Ctrl-C the curl):

2022/08/12 19:29:17 [debug] 17157#17160: Nothing to add on the body
inspection, reclaiming a NGX_DECLINED
2022/08/12 19:29:17 [debug] 17157#17160: complete task #2 in thread
pool "default"
2022/08/12 19:29:17 [debug] 17157#17157: epoll: fd:11 ev:0001 d:00005558A9EA4560
2022/08/12 19:29:17 [debug] 17157#17157: thread pool handler
2022/08/12 19:29:17 [debug] 17157#17157: run completion handler for task #2
2022/08/12 19:29:17 [debug] 17157#17157: *2 generic phase: 6
2022/08/12 19:29:17 [debug] 17157#17157: *2 generic phase: 7
2022/08/12 19:29:17 [debug] 17157#17157: *2 access phase: 8
2022/08/12 19:29:17 [debug] 17157#17157: *2 access phase: 9
2022/08/12 19:29:17 [debug] 17157#17157: *2 access phase: 10
2022/08/12 19:29:17 [debug] 17157#17157: *2 auth request handler
2022/08/12 19:29:17 [debug] 17157#17157: *2 http subrequest "/auth?"
2022/08/12 19:29:17 [debug] 17157#17157: timer delta: 0
2022/08/12 19:29:17 [debug] 17157#17157: worker cycle
2022/08/12 19:29:17 [debug] 17157#17157: epoll timer: -1
2022/08/12 19:29:23 [debug] 17157#17157: epoll: fd:3 ev:2001 d:00005558AB3ED909
2022/08/12 19:29:23 [debug] 17157#17157: *2 http run request: "/auth?"
2022/08/12 19:29:23 [debug] 17157#17157: *2 http request empty handler
2022/08/12 19:29:23 [debug] 17157#17157: *2 http posted request: "/auth?"
2022/08/12 19:29:23 [debug] 17157#17157: *2 rewrite phase: 0
2022/08/12 19:29:23 [debug] 17157#17157: *2 test location: "/bodylimitreject"
2022/08/12 19:29:23 [debug] 17157#17157: *2 test location: "/bodyaccess"
2022/08/12 19:29:23 [debug] 17157#17157: *2 test location: "/auth"
2022/08/12 19:29:23 [debug] 17157#17157: *2 using configuration "=/auth"
2022/08/12 19:29:23 [debug] 17157#17157: *2 http cl:-1 max:1048576
2022/08/12 19:29:23 [debug] 17157#17157: *2 rewrite phase: 2
2022/08/12 19:29:23 [debug] 17157#17157: *2 catching a new _rewrite_
phase handler

This is the log when I don’t bother with the thread pool:

2022/08/12 19:29:45 [debug] 18087#18087: *1 catching a new _rewrite_
phase handler
2022/08/12 19:29:45 [error] 18087#18087: *1 ModSecurity not enabled...
returning, client: 127.0.0.1, server: localhost, request: "GET
/useauth HTTP/1.1", host: "127.0.0.1:8080"
2022/08/12 19:29:45 [debug] 18087#18087: *1 rewrite phase: 3
2022/08/12 19:29:45 [debug] 18087#18087: *1 post rewrite phase: 4
2022/08/12 19:29:45 [debug] 18087#18087: *1 generic phase: 5
2022/08/12 19:29:45 [debug] 18087#18087: *1 catching a new _preaccess_
phase handler
2022/08/12 19:29:45 [error] 18087#18087: *1 ModSecurity not enabled...
returning, client: 127.0.0.1, server: localhost, request: "GET
/useauth HTTP/1.1", host: "127.0.0.1:8080"
2022/08/12 19:29:45 [debug] 18087#18087: *1 generic phase: 6
2022/08/12 19:29:45 [debug] 18087#18087: *1 generic phase: 7
2022/08/12 19:29:45 [debug] 18087#18087: *1 access phase: 8
2022/08/12 19:29:45 [debug] 18087#18087: *1 access phase: 9
2022/08/12 19:29:45 [debug] 18087#18087: *1 access phase: 10
2022/08/12 19:29:45 [debug] 18087#18087: *1 auth request handler
2022/08/12 19:29:45 [debug] 18087#18087: *1 http subrequest "/auth?"
2022/08/12 19:29:45 [debug] 18087#18087: *1 http posted request: "/auth?"
2022/08/12 19:29:45 [debug] 18087#18087: *1 rewrite phase: 0
2022/08/12 19:29:45 [debug] 18087#18087: *1 test location: "/bodylimitreject"
2022/08/12 19:29:45 [debug] 18087#18087: *1 test location: "/bodyaccess"
2022/08/12 19:29:45 [debug] 18087#18087: *1 test location: "/auth"
2022/08/12 19:29:45 [debug] 18087#18087: *1 using configuration "=/auth"
2022/08/12 19:29:45 [debug] 18087#18087: *1 http cl:-1 max:1048576
2022/08/12 19:29:45 [debug] 18087#18087: *1 rewrite phase: 2
2022/08/12 19:29:45 [debug] 18087#18087: *1 catching a new _rewrite_
phase handler
2022/08/12 19:29:45 [error] 18087#18087: *1 ModSecurity not enabled...
returning, client: 127.0.0.1, server: localhost, request: "GET
/useauth HTTP/1.1", subrequest: "/auth", host: "127.0.0.1:8080"
2022/08/12 19:29:45 [debug] 18087#18087: *1 rewrite phase: 3
2022/08/12 19:29:45 [debug] 18087#18087: *1 header filter, recovering
ctx: 0000000000000000


From xeioex at nginx.com  Sat Aug 20 04:40:26 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Sat, 20 Aug 2022 04:40:26 +0000
Subject: [njs] Tests: improved fs test262 tests robustness with different
 environment.
Message-ID: <hg.ae971e65f638.1660970426.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/ae971e65f638
branches:  
changeset: 1928:ae971e65f638
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Aug 19 21:38:55 2022 -0700
description:
Tests: improved fs test262 tests robustness with different environment.

diffstat:

 test/fs/methods.t.js            |  12 +++---
 test/fs/promises_01.t.js        |  12 +++++-
 test/fs/promises_02.t.js        |  75 ++++++++++++++++++++++++----------------
 test/fs/promises_03.t.js        |  37 ++++++++++---------
 test/fs/promises_04.t.js        |  60 +++++++++++++++-----------------
 test/fs/promises_05.t.js        |  37 ++++++++++---------
 test/fs/promises_06.t.js        |  69 ++++++++++++++++++-------------------
 test/fs/promises_07.t.js        |  35 +++++++++---------
 test/fs/promises_08.t.js        |  23 ++++++------
 test/fs/promises_09.t.js        |  19 +++++----
 test/harness/compatFs.js        |  32 ++++++++++++++++-
 test/harness/compatPrint.js     |   2 +-
 test/harness/compatWebcrypto.js |   2 +-
 13 files changed, 232 insertions(+), 183 deletions(-)

diffs (822 lines):

diff -r 446a1cb64a6a -r ae971e65f638 test/fs/methods.t.js
--- a/test/fs/methods.t.js	Wed Aug 10 20:04:40 2022 -0700
+++ b/test/fs/methods.t.js	Fri Aug 19 21:38:55 2022 -0700
@@ -535,7 +535,7 @@ let stat_tests = () => [
 
 let stat_tsuite = {
     name: "fs stat",
-    skip: () => (!has_fs() || !has_buffer()),
+    skip: () => (!has_fs() || !has_fs_symbolic_link() || !has_buffer()),
     T: stat_test,
     prepare_args: p,
     opts: { type: "callback", method: "stat" },
@@ -544,7 +544,7 @@ let stat_tsuite = {
 
 let statSync_tsuite = {
     name: "fs statSync",
-    skip: () => (!has_fs() || !has_buffer()),
+    skip: () => (!has_fs() || !has_fs_symbolic_link() || !has_buffer()),
     T: stat_test,
     prepare_args: p,
     opts: { type: "sync", method: "stat" },
@@ -553,7 +553,7 @@ let statSync_tsuite = {
 
 let statP_tsuite = {
     name: "fsp stat",
-    skip: () => (!has_fs() || !has_buffer()),
+    skip: () => (!has_fs() || !has_fs_symbolic_link() || !has_buffer()),
     T: stat_test,
     prepare_args: p,
     opts: { type: "promise", method: "stat" },
@@ -562,7 +562,7 @@ let statP_tsuite = {
 
 let lstat_tsuite = {
     name: "fs lstat",
-    skip: () => (!has_fs() || !has_buffer()),
+    skip: () => (!has_fs() || !has_fs_symbolic_link() || !has_buffer()),
     T: stat_test,
     prepare_args: p,
     opts: { type: "callback", method: "lstat" },
@@ -571,7 +571,7 @@ let lstat_tsuite = {
 
 let lstatSync_tsuite = {
     name: "fs lstatSync",
-    skip: () => (!has_fs() || !has_buffer()),
+    skip: () => (!has_fs() || !has_fs_symbolic_link() || !has_buffer()),
     T: stat_test,
     prepare_args: p,
     opts: { type: "sync", method: "lstat" },
@@ -580,7 +580,7 @@ let lstatSync_tsuite = {
 
 let lstatP_tsuite = {
     name: "fsp lstat",
-    skip: () => (!has_fs() || !has_buffer()),
+    skip: () => (!has_fs() || !has_fs_symbolic_link() || !has_buffer()),
     T: stat_test,
     prepare_args: p,
     opts: { type: "promise", method: "lstat" },
diff -r 446a1cb64a6a -r ae971e65f638 test/fs/promises_01.t.js
--- a/test/fs/promises_01.t.js	Wed Aug 10 20:04:40 2022 -0700
+++ b/test/fs/promises_01.t.js	Fri Aug 19 21:38:55 2022 -0700
@@ -7,7 +7,7 @@ var fname = `${test_dir}/fs_promises_01`
 
 let stages = [];
 
-Promise.resolve()
+let test = () => Promise.resolve()
 .then(() => {
     return fsp.writeFile(fname, fname);
 })
@@ -39,4 +39,12 @@ Promise.resolve()
 .then(() => {
     assert.compareArray(stages, ["init", "short circut", "chain", "errors ok"]);
 })
-.then($DONE, $DONE);
+
+let p = Promise.resolve()
+if (has_fs()) {
+    p = p
+        .then(test)
+        .then(() => assert.compareArray(stages, ["init", "short circut", "chain", "errors ok"]))
+}
+
+p.then($DONE, $DONE);
diff -r 446a1cb64a6a -r ae971e65f638 test/fs/promises_02.t.js
--- a/test/fs/promises_02.t.js	Wed Aug 10 20:04:40 2022 -0700
+++ b/test/fs/promises_02.t.js	Fri Aug 19 21:38:55 2022 -0700
@@ -5,8 +5,9 @@ flags: [async]
 
 var fname = `${test_dir}/fs_promises_02`;
 
-var testSync = new Promise((resolve, reject) => {
-    var failed = false;
+let stages = [];
+
+var testSync = () => new Promise((resolve, reject) => {
     try {
         fs.writeFileSync(fname, fname);
 
@@ -15,47 +16,52 @@ var testSync = new Promise((resolve, rej
 
         try {
             fs.accessSync(fname + '___');
-            failed = true;
-        } catch(e) {
-            failed = (e.syscall != 'access') || e.code != 'ENOENT';
+            reject(new Error('fs.accessSync error 1'));
+        } catch (e) {
+            if (e.syscall != 'access' || e.code != 'ENOENT') {
+                reject(new Error('fs.accessSync error 2'));
+            }
         }
-        resolve(Boolean(failed));
+
+        stages.push('testSync');
+
+        resolve();
+
     } catch (e) {
         reject(e);
     }
 });
 
-var testCallback = new Promise((resolve, reject) => {
-    var failed = false;
-
+var testCallback = () => new Promise((resolve, reject) => {
     fs.writeFileSync(fname, fname);
 
     fs.access(fname, (err) => {
-        failed = (err !== undefined);
+        if (err) {
+            reject(new Error('fs.access error 1'));
+        }
+
         fs.access(fname, fs.constants.R_OK | fs.constants.W_OK, (err) => {
-            failed |= (err !== undefined);
+            if (err) {
+                reject(err);
+            }
+
             fs.access(fname + '___', (err) => {
-                failed |= ((err === undefined) || (err.syscall != 'access')
-                                               || err.code != 'ENOENT');
-                resolve(Boolean(failed));
+                if (!err
+                    || err.syscall != 'access'
+                    || err.code != 'ENOENT')
+                    {
+                        reject(new Error('fs.access error 2'));
+                    }
+
+                stages.push('testCallback');
+
+                resolve();
             });
         });
     });
 });
 
-let stages = [];
-
-Promise.resolve()
-.then(() => testSync)
-.then(failed => {
-    stages.push('testSync');
-    assert.sameValue(failed, false, 'testSync');
-})
-.then(() => testCallback)
-.then(failed => {
-    stages.push('testCallback');
-    assert.sameValue(failed, false, 'testCallback');
-})
+let testFsp = () => Promise.resolve()
 .then(() => {
     fs.writeFileSync(fname, fname);
 
@@ -72,7 +78,14 @@ Promise.resolve()
     assert.sameValue(e.path, fname + '___', 'testPromise');
     assert.sameValue(e.code, 'ENOENT', 'testPromise');
 })
-.then(() => {
-    assert.compareArray(stages, ["testSync", "testCallback", "testPromise"]);
-})
-.then($DONE, $DONE);
+
+let p = Promise.resolve()
+if (has_fs()) {
+    p = p
+        .then(testSync)
+        .then(testCallback)
+        .then(testFsp)
+        .then(() => assert.compareArray(stages, ["testSync", "testCallback", "testPromise"]))
+}
+
+p.then($DONE, $DONE);
diff -r 446a1cb64a6a -r ae971e65f638 test/fs/promises_03.t.js
--- a/test/fs/promises_03.t.js	Wed Aug 10 20:04:40 2022 -0700
+++ b/test/fs/promises_03.t.js	Fri Aug 19 21:38:55 2022 -0700
@@ -5,6 +5,8 @@ flags: [async]
 
 var fname = `${test_dir}/fs_promises_03`;
 
+let stages = [];
+
 var testSync = () => new Promise((resolve, reject) => {
     try {
         try {
@@ -32,13 +34,14 @@ var testSync = () => new Promise((resolv
             void e;
         }
 
+        stages.push("unlinkSync");
+
         resolve();
     } catch (e) {
         reject(e);
     }
 });
 
-
 var testCallback = () => new Promise((resolve, reject) => {
     fs.unlink(fname, () => {
         fs.unlink(fname, (err) => {
@@ -64,31 +67,21 @@ var testCallback = () => new Promise((re
                 } catch (e) {
                     void e;
                 }
+
+                stages.push("unlink");
+
                 resolve();
             });
         });
     });
 });
 
-let stages = [];
-
-Promise.resolve()
-.then(testSync)
-.then(() => {
-    stages.push("unlinkSync");
-})
-
-.then(testCallback)
-.then(() => {
-    stages.push("unlink");
-})
-
+let testFsp = () => Promise.resolve()
 .then(() => fsp.unlink(fname)
             .catch(() => {}))
 .then(() => fsp.unlink(fname))
             .then(() => { throw new Error('fsp.unlink error 1'); })
 .catch((e) => { if (e.syscall != 'unlink') { throw e; } })
-
 .then(() => {
     fs.writeFileSync(fname, fname);
     return fsp.unlink(fname);
@@ -96,9 +89,17 @@ Promise.resolve()
 .then(() => fsp.access(fname))
             .then(() => { throw new Error('fsp.unlink error 2'); })
 .catch((e) => { if (e.syscall != 'access') { throw e; } })
-
 .then(() => {
     stages.push("fsp.unlink");
 })
-.then(() => assert.compareArray(stages, ['unlinkSync', 'unlink', 'fsp.unlink']))
-.then($DONE, $DONE);
+
+let p = Promise.resolve()
+if (has_fs()) {
+    p = p
+        .then(testSync)
+        .then(testCallback)
+        .then(testFsp)
+        .then(() => assert.compareArray(stages, ['unlinkSync', 'unlink', 'fsp.unlink']))
+}
+
+p.then($DONE, $DONE);
diff -r 446a1cb64a6a -r ae971e65f638 test/fs/promises_04.t.js
--- a/test/fs/promises_04.t.js	Wed Aug 10 20:04:40 2022 -0700
+++ b/test/fs/promises_04.t.js	Fri Aug 19 21:38:55 2022 -0700
@@ -7,25 +7,21 @@ var fname = `${test_dir}/fs_promises_04`
 var fname_utf8 = `${test_dir}/fs_promises_αβγ_04`;
 var lname = `${test_dir}/fs_promises_lnk_04`;
 
+let stages = [];
+
 var testSync = () => new Promise((resolve, reject) => {
     try {
-        try {
-            fs.unlinkSync(fname);
-        } catch (e) {
-            void e;
-        }
-        try {
-            fs.unlinkSync(lname);
-        } catch (e) {
-            void e;
-        }
+        try { fs.unlinkSync(fname); } catch (e) {}
+        try { fs.unlinkSync(lname); } catch (e) {}
 
         try {
             fs.realpathSync(fname);
+
             throw new Error('fs.realpathSync error 1');
+
         } catch (e) {
-            if (e.syscall != 'realpath' || e.code != 'ENOENT') {
-                throw e;
+            if (e.code != 'ENOENT') {
+                throw new Error('fs.realpathSync error 2');
             }
         }
 
@@ -46,7 +42,7 @@ var testSync = () => new Promise((resolv
 
         var rname_utf8 = fs.realpathSync(fname_utf8);
         if (rname_utf8.slice(-6,-3) != 'αβγ') {
-            throw new Error('fs.realpathSync error 2');
+            throw new Error('fs.realpathSync error 3');
         }
 
         fs.unlinkSync(lname);
@@ -54,6 +50,8 @@ var testSync = () => new Promise((resolv
         fs.unlinkSync(fname);
         fs.unlinkSync(fname_utf8);
 
+        stages.push("symlinkSync");
+
         resolve();
 
     } catch (e) {
@@ -80,7 +78,8 @@ var testCallback = () => new Promise((re
                 reject(new Error('fs.realpath error 1'));
                 return;
             }
-            if (err.syscall != 'realpath' || err.code != 'ENOENT') {
+
+            if (err.code != 'ENOENT') {
                 reject(err);
                 return;
             }
@@ -130,6 +129,8 @@ var testCallback = () => new Promise((re
                             return;
                         }
 
+                        stages.push("symlink");
+
                         resolve();
                     });
                 });
@@ -141,19 +142,7 @@ var testCallback = () => new Promise((re
     }
 });
 
-let stages = [];
-
-Promise.resolve()
-.then(testSync)
-.then(() => {
-    stages.push("symlinkSync");
-})
-
-.then(testCallback)
-.then(() => {
-    stages.push("symlink");
-})
-
+let testFsp = () => Promise.resolve()
 .then(() => fsp.unlink(fname)
             .catch(() => {}))
 .then(() => fsp.unlink(lname)
@@ -186,10 +175,17 @@ Promise.resolve()
     fs.unlinkSync(lname);
     fs.accessSync(fname);
     fs.unlinkSync(fname);
+
+    stages.push("fsp.symlink");
 })
 
-.then(() => {
-    stages.push("fsp.symlink");
-})
-.then(() => assert.compareArray(stages, ['symlinkSync', 'symlink', 'fsp.symlink']))
-.then($DONE, $DONE);
+let p = Promise.resolve()
+if (has_fs() && has_fs_symbolic_link()) {
+    p = p
+        .then(testSync)
+        .then(testCallback)
+        .then(testFsp)
+        .then(() => assert.compareArray(stages, ['symlinkSync', 'symlink', 'fsp.symlink']))
+}
+
+p.then($DONE, $DONE);
diff -r 446a1cb64a6a -r ae971e65f638 test/fs/promises_05.t.js
--- a/test/fs/promises_05.t.js	Wed Aug 10 20:04:40 2022 -0700
+++ b/test/fs/promises_05.t.js	Fri Aug 19 21:38:55 2022 -0700
@@ -7,6 +7,8 @@ var dname = `${test_dir}/fs_promises_05`
 var dname_utf8 = `${test_dir}/fs_promises_αβγ_05`;
 var fname = (d) => d + '/fs_promises_05_file';
 
+let stages = [];
+
 var testSync = () => new Promise((resolve, reject) => {
     try {
         try { fs.unlinkSync(fname(dname)); } catch (e) {}
@@ -47,6 +49,8 @@ var testSync = () => new Promise((resolv
         try {
             fs.writeFileSync(fname(dname_utf8), fname(dname_utf8));
 
+            throw new Error('fs.mkdirSync error 1');
+
         } catch (e) {
             if (e.syscall != 'open' || e.code != 'EACCES') {
                 throw e;
@@ -64,6 +68,8 @@ var testSync = () => new Promise((resolv
 
         fs.rmdirSync(dname_utf8);
 
+        stages.push("mkdirSync");
+
         resolve();
 
     } catch (e) {
@@ -94,6 +100,8 @@ var testCallback = () => new Promise((re
                         reject(err);
                     }
 
+                    stages.push("mkdir");
+
                     resolve();
                 });
             });
@@ -104,21 +112,7 @@ var testCallback = () => new Promise((re
     }
 });
 
-
-let stages = [];
-
-Promise.resolve()
-.then(testSync)
-.then(() => {
-    stages.push("mkdirSync");
-})
-
-
-.then(testCallback)
-.then(() => {
-    stages.push("mkdir");
-})
-
+let testFsp = () => Promise.resolve()
 .then(() => {
     try { fs.unlinkSync(fname(dname)); } catch (e) {}
     try { fs.unlinkSync(fname(dname_utf8)); } catch (e) {}
@@ -136,5 +130,14 @@ Promise.resolve()
 .then(() => {
     stages.push("fsp.mkdir");
 })
-.then(() => assert.compareArray(stages, ['mkdirSync', 'mkdir', 'fsp.mkdir']))
-.then($DONE, $DONE);
+
+let p = Promise.resolve()
+if (has_fs()) {
+    p = p
+        .then(testSync)
+        .then(testCallback)
+        .then(testFsp)
+        .then(() => assert.compareArray(stages, ['mkdirSync', 'mkdir', 'fsp.mkdir']))
+}
+
+p.then($DONE, $DONE);
diff -r 446a1cb64a6a -r ae971e65f638 test/fs/promises_06.t.js
--- a/test/fs/promises_06.t.js	Wed Aug 10 20:04:40 2022 -0700
+++ b/test/fs/promises_06.t.js	Fri Aug 19 21:38:55 2022 -0700
@@ -7,7 +7,9 @@ var dname = `${test_dir}/`;
 var fname = (d) => d + '/fs_promises_06_file';
 var fname_utf8 = (d) => d + '/fs_promises_αβγ_06';
 
-var testSync = new Promise((resolve, reject) => {
+let stages = [];
+
+var testSync = () => new Promise((resolve, reject) => {
     try {
         try { fs.unlinkSync(fname(dname)); } catch (e) {}
         try { fs.unlinkSync(fname_utf8(dname)); } catch (e) {}
@@ -25,10 +27,12 @@ var testSync = new Promise((resolve, rej
             if (e.syscall != 'rename'
                 || (e.code != 'ENOTDIR' && e.code != 'EISDIR'))
             {
-                throw e;
+                reject(new Error('fs.unlinkSync error 1'));
             }
         }
 
+        stages.push("renameSync");
+
         resolve();
 
     } catch (e) {
@@ -36,7 +40,7 @@ var testSync = new Promise((resolve, rej
     }
 });
 
-var testCallback = new Promise((resolve, reject) => {
+var testCallback = () => new Promise((resolve, reject) => {
     try {
         try { fs.unlinkSync(fname(dname)); } catch (e) {}
         try { fs.unlinkSync(fname_utf8(dname)); } catch (e) {}
@@ -45,46 +49,30 @@ var testCallback = new Promise((resolve,
 
         fs.rename(fname(dname), fname_utf8(dname), err => {
             if (err) {
-                throw err;
+                reject(new Error('fs.unlink error 1'));
             }
-        });
 
-        fs.accessSync(fname_utf8(dname));
+            fs.accessSync(fname_utf8(dname));
 
-        fs.rename(fname_utf8(dname), dname, err => {
-            if (err.syscall != 'rename'
-                || (err.code != 'ENOTDIR' && err.code != 'EISDIR'))
-            {
-                throw err;
-            }
+            fs.rename(fname_utf8(dname), dname, err => {
+                if (err.syscall != 'rename'
+                    || (err.code != 'ENOTDIR' && err.code != 'EISDIR'))
+                {
+                    reject(new Error('fs.unlink error 2'));
+                }
+            });
+
+            stages.push("rename");
+
+            resolve();
         });
 
-        resolve();
-
     } catch (e) {
         reject(e);
     }
 });
 
-let stages = [];
-
-Promise.resolve()
-.then(() => testSync)
-.then(() => {
-    stages.push("renameSync");
-})
-.catch((e) => {
-    console.log('test fs.renameSync failed', JSON.stringify(e));
-})
-
-.then(testCallback)
-.then(() => {
-    stages.push("rename");
-})
-.catch((e) => {
-    console.log('test fs.rename failed', JSON.stringify(e));
-})
-
+let testFsp = () => Promise.resolve()
 .then(() => {
     try { fs.unlinkSync(fname(dname)); } catch (e) {}
     try { fs.unlinkSync(fname_utf8(dname)); } catch (e) {}
@@ -98,11 +86,20 @@ Promise.resolve()
     if (e.syscall != 'rename'
         || (e.code != 'ENOTDIR' && e.code != 'EISDIR'))
     {
-        throw e;
+        throw new Error('fsp.rename error 1');
     }
 })
 .then(() => {
     stages.push("fsp.rename");
 })
-.then(() => assert.compareArray(stages, ["renameSync", "rename", "fsp.rename"]))
-.then($DONE, $DONE);
+
+let p = Promise.resolve()
+if (has_fs()) {
+    p = p
+        .then(testSync)
+        .then(testCallback)
+        .then(testFsp)
+        .then(() => assert.compareArray(stages, ["renameSync", "rename", "fsp.rename"]))
+}
+
+p.then($DONE, $DONE);
diff -r 446a1cb64a6a -r ae971e65f638 test/fs/promises_07.t.js
--- a/test/fs/promises_07.t.js	Wed Aug 10 20:04:40 2022 -0700
+++ b/test/fs/promises_07.t.js	Fri Aug 19 21:38:55 2022 -0700
@@ -15,7 +15,7 @@ var match = (entry) => {
     var idx = dir_test.indexOf(entry.name);
 
     try {
-        switch(idx) {
+        switch (idx) {
         case 0:
             return entry.isDirectory();
         case 1:
@@ -34,6 +34,7 @@ var match = (entry) => {
     }
 };
 
+let stages = [];
 
 var testSync = () => new Promise((resolve, reject) => {
     try {
@@ -109,6 +110,8 @@ var testSync = () => new Promise((resolv
             throw new Error('fs.readdirSync - error 8');
         }
 
+        stages.push("readdirSync");
+
         resolve();
 
     } catch (e) {
@@ -167,6 +170,8 @@ var testCallback = () => new Promise((re
                         reject(new Error('fs.readdir - error 5'));
                     }
 
+                    stages.push("readdir");
+
                     resolve();
                 });
             });
@@ -177,20 +182,7 @@ var testCallback = () => new Promise((re
     }
 });
 
-
-let stages = [];
-
-Promise.resolve()
-.then(testSync)
-.then(() => {
-    stages.push("readdirSync");
-})
-
-.then(testCallback)
-.then(() => {
-    stages.push("readdir");
-})
-
+let testFsp = () => Promise.resolve()
 .then(() => {
     try { fs.rmdirSync(cname(dname)); } catch (e) {}
     try { fs.unlinkSync(lname(dname)); } catch (e) {}
@@ -235,5 +227,14 @@ Promise.resolve()
 .then(() => {
     stages.push("fsp.readdir");
 })
-.then(() => assert.compareArray(stages, ["readdirSync", "readdir", "fsp.readdir"]))
-.then($DONE, $DONE);
+
+let p = Promise.resolve()
+if (has_fs() && has_fs_symbolic_link()) {
+    p = p
+        .then(testSync)
+        .then(testCallback)
+        .then(testFsp)
+        .then(() => assert.compareArray(stages, ['readdirSync', 'readdir', 'fsp.readdir']))
+}
+
+p.then($DONE, $DONE);
diff -r 446a1cb64a6a -r ae971e65f638 test/fs/promises_08.t.js
--- a/test/fs/promises_08.t.js	Wed Aug 10 20:04:40 2022 -0700
+++ b/test/fs/promises_08.t.js	Fri Aug 19 21:38:55 2022 -0700
@@ -24,6 +24,8 @@ var wipePath = (root, path, nofail) => {
         });
 };
 
+let stages = [];
+
 var testSync = () => new Promise((resolve, reject) => {
     try {
         wipePath(dname, path + '/' + path, true);
@@ -66,21 +68,20 @@ var testSync = () => new Promise((resolv
         wipePath(dname, path);
 
         fs.rmdirSync(dname);
+
+        stages.push("mkdirSync")
+
         resolve();
     } catch (e) {
         reject(e);
     }
 });
 
-let stages = [];
+let p = Promise.resolve()
+if (has_fs()) {
+    p = p
+        .then(testSync)
+        .then(() => assert.compareArray(stages, ["mkdirSync"]))
+}
 
-Promise.resolve()
-.then(testSync)
-.then(() => {
-    stages.push("mkdirSync");
-})
-.catch((e) => {
-    $DONOTEVALUATE();
-})
-.then(() => assert.compareArray(stages, ["mkdirSync"]))
-.then($DONE, $DONE);
+p.then($DONE, $DONE);
diff -r 446a1cb64a6a -r ae971e65f638 test/fs/promises_09.t.js
--- a/test/fs/promises_09.t.js	Wed Aug 10 20:04:40 2022 -0700
+++ b/test/fs/promises_09.t.js	Fri Aug 19 21:38:55 2022 -0700
@@ -23,6 +23,7 @@ var setContent = (root, path) => {
 
 var isNode = () => process.argv[0].includes('node');
 
+let stages = [];
 
 var testSync = () => new Promise((resolve, reject) => {
     try {
@@ -90,6 +91,8 @@ var testSync = () => new Promise((resolv
             }
         }
 
+        stages.push("rmdirSync");
+
         resolve();
 
     } catch (e) {
@@ -97,13 +100,11 @@ var testSync = () => new Promise((resolv
     }
 });
 
-
-let stages = [];
+let p = Promise.resolve()
+if (has_fs() && has_fs_symbolic_link()) {
+    p = p
+        .then(testSync)
+        .then(() => assert.compareArray(stages, ['rmdirSync']))
+}
 
-Promise.resolve()
-.then(testSync)
-.then(() => {
-    stages.push("rmdirSync");
-})
-.then(() => assert.compareArray(stages, ["rmdirSync"]))
-.then($DONE, $DONE);
+p.then($DONE, $DONE);
diff -r 446a1cb64a6a -r ae971e65f638 test/harness/compatFs.js
--- a/test/harness/compatFs.js	Wed Aug 10 20:04:40 2022 -0700
+++ b/test/harness/compatFs.js	Fri Aug 19 21:38:55 2022 -0700
@@ -6,9 +6,37 @@ if (typeof require == 'function') {
     fsp = fs.promises;
 }
 
+if (typeof process == 'undefined') {
+    globalThis.process = {};
+}
+
+let test_dir = process.env && process.env['NJS_TEST_DIR'] || 'build';
+test_dir = `${test_dir}/test`;
+
 function has_fs() {
     return fs;
 }
 
-let test_dir = process.env && process.env['NJS_TEST_DIR'] || 'build';
-test_dir = `${test_dir}/test`;
+function has_fs_symbolic_link() {
+    if (!fs) {
+        return false;
+    }
+
+    let fname = test_dir + '/a';
+    let lname = test_dir + '/b';
+
+    try { fs.unlinkSync(fname); fs.unlinkSync(lname); } catch (e) {}
+
+    fs.writeFileSync(fname, fname);
+
+    fname = fs.realpathSync(fname);
+
+    try {
+        fs.symlinkSync(fname, lname);
+    } catch (e) {
+        return false;
+    }
+
+    return true;
+}
+
diff -r 446a1cb64a6a -r ae971e65f638 test/harness/compatPrint.js
--- a/test/harness/compatPrint.js	Wed Aug 10 20:04:40 2022 -0700
+++ b/test/harness/compatPrint.js	Fri Aug 19 21:38:55 2022 -0700
@@ -1,3 +1,3 @@
 if (typeof print !== 'function') {
-    print = console.log;
+    globalThis.print = console.log;
 }
diff -r 446a1cb64a6a -r ae971e65f638 test/harness/compatWebcrypto.js
--- a/test/harness/compatWebcrypto.js	Wed Aug 10 20:04:40 2022 -0700
+++ b/test/harness/compatWebcrypto.js	Fri Aug 19 21:38:55 2022 -0700
@@ -1,5 +1,5 @@
 if (typeof crypto == 'undefined' && typeof require == 'function') {
-    crypto = require('crypto').webcrypto;
+    globalThis.crypto = require('crypto').webcrypto;
 }
 
 function has_webcrypto() {

From xeioex at nginx.com  Sat Aug 20 05:08:18 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Sat, 20 Aug 2022 05:08:18 +0000
Subject: [njs] Modules: improved generic external getters.
Message-ID: <hg.7d2fb383e424.1660972098.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/7d2fb383e424
branches:  
changeset: 1929:7d2fb383e424
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Aug 19 22:02:37 2022 -0700
description:
Modules: improved generic external getters.

diffstat:

 nginx/ngx_js.c       |  28 +++++++++++++++++-----------
 nginx/ngx_js.h       |   2 --
 nginx/ngx_js_fetch.c |   3 ++-
 3 files changed, 19 insertions(+), 14 deletions(-)

diffs (92 lines):

diff -r ae971e65f638 -r 7d2fb383e424 nginx/ngx_js.c
--- a/nginx/ngx_js.c	Fri Aug 19 21:38:55 2022 -0700
+++ b/nginx/ngx_js.c	Fri Aug 19 22:02:37 2022 -0700
@@ -34,6 +34,7 @@ static njs_external_t  ngx_js_ext_core[]
         .u.property = {
             .handler = ngx_js_ext_constant,
             .magic32 = NGX_LOG_INFO,
+            .magic16 = NGX_JS_NUMBER,
         }
     },
 
@@ -43,6 +44,7 @@ static njs_external_t  ngx_js_ext_core[]
         .u.property = {
             .handler = ngx_js_ext_constant,
             .magic32 = NGX_LOG_WARN,
+            .magic16 = NGX_JS_NUMBER,
         }
     },
 
@@ -52,6 +54,7 @@ static njs_external_t  ngx_js_ext_core[]
         .u.property = {
             .handler = ngx_js_ext_constant,
             .magic32 = NGX_LOG_ERR,
+            .magic16 = NGX_JS_NUMBER,
         }
     },
 
@@ -256,7 +259,20 @@ njs_int_t
 ngx_js_ext_constant(njs_vm_t *vm, njs_object_prop_t *prop,
     njs_value_t *value, njs_value_t *setval, njs_value_t *retval)
 {
-    njs_value_number_set(retval, njs_vm_prop_magic32(prop));
+    uint32_t  magic32;
+
+    magic32 = njs_vm_prop_magic32(prop);
+
+    switch (njs_vm_prop_magic16(prop)) {
+    case NGX_JS_NUMBER:
+        njs_value_number_set(retval, magic32);
+        break;
+
+    case NGX_JS_BOOLEAN:
+    default:
+        njs_value_boolean_set(retval, magic32);
+        break;
+    }
 
     return NJS_OK;
 }
@@ -288,16 +304,6 @@ ngx_js_ext_flags(njs_vm_t *vm, njs_objec
 
 
 njs_int_t
-ngx_js_ext_boolean(njs_vm_t *vm, njs_object_prop_t *prop,
-    njs_value_t *value, njs_value_t *setval, njs_value_t *retval)
-{
-    njs_value_boolean_set(retval, njs_vm_prop_magic32(prop));
-
-    return NJS_OK;
-}
-
-
-njs_int_t
 ngx_js_ext_log(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
     njs_index_t level)
 {
diff -r ae971e65f638 -r 7d2fb383e424 nginx/ngx_js.h
--- a/nginx/ngx_js.h	Fri Aug 19 21:38:55 2022 -0700
+++ b/nginx/ngx_js.h	Fri Aug 19 22:02:37 2022 -0700
@@ -84,8 +84,6 @@ njs_int_t ngx_js_ext_constant(njs_vm_t *
     njs_value_t *value, njs_value_t *setval, njs_value_t *retval);
 njs_int_t ngx_js_ext_flags(njs_vm_t *vm, njs_object_prop_t *prop,
     njs_value_t *value, njs_value_t *setval, njs_value_t *retval);
-njs_int_t ngx_js_ext_boolean(njs_vm_t *vm, njs_object_prop_t *prop,
-    njs_value_t *value, njs_value_t *setval, njs_value_t *retval);
 
 ngx_int_t ngx_js_core_init(njs_vm_t *vm, ngx_log_t *log);
 
diff -r ae971e65f638 -r 7d2fb383e424 nginx/ngx_js_fetch.c
--- a/nginx/ngx_js_fetch.c	Fri Aug 19 21:38:55 2022 -0700
+++ b/nginx/ngx_js_fetch.c	Fri Aug 19 22:02:37 2022 -0700
@@ -276,8 +276,9 @@ static njs_external_t  ngx_js_ext_http_r
         .name.string = njs_str("redirected"),
         .enumerable = 1,
         .u.property = {
-            .handler = ngx_js_ext_boolean,
+            .handler = ngx_js_ext_constant,
             .magic32 = 0,
+            .magic16 = NGX_JS_BOOLEAN,
         }
     },
 


From xeioex at nginx.com  Sat Aug 20 05:08:20 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Sat, 20 Aug 2022 05:08:20 +0000
Subject: [njs] Added generic handler for external objects.
Message-ID: <hg.aecd8b5387d2.1660972100.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/aecd8b5387d2
branches:  
changeset: 1930:aecd8b5387d2
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Aug 19 22:03:06 2022 -0700
description:
Added generic handler for external objects.

diffstat:

 src/njs.h        |  10 ++++++++++
 src/njs_extern.c |  36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+), 0 deletions(-)

diffs (70 lines):

diff -r 7d2fb383e424 -r aecd8b5387d2 src/njs.h
--- a/src/njs.h	Fri Aug 19 22:02:37 2022 -0700
+++ b/src/njs.h	Fri Aug 19 22:03:06 2022 -0700
@@ -139,6 +139,13 @@ typedef enum {
 } njs_extern_flag_t;
 
 
+typedef enum {
+    NJS_EXTERN_TYPE_INT = 0,
+    NJS_EXTERN_TYPE_UINT = 1,
+    NJS_EXTERN_TYPE_VALUE = 2,
+} njs_extern_type_t;
+
+
 struct njs_external_s {
     njs_extern_flag_t               flags;
 
@@ -361,6 +368,9 @@ NJS_EXPORT njs_int_t njs_vm_external_cre
     njs_int_t proto_id, njs_external_ptr_t external, njs_bool_t shared);
 NJS_EXPORT njs_external_ptr_t njs_vm_external(njs_vm_t *vm,
     njs_int_t proto_id, const njs_value_t *value);
+NJS_EXPORT njs_int_t njs_external_property(njs_vm_t *vm,
+    njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval,
+    njs_value_t *retval);
 NJS_EXPORT uintptr_t njs_vm_meta(njs_vm_t *vm, njs_uint_t index);
 
 NJS_EXPORT njs_function_t *njs_vm_function_alloc(njs_vm_t *vm,
diff -r 7d2fb383e424 -r aecd8b5387d2 src/njs_extern.c
--- a/src/njs_extern.c	Fri Aug 19 22:02:37 2022 -0700
+++ b/src/njs_extern.c	Fri Aug 19 22:03:06 2022 -0700
@@ -343,3 +343,39 @@ njs_value_external_tag(const njs_value_t
 
     return -1;
 }
+
+
+njs_int_t
+njs_external_property(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value,
+    njs_value_t *setval, njs_value_t *retval)
+{
+    char        *p;
+    njs_int_t   i;
+    njs_uint_t  ui;
+
+    p = njs_vm_external(vm, NJS_PROTO_ID_ANY, value);
+    if (p == NULL) {
+        njs_value_undefined_set(retval);
+        return NJS_DECLINED;
+    }
+
+    switch (njs_vm_prop_magic16(prop)) {
+    case NJS_EXTERN_TYPE_INT:
+        i = *(njs_int_t *) (p + njs_vm_prop_magic32(prop));
+        njs_value_number_set(retval, i);
+        break;
+
+    case NJS_EXTERN_TYPE_UINT:
+        ui = *(njs_uint_t *) (p + njs_vm_prop_magic32(prop));
+        njs_value_number_set(retval, ui);
+        break;
+
+    case NJS_EXTERN_TYPE_VALUE:
+    default:
+        njs_value_assign(retval,
+                         (njs_value_t *) (p + njs_vm_prop_magic32(prop)));
+
+    }
+
+    return NJS_OK;
+}


From xeioex at nginx.com  Tue Aug 23 00:23:26 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 23 Aug 2022 00:23:26 +0000
Subject: [njs] Added fs.FileHandle.
Message-ID: <hg.e9ce6daa07fa.1661214206.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/e9ce6daa07fa
branches:  
changeset: 1931:e9ce6daa07fa
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Aug 22 16:49:27 2022 -0700
description:
Added fs.FileHandle.

The following methods are implemented:
    - fs.openSync(path[, flag[, mode]])
    - fs.promises.open(path[, flag[, mode]])
    - fs.fstatSync(fd)
    - fs.readSync(fd, buffer, offset[, length[, position]])
    - fs.writeSync(fd, buffer, offset[, length[, position]])
    - fs.writeSync(fd, string[, position[, encoding]])

The following properties of FileHandle are implemented:
    - filehandle.fd
    - filehandle.read(buffer, offset, [length[, position]])
    - filehandle.stat()
    - filehandle.write(buffer, offset, [length[, position]])
    - filehandle.write(string[, position[, encoding]])
    - filehandle.close()

diffstat:

 external/njs_fs_module.c |  831 ++++++++++++++++++++++++++++++++++++++++++++++-
 src/njs_buffer.c         |    2 +-
 src/njs_buffer.h         |    2 +
 test/fs/methods.t.js     |  623 +++++++++++++++++++++++++++++++++-
 4 files changed, 1412 insertions(+), 46 deletions(-)

diffs (truncated from 1825 to 1000 lines):

diff -r aecd8b5387d2 -r e9ce6daa07fa external/njs_fs_module.c
--- a/external/njs_fs_module.c	Fri Aug 19 22:03:06 2022 -0700
+++ b/external/njs_fs_module.c	Mon Aug 22 16:49:27 2022 -0700
@@ -56,6 +56,7 @@ typedef enum {
 typedef enum {
     NJS_FS_STAT,
     NJS_FS_LSTAT,
+    NJS_FS_FSTAT,
 } njs_fs_statmode_t;
 
 
@@ -125,6 +126,18 @@ typedef enum {
 } njs_stat_prop_t;
 
 
+typedef struct {
+    njs_int_t       fd;
+    njs_vm_t        *vm;
+} njs_filehandle_t;
+
+
+typedef struct {
+    njs_int_t       bytes;
+    njs_value_t     buffer;
+} njs_bytes_struct_t;
+
+
 typedef njs_int_t (*njs_file_tree_walk_cb_t)(const char *, const struct stat *,
      njs_ftw_type_t);
 
@@ -133,7 +146,13 @@ static njs_int_t njs_fs_access(njs_vm_t 
     njs_uint_t nargs, njs_index_t calltype);
 static njs_int_t njs_fs_mkdir(njs_vm_t *vm, njs_value_t *args,
     njs_uint_t nargs, njs_index_t calltype);
-static njs_int_t njs_fs_read(njs_vm_t *vm, njs_value_t *args,
+static njs_int_t njs_fs_open(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
+    njs_index_t calltype);
+static njs_int_t njs_fs_close(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
+    njs_index_t calltype);
+static njs_int_t njs_fs_read(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
+    njs_index_t calltype);
+static njs_int_t njs_fs_read_file(njs_vm_t *vm, njs_value_t *args,
     njs_uint_t nargs, njs_index_t calltype);
 static njs_int_t njs_fs_readdir(njs_vm_t *vm, njs_value_t *args,
     njs_uint_t nargs, njs_index_t calltype);
@@ -149,7 +168,9 @@ static njs_int_t njs_fs_symlink(njs_vm_t
     njs_uint_t nargs, njs_index_t calltype);
 static njs_int_t njs_fs_unlink(njs_vm_t *vm, njs_value_t *args,
     njs_uint_t nargs, njs_index_t calltype);
-static njs_int_t njs_fs_write(njs_vm_t *vm, njs_value_t *args,
+static njs_int_t njs_fs_write(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
+    njs_index_t calltype);
+static njs_int_t njs_fs_write_file(njs_vm_t *vm, njs_value_t *args,
     njs_uint_t nargs, njs_index_t calltype);
 
 static njs_int_t njs_fs_constants(njs_vm_t *vm, njs_object_prop_t *prop,
@@ -169,6 +190,18 @@ static njs_int_t njs_fs_stats_prop(njs_v
 static njs_int_t njs_fs_stats_create(njs_vm_t *vm, struct stat *st,
     njs_value_t *retval);
 
+static njs_int_t njs_fs_filehandle_close(njs_vm_t *vm, njs_value_t *args,
+    njs_uint_t nargs, njs_index_t unused);
+static njs_int_t njs_fs_filehandle_value_of(njs_vm_t *vm, njs_value_t *args,
+    njs_uint_t nargs, njs_index_t unused);
+static njs_int_t njs_fs_filehandle_create(njs_vm_t *vm, int fd,
+    njs_bool_t shadow, njs_value_t *retval);
+
+static njs_int_t njs_fs_bytes_read_create(njs_vm_t *vm, int bytes,
+    njs_value_t *buffer, njs_value_t *retval);
+static njs_int_t njs_fs_bytes_written_create(njs_vm_t *vm, int bytes,
+    njs_value_t *buffer, njs_value_t *retval);
+
 static njs_int_t njs_fs_fd_read(njs_vm_t *vm, int fd, njs_str_t *data);
 
 static njs_int_t njs_fs_error(njs_vm_t *vm, const char *syscall,
@@ -263,7 +296,7 @@ static njs_external_t  njs_ext_fs[] = {
         .writable = 1,
         .configurable = 1,
         .u.method = {
-            .native = njs_fs_write,
+            .native = njs_fs_write_file,
             .magic8 = njs_fs_magic(NJS_FS_CALLBACK, NJS_FS_APPEND),
         }
     },
@@ -274,12 +307,23 @@ static njs_external_t  njs_ext_fs[] = {
         .writable = 1,
         .configurable = 1,
         .u.method = {
-            .native = njs_fs_write,
+            .native = njs_fs_write_file,
             .magic8 = njs_fs_magic(NJS_FS_DIRECT, NJS_FS_APPEND),
         }
     },
 
     {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("closeSync"),
+        .writable = 1,
+        .configurable = 1,
+        .u.method = {
+            .native = njs_fs_close,
+            .magic8 = NJS_FS_DIRECT,
+        }
+    },
+
+    {
         .flags = NJS_EXTERN_PROPERTY,
         .name.string = njs_str("constants"),
         .enumerable = 1,
@@ -301,6 +345,17 @@ static njs_external_t  njs_ext_fs[] = {
 
     {
         .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("fstatSync"),
+        .writable = 1,
+        .configurable = 1,
+        .u.method = {
+            .native = njs_fs_stat,
+            .magic8 = njs_fs_magic(NJS_FS_DIRECT, NJS_FS_FSTAT),
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
         .name.string = njs_str("lstat"),
         .writable = 1,
         .configurable = 1,
@@ -344,6 +399,17 @@ static njs_external_t  njs_ext_fs[] = {
     },
 
     {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("openSync"),
+        .writable = 1,
+        .configurable = 1,
+        .u.method = {
+            .native = njs_fs_open,
+            .magic8 = NJS_FS_DIRECT,
+        }
+    },
+
+    {
         .flags = NJS_EXTERN_PROPERTY,
         .name.string = njs_str("promises"),
         .enumerable = 1,
@@ -380,7 +446,7 @@ static njs_external_t  njs_ext_fs[] = {
         .writable = 1,
         .configurable = 1,
         .u.method = {
-            .native = njs_fs_read,
+            .native = njs_fs_read_file,
             .magic8 = NJS_FS_CALLBACK,
         }
     },
@@ -391,6 +457,17 @@ static njs_external_t  njs_ext_fs[] = {
         .writable = 1,
         .configurable = 1,
         .u.method = {
+            .native = njs_fs_read_file,
+            .magic8 = NJS_FS_DIRECT,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("readSync"),
+        .writable = 1,
+        .configurable = 1,
+        .u.method = {
             .native = njs_fs_read,
             .magic8 = NJS_FS_DIRECT,
         }
@@ -534,7 +611,7 @@ static njs_external_t  njs_ext_fs[] = {
         .writable = 1,
         .configurable = 1,
         .u.method = {
-            .native = njs_fs_write,
+            .native = njs_fs_write_file,
             .magic8 = njs_fs_magic(NJS_FS_CALLBACK, NJS_FS_TRUNC),
         }
     },
@@ -545,8 +622,19 @@ static njs_external_t  njs_ext_fs[] = {
         .writable = 1,
         .configurable = 1,
         .u.method = {
+            .native = njs_fs_write_file,
+            .magic8 = njs_fs_magic(NJS_FS_DIRECT, NJS_FS_TRUNC),
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("writeSync"),
+        .writable = 1,
+        .configurable = 1,
+        .u.method = {
             .native = njs_fs_write,
-            .magic8 = njs_fs_magic(NJS_FS_DIRECT, NJS_FS_TRUNC),
+            .magic8 = NJS_FS_DIRECT,
         }
     },
 
@@ -922,8 +1010,158 @@ static njs_external_t  njs_ext_stats[] =
 };
 
 
+static njs_external_t  njs_ext_filehandle[] = {
+
+    {
+        .flags = NJS_EXTERN_PROPERTY | NJS_EXTERN_SYMBOL,
+        .name.symbol = NJS_SYMBOL_TO_STRING_TAG,
+        .u.property = {
+            .value = "FileHandle",
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("close"),
+        .writable = 1,
+        .configurable = 1,
+        .u.method = {
+            .native = njs_fs_filehandle_close,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("fd"),
+        .enumerable = 1,
+        .u.property = {
+            .handler = njs_external_property,
+            .magic32 = offsetof(njs_filehandle_t, fd),
+            .magic16 = NJS_EXTERN_TYPE_INT,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("read"),
+        .writable = 1,
+        .configurable = 1,
+        .u.method = {
+            .native = njs_fs_read,
+            .magic8 = NJS_FS_PROMISE,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("stat"),
+        .writable = 1,
+        .configurable = 1,
+        .u.method = {
+            .native = njs_fs_stat,
+            .magic8 = njs_fs_magic(NJS_FS_PROMISE, NJS_FS_FSTAT),
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("valueOf"),
+        .writable = 1,
+        .configurable = 1,
+        .u.method = {
+            .native = njs_fs_filehandle_value_of,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("write"),
+        .writable = 1,
+        .configurable = 1,
+        .u.method = {
+            .native = njs_fs_write,
+            .magic8 = NJS_FS_PROMISE,
+        }
+    },
+
+};
+
+
+static njs_external_t  njs_ext_bytes_read[] = {
+
+    {
+        .flags = NJS_EXTERN_PROPERTY | NJS_EXTERN_SYMBOL,
+        .name.symbol = NJS_SYMBOL_TO_STRING_TAG,
+        .u.property = {
+            .value = "BytesRead",
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("buffer"),
+        .enumerable = 1,
+        .u.property = {
+            .handler = njs_external_property,
+            .magic32 = offsetof(njs_bytes_struct_t, buffer),
+            .magic16 = NJS_EXTERN_TYPE_VALUE,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("bytesRead"),
+        .enumerable = 1,
+        .u.property = {
+            .handler = njs_external_property,
+            .magic32 = offsetof(njs_bytes_struct_t, bytes),
+            .magic16 = NJS_EXTERN_TYPE_INT,
+        }
+    },
+
+};
+
+
+static njs_external_t  njs_ext_bytes_written[] = {
+
+    {
+        .flags = NJS_EXTERN_PROPERTY | NJS_EXTERN_SYMBOL,
+        .name.symbol = NJS_SYMBOL_TO_STRING_TAG,
+        .u.property = {
+            .value = "BytesWritten",
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("buffer"),
+        .enumerable = 1,
+        .u.property = {
+            .handler = njs_external_property,
+            .magic32 = offsetof(njs_bytes_struct_t, buffer),
+            .magic16 = NJS_EXTERN_TYPE_VALUE,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("bytesWritten"),
+        .enumerable = 1,
+        .u.property = {
+            .handler = njs_external_property,
+            .magic32 = offsetof(njs_bytes_struct_t, bytes),
+            .magic16 = NJS_EXTERN_TYPE_INT,
+        }
+    },
+
+};
+
+
 static njs_int_t    njs_fs_stats_proto_id;
 static njs_int_t    njs_fs_dirent_proto_id;
+static njs_int_t    njs_fs_filehandle_proto_id;
+static njs_int_t    njs_fs_bytes_read_proto_id;
+static njs_int_t    njs_fs_bytes_written_proto_id;
 
 
 njs_module_t  njs_fs_module = {
@@ -992,6 +1230,101 @@ njs_fs_access(njs_vm_t *vm, njs_value_t 
 
 
 static njs_int_t
+njs_fs_open(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
+    njs_index_t calltype)
+{
+    int          fd, flags;
+    mode_t       md;
+    njs_int_t    ret;
+    const char   *path;
+    njs_value_t  retval, *value;
+    char         path_buf[NJS_MAX_PATH + 1];
+
+    path = njs_fs_path(vm, path_buf, njs_arg(args, nargs, 1), "path");
+    if (njs_slow_path(path == NULL)) {
+        return NJS_ERROR;
+    }
+
+    value = njs_arg(args, nargs, 2);
+    if (njs_is_function(value)) {
+        value = njs_value_arg(&njs_value_undefined);
+    }
+
+    flags = njs_fs_flags(vm, value, O_RDONLY);
+    if (njs_slow_path(flags == -1)) {
+        return NJS_ERROR;
+    }
+
+    value = njs_arg(args, nargs, 3);
+    if (njs_is_function(value)) {
+        value = njs_value_arg(&njs_value_undefined);
+    }
+
+    md = njs_fs_mode(vm, value, 0666);
+    if (njs_slow_path(md == (mode_t) -1)) {
+        return NJS_ERROR;
+    }
+
+    fd = open(path, flags, md);
+    if (njs_slow_path(fd < 0)) {
+        ret = njs_fs_error(vm, "open", strerror(errno), path, errno, &retval);
+        goto done;
+    }
+
+    ret = njs_fs_filehandle_create(vm, fd, calltype == NJS_FS_DIRECT, &retval);
+    if (njs_slow_path(ret != NJS_OK)) {
+        goto done;
+    }
+
+    if (calltype == NJS_FS_DIRECT) {
+        njs_value_number_set(&retval, fd);
+    }
+
+done:
+
+    if (ret == NJS_OK) {
+        return njs_fs_result(vm, &retval, calltype, NULL, 2);
+    }
+
+    if (fd != -1) {
+        (void) close(fd);
+    }
+
+    return NJS_ERROR;
+}
+
+
+static njs_int_t
+njs_fs_close(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
+    njs_index_t calltype)
+{
+    int64_t      fd;
+    njs_int_t    ret;
+    njs_value_t  retval, *fh;
+
+    fh = njs_arg(args, nargs, 1);
+
+    ret = njs_value_to_integer(vm, fh, &fd);
+    if (njs_slow_path(ret != NJS_OK)) {
+        return ret;
+    }
+
+    njs_set_undefined(&retval);
+
+    ret = close((int) fd);
+    if (njs_slow_path(ret != 0)) {
+        ret = njs_fs_error(vm, "close", strerror(errno), NULL, errno, &retval);
+    }
+
+    if (ret == NJS_OK) {
+        return njs_fs_result(vm, &retval, calltype, NULL, 1);
+    }
+
+    return NJS_ERROR;
+}
+
+
+static njs_int_t
 njs_fs_mkdir(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
     njs_index_t calltype)
 {
@@ -1071,6 +1404,117 @@ static njs_int_t
 njs_fs_read(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
     njs_index_t calltype)
 {
+    int64_t             fd, length, pos, offset;
+    ssize_t             n;
+    njs_int_t           ret;
+    njs_str_t           data;
+    njs_uint_t          fd_offset;
+    njs_value_t         retval, *buffer, *value;
+    njs_typed_array_t   *array;
+    njs_array_buffer_t  *array_buffer;
+
+    fd_offset = !!(calltype == NJS_FS_DIRECT);
+
+    ret = njs_value_to_integer(vm, njs_arg(args, nargs, fd_offset), &fd);
+    if (njs_slow_path(ret != NJS_OK)) {
+        return ret;
+    }
+
+    pos = -1;
+
+    /*
+     * fh.read(buffer, offset[, length[, position]])
+     * fs.readSync(fd, buffer, offset[, length[, position]])
+     */
+
+    buffer = njs_arg(args, nargs, fd_offset + 1);
+    array = njs_buffer_slot(vm, buffer, "buffer");
+    if (njs_slow_path(array == NULL)) {
+        return NJS_ERROR;
+    }
+
+    array_buffer = njs_typed_array_writable(vm, array);
+    if (njs_slow_path(array_buffer == NULL)) {
+        return NJS_ERROR;
+    }
+
+    ret = njs_value_to_integer(vm, njs_arg(args, nargs, fd_offset + 2),
+                               &offset);
+    if (njs_slow_path(ret != NJS_OK)) {
+        return ret;
+    }
+
+    if (njs_slow_path(offset < 0 || (size_t) offset > array->byte_length)) {
+        njs_range_error(vm, "offset is out of range (must be <= %z)",
+                        array->byte_length);
+        return NJS_ERROR;
+    }
+
+    data.length = array->byte_length - offset;
+    data.start = &array_buffer->u.u8[array->offset + offset];
+
+    value = njs_arg(args, nargs, fd_offset + 3);
+
+    if (njs_is_defined(value)) {
+        ret = njs_value_to_integer(vm, value, &length);
+        if (njs_slow_path(ret != NJS_OK)) {
+            return ret;
+        }
+
+        if (njs_slow_path(length < 0 || (size_t) length > data.length)) {
+            njs_range_error(vm, "length is out of range (must be <= %z)",
+                            data.length);
+            return NJS_ERROR;
+        }
+
+        data.length = length;
+    }
+
+    value = njs_arg(args, nargs, fd_offset + 4);
+
+    if (!njs_is_null_or_undefined(value)) {
+        ret = njs_value_to_integer(vm, value, &pos);
+        if (njs_slow_path(ret != NJS_OK)) {
+            return ret;
+        }
+    }
+
+    if (pos == -1) {
+        n = read(fd, data.start, data.length);
+
+    } else {
+        n = pread(fd, data.start, data.length, pos);
+    }
+
+    if (njs_slow_path(n == -1)) {
+        ret = njs_fs_error(vm, "read", strerror(errno), NULL, errno, &retval);
+        goto done;
+    }
+
+    if (calltype == NJS_FS_PROMISE) {
+        ret = njs_fs_bytes_read_create(vm, n, buffer, &retval);
+        if (njs_slow_path(ret != NJS_OK)) {
+            goto done;
+        }
+
+    } else {
+        njs_value_number_set(&retval, n);
+    }
+
+done:
+
+    if (ret == NJS_OK) {
+        return njs_fs_result(vm, &retval, calltype, NULL, 1);
+    }
+
+    return NJS_ERROR;
+}
+
+
+static njs_int_t
+njs_fs_read_file(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
+    njs_index_t calltype)
+{
     int                          fd, flags;
     njs_str_t                    data;
     njs_int_t                    ret;
@@ -1552,7 +1996,9 @@ static njs_int_t
 njs_fs_stat(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
     njs_index_t magic)
 {
+    int64_t            fd;
     njs_int_t          ret;
+    njs_uint_t         fd_offset;
     njs_bool_t         throw;
     struct stat        sb;
     const char         *path;
@@ -1563,14 +2009,29 @@ njs_fs_stat(njs_vm_t *vm, njs_value_t *a
     static const njs_value_t  string_bigint = njs_string("bigint");
     static const njs_value_t  string_throw = njs_string("throwIfNoEntry");
 
-    path = njs_fs_path(vm, path_buf, njs_arg(args, nargs, 1), "path");
-    if (njs_slow_path(path == NULL)) {
-        return NJS_ERROR;
+    fd = -1;
+    path = NULL;
+    calltype = magic & 3;
+
+    if ((magic >> 2) != NJS_FS_FSTAT) {
+        path = njs_fs_path(vm, path_buf, njs_arg(args, nargs, 1), "path");
+        if (njs_slow_path(path == NULL)) {
+            return NJS_ERROR;
+        }
+
+        options = njs_arg(args, nargs, 2);
+
+    } else {
+        fd_offset = !!(calltype == NJS_FS_DIRECT);
+        ret = njs_value_to_integer(vm, njs_argument(args, fd_offset), &fd);
+        if (njs_slow_path(ret != NJS_OK)) {
+            return ret;
+        }
+
+        options = njs_arg(args, nargs, fd_offset + 1);
     }
 
     callback = NULL;
-    calltype = magic & 3;
-    options = njs_arg(args, nargs, 2);
 
     if (njs_slow_path(calltype == NJS_FS_CALLBACK)) {
         callback = njs_arg(args, nargs, njs_min(nargs - 1, 3));
@@ -1619,7 +2080,21 @@ njs_fs_stat(njs_vm_t *vm, njs_value_t *a
         }
     }
 
-    ret = ((magic >> 2) == NJS_FS_STAT) ? stat(path, &sb) : lstat(path, &sb);
+    switch (magic >> 2) {
+    case NJS_FS_STAT:
+        ret = stat(path, &sb);
+        break;
+
+    case NJS_FS_LSTAT:
+        ret = lstat(path, &sb);
+        break;
+
+    case NJS_FS_FSTAT:
+    default:
+        ret = fstat(fd, &sb);
+        break;
+    }
+
     if (njs_slow_path(ret != 0)) {
         if (errno != ENOENT || throw) {
             ret = njs_fs_error(vm,
@@ -1740,6 +2215,151 @@ njs_fs_unlink(njs_vm_t *vm, njs_value_t 
 
 static njs_int_t
 njs_fs_write(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
+    njs_index_t calltype)
+{
+    int64_t                      fd, length, pos, offset;
+    ssize_t                      n;
+    njs_int_t                    ret;
+    njs_str_t                    data;
+    njs_uint_t                   fd_offset;
+    njs_value_t                  retval, *buffer, *value;
+    const njs_buffer_encoding_t  *encoding;
+
+    fd_offset = !!(calltype == NJS_FS_DIRECT);
+
+    ret = njs_value_to_integer(vm, njs_arg(args, nargs, fd_offset), &fd);
+    if (njs_slow_path(ret != NJS_OK)) {
+        return ret;
+    }
+
+    buffer = njs_arg(args, nargs, fd_offset + 1);
+
+    pos = -1;
+    encoding = NULL;
+
+    /*
+     * fs.writeSync(fd, string[, position[, encoding]])
+     * fh.write(string[, position[, encoding]])
+     */
+
+    if (njs_is_string(buffer)) {
+        value = njs_arg(args, nargs, fd_offset + 2);
+
+        if (!njs_is_null_or_undefined(value)) {
+            ret = njs_value_to_integer(vm, value, &pos);
+            if (njs_slow_path(ret != NJS_OK)) {
+                return ret;
+            }
+        }
+
+        encoding = njs_buffer_encoding(vm, njs_arg(args, nargs, fd_offset + 3));
+        if (njs_slow_path(encoding == NULL)) {
+            return NJS_ERROR;
+        }
+
+        ret = njs_buffer_decode_string(vm, buffer, &retval, encoding);
+        if (njs_slow_path(ret != NJS_OK)) {
+            return NJS_ERROR;
+        }
+
+        njs_string_get(&retval, &data);
+
+        goto process;
+    }
+
+    /*
+     * fh.write(buffer, offset[, length[, position]])
+     * fs.writeSync(fd, buffer, offset[, length[, position]])
+     */
+
+    ret = njs_vm_value_to_bytes(vm, &data, buffer);
+    if (njs_slow_path(ret != NJS_OK)) {
+        return NJS_ERROR;
+    }
+
+    ret = njs_value_to_integer(vm, njs_arg(args, nargs, fd_offset + 2),
+                               &offset);
+    if (njs_slow_path(ret != NJS_OK)) {
+        return ret;
+    }
+
+    if (njs_slow_path(offset < 0 || (size_t) offset > data.length)) {
+        njs_range_error(vm, "offset is out of range (must be <= %z)",
+                        data.length);
+        return NJS_ERROR;
+    }
+
+    data.length -= offset;
+    data.start += offset;
+
+    value = njs_arg(args, nargs, fd_offset + 3);
+
+    if (njs_is_defined(value)) {
+        ret = njs_value_to_integer(vm, value, &length);
+        if (njs_slow_path(ret != NJS_OK)) {
+            return ret;
+        }
+
+        if (njs_slow_path(length < 0 || (size_t) length > data.length)) {
+            njs_range_error(vm, "length is out of range (must be <= %z)",
+                            data.length);
+            return NJS_ERROR;
+        }
+
+        data.length = length;
+    }
+
+    value = njs_arg(args, nargs, fd_offset + 4);
+
+    if (!njs_is_null_or_undefined(value)) {
+        ret = njs_value_to_integer(vm, value, &pos);
+        if (njs_slow_path(ret != NJS_OK)) {
+            return ret;
+        }
+    }
+
+process:
+
+    if (pos == -1) {
+        n = write(fd, data.start, data.length);
+
+    } else {
+        n = pwrite(fd, data.start, data.length, pos);
+    }
+
+    if (njs_slow_path(n == -1)) {
+        ret = njs_fs_error(vm, "write", strerror(errno), NULL, errno, &retval);
+        goto done;
+    }
+
+    if (njs_slow_path((size_t) n != data.length)) {
+        ret = njs_fs_error(vm, "write", "failed to write all the data", NULL,
+                           0, &retval);
+        goto done;
+    }
+
+    if (calltype == NJS_FS_PROMISE) {
+        ret = njs_fs_bytes_written_create(vm, n, buffer, &retval);
+        if (njs_slow_path(ret != NJS_OK)) {
+            goto done;
+        }
+
+    } else {
+        njs_value_number_set(&retval, n);
+    }
+
+done:
+
+    if (ret == NJS_OK) {
+        return njs_fs_result(vm, &retval, calltype, NULL, 1);
+    }
+
+    return NJS_ERROR;
+}
+
+
+static njs_int_t
+njs_fs_write_file(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
     njs_index_t magic)
 {
     int                          fd, flags;
@@ -2886,11 +3506,146 @@ njs_fs_stats_prop(njs_vm_t *vm, njs_obje
 }
 
 
+static njs_int_t
+njs_fs_filehandle_close(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
+    njs_index_t unused)
+{
+    njs_value_t       retval;
+    njs_filehandle_t  *fh;
+
+    fh = njs_vm_external(vm, njs_fs_filehandle_proto_id, njs_argument(args, 0));
+    if (njs_slow_path(fh == NULL)) {
+        njs_type_error(vm, "\"this\" is not a filehandle object");
+        return NJS_ERROR;
+    }
+
+    if (njs_slow_path(fh->fd == -1)) {
+        njs_type_error(vm, "file was already closed");
+        return NJS_ERROR;
+    }
+
+    (void) close(fh->fd);
+    fh->fd = -1;
+
+    njs_set_undefined(&retval);
+
+    return njs_fs_result(vm, &retval, NJS_FS_PROMISE, NULL, 1);
+}
+
+
+static njs_int_t
+njs_fs_filehandle_value_of(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
+    njs_index_t unused)
+{
+    njs_filehandle_t  *fh;
+
+    fh = njs_vm_external(vm, njs_fs_filehandle_proto_id, njs_argument(args, 0));
+    if (njs_slow_path(fh == NULL)) {
+        njs_type_error(vm, "\"this\" is not a filehandle object");
+        return NJS_ERROR;
+    }
+
+    njs_set_number(njs_vm_retval(vm), fh->fd);
+
+    return NJS_OK;
+}
+
+
+static void
+njs_fs_filehandle_cleanup(void *data)
+{
+    njs_filehandle_t  *fh = data;
+
+    if (fh->vm != NULL && fh->fd != -1) {
+        njs_vm_warn(fh->vm, "closing file description %d on cleanup\n", fh->fd);
+        (void) close(fh->fd);
+    }
+}
+
+
+static njs_int_t
+njs_fs_filehandle_create(njs_vm_t *vm, int fd, njs_bool_t shadow,
+    njs_value_t *retval)
+{
+    njs_filehandle_t  *fh;
+    njs_mp_cleanup_t  *cln;
+
+    fh = njs_mp_alloc(vm->mem_pool, sizeof(njs_filehandle_t));
+    if (njs_slow_path(fh == NULL)) {
+        njs_memory_error(vm);
+        return NJS_ERROR;
+    }
+
+    fh->fd = fd;
+    fh->vm = !shadow ? vm : NULL;
+
+    cln = njs_mp_cleanup_add(njs_vm_memory_pool(vm), 0);
+    if (cln == NULL) {
+        njs_memory_error(vm);
+        return NJS_ERROR;
+    }
+
+    cln->handler = njs_fs_filehandle_cleanup;
+    cln->data = fh;
+
+    return njs_vm_external_create(vm, retval, njs_fs_filehandle_proto_id,
+                                  fh, 0);
+}
+
+
+static njs_int_t
+njs_fs_bytes_read_create(njs_vm_t *vm, int bytes, njs_value_t *buffer,
+    njs_value_t *retval)
+{
+    njs_bytes_struct_t  *bs;
+
+    bs = njs_mp_alloc(vm->mem_pool, sizeof(njs_bytes_struct_t));
+    if (njs_slow_path(bs == NULL)) {
+        njs_memory_error(vm);
+        return NJS_ERROR;
+    }
+
+    bs->bytes = bytes;
+    njs_value_assign(&bs->buffer, buffer);
+
+    return njs_vm_external_create(vm, retval, njs_fs_bytes_read_proto_id,
+                                  bs, 0);
+}
+
+
+static njs_int_t
+njs_fs_bytes_written_create(njs_vm_t *vm, int bytes, njs_value_t *buffer,
+    njs_value_t *retval)
+{
+    njs_bytes_struct_t  *bs;
+
+    bs = njs_mp_alloc(vm->mem_pool, sizeof(njs_bytes_struct_t));
+    if (njs_slow_path(bs == NULL)) {
+        njs_memory_error(vm);
+        return NJS_ERROR;
+    }
+
+    bs->bytes = bytes;
+    njs_value_assign(&bs->buffer, buffer);
+
+    return njs_vm_external_create(vm, retval, njs_fs_bytes_written_proto_id,
+                                  bs, 0);
+}
+
+
 static const njs_object_prop_t  njs_fs_promises_properties[] =
 {
     {
         .type = NJS_PROPERTY,
         .name = njs_string("readFile"),
+        .value = njs_native_function2(njs_fs_read_file, 0, NJS_FS_PROMISE),
+        .writable = 1,
+        .configurable = 1,
+    },
+
+    {
+        .type = NJS_PROPERTY,
+        .name = njs_string("readSync"),
         .value = njs_native_function2(njs_fs_read, 0, NJS_FS_PROMISE),
         .writable = 1,
         .configurable = 1,
@@ -2899,7 +3654,7 @@ static const njs_object_prop_t  njs_fs_p
     {
         .type = NJS_PROPERTY,
         .name = njs_string("appendFile"),
-        .value = njs_native_function2(njs_fs_write, 0,
+        .value = njs_native_function2(njs_fs_write_file, 0,
                                   njs_fs_magic(NJS_FS_PROMISE, NJS_FS_APPEND)),
         .writable = 1,
         .configurable = 1,
@@ -2908,7 +3663,7 @@ static const njs_object_prop_t  njs_fs_p
     {
         .type = NJS_PROPERTY,
         .name = njs_string("writeFile"),
-        .value = njs_native_function2(njs_fs_write, 0,
+        .value = njs_native_function2(njs_fs_write_file, 0,
                                    njs_fs_magic(NJS_FS_PROMISE, NJS_FS_TRUNC)),
         .writable = 1,
         .configurable = 1,
@@ -2932,6 +3687,22 @@ static const njs_object_prop_t  njs_fs_p
 
     {
         .type = NJS_PROPERTY,
+        .name = njs_string("open"),
+        .value = njs_native_function2(njs_fs_open, 0, NJS_FS_PROMISE),
+        .writable = 1,
+        .configurable = 1,
+    },
+
+    {
+        .type = NJS_PROPERTY,
+        .name = njs_string("close"),
+        .value = njs_native_function2(njs_fs_close, 0, NJS_FS_PROMISE),
+        .writable = 1,
+        .configurable = 1,
+    },
+
+    {
+        .type = NJS_PROPERTY,
         .name = njs_string("rename"),
         .value = njs_native_function2(njs_fs_rename, 0, NJS_FS_PROMISE),
         .writable = 1,
@@ -2956,6 +3727,15 @@ static const njs_object_prop_t  njs_fs_p
 
     {
         .type = NJS_PROPERTY,
+        .name = njs_string("fstat"),
+        .value = njs_native_function2(njs_fs_stat, 0,
+                                   njs_fs_magic(NJS_FS_PROMISE, NJS_FS_FSTAT)),
+        .writable = 1,
+        .configurable = 1,
+    },
+


From xeioex at nginx.com  Tue Aug 23 05:06:12 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 23 Aug 2022 05:06:12 +0000
Subject: [njs] Types: added ts types for the recent fs API.
Message-ID: <hg.2f0056631e75.1661231172.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/2f0056631e75
branches:  
changeset: 1932:2f0056631e75
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Aug 22 22:03:15 2022 -0700
description:
Types: added ts types for the recent fs API.

diffstat:

 test/ts/test.ts        |   23 ++++++-
 ts/njs_core.d.ts       |    1 +
 ts/njs_modules/fs.d.ts |  174 +++++++++++++++++++++++++++++++++++++++++++++---
 3 files changed, 185 insertions(+), 13 deletions(-)

diffs (348 lines):

diff -r e9ce6daa07fa -r 2f0056631e75 test/ts/test.ts
--- a/test/ts/test.ts	Mon Aug 22 16:49:27 2022 -0700
+++ b/test/ts/test.ts	Mon Aug 22 22:03:15 2022 -0700
@@ -103,13 +103,34 @@ async function http_module(r: NginxHTTPR
     r.done();
 }
 
-function fs_module() {
+async function fs_module() {
     var s:string;
 
     s = fs.readFileSync('/path', 'utf8');
     s = fs.readFileSync(Buffer.from('/path'), {encoding:'hex'});
 
     fs.writeFileSync('/path', Buffer.from('abc'));
+
+    let fh = await fs.promises.open('/path', 'r+');
+
+    let bw = await fh.write(Buffer.from('abc'), 0, 1, 3);
+    let bytes = bw.bytesWritten;
+    bw = await fh.write(Buffer.from('abc'), 2, 2, null);
+    let stat = fh.stat();
+
+    let buffer = Buffer.alloc(16);
+    let br = await fh.read(buffer, 0, 16, null);
+    bytes = br.bytesRead;
+
+    await fh.close();
+
+    let fd = fs.openSync('/path', 'r+');
+    let stat2 = fs.fstatSync(fd);
+
+    fs.readSync(fd, buffer, 0, 16, 4);
+    buffer[1] += 2;
+    fs.writeSync(fd, buffer, 0, 16, 4);
+    fs.closeSync(fd);
 }
 
 function qs_module(str: NjsByteString) {
diff -r e9ce6daa07fa -r 2f0056631e75 ts/njs_core.d.ts
--- a/ts/njs_core.d.ts	Mon Aug 22 16:49:27 2022 -0700
+++ b/ts/njs_core.d.ts	Mon Aug 22 22:03:15 2022 -0700
@@ -595,6 +595,7 @@ declare class Buffer extends Uint8Array 
 }
 
 type NjsStringOrBuffer = NjsStringLike | Buffer | DataView | TypedArray | ArrayBuffer;
+type NjsBuffer = Buffer | DataView | TypedArray;
 
 // Global objects
 
diff -r e9ce6daa07fa -r 2f0056631e75 ts/njs_modules/fs.d.ts
--- a/ts/njs_modules/fs.d.ts	Mon Aug 22 16:49:27 2022 -0700
+++ b/ts/njs_modules/fs.d.ts	Mon Aug 22 22:03:15 2022 -0700
@@ -34,7 +34,7 @@ declare module "fs" {
      * When `readdirSync()` is called with the `withFileTypes` option, the resulting array contains
      * `fs.Dirent` objects.
      */
-    export interface Dirent {
+    export interface NjsDirent {
         /**
          * @returns `true` if the object describes a block device.
          */
@@ -75,7 +75,7 @@ declare module "fs" {
      *
      * The objects is returned from fs.stat(), fs.lstat() and friends.
      */
-    export interface Stats {
+    export interface NjsStats {
         /**
          * @returns `true` if the object describes a block device.
          */
@@ -205,7 +205,7 @@ declare module "fs" {
         flag?: OpenMode;
     };
 
-    type Constants = {
+    type NjsFsConstants = {
         /**
          * Indicates that the file is visible to the calling process, used by default if no mode
          * is specified.
@@ -225,7 +225,7 @@ declare module "fs" {
         X_OK: 1;
     };
 
-    interface Promises {
+    interface NjsFsPromises {
         /**
          * Asynchronously tests permissions for a file or directory specified in the `path`.
          * If the check fails, an error will be returned, otherwise, the method will return undefined.
@@ -244,6 +244,21 @@ declare module "fs" {
         access(path: PathLike, mode?: number): Promise<void>;
 
         /**
+         * Asynchronously opens a file specified in the `path`.
+         *
+         * @example
+         *   import fs from 'fs'
+         *   let fh = await fs.promises.open('/file/path', 'w');
+         *   let bw = await fh.write("data to write", 10);
+         *
+         * @since 0.7.7
+         * @param path A path to a file.
+         * @param flags File system flags, defaults to `r`.
+         * @param mode The file mode, defaults to 0o666.
+         */
+        open(path: PathLike, flags?: OpenMode, mode?: number): Promise<NjsFsFileHandle>;
+
+        /**
          * Asynchronously appends specified `data` to a file with provided `filename`.
          * If the file does not exist, it will be created.
          *
@@ -266,7 +281,7 @@ declare module "fs" {
          *   - `throwIfNoEntry` - Whether an exception will be thrown if no file system entry exists,
          *      rather than returning undefined, defaults to `true`.
          */
-        lstat(path: PathLike, options?: { throwIfNoEntry?: boolean; }): Promise<Stats>;
+        lstat(path: PathLike, options?: { throwIfNoEntry?: boolean; }): Promise<NjsStats>;
 
         /**
          * Asynchronously creates a directory at the specified `path`.
@@ -289,7 +304,7 @@ declare module "fs" {
          */
         readdir(path: PathLike, options?: { encoding?: "utf8"; withFileTypes?: false; } | "utf8"): Promise<string[]>;
         readdir(path: PathLike, options: { encoding: "buffer"; withFileTypes?: false; } | "buffer"): Promise<Buffer[]>;
-        readdir(path: PathLike, options: { encoding?: "utf8" | "buffer"; withFileTypes: true; }): Promise<Dirent[]>;
+        readdir(path: PathLike, options: { encoding?: "utf8" | "buffer"; withFileTypes: true; }): Promise<NjsDirent[]>;
 
         /**
          * Asynchronously returns the contents of the file with provided `filename`.
@@ -341,7 +356,7 @@ declare module "fs" {
          *   - `throwIfNoEntry` - Whether an exception will be thrown if no file system entry exists,
          *      rather than returning undefined, defaults to `true`.
          */
-        stat(path: PathLike, options?: { throwIfNoEntry?: boolean; }): Promise<Stats>;
+        stat(path: PathLike, options?: { throwIfNoEntry?: boolean; }): Promise<NjsStats>;
 
         /**
          * Asynchronously creates the link called `path` pointing to `target` using `symlink(2)`.
@@ -375,17 +390,95 @@ declare module "fs" {
         writeFile(path: PathLike, data: NjsStringOrBuffer, options?: WriteFileOptions): Promise<void>;
     }
 
+    interface NjsFsBytesRead {
+        /**
+         * The number of bytes read.
+         */
+        bytesRead: number;
+
+        /**
+         * A reference to the passed in buffer argument.
+         */
+        buffer: NjsBuffer;
+    }
+
+    interface NjsFsBytesWritten {
+        /**
+         * The number of bytes written.
+         */
+        bytesWritten: number;
+
+        /**
+         * A reference to the buffer written.
+         */
+        buffer: NjsBuffer;
+    }
+
+    interface NjsFsFileHandle {
+        /**
+         * Asynchronously closes the file handle after waiting for any pending operation
+         * on the handle to complete.
+         */
+        close(): Promise<void>;
+
+        /**
+         * The file descriptor number.
+         */
+        fd: number;
+
+        /**
+         * Asynchronously reads data from the file and stores that in the given buffer.
+         *
+         * @param buffer A buffer that will be filled with the file data read.
+         * @param offset The location in the buffer at which to start filling.
+         * @param length The number of bytes to read.
+         * @param position The location where to begin reading data from the file.
+         *    If null, data will be read from the current file position, and the position will be updated.
+         *    If position is an integer, the current file position will remain unchanged.
+         */
+        read(buffer: NjsBuffer, offset: number, length: number, position: number | null): Promise<NjsFsBytesRead>;
+
+        /**
+         * Asynchronously retrieves `fs.Stats` for the underlying descriptor.
+         */
+        stat(): Promise<NjsStats>;
+
+        /**
+         * Asynchronously writes buffer to the file.
+         *
+         * @param buffer A buffer to write.
+         * @param offset The start position from within buffer where the data to write begins.
+         * @param The number of bytes from buffer to write.
+         *    Defaults to buffer.byteLength - offset
+         * @param position The offset from the beginning of the file where the data from buffer
+         *    should be written. If position is not a number, the data will be written at the current position.
+         * @param encoding  One of the `'utf8'`, `'hex'`, `'base64'`, or `'base64url'`.
+         *    Defaults to 'utf8'.
+         */
+        write(buffer: NjsBuffer, offset: number, length?: number, position?: number | null): Promise<NjsFsBytesWritten>;
+        write(buffer: NjsStringLike, position?: number | null, encoding?: FileEncoding): Promise<NjsFsBytesWritten>;
+    }
+
     interface NjsFS {
         /**
          * Promissified versions of file system methods.
          *
          * @since 0.3.9
          */
-        promises: Promises
+        promises: NjsFsPromises
+
+        /**
+         * Synchronously closes specified file descriptor.
+         *
+         * @since 0.7.7
+         * @param fd A file descriptor.
+         */
+        closeSync(fd: number): undefined;
+
         /**
          * File Access Constants
          */
-        constants: Constants
+        constants: NjsFsConstants
 
         /**
          * Synchronously tests permissions for a file or directory specified in the `path`.
@@ -420,6 +513,14 @@ declare module "fs" {
         appendFileSync(path: PathLike, data: NjsStringOrBuffer, options?: WriteFileOptions): void;
 
         /**
+         * Synchronously retrieves `fs.Stats` object for specified file descriptor.
+         *
+         * @since 0.7.7
+         * @param fd A file descriptor.
+         */
+        fstatSync(fd: number): NjsStats;
+
+        /**
          * Synchronously retrieves `fs.Stats` object for the symbolic link referred to by path.
          * See `lstat(2)` for more details.
          *
@@ -429,7 +530,7 @@ declare module "fs" {
          *   - `throwIfNoEntry` - Whether an exception will be thrown if no file system entry exists,
          *      rather than returning undefined, defaults to `true`.
          */
-        lstatSync(path: PathLike, options?: { throwIfNoEntry?: boolean; }): Stats;
+        lstatSync(path: PathLike, options?: { throwIfNoEntry?: boolean; }): NjsStats;
 
         /**
          * Synchronously creates a directory at the specified `path`.
@@ -441,6 +542,21 @@ declare module "fs" {
         mkdirSync(path: PathLike, options?: { mode?: number } | number): void;
 
         /**
+         * Synchronously opens a file specified in the `path`.
+         *
+         * @example
+         *   import fs from 'fs'
+         *   let fd = fs.openSync('/file/path', 'w');
+         *   let bytesWritten = fs.writeSync("data to write", 10);
+         *
+         * @since 0.7.7
+         * @param path A path to a file.
+         * @param flags file system flags, defaults to `r`.
+         * @param mode Thre file mode, defaults to 0o666.
+         */
+        openSync(path: PathLike, flags?: OpenMode, mode?: number): number;
+
+        /**
          * Synchronously reads the contents of a directory at the specified `path`.
          *
          * @since 0.4.2
@@ -453,7 +569,7 @@ declare module "fs" {
          */
         readdirSync(path: PathLike, options?: { encoding?: "utf8"; withFileTypes?: false; } | "utf8"): string[];
         readdirSync(path: PathLike, options: { encoding: "buffer"; withFileTypes?: false; } | "buffer"): Buffer[];
-        readdirSync(path: PathLike, options: { encoding?: "utf8" | "buffer"; withFileTypes: true; }): Dirent[];
+        readdirSync(path: PathLike, options: { encoding?: "utf8" | "buffer"; withFileTypes: true; }): NjsDirent[];
 
         /**
          * Synchronously returns the contents of the file with provided `filename`.
@@ -474,6 +590,23 @@ declare module "fs" {
         readFileSync(path: PathLike, options: { encoding?: FileEncoding; flag?: OpenMode; } | FileEncoding): string;
 
         /**
+         * Synchronously reads data from the file and stores that in the given buffer.
+         *
+         * @since 0.7.7
+         * @param fd A file descriptor.
+         * @param buffer A buffer that will be filled with the file data read.
+         * @param offset The location in the buffer at which to start filling.
+         * @param length The number of bytes to read.
+         * @param position The location where to begin reading data from the file.
+         *    If null, data will be read from the current file position, and the position will be updated.
+         *    If position is an integer, the current file position will remain unchanged.
+         * @param encoding  One of the `'utf8'`, `'hex'`, `'base64'`, or `'base64url'`.
+         *    Defaults to 'utf8'.
+         */
+        readSync(fd: number, buffer: NjsBuffer, offset: number, length?: number, position?: number | null): number;
+        readSync(fd: number, string: NjsStringLike, position?: number | null, encoding?: FileEncoding): number;
+
+        /**
          * Synchronously computes the canonical pathname by resolving `.`, `..` and symbolic links using
          * `realpath(3)`.
          *
@@ -514,7 +647,7 @@ declare module "fs" {
          *   - `throwIfNoEntry` - Whether an exception will be thrown if no file system entry exists,
          *      rather than returning undefined, defaults to `true`.
          */
-        statSync(path: PathLike, options?: { throwIfNoEntry?: boolean; }): Stats;
+        statSync(path: PathLike, options?: { throwIfNoEntry?: boolean; }): NjsStats;
 
         /**
          * Synchronously creates the link called `path` pointing to `target` using `symlink(2)`.
@@ -550,6 +683,23 @@ declare module "fs" {
          *   If `flag` is not supplied, the default of `'w'` is used.
          */
         writeFileSync(path: PathLike, data: NjsStringOrBuffer, options?: WriteFileOptions): void;
+
+        /**
+         * Synchronously writes `buffer` data to a file.
+         *
+         * @since 0.7.7
+         * @param fd A file descriptor.
+         * @param buffer A buffer that will be filled with the file data read.
+         * @param offset The location in the buffer at which to start filling.
+         * @param length The number of bytes to read.
+         * @param position The location where to begin reading data from the file.
+         *    If null, data will be read from the current file position, and the position will be updated.
+         *    If position is an integer, the current file position will remain unchanged.
+         * @param encoding  One of the `'utf8'`, `'hex'`, `'base64'`, or `'base64url'`.
+         *    Defaults to 'utf8'.
+         */
+        writeSync(fd: number, buffer: NjsBuffer, offset: number, length?: number, position?: number | null): number;
+        writeSync(fd: number, string: NjsStringLike, position?: number | null, encoding?: FileEncoding): number;
     }
 
     const fs: NjsFS;


From arut at nginx.com  Tue Aug 23 12:21:19 2022
From: arut at nginx.com (Roman Arutyunyan)
Date: Tue, 23 Aug 2022 16:21:19 +0400
Subject: [PATCH 1 of 8] QUIC: treat qc->error == -1 as a missing error
In-Reply-To: <20220802134856.y5uju2ko33y45opf@Y9MQ9X2QVV>
References: <patchbomb.1655999918@arut-laptop>
 <951d7116f37dc39d9eba.1655999919@arut-laptop>
 <20220802134856.y5uju2ko33y45opf@Y9MQ9X2QVV>
Message-ID: <20220823122119.bbrw74mbds2urtpg@N00W24XTQX>

On Tue, Aug 02, 2022 at 05:48:56PM +0400, Sergey Kandaurov wrote:
> On Thu, Jun 23, 2022 at 07:58:39PM +0400, Roman Arutyunyan wrote:
> > # HG changeset patch
> > # User Roman Arutyunyan <arut at nginx.com>
> > # Date 1655889315 -14400
> > #      Wed Jun 22 13:15:15 2022 +0400
> > # Branch quic
> > # Node ID 951d7116f37dc39d9eba20ceae49434592ce4677
> > # Parent  5b1011b5702b5c5db2ba3d392a4da25596183cc2
> > QUIC: treat qc->error == -1 as a missing error.
> > 
> > Previously, zero was used for this purpose.  However, NGX_QUIC_ERR_NO_ERROR is
> > zero too.  As a result, NGX_QUIC_ERR_NO_ERROR was changed to
> > NGX_QUIC_ERR_INTERNAL_ERROR when closing a QUIC connection.
> > 
> > diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c
> > --- a/src/event/quic/ngx_event_quic.c
> > +++ b/src/event/quic/ngx_event_quic.c
> > @@ -317,6 +317,8 @@ ngx_quic_new_connection(ngx_connection_t
> >      qc->congestion.ssthresh = (size_t) -1;
> >      qc->congestion.recovery_start = ngx_current_msec;
> >  
> > +    qc->error = (ngx_uint_t) -1;
> > +
> >      if (pkt->validated && pkt->retried) {
> >          qc->tp.retry_scid.len = pkt->dcid.len;
> >          qc->tp.retry_scid.data = ngx_pstrdup(c->pool, &pkt->dcid);
> > @@ -523,7 +525,7 @@ ngx_quic_close_connection(ngx_connection
> >                  qc->error = NGX_QUIC_ERR_NO_ERROR;
> >  
> >              } else {
> > -                if (qc->error == 0 && !qc->error_app) {
> > +                if (qc->error == (ngx_uint_t) -1 && !qc->error_app) {
> >                      qc->error = NGX_QUIC_ERR_INTERNAL_ERROR;
> >                  }
> >  
> 
> A former initialization is in ngx_quic_handle_payload().
> Leaving it there makes connection close on the error path, for example,
> on ngx_quic_set_path() failure, with a wrong NO_ERROR code.
> 
> I'd just leave (a new value of) initialization in place:
> ngx_quic_new_connection() gives an established quic connection or nothing,
> it cannot fail with qc initialized. ngx_quic_handle_payload() is called
> right after, and is a point of packet decryption, which implies qc->error
> reset.  This makes it a suitable place.

You're right, let's leave the initializaion where it is now, but with a new
value.


From arut at nginx.com  Tue Aug 23 12:28:51 2022
From: arut at nginx.com (Roman Arutyunyan)
Date: Tue, 23 Aug 2022 16:28:51 +0400
Subject: [PATCH 4 of 8] QUIC: removed ngx_quic_shutdown_connection()
In-Reply-To: <20220802154411.l5hfvk5wnc2pmklb@Y9MQ9X2QVV>
References: <patchbomb.1655999918@arut-laptop>
 <1912c09e0e4d746ec0a4.1655999922@arut-laptop>
 <20220802154411.l5hfvk5wnc2pmklb@Y9MQ9X2QVV>
Message-ID: <20220823122851.u4yud3rfyrvstd5q@N00W24XTQX>

On Tue, Aug 02, 2022 at 07:44:11PM +0400, Sergey Kandaurov wrote:
> On Thu, Jun 23, 2022 at 07:58:42PM +0400, Roman Arutyunyan wrote:
> > # HG changeset patch
> > # User Roman Arutyunyan <arut at nginx.com>
> > # Date 1655904279 -14400
> > #      Wed Jun 22 17:24:39 2022 +0400
> > # Branch quic
> > # Node ID 1912c09e0e4d746ec0a4c2140b6d2046b283b647
> > # Parent  ea555f7caec372e93e1b9a11ed822002e97ed58b
> > QUIC: removed ngx_quic_shutdown_connection().
> > 
> > HTTP/3 shutdown is now controlled by HTTP/3 layer.  QUIC connection is
> > finalized after reaching requests limit when all request streams have finished.
> 
> Finalizing QUIC connection from HTTP/3 connection cleanup handler
> prevents the last stream cleanup handler from regular run.  Normally
> it is called next, but after QUIC shutdown initiated, that's too late,
> and stream FIN won't be sent.

Looks like moving shutdown out of QUIC to HTTP/3 is not so easy.  Let's give
up this idea.  I've reworked the patchset a little bit.


From arut at nginx.com  Tue Aug 23 12:51:09 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 23 Aug 2022 16:51:09 +0400
Subject: [PATCH 0 of 9] QUIC connection reuse
In-Reply-To: <patchbomb.1655999918@arut-laptop>
References: <patchbomb.1655999918@arut-laptop>
Message-ID: <patchbomb.1661259069@arut-laptop>

A new reworked series.


From arut at nginx.com  Tue Aug 23 12:51:10 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 23 Aug 2022 16:51:10 +0400
Subject: [PATCH 1 of 9] QUIC: treat qc->error == -1 as a missing error
In-Reply-To: <patchbomb.1661259069@arut-laptop>
References: <patchbomb.1661259069@arut-laptop>
Message-ID: <d928c8ca3415be4e9cce.1661259070@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1660223647 -14400
#      Thu Aug 11 17:14:07 2022 +0400
# Branch quic
# Node ID d928c8ca3415be4e9cced59a3cb9fc70249a0d9d
# Parent  f9d7930d0eedae28defd0803cb95dc8ab68e56b3
QUIC: treat qc->error == -1 as a missing error.

Previously, zero was used for this purpose.  However, NGX_QUIC_ERR_NO_ERROR is
zero too.  As a result, NGX_QUIC_ERR_NO_ERROR was changed to
NGX_QUIC_ERR_INTERNAL_ERROR when closing a QUIC connection.

diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c
--- a/src/event/quic/ngx_event_quic.c
+++ b/src/event/quic/ngx_event_quic.c
@@ -523,7 +523,7 @@ ngx_quic_close_connection(ngx_connection
                 qc->error = NGX_QUIC_ERR_NO_ERROR;
 
             } else {
-                if (qc->error == 0 && !qc->error_app) {
+                if (qc->error == (ngx_uint_t) -1 && !qc->error_app) {
                     qc->error = NGX_QUIC_ERR_INTERNAL_ERROR;
                 }
 
@@ -939,7 +939,7 @@ ngx_quic_handle_payload(ngx_connection_t
 
     qc = ngx_quic_get_connection(c);
 
-    qc->error = 0;
+    qc->error = (ngx_uint_t) -1;
     qc->error_reason = 0;
 
     c->log->action = "decrypting packet";


From arut at nginx.com  Tue Aug 23 12:51:11 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 23 Aug 2022 16:51:11 +0400
Subject: [PATCH 2 of 9] QUIC: made ngx_quic_finalize_connecion() more graceful
In-Reply-To: <patchbomb.1661259069@arut-laptop>
References: <patchbomb.1661259069@arut-laptop>
Message-ID: <923c0352912d80b22433.1661259071@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1661167731 -14400
#      Mon Aug 22 15:28:51 2022 +0400
# Branch quic
# Node ID 923c0352912d80b224332c508379e047ec08dc4e
# Parent  d928c8ca3415be4e9cced59a3cb9fc70249a0d9d
QUIC: made ngx_quic_finalize_connecion() more graceful.

Previously, ngx_quic_finalize_connection() closed the connection with NGX_ERROR
code, which resulted in immediate connection closure.  Now the code is NGX_OK,
which provides a more graceful shutdown with a timeout.

diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c
--- a/src/event/quic/ngx_event_quic.c
+++ b/src/event/quic/ngx_event_quic.c
@@ -414,6 +414,7 @@ ngx_quic_input_handler(ngx_event_t *rev)
     }
 
     if (c->close) {
+        qc->error = NGX_QUIC_ERR_NO_ERROR;
         qc->error_reason = "graceful shutdown";
         ngx_quic_close_connection(c, NGX_OK);
         return;
@@ -506,31 +507,26 @@ ngx_quic_close_connection(ngx_connection
             qc->error_level = c->ssl ? SSL_quic_read_level(c->ssl->connection)
                                      : ssl_encryption_initial;
 
+            if (qc->error == (ngx_uint_t) -1) {
+                qc->error = NGX_QUIC_ERR_INTERNAL_ERROR;
+                qc->error_app = 0;
+            }
+
+            ngx_log_debug5(NGX_LOG_DEBUG_EVENT, c->log, 0,
+                           "quic close immediate term:%d drain:%d "
+                           "%serror:%ui \"%s\"",
+                           rc == NGX_ERROR ? 1 : 0, qc->draining,
+                           qc->error_app ? "app " : "", qc->error,
+                           qc->error_reason ? qc->error_reason : "");
+
             if (rc == NGX_OK) {
-                ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0,
-                               "quic close immediate drain:%d",
-                               qc->draining);
-
                 qc->close.log = c->log;
                 qc->close.data = c;
                 qc->close.handler = ngx_quic_close_timer_handler;
                 qc->close.cancelable = 1;
 
                 ctx = ngx_quic_get_send_ctx(qc, qc->error_level);
-
                 ngx_add_timer(&qc->close, 3 * ngx_quic_pto(c, ctx));
-
-                qc->error = NGX_QUIC_ERR_NO_ERROR;
-
-            } else {
-                if (qc->error == (ngx_uint_t) -1 && !qc->error_app) {
-                    qc->error = NGX_QUIC_ERR_INTERNAL_ERROR;
-                }
-
-                ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0,
-                               "quic close immediate due to %serror: %ui %s",
-                               qc->error_app ? "app " : "", qc->error,
-                               qc->error_reason ? qc->error_reason : "");
             }
 
             (void) ngx_quic_send_cc(c);
@@ -617,7 +613,7 @@ ngx_quic_finalize_connection(ngx_connect
     qc->error_app = 1;
     qc->error_ftype = 0;
 
-    ngx_quic_close_connection(c, NGX_ERROR);
+    ngx_quic_close_connection(c, NGX_OK);
 }
 
 


From arut at nginx.com  Tue Aug 23 12:51:12 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 23 Aug 2022 16:51:12 +0400
Subject: [PATCH 3 of 9] QUIC: reusable mode for main connection
In-Reply-To: <patchbomb.1661259069@arut-laptop>
References: <patchbomb.1661259069@arut-laptop>
Message-ID: <6d444d4add98478c92a2.1661259072@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1661169774 -14400
#      Mon Aug 22 16:02:54 2022 +0400
# Branch quic
# Node ID 6d444d4add98478c92a2ca84dde86005394ae0ea
# Parent  923c0352912d80b224332c508379e047ec08dc4e
QUIC: reusable mode for main connection.

The mode is controlled by application layer.  For HTTP/3, it is enabled
when there are no active request streams.

diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c
--- a/src/event/quic/ngx_event_quic.c
+++ b/src/event/quic/ngx_event_quic.c
@@ -416,7 +416,7 @@ ngx_quic_input_handler(ngx_event_t *rev)
     if (c->close) {
         qc->error = NGX_QUIC_ERR_NO_ERROR;
         qc->error_reason = "graceful shutdown";
-        ngx_quic_close_connection(c, NGX_OK);
+        ngx_quic_close_connection(c, NGX_ERROR);
         return;
     }
 
diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c
--- a/src/event/quic/ngx_event_quic_streams.c
+++ b/src/event/quic/ngx_event_quic_streams.c
@@ -161,13 +161,10 @@ ngx_quic_close_streams(ngx_connection_t 
     ngx_pool_t         *pool;
     ngx_queue_t        *q;
     ngx_rbtree_t       *tree;
+    ngx_connection_t   *sc;
     ngx_rbtree_node_t  *node;
     ngx_quic_stream_t  *qs;
 
-#if (NGX_DEBUG)
-    ngx_uint_t          ns;
-#endif
-
     while (!ngx_queue_empty(&qc->streams.uninitialized)) {
         q = ngx_queue_head(&qc->streams.uninitialized);
         ngx_queue_remove(q);
@@ -185,34 +182,31 @@ ngx_quic_close_streams(ngx_connection_t 
         return NGX_OK;
     }
 
-#if (NGX_DEBUG)
-    ns = 0;
-#endif
-
     node = ngx_rbtree_min(tree->root, tree->sentinel);
 
     while (node) {
         qs = (ngx_quic_stream_t *) node;
         node = ngx_rbtree_next(tree, node);
+        sc = qs->connection;
 
-        qs->recv_state = NGX_QUIC_STREAM_RECV_RESET_RECVD;
-        qs->send_state = NGX_QUIC_STREAM_SEND_RESET_SENT;
-
-        if (qs->connection == NULL) {
+        if (sc == NULL) {
             ngx_quic_close_stream(qs);
             continue;
         }
 
-        ngx_quic_set_event(qs->connection->read);
-        ngx_quic_set_event(qs->connection->write);
+        ngx_quic_set_event(sc->read);
+        ngx_quic_set_event(sc->write);
 
-#if (NGX_DEBUG)
-        ns++;
-#endif
+        sc->close = 1;
+        sc->read->handler(sc->read);
     }
 
-    ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0,
-                   "quic connection has %ui active streams", ns);
+    if (tree->root == tree->sentinel) {
+        return NGX_OK;
+    }
+
+    ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0,
+                   "quic connection has active streams");
 
     return NGX_AGAIN;
 }
@@ -587,6 +581,7 @@ ngx_quic_create_stream(ngx_connection_t 
 {
     ngx_log_t              *log;
     ngx_pool_t             *pool;
+    ngx_uint_t              reusable;
     ngx_queue_t            *q;
     ngx_connection_t       *sc;
     ngx_quic_stream_t      *qs;
@@ -639,7 +634,13 @@ ngx_quic_create_stream(ngx_connection_t 
     *log = *c->log;
     pool->log = log;
 
+    reusable = c->reusable;
+    ngx_reusable_connection(c, 0);
+
     sc = ngx_get_connection(c->fd, log);
+
+    ngx_reusable_connection(c, reusable);
+
     if (sc == NULL) {
         ngx_destroy_pool(pool);
         ngx_queue_insert_tail(&qc->streams.free, &qs->queue);
@@ -733,15 +734,22 @@ ngx_quic_empty_handler(ngx_event_t *ev)
 static ssize_t
 ngx_quic_stream_recv(ngx_connection_t *c, u_char *buf, size_t size)
 {
-    ssize_t             len;
-    ngx_buf_t          *b;
-    ngx_chain_t        *cl, *in;
-    ngx_event_t        *rev;
-    ngx_connection_t   *pc;
-    ngx_quic_stream_t  *qs;
+    ssize_t                 len;
+    ngx_buf_t              *b;
+    ngx_chain_t            *cl, *in;
+    ngx_event_t            *rev;
+    ngx_connection_t       *pc;
+    ngx_quic_stream_t      *qs;
+    ngx_quic_connection_t  *qc;
 
     qs = c->quic;
     pc = qs->parent;
+    qc = ngx_quic_get_connection(pc);
+
+    if (qc->closing) {
+        return NGX_ERROR;
+    }
+
     rev = c->read;
 
     if (qs->recv_state == NGX_QUIC_STREAM_RECV_RESET_RECVD
@@ -807,8 +815,17 @@ ngx_quic_stream_recv(ngx_connection_t *c
 static ssize_t
 ngx_quic_stream_send(ngx_connection_t *c, u_char *buf, size_t size)
 {
-    ngx_buf_t    b;
-    ngx_chain_t  cl;
+    ngx_buf_t               b;
+    ngx_chain_t             cl;
+    ngx_connection_t       *pc;
+    ngx_quic_connection_t  *qc;
+
+    pc = c->quic->parent;
+    qc = ngx_quic_get_connection(pc);
+
+    if (qc->closing) {
+        return NGX_ERROR;
+    }
 
     ngx_memzero(&b, sizeof(ngx_buf_t));
 
@@ -843,6 +860,11 @@ ngx_quic_stream_send_chain(ngx_connectio
     qs = c->quic;
     pc = qs->parent;
     qc = ngx_quic_get_connection(pc);
+
+    if (qc->closing) {
+        return NGX_CHAIN_ERROR;
+    }
+
     wev = c->write;
 
     if (qs->send_state != NGX_QUIC_STREAM_SEND_READY
diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c
--- a/src/http/v3/ngx_http_v3_request.c
+++ b/src/http/v3/ngx_http_v3_request.c
@@ -258,7 +258,7 @@ ngx_http_v3_wait_request_handler(ngx_eve
     size_t                     size;
     ssize_t                    n;
     ngx_buf_t                 *b;
-    ngx_connection_t          *c;
+    ngx_connection_t          *c, *pc;
     ngx_pool_cleanup_t        *cln;
     ngx_http_request_t        *r;
     ngx_http_connection_t     *hc;
@@ -385,6 +385,9 @@ ngx_http_v3_wait_request_handler(ngx_eve
     h3c = ngx_http_v3_get_session(c);
     h3c->nrequests++;
 
+    pc = c->quic->parent;
+    ngx_reusable_connection(pc, 0);
+
     if (h3c->keepalive.timer_set) {
         ngx_del_timer(&h3c->keepalive);
     }
@@ -430,7 +433,7 @@ ngx_http_v3_cleanup_request(void *data)
 {
     ngx_http_request_t  *r = data;
 
-    ngx_connection_t          *c;
+    ngx_connection_t          *c, *pc;
     ngx_http_v3_session_t     *h3c;
     ngx_http_core_loc_conf_t  *clcf;
 
@@ -443,6 +446,9 @@ ngx_http_v3_cleanup_request(void *data)
     h3c = ngx_http_v3_get_session(c);
 
     if (--h3c->nrequests == 0) {
+        pc = c->quic->parent;
+        ngx_reusable_connection(pc, 1);
+
         clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module);
         ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout);
     }
diff --git a/src/http/v3/ngx_http_v3_uni.c b/src/http/v3/ngx_http_v3_uni.c
--- a/src/http/v3/ngx_http_v3_uni.c
+++ b/src/http/v3/ngx_http_v3_uni.c
@@ -182,6 +182,11 @@ ngx_http_v3_uni_read_handler(ngx_event_t
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 read handler");
 
+    if (c->close) {
+        ngx_http_v3_close_uni_stream(c);
+        return;
+    }
+
     ngx_memzero(&b, sizeof(ngx_buf_t));
 
     while (rev->ready) {
@@ -262,6 +267,11 @@ ngx_http_v3_uni_dummy_read_handler(ngx_e
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 dummy read handler");
 
+    if (c->close) {
+        ngx_http_v3_close_uni_stream(c);
+        return;
+    }
+
     if (rev->ready) {
         if (c->recv(c, &ch, 1) != 0) {
             ngx_http_v3_finalize_connection(c, NGX_HTTP_V3_ERR_NO_ERROR, NULL);


From arut at nginx.com  Tue Aug 23 12:51:13 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 23 Aug 2022 16:51:13 +0400
Subject: [PATCH 4 of 9] QUIC: defer stream removal until all its data is acked
In-Reply-To: <patchbomb.1661259069@arut-laptop>
References: <patchbomb.1661259069@arut-laptop>
Message-ID: <d95af176c8e90b6f5fbc.1661259073@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1661168003 -14400
#      Mon Aug 22 15:33:23 2022 +0400
# Branch quic
# Node ID d95af176c8e90b6f5fbcc7650aab8c3fddcfac2b
# Parent  6d444d4add98478c92a2ca84dde86005394ae0ea
QUIC: defer stream removal until all its data is acked.

Previously, stream was kept alive until all its data is sent.  This resulted
in disabling retransmission of final part of stream when QUIC connection
was closed right after closing stream connection.

diff --git a/src/event/quic/ngx_event_quic.h b/src/event/quic/ngx_event_quic.h
--- a/src/event/quic/ngx_event_quic.h
+++ b/src/event/quic/ngx_event_quic.h
@@ -85,6 +85,7 @@ struct ngx_quic_stream_s {
     ngx_connection_t              *parent;
     ngx_connection_t              *connection;
     uint64_t                       id;
+    uint64_t                       sent;
     uint64_t                       acked;
     uint64_t                       send_max_data;
     uint64_t                       send_offset;
@@ -98,7 +99,8 @@ struct ngx_quic_stream_s {
     ngx_quic_buffer_t              recv;
     ngx_quic_stream_send_state_e   send_state;
     ngx_quic_stream_recv_state_e   recv_state;
-    ngx_uint_t                     cancelable;  /* unsigned  cancelable:1; */
+    unsigned                       cancelable:1;
+    unsigned                       fin_acked:1;
 };
 
 
diff --git a/src/event/quic/ngx_event_quic_ack.c b/src/event/quic/ngx_event_quic_ack.c
--- a/src/event/quic/ngx_event_quic_ack.c
+++ b/src/event/quic/ngx_event_quic_ack.c
@@ -253,6 +253,7 @@ ngx_quic_handle_ack_frame_range(ngx_conn
                 break;
 
             case NGX_QUIC_FT_STREAM:
+            case NGX_QUIC_FT_RESET_STREAM:
                 ngx_quic_handle_stream_ack(c, f);
                 break;
             }
diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c
--- a/src/event/quic/ngx_event_quic_streams.c
+++ b/src/event/quic/ngx_event_quic_streams.c
@@ -876,7 +876,7 @@ ngx_quic_stream_send_chain(ngx_connectio
 
     qs->send_state = NGX_QUIC_STREAM_SEND_SEND;
 
-    flow = qs->acked + qc->conf->stream_buffer_size - c->sent;
+    flow = qs->acked + qc->conf->stream_buffer_size - qs->sent;
 
     if (flow == 0) {
         wev->ready = 0;
@@ -889,13 +889,14 @@ ngx_quic_stream_send_chain(ngx_connectio
 
     n = qs->send.size;
 
-    in = ngx_quic_write_buffer(pc, &qs->send, in, limit, c->sent);
+    in = ngx_quic_write_buffer(pc, &qs->send, in, limit, qs->sent);
     if (in == NGX_CHAIN_ERROR) {
         return NGX_CHAIN_ERROR;
     }
 
     n = qs->send.size - n;
     c->sent += n;
+    qs->sent += n;
     qc->streams.sent += n;
 
     if (flow == n) {
@@ -1034,9 +1035,12 @@ ngx_quic_close_stream(ngx_quic_stream_t 
     if (!qc->closing) {
         /* make sure everything is sent and final size is received */
 
-        if (qs->recv_state == NGX_QUIC_STREAM_RECV_RECV
-            || qs->send_state == NGX_QUIC_STREAM_SEND_READY
-            || qs->send_state == NGX_QUIC_STREAM_SEND_SEND)
+        if (qs->recv_state == NGX_QUIC_STREAM_RECV_RECV) {
+            return NGX_OK;
+        }
+
+        if (qs->send_state != NGX_QUIC_STREAM_SEND_DATA_RECVD
+            && qs->send_state != NGX_QUIC_STREAM_SEND_RESET_RECVD)
         {
             return NGX_OK;
         }
@@ -1444,36 +1448,70 @@ ngx_quic_handle_max_streams_frame(ngx_co
 void
 ngx_quic_handle_stream_ack(ngx_connection_t *c, ngx_quic_frame_t *f)
 {
-    uint64_t                sent, unacked;
+    uint64_t                acked;
     ngx_quic_stream_t      *qs;
     ngx_quic_connection_t  *qc;
 
     qc = ngx_quic_get_connection(c);
 
-    qs = ngx_quic_find_stream(&qc->streams.tree, f->u.stream.stream_id);
-    if (qs == NULL) {
+    switch (f->type) {
+
+    case NGX_QUIC_FT_RESET_STREAM:
+
+        qs = ngx_quic_find_stream(&qc->streams.tree, f->u.reset_stream.id);
+        if (qs == NULL) {
+            return;
+        }
+
+        qs->send_state = NGX_QUIC_STREAM_SEND_RESET_RECVD;
+
+        ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0,
+                       "quic stream id:0x%xL ack reset final_size:%uL",
+                       qs->id, f->u.reset_stream.final_size);
+
+        break;
+
+    case NGX_QUIC_FT_STREAM:
+
+        qs = ngx_quic_find_stream(&qc->streams.tree, f->u.stream.stream_id);
+        if (qs == NULL) {
+            return;
+        }
+
+        acked = qs->acked;
+        qs->acked += f->u.stream.length;
+
+        if (f->u.stream.fin) {
+            qs->fin_acked = 1;
+        }
+
+        if (qs->send_state == NGX_QUIC_STREAM_SEND_DATA_SENT
+            && qs->acked == qs->sent && qs->fin_acked)
+        {
+            qs->send_state = NGX_QUIC_STREAM_SEND_DATA_RECVD;
+        }
+
+        ngx_log_debug4(NGX_LOG_DEBUG_EVENT, c->log, 0,
+                       "quic stream id:0x%xL ack len:%uL fin:%d unacked:%uL",
+                       qs->id, f->u.stream.length, f->u.stream.fin,
+                       qs->sent - qs->acked);
+
+        if (qs->connection
+            && qs->sent - acked == qc->conf->stream_buffer_size
+            && f->u.stream.length > 0)
+        {
+            ngx_quic_set_event(qs->connection->write);
+        }
+
+        break;
+
+    default:
         return;
     }
 
     if (qs->connection == NULL) {
-        qs->acked += f->u.stream.length;
-        return;
+        ngx_quic_close_stream(qs);
     }
-
-    sent = qs->connection->sent;
-    unacked = sent - qs->acked;
-    qs->acked += f->u.stream.length;
-
-    ngx_log_debug4(NGX_LOG_DEBUG_EVENT, c->log, 0,
-                   "quic stream id:0x%xL ack len:%uL acked:%uL unacked:%uL",
-                   qs->id, f->u.stream.length, qs->acked, sent - qs->acked);
-
-    if (unacked != qc->conf->stream_buffer_size) {
-        /* not blocked on buffer size */
-        return;
-    }
-
-    ngx_quic_set_event(qs->connection->write);
 }
 
 


From arut at nginx.com  Tue Aug 23 12:51:14 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 23 Aug 2022 16:51:14 +0400
Subject: [PATCH 5 of 9] QUIC: do not send MAX_STREAMS in shutdown state
In-Reply-To: <patchbomb.1661259069@arut-laptop>
References: <patchbomb.1661259069@arut-laptop>
Message-ID: <8b0ffd69cca1a9aff606.1661259074@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1660909446 -14400
#      Fri Aug 19 15:44:06 2022 +0400
# Branch quic
# Node ID 8b0ffd69cca1a9aff606f1c8e416024b1e818a5e
# Parent  d95af176c8e90b6f5fbcc7650aab8c3fddcfac2b
QUIC: do not send MAX_STREAMS in shutdown state.

No more streams are expected from client.

diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c
--- a/src/event/quic/ngx_event_quic_streams.c
+++ b/src/event/quic/ngx_event_quic_streams.c
@@ -1055,8 +1055,7 @@ ngx_quic_close_stream(ngx_quic_stream_t 
     ngx_rbtree_delete(&qc->streams.tree, &qs->node);
     ngx_queue_insert_tail(&qc->streams.free, &qs->queue);
 
-    if (qc->closing) {
-        /* schedule handler call to continue ngx_quic_close_connection() */
+    if (qc->closing || qc->shutdown) {
         ngx_post_event(pc->read, &ngx_posted_events);
         return NGX_OK;
     }
@@ -1082,10 +1081,6 @@ ngx_quic_close_stream(ngx_quic_stream_t 
         ngx_quic_queue_frame(qc, frame);
     }
 
-    if (qc->shutdown) {
-        ngx_post_event(pc->read, &ngx_posted_events);
-    }
-
     return NGX_OK;
 }
 


From arut at nginx.com  Tue Aug 23 12:51:15 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 23 Aug 2022 16:51:15 +0400
Subject: [PATCH 6 of 9] HTTP/3: unified hq code with HTTP/3 code
In-Reply-To: <patchbomb.1661259069@arut-laptop>
References: <patchbomb.1661259069@arut-laptop>
Message-ID: <462b1520751e1c7a06af.1661259075@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1661249481 -14400
#      Tue Aug 23 14:11:21 2022 +0400
# Branch quic
# Node ID 462b1520751e1c7a06af9de3004358a0246f8792
# Parent  8b0ffd69cca1a9aff606f1c8e416024b1e818a5e
HTTP/3: unified hq code with HTTP/3 code.

The change removes hq-specific request handler.  Now hq requests are handled
by the HTTP/3 request handler.  This brings missing features like keepalive
timeout and reusable mode to hq.

diff --git a/src/http/v3/ngx_http_v3.c b/src/http/v3/ngx_http_v3.c
--- a/src/http/v3/ngx_http_v3.c
+++ b/src/http/v3/ngx_http_v3.c
@@ -17,10 +17,13 @@ static void ngx_http_v3_cleanup_session(
 ngx_int_t
 ngx_http_v3_init_session(ngx_connection_t *c)
 {
-    ngx_connection_t       *pc;
-    ngx_pool_cleanup_t     *cln;
-    ngx_http_connection_t  *hc;
-    ngx_http_v3_session_t  *h3c;
+    ngx_connection_t        *pc;
+    ngx_pool_cleanup_t      *cln;
+    ngx_http_connection_t   *hc;
+    ngx_http_v3_session_t   *h3c;
+#if (NGX_HTTP_V3_HQ)
+    ngx_http_v3_srv_conf_t  *h3scf;
+#endif
 
     pc = c->quic->parent;
     hc = pc->data;
@@ -39,6 +42,13 @@ ngx_http_v3_init_session(ngx_connection_
     h3c->max_push_id = (uint64_t) -1;
     h3c->goaway_push_id = (uint64_t) -1;
 
+#if (NGX_HTTP_V3_HQ)
+    h3scf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_v3_module);
+    if (h3scf->hq) {
+        h3c->hq = 1;
+    }
+#endif
+
     ngx_queue_init(&h3c->blocked);
     ngx_queue_init(&h3c->pushing);
 
@@ -61,6 +71,12 @@ ngx_http_v3_init_session(ngx_connection_
 
     hc->v3_session = h3c;
 
+#if (NGX_HTTP_V3_HQ)
+    if (h3c->hq) {
+        return NGX_OK;
+    }
+#endif
+
     return ngx_http_v3_send_settings(c);
 
 failed:
diff --git a/src/http/v3/ngx_http_v3.h b/src/http/v3/ngx_http_v3.h
--- a/src/http/v3/ngx_http_v3.h
+++ b/src/http/v3/ngx_http_v3.h
@@ -145,7 +145,10 @@ struct ngx_http_v3_session_s {
     off_t                         total_bytes;
     off_t                         payload_bytes;
 
-    ngx_uint_t                    goaway;  /* unsigned  goaway:1; */
+    unsigned                      goaway:1;
+#if (NGX_HTTP_V3_HQ)
+    unsigned                      hq:1;
+#endif
 
     ngx_connection_t             *known_streams[NGX_HTTP_V3_MAX_KNOWN_STREAM];
 };
diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c
--- a/src/http/v3/ngx_http_v3_request.c
+++ b/src/http/v3/ngx_http_v3_request.c
@@ -10,10 +10,8 @@
 #include <ngx_http.h>
 
 
-#if (NGX_HTTP_V3_HQ)
-static void ngx_http_v3_init_hq_stream(ngx_connection_t *c);
-#endif
 static void ngx_http_v3_init_request_stream(ngx_connection_t *c);
+static void ngx_http_v3_cleanup_connection(void *data);
 static void ngx_http_v3_wait_request_handler(ngx_event_t *rev);
 static void ngx_http_v3_cleanup_request(void *data);
 static void ngx_http_v3_process_request(ngx_event_t *rev);
@@ -86,13 +84,6 @@ ngx_http_v3_init(ngx_connection_t *c)
         ngx_set_connection_log(c, clcf->error_log);
     }
 
-#if (NGX_HTTP_V3_HQ)
-    if (h3scf->hq) {
-        ngx_http_v3_init_hq_stream(c);
-        return;
-    }
-#endif
-
     if (ngx_http_v3_init_session(c) != NGX_OK) {
         ngx_http_close_connection(c);
         return;
@@ -107,83 +98,13 @@ ngx_http_v3_init(ngx_connection_t *c)
 }
 
 
-#if (NGX_HTTP_V3_HQ)
-
-static void
-ngx_http_v3_init_hq_stream(ngx_connection_t *c)
-{
-    uint64_t                   n;
-    ngx_event_t               *rev;
-    ngx_http_connection_t     *hc;
-    ngx_http_core_loc_conf_t  *clcf;
-    ngx_http_core_srv_conf_t  *cscf;
-
-    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 init hq stream");
-
-#if (NGX_STAT_STUB)
-    (void) ngx_atomic_fetch_add(ngx_stat_active, 1);
-#endif
-
-    hc = c->data;
-
-    /* Use HTTP/3 General Protocol Error Code 0x101 for finalization */
-
-    if (c->quic->id & NGX_QUIC_STREAM_UNIDIRECTIONAL) {
-        ngx_quic_finalize_connection(c->quic->parent,
-                                     NGX_HTTP_V3_ERR_GENERAL_PROTOCOL_ERROR,
-                                     "unexpected uni stream");
-        ngx_http_close_connection(c);
-        return;
-    }
-
-    clcf = ngx_http_get_module_loc_conf(hc->conf_ctx, ngx_http_core_module);
-
-    n = c->quic->id >> 2;
-
-    if (n >= clcf->keepalive_requests) {
-        ngx_quic_finalize_connection(c->quic->parent,
-                                     NGX_HTTP_V3_ERR_GENERAL_PROTOCOL_ERROR,
-                                     "reached maximum number of requests");
-        ngx_http_close_connection(c);
-        return;
-    }
-
-    if (ngx_current_msec - c->quic->parent->start_time
-        > clcf->keepalive_time)
-    {
-        ngx_quic_finalize_connection(c->quic->parent,
-                                     NGX_HTTP_V3_ERR_GENERAL_PROTOCOL_ERROR,
-                                     "reached maximum time for requests");
-        ngx_http_close_connection(c);
-        return;
-    }
-
-    rev = c->read;
-
-    if (rev->ready) {
-        rev->handler(rev);
-        return;
-    }
-
-    cscf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_core_module);
-
-    ngx_add_timer(rev, cscf->client_header_timeout);
-    ngx_reusable_connection(c, 1);
-
-    if (ngx_handle_read_event(rev, 0) != NGX_OK) {
-        ngx_http_close_connection(c);
-        return;
-    }
-}
-
-#endif
-
-
 static void
 ngx_http_v3_init_request_stream(ngx_connection_t *c)
 {
     uint64_t                   n;
     ngx_event_t               *rev;
+    ngx_connection_t          *pc;
+    ngx_pool_cleanup_t        *cln;
     ngx_http_connection_t     *hc;
     ngx_http_v3_session_t     *h3c;
     ngx_http_core_loc_conf_t  *clcf;
@@ -216,24 +137,55 @@ ngx_http_v3_init_request_stream(ngx_conn
         return;
     }
 
+    pc = c->quic->parent;
+
     if (n + 1 == clcf->keepalive_requests
-        || ngx_current_msec - c->quic->parent->start_time
-           > clcf->keepalive_time)
+        || ngx_current_msec - pc->start_time > clcf->keepalive_time)
     {
         h3c->goaway = 1;
 
-        if (ngx_http_v3_send_goaway(c, (n + 1) << 2) != NGX_OK) {
-            ngx_http_close_connection(c);
-            return;
+#if (NGX_HTTP_V3_HQ)
+        if (!h3c->hq)
+#endif
+        {
+            if (ngx_http_v3_send_goaway(c, (n + 1) << 2) != NGX_OK) {
+                ngx_http_close_connection(c);
+                return;
+            }
         }
 
         ngx_http_v3_shutdown_connection(c, NGX_HTTP_V3_ERR_NO_ERROR,
                                         "reached maximum number of requests");
     }
 
+    cln = ngx_pool_cleanup_add(c->pool, 0);
+    if (cln == NULL) {
+        ngx_quic_finalize_connection(pc, NGX_HTTP_V3_ERR_INTERNAL_ERROR,
+                                     "internal error");
+        ngx_http_close_connection(c);
+        return;
+    }
+
+    cln->handler = ngx_http_v3_cleanup_connection;
+    cln->data = c;
+
+    if (h3c->nrequests++ == 0) {
+        ngx_reusable_connection(pc, 0);
+    }
+
+    if (h3c->keepalive.timer_set) {
+        ngx_del_timer(&h3c->keepalive);
+    }
+
     rev = c->read;
-    rev->handler = ngx_http_v3_wait_request_handler;
-    c->write->handler = ngx_http_empty_handler;
+
+#if (NGX_HTTP_V3_HQ)
+    if (!h3c->hq)
+#endif
+    {
+        rev->handler = ngx_http_v3_wait_request_handler;
+        c->write->handler = ngx_http_empty_handler;
+    }
 
     if (rev->ready) {
         rev->handler(rev);
@@ -253,16 +205,36 @@ ngx_http_v3_init_request_stream(ngx_conn
 
 
 static void
+ngx_http_v3_cleanup_connection(void *data)
+{
+    ngx_connection_t  *c = data;
+
+    ngx_connection_t          *pc;
+    ngx_http_v3_session_t     *h3c;
+    ngx_http_core_loc_conf_t  *clcf;
+
+    h3c = ngx_http_v3_get_session(c);
+
+    if (--h3c->nrequests == 0) {
+        pc = c->quic->parent;
+        ngx_reusable_connection(pc, 1);
+
+        clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module);
+        ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout);
+    }
+}
+
+
+static void
 ngx_http_v3_wait_request_handler(ngx_event_t *rev)
 {
     size_t                     size;
     ssize_t                    n;
     ngx_buf_t                 *b;
-    ngx_connection_t          *c, *pc;
+    ngx_connection_t          *c;
     ngx_pool_cleanup_t        *cln;
     ngx_http_request_t        *r;
     ngx_http_connection_t     *hc;
-    ngx_http_v3_session_t     *h3c;
     ngx_http_core_srv_conf_t  *cscf;
 
     c = rev->data;
@@ -382,16 +354,6 @@ ngx_http_v3_wait_request_handler(ngx_eve
     cln->handler = ngx_http_v3_cleanup_request;
     cln->data = r;
 
-    h3c = ngx_http_v3_get_session(c);
-    h3c->nrequests++;
-
-    pc = c->quic->parent;
-    ngx_reusable_connection(pc, 0);
-
-    if (h3c->keepalive.timer_set) {
-        ngx_del_timer(&h3c->keepalive);
-    }
-
     rev->handler = ngx_http_v3_process_request;
     ngx_http_v3_process_request(rev);
 }
@@ -404,13 +366,10 @@ ngx_http_v3_reset_connection(ngx_connect
 
     h3scf = ngx_http_v3_get_module_srv_conf(c, ngx_http_v3_module);
 
+    if (h3scf->max_table_capacity > 0 && !c->read->eof
 #if (NGX_HTTP_V3_HQ)
-    if (h3scf->hq) {
-        return;
-    }
+        && !h3scf->hq
 #endif
-
-    if (h3scf->max_table_capacity > 0 && !c->read->eof
         && (c->quic->id & NGX_QUIC_STREAM_UNIDIRECTIONAL) == 0)
     {
         (void) ngx_http_v3_send_cancel_stream(c, c->quic->id);
@@ -433,24 +392,8 @@ ngx_http_v3_cleanup_request(void *data)
 {
     ngx_http_request_t  *r = data;
 
-    ngx_connection_t          *c, *pc;
-    ngx_http_v3_session_t     *h3c;
-    ngx_http_core_loc_conf_t  *clcf;
-
-    c = r->connection;
-
     if (!r->response_sent) {
-        c->error = 1;
-    }
-
-    h3c = ngx_http_v3_get_session(c);
-
-    if (--h3c->nrequests == 0) {
-        pc = c->quic->parent;
-        ngx_reusable_connection(pc, 1);
-
-        clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module);
-        ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout);
+        r->connection->error = 1;
     }
 }
 
diff --git a/src/http/v3/ngx_http_v3_uni.c b/src/http/v3/ngx_http_v3_uni.c
--- a/src/http/v3/ngx_http_v3_uni.c
+++ b/src/http/v3/ngx_http_v3_uni.c
@@ -37,8 +37,23 @@ void
 ngx_http_v3_init_uni_stream(ngx_connection_t *c)
 {
     uint64_t                   n;
+#if (NGX_HTTP_V3_HQ)
+    ngx_http_v3_session_t     *h3c;
+#endif
     ngx_http_v3_uni_stream_t  *us;
 
+#if (NGX_HTTP_V3_HQ)
+    h3c = ngx_http_v3_get_session(c);
+    if (h3c->hq) {
+        ngx_http_v3_finalize_connection(c,
+                                        NGX_HTTP_V3_ERR_STREAM_CREATION_ERROR,
+                                        "uni stream in hq mode");
+        c->data = NULL;
+        ngx_http_v3_close_uni_stream(c);
+        return;
+    }
+#endif
+
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 init uni stream");
 
     n = c->quic->id >> 2;


From arut at nginx.com  Tue Aug 23 12:51:16 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 23 Aug 2022 16:51:16 +0400
Subject: [PATCH 7 of 9] QUIC: idle mode for main connection
In-Reply-To: <patchbomb.1661259069@arut-laptop>
References: <patchbomb.1661259069@arut-laptop>
Message-ID: <0462a4da1f5bf96fc740.1661259076@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1661248462 -14400
#      Tue Aug 23 13:54:22 2022 +0400
# Branch quic
# Node ID 0462a4da1f5bf96fc74064f2c45b384eead6ef63
# Parent  462b1520751e1c7a06af9de3004358a0246f8792
QUIC: idle mode for main connection.

Now main QUIC connection for HTTP/3 always has c->idle flag set.  This allows
the connection to receive worker shutdown notification.  It is passed to
application level via a new conf->shutdown() callback.

The HTTP/3 shutdown callback sends GOAWAY to client and gracefully shuts down
the QUIC connection.

diff --git a/src/event/quic/ngx_event_quic.c b/src/event/quic/ngx_event_quic.c
--- a/src/event/quic/ngx_event_quic.c
+++ b/src/event/quic/ngx_event_quic.c
@@ -414,9 +414,9 @@ ngx_quic_input_handler(ngx_event_t *rev)
     }
 
     if (c->close) {
-        qc->error = NGX_QUIC_ERR_NO_ERROR;
-        qc->error_reason = "graceful shutdown";
-        ngx_quic_close_connection(c, NGX_ERROR);
+        if (qc->conf->shutdown) {
+            qc->conf->shutdown(c);
+        }
         return;
     }
 
diff --git a/src/event/quic/ngx_event_quic.h b/src/event/quic/ngx_event_quic.h
--- a/src/event/quic/ngx_event_quic.h
+++ b/src/event/quic/ngx_event_quic.h
@@ -28,6 +28,9 @@
 #define NGX_QUIC_STREAM_UNIDIRECTIONAL       0x02
 
 
+typedef void (*ngx_quic_shutdown_pt)(ngx_connection_t *c);
+
+
 typedef enum {
     NGX_QUIC_STREAM_SEND_READY = 0,
     NGX_QUIC_STREAM_SEND_SEND,
@@ -74,6 +77,8 @@ typedef struct {
     ngx_int_t                      stream_reject_code_uni;
     ngx_int_t                      stream_reject_code_bidi;
 
+    ngx_quic_shutdown_pt           shutdown;
+
     u_char                         av_token_key[NGX_QUIC_AV_KEY_LEN];
     u_char                         sr_token_key[NGX_QUIC_SR_KEY_LEN];
 } ngx_quic_conf_t;
diff --git a/src/http/v3/ngx_http_v3.h b/src/http/v3/ngx_http_v3.h
--- a/src/http/v3/ngx_http_v3.h
+++ b/src/http/v3/ngx_http_v3.h
@@ -141,6 +141,7 @@ struct ngx_http_v3_session_s {
     uint64_t                      next_push_id;
     uint64_t                      max_push_id;
     uint64_t                      goaway_push_id;
+    uint64_t                      next_request_id;
 
     off_t                         total_bytes;
     off_t                         payload_bytes;
@@ -158,6 +159,7 @@ void ngx_http_v3_init(ngx_connection_t *
 void ngx_http_v3_reset_connection(ngx_connection_t *c);
 ngx_int_t ngx_http_v3_init_session(ngx_connection_t *c);
 ngx_int_t ngx_http_v3_check_flood(ngx_connection_t *c);
+void ngx_http_v3_shutdown(ngx_connection_t *c);
 
 ngx_int_t ngx_http_v3_read_request_body(ngx_http_request_t *r);
 ngx_int_t ngx_http_v3_read_unbuffered_request_body(ngx_http_request_t *r);
diff --git a/src/http/v3/ngx_http_v3_module.c b/src/http/v3/ngx_http_v3_module.c
--- a/src/http/v3/ngx_http_v3_module.c
+++ b/src/http/v3/ngx_http_v3_module.c
@@ -249,6 +249,8 @@ ngx_http_v3_create_srv_conf(ngx_conf_t *
     h3scf->quic.stream_reject_code_bidi = NGX_HTTP_V3_ERR_REQUEST_REJECTED;
     h3scf->quic.active_connection_id_limit = NGX_CONF_UNSET_UINT;
 
+    h3scf->quic.shutdown = ngx_http_v3_shutdown;
+
     return h3scf;
 }
 
diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c
--- a/src/http/v3/ngx_http_v3_request.c
+++ b/src/http/v3/ngx_http_v3_request.c
@@ -70,6 +70,7 @@ ngx_http_v3_init(ngx_connection_t *c)
     h3scf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_v3_module);
 
     if (c->quic == NULL) {
+        c->idle = 1;
         h3scf->quic.timeout = clcf->keepalive_timeout;
         ngx_quic_run(c, &h3scf->quic);
         return;
@@ -98,6 +99,37 @@ ngx_http_v3_init(ngx_connection_t *c)
 }
 
 
+void
+ngx_http_v3_shutdown(ngx_connection_t *c)
+{
+    ngx_http_v3_session_t   *h3c;
+
+    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 shutdown");
+
+    h3c = ngx_http_v3_get_session(c);
+
+    if (h3c == NULL) {
+        ngx_quic_finalize_connection(c, NGX_HTTP_V3_ERR_NO_ERROR,
+                                     "connection shutdown");
+        return;
+    }
+
+    if (!h3c->goaway) {
+        h3c->goaway = 1;
+
+#if (NGX_HTTP_V3_HQ)
+        if (!h3c->hq)
+#endif
+        {
+            (void) ngx_http_v3_send_goaway(c, h3c->next_request_id);
+        }
+
+        ngx_http_v3_shutdown_connection(c, NGX_HTTP_V3_ERR_NO_ERROR,
+                                        "connection shutdown");
+    }
+}
+
+
 static void
 ngx_http_v3_init_request_stream(ngx_connection_t *c)
 {
@@ -139,6 +171,8 @@ ngx_http_v3_init_request_stream(ngx_conn
 
     pc = c->quic->parent;
 
+    h3c->next_request_id = c->quic->id + 0x04;
+
     if (n + 1 == clcf->keepalive_requests
         || ngx_current_msec - pc->start_time > clcf->keepalive_time)
     {
@@ -148,7 +182,7 @@ ngx_http_v3_init_request_stream(ngx_conn
         if (!h3c->hq)
 #endif
         {
-            if (ngx_http_v3_send_goaway(c, (n + 1) << 2) != NGX_OK) {
+            if (ngx_http_v3_send_goaway(c, h3c->next_request_id) != NGX_OK) {
                 ngx_http_close_connection(c);
                 return;
             }


From arut at nginx.com  Tue Aug 23 12:51:17 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 23 Aug 2022 16:51:17 +0400
Subject: [PATCH 8 of 9] HTTP/3: renamed functions
In-Reply-To: <patchbomb.1661259069@arut-laptop>
References: <patchbomb.1661259069@arut-laptop>
Message-ID: <a79b10f6ad221447c528.1661259077@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1661162943 -14400
#      Mon Aug 22 14:09:03 2022 +0400
# Branch quic
# Node ID a79b10f6ad221447c52870a90c856a04abfb07a6
# Parent  0462a4da1f5bf96fc74064f2c45b384eead6ef63
HTTP/3: renamed functions.

ngx_http_v3_init() is renamed ngx_http_v3_init_stream().
ngx_http_v3_reset_connection() is renamed to ngx_http_v3_reset_stream().

diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c
+++ b/src/http/ngx_http_request.c
@@ -326,7 +326,7 @@ ngx_http_init_connection(ngx_connection_
 
 #if (NGX_HTTP_V3)
     if (hc->addr_conf->http3) {
-        ngx_http_v3_init(c);
+        ngx_http_v3_init_stream(c);
         return;
     }
 #endif
@@ -3786,7 +3786,7 @@ ngx_http_close_connection(ngx_connection
 
 #if (NGX_HTTP_V3)
     if (c->quic) {
-        ngx_http_v3_reset_connection(c);
+        ngx_http_v3_reset_stream(c);
     }
 #endif
 
diff --git a/src/http/v3/ngx_http_v3.h b/src/http/v3/ngx_http_v3.h
--- a/src/http/v3/ngx_http_v3.h
+++ b/src/http/v3/ngx_http_v3.h
@@ -155,8 +155,8 @@ struct ngx_http_v3_session_s {
 };
 
 
-void ngx_http_v3_init(ngx_connection_t *c);
-void ngx_http_v3_reset_connection(ngx_connection_t *c);
+void ngx_http_v3_init_stream(ngx_connection_t *c);
+void ngx_http_v3_reset_stream(ngx_connection_t *c);
 ngx_int_t ngx_http_v3_init_session(ngx_connection_t *c);
 ngx_int_t ngx_http_v3_check_flood(ngx_connection_t *c);
 void ngx_http_v3_shutdown(ngx_connection_t *c);
diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c
--- a/src/http/v3/ngx_http_v3_request.c
+++ b/src/http/v3/ngx_http_v3_request.c
@@ -56,7 +56,7 @@ static const struct {
 
 
 void
-ngx_http_v3_init(ngx_connection_t *c)
+ngx_http_v3_init_stream(ngx_connection_t *c)
 {
     ngx_http_connection_t     *hc, *phc;
     ngx_http_v3_srv_conf_t    *h3scf;
@@ -394,7 +394,7 @@ ngx_http_v3_wait_request_handler(ngx_eve
 
 
 void
-ngx_http_v3_reset_connection(ngx_connection_t *c)
+ngx_http_v3_reset_stream(ngx_connection_t *c)
 {
     ngx_http_v3_srv_conf_t  *h3scf;
 


From arut at nginx.com  Tue Aug 23 12:51:18 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Tue, 23 Aug 2022 16:51:18 +0400
Subject: [PATCH 9 of 9] QUIC: application init() callback
In-Reply-To: <patchbomb.1661259069@arut-laptop>
References: <patchbomb.1661259069@arut-laptop>
Message-ID: <59eac6d01094b992ffcf.1661259078@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1661249881 -14400
#      Tue Aug 23 14:18:01 2022 +0400
# Branch quic
# Node ID 59eac6d01094b992ffcfe70a11a214586f18e9f2
# Parent  a79b10f6ad221447c52870a90c856a04abfb07a6
QUIC: application init() callback.

It's called after handshake completion or prior to the first early data stream
creation.  The callback should initialize application-level data before
creating streams.

HTTP/3 callback implementation sets keepalive timer and marks connection as
reusable.

Also, this allows to limit max handshake time in ngx_http_v3_init_stream().

diff --git a/src/event/quic/ngx_event_quic.h b/src/event/quic/ngx_event_quic.h
--- a/src/event/quic/ngx_event_quic.h
+++ b/src/event/quic/ngx_event_quic.h
@@ -28,6 +28,7 @@
 #define NGX_QUIC_STREAM_UNIDIRECTIONAL       0x02
 
 
+typedef ngx_int_t (*ngx_quic_init_pt)(ngx_connection_t *c);
 typedef void (*ngx_quic_shutdown_pt)(ngx_connection_t *c);
 
 
@@ -77,6 +78,7 @@ typedef struct {
     ngx_int_t                      stream_reject_code_uni;
     ngx_int_t                      stream_reject_code_bidi;
 
+    ngx_quic_init_pt               init;
     ngx_quic_shutdown_pt           shutdown;
 
     u_char                         av_token_key[NGX_QUIC_AV_KEY_LEN];
diff --git a/src/event/quic/ngx_event_quic_streams.c b/src/event/quic/ngx_event_quic_streams.c
--- a/src/event/quic/ngx_event_quic_streams.c
+++ b/src/event/quic/ngx_event_quic_streams.c
@@ -21,6 +21,7 @@ static ngx_quic_stream_t *ngx_quic_get_s
 static ngx_int_t ngx_quic_reject_stream(ngx_connection_t *c, uint64_t id);
 static void ngx_quic_init_stream_handler(ngx_event_t *ev);
 static void ngx_quic_init_streams_handler(ngx_connection_t *c);
+static ngx_int_t ngx_quic_do_init_streams(ngx_connection_t *c);
 static ngx_quic_stream_t *ngx_quic_create_stream(ngx_connection_t *c,
     uint64_t id);
 static void ngx_quic_empty_handler(ngx_event_t *ev);
@@ -547,15 +548,22 @@ ngx_quic_init_streams(ngx_connection_t *
         return NGX_OK;
     }
 
-    ngx_quic_init_streams_handler(c);
-
-    return NGX_OK;
+    return ngx_quic_do_init_streams(c);
 }
 
 
 static void
 ngx_quic_init_streams_handler(ngx_connection_t *c)
 {
+    if (ngx_quic_do_init_streams(c) != NGX_OK) {
+        ngx_quic_close_connection(c, NGX_ERROR);
+    }
+}
+
+
+static ngx_int_t
+ngx_quic_do_init_streams(ngx_connection_t *c)
+{
     ngx_queue_t            *q;
     ngx_quic_stream_t      *qs;
     ngx_quic_connection_t  *qc;
@@ -564,6 +572,12 @@ ngx_quic_init_streams_handler(ngx_connec
 
     qc = ngx_quic_get_connection(c);
 
+    if (qc->conf->init) {
+        if (qc->conf->init(c) != NGX_OK) {
+            return NGX_ERROR;
+        }
+    }
+
     for (q = ngx_queue_head(&qc->streams.uninitialized);
          q != ngx_queue_sentinel(&qc->streams.uninitialized);
          q = ngx_queue_next(q))
@@ -573,6 +587,8 @@ ngx_quic_init_streams_handler(ngx_connec
     }
 
     qc->streams.initialized = 1;
+
+    return NGX_OK;
 }
 
 
diff --git a/src/http/v3/ngx_http_v3.c b/src/http/v3/ngx_http_v3.c
--- a/src/http/v3/ngx_http_v3.c
+++ b/src/http/v3/ngx_http_v3.c
@@ -17,7 +17,6 @@ static void ngx_http_v3_cleanup_session(
 ngx_int_t
 ngx_http_v3_init_session(ngx_connection_t *c)
 {
-    ngx_connection_t        *pc;
     ngx_pool_cleanup_t      *cln;
     ngx_http_connection_t   *hc;
     ngx_http_v3_session_t   *h3c;
@@ -25,16 +24,11 @@ ngx_http_v3_init_session(ngx_connection_
     ngx_http_v3_srv_conf_t  *h3scf;
 #endif
 
-    pc = c->quic->parent;
-    hc = pc->data;
-
-    if (hc->v3_session) {
-        return NGX_OK;
-    }
+    hc = c->data;
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 init session");
 
-    h3c = ngx_pcalloc(pc->pool, sizeof(ngx_http_v3_session_t));
+    h3c = ngx_pcalloc(c->pool, sizeof(ngx_http_v3_session_t));
     if (h3c == NULL) {
         goto failed;
     }
@@ -52,16 +46,16 @@ ngx_http_v3_init_session(ngx_connection_
     ngx_queue_init(&h3c->blocked);
     ngx_queue_init(&h3c->pushing);
 
-    h3c->keepalive.log = pc->log;
-    h3c->keepalive.data = pc;
+    h3c->keepalive.log = c->log;
+    h3c->keepalive.data = c;
     h3c->keepalive.handler = ngx_http_v3_keepalive_handler;
     h3c->keepalive.cancelable = 1;
 
-    h3c->table.send_insert_count.log = pc->log;
-    h3c->table.send_insert_count.data = pc;
+    h3c->table.send_insert_count.log = c->log;
+    h3c->table.send_insert_count.data = c;
     h3c->table.send_insert_count.handler = ngx_http_v3_inc_insert_count_handler;
 
-    cln = ngx_pool_cleanup_add(pc->pool, 0);
+    cln = ngx_pool_cleanup_add(c->pool, 0);
     if (cln == NULL) {
         goto failed;
     }
@@ -71,13 +65,7 @@ ngx_http_v3_init_session(ngx_connection_
 
     hc->v3_session = h3c;
 
-#if (NGX_HTTP_V3_HQ)
-    if (h3c->hq) {
-        return NGX_OK;
-    }
-#endif
-
-    return ngx_http_v3_send_settings(c);
+    return NGX_OK;
 
 failed:
 
diff --git a/src/http/v3/ngx_http_v3.h b/src/http/v3/ngx_http_v3.h
--- a/src/http/v3/ngx_http_v3.h
+++ b/src/http/v3/ngx_http_v3.h
@@ -159,6 +159,7 @@ void ngx_http_v3_init_stream(ngx_connect
 void ngx_http_v3_reset_stream(ngx_connection_t *c);
 ngx_int_t ngx_http_v3_init_session(ngx_connection_t *c);
 ngx_int_t ngx_http_v3_check_flood(ngx_connection_t *c);
+ngx_int_t ngx_http_v3_init(ngx_connection_t *c);
 void ngx_http_v3_shutdown(ngx_connection_t *c);
 
 ngx_int_t ngx_http_v3_read_request_body(ngx_http_request_t *r);
diff --git a/src/http/v3/ngx_http_v3_module.c b/src/http/v3/ngx_http_v3_module.c
--- a/src/http/v3/ngx_http_v3_module.c
+++ b/src/http/v3/ngx_http_v3_module.c
@@ -249,6 +249,7 @@ ngx_http_v3_create_srv_conf(ngx_conf_t *
     h3scf->quic.stream_reject_code_bidi = NGX_HTTP_V3_ERR_REQUEST_REJECTED;
     h3scf->quic.active_connection_id_limit = NGX_CONF_UNSET_UINT;
 
+    h3scf->quic.init = ngx_http_v3_init;
     h3scf->quic.shutdown = ngx_http_v3_shutdown;
 
     return h3scf;
diff --git a/src/http/v3/ngx_http_v3_request.c b/src/http/v3/ngx_http_v3_request.c
--- a/src/http/v3/ngx_http_v3_request.c
+++ b/src/http/v3/ngx_http_v3_request.c
@@ -58,18 +58,29 @@ static const struct {
 void
 ngx_http_v3_init_stream(ngx_connection_t *c)
 {
+    ngx_http_v3_session_t     *h3c;
     ngx_http_connection_t     *hc, *phc;
     ngx_http_v3_srv_conf_t    *h3scf;
     ngx_http_core_loc_conf_t  *clcf;
+    ngx_http_core_srv_conf_t  *cscf;
 
     hc = c->data;
 
     hc->ssl = 1;
 
     clcf = ngx_http_get_module_loc_conf(hc->conf_ctx, ngx_http_core_module);
+    cscf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_core_module);
     h3scf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_v3_module);
 
     if (c->quic == NULL) {
+        if (ngx_http_v3_init_session(c) != NGX_OK) {
+            ngx_http_close_connection(c);
+            return;
+        }
+
+        h3c = hc->v3_session;
+        ngx_add_timer(&h3c->keepalive, cscf->client_header_timeout);
+
         c->idle = 1;
         h3scf->quic.timeout = clcf->keepalive_timeout;
         ngx_quic_run(c, &h3scf->quic);
@@ -85,11 +96,6 @@ ngx_http_v3_init_stream(ngx_connection_t
         ngx_set_connection_log(c, clcf->error_log);
     }
 
-    if (ngx_http_v3_init_session(c) != NGX_OK) {
-        ngx_http_close_connection(c);
-        return;
-    }
-
     if (c->quic->id & NGX_QUIC_STREAM_UNIDIRECTIONAL) {
         ngx_http_v3_init_uni_stream(c);
 
@@ -99,6 +105,30 @@ ngx_http_v3_init_stream(ngx_connection_t
 }
 
 
+ngx_int_t
+ngx_http_v3_init(ngx_connection_t *c)
+{
+    ngx_http_v3_session_t     *h3c;
+    ngx_http_core_loc_conf_t  *clcf;
+
+    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http3 init");
+
+    ngx_reusable_connection(c, 1);
+
+    h3c = ngx_http_v3_get_session(c);
+    clcf = ngx_http_v3_get_module_loc_conf(c, ngx_http_core_module);
+    ngx_add_timer(&h3c->keepalive, clcf->keepalive_timeout);
+
+#if (NGX_HTTP_V3_HQ)
+    if (h3c->hq) {
+        return NGX_OK;
+    }
+#endif
+
+    return ngx_http_v3_send_settings(c);
+}
+
+
 void
 ngx_http_v3_shutdown(ngx_connection_t *c)
 {


From alx.manpages at gmail.com  Tue Aug 23 23:00:26 2022
From: alx.manpages at gmail.com (Alejandro Colomar)
Date: Wed, 24 Aug 2022 01:00:26 +0200
Subject: [WIP/RFC v4 0/5] Support abstract Unix sockets
In-Reply-To: <20220823211805.4509-1-alx.manpages@gmail.com>
References: <20220819142144.13915-1-alx.manpages@gmail.com>
 <20220823211805.4509-1-alx.manpages@gmail.com>
Message-ID: <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>

I'll resend v4 in a moment, since it didn't arrive to the list.

On 8/23/22 23:18, Alejandro Colomar wrote:
> Hi,
> 
> Patch set ready for review.
> Both 'listen' and 'proxy_pass' work fine, with the only caveat
> that writing a NUL character in the nginx.conf file is not the
> friendliest interface.  I will add a few more patches to make '@'
> a synonym for NUL, as many other projects do.
> 
> However, this patch set can be applied on its own if you prefer
> to make them separate, since this one adds the feature in its
> simplest way, and the upcoming patches are just making it more
> user friendly.  Please advice on your preferred way of proceeding.
> 
> Cheers,
> 
> Alex
> 
> 
> Changes:
> 
> v2:
>   - Complete rewrite from scratch (v1 was a month ago).
> 
> v3:
>   - Fix bugs in listener socket [Andrew Clayton].
>   - This version of the patch set wasn't sent to the list.
> 
> v4:
>   - Reorder patches.
>   - Improve commit logs.
> 
> Alejandro Colomar (5):
>    Optimize string copy in Unix sockets.
>    Use a minimal socklen.
>    Don't add a trailing '\0' for abstract Unix domain sockets.
>    Log abstract Unix domain sockets with a leading '@'.
>    Don't try to chmod(2) abstract Unix domain sockets.
> 
>   src/core/ngx_connection.c | 28 ++++++++++++++++------------
>   src/core/ngx_inet.c       | 33 +++++++++++++++++++++++++++------
>   2 files changed, 43 insertions(+), 18 deletions(-)
> 

-- 
Alejandro Colomar
<http://www.alejandro-colomar.es/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220824/98115466/attachment.bin>

From alx.manpages at gmail.com  Tue Aug 23 23:02:34 2022
From: alx.manpages at gmail.com (Alejandro Colomar)
Date: Wed, 24 Aug 2022 01:02:34 +0200
Subject: [PATCH v4 RESEND 1/5] Optimize string copy in Unix sockets.
In-Reply-To: <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
References: <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
Message-ID: <20220823230237.7194-1-alx.manpages@gmail.com>

ngx_cpystrn() checks at every character if it is '\0' after
copying it.  We don't really need that check for copying the
sun_path, since it's unlikely that the path contains a null
character.  And even if that happened, the kernel will ignore any
trailing bytes after the first null character, so we can safely
copy things like "foo\0bar", and the kernel will interpret it as
if we had passed just "foo".

memcpy(3) will usually be the fastest copy method for the system,
so it optimizes a little bit.

But the main reason for this change is not really optimizing, but
instead it's for adding support for abstract Unix domain sockets,
which use null characters as the first character in the string,
and possibly also use it in the middle of the string.

Signed-off-by: Alejandro Colomar <alx.manpages at gmail.com>
Cc: Andrew Clayton <andrew at digital-domain.net>
Cc: Bjornar Ness <bjornar.ness at gmail.com>
---
 src/core/ngx_inet.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/src/core/ngx_inet.c b/src/core/ngx_inet.c
index 4228504a..2b062b88 100644
--- a/src/core/ngx_inet.c
+++ b/src/core/ngx_inet.c
@@ -744,7 +744,8 @@ ngx_parse_unix_domain_url(ngx_pool_t *pool, ngx_url_t *u)
     u->socklen = sizeof(struct sockaddr_un);
     saun = (struct sockaddr_un *) &u->sockaddr;
     saun->sun_family = AF_UNIX;
-    (void) ngx_cpystrn((u_char *) saun->sun_path, path, len);
+    memcpy(saun->sun_path, path, len - 1);
+    saun->sun_path[len] = '\0';
 
     u->addrs = ngx_pcalloc(pool, sizeof(ngx_addr_t));
     if (u->addrs == NULL) {
@@ -760,7 +761,8 @@ ngx_parse_unix_domain_url(ngx_pool_t *pool, ngx_url_t *u)
     u->naddrs = 1;
 
     saun->sun_family = AF_UNIX;
-    (void) ngx_cpystrn((u_char *) saun->sun_path, path, len);
+    memcpy(saun->sun_path, path, len - 1);
+    saun->sun_path[len] = '\0';
 
     u->addrs[0].sockaddr = (struct sockaddr *) saun;
     u->addrs[0].socklen = sizeof(struct sockaddr_un);
-- 
2.37.2


From alx.manpages at gmail.com  Tue Aug 23 23:02:35 2022
From: alx.manpages at gmail.com (Alejandro Colomar)
Date: Wed, 24 Aug 2022 01:02:35 +0200
Subject: [PATCH v4 RESEND 2/5] Use a minimal socklen.
In-Reply-To: <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
References: <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
Message-ID: <20220823230237.7194-2-alx.manpages@gmail.com>

Instead of using a socklen that is the size of the structure,
limit the value to the part of the structure that is really used.

This is necessary for supporting abstract Unix domain sockets,
since trailing null bytes are part of the socket name, and having
a socket name with null bytes in it makes it basically unusable,
since most tools (e.g., curl(1)) can't refer to them.

Signed-off-by: Alejandro Colomar <alx.manpages at gmail.com>
Cc: Andrew Clayton <andrew at digital-domain.net>
Cc: Bjornar Ness <bjornar.ness at gmail.com>
---
 src/core/ngx_inet.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/core/ngx_inet.c b/src/core/ngx_inet.c
index 2b062b88..0be875a3 100644
--- a/src/core/ngx_inet.c
+++ b/src/core/ngx_inet.c
@@ -741,7 +741,7 @@ ngx_parse_unix_domain_url(ngx_pool_t *pool, ngx_url_t *u)
         return NGX_ERROR;
     }
 
-    u->socklen = sizeof(struct sockaddr_un);
+    u->socklen = offsetof(struct sockaddr_un, sun_path) + len;
     saun = (struct sockaddr_un *) &u->sockaddr;
     saun->sun_family = AF_UNIX;
     memcpy(saun->sun_path, path, len - 1);
@@ -765,7 +765,7 @@ ngx_parse_unix_domain_url(ngx_pool_t *pool, ngx_url_t *u)
     saun->sun_path[len] = '\0';
 
     u->addrs[0].sockaddr = (struct sockaddr *) saun;
-    u->addrs[0].socklen = sizeof(struct sockaddr_un);
+    u->addrs[0].socklen = offsetof(struct sockaddr_un, sun_path) + len;
     u->addrs[0].name.len = len + 4;
     u->addrs[0].name.data = u->url.data;
 
-- 
2.37.2


From alx.manpages at gmail.com  Tue Aug 23 23:02:36 2022
From: alx.manpages at gmail.com (Alejandro Colomar)
Date: Wed, 24 Aug 2022 01:02:36 +0200
Subject: [PATCH v4 RESEND 3/5] Don't add a trailing '\0' for abstract Unix domain sockets.
In-Reply-To: <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
References: <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
Message-ID: <20220823230237.7194-3-alx.manpages@gmail.com>

Since '\0' doesn't have a special meaning for abstract Unix domain
sockets, the '\0' would be part of the name.  This makes it hard
to use tools that can handle abstract sockets, such as
curl(1) --abstract-unix-sock, since it's not possible to embed a
'\0' in a string passed to execve(2).

Signed-off-by: Alejandro Colomar <alx.manpages at gmail.com>
Cc: Andrew Clayton <andrew at digital-domain.net>
Cc: Bjornar Ness <bjornar.ness at gmail.com>
---
 src/core/ngx_inet.c | 24 +++++++++++++++++++-----
 1 file changed, 19 insertions(+), 5 deletions(-)

diff --git a/src/core/ngx_inet.c b/src/core/ngx_inet.c
index 0be875a3..4a455e5b 100644
--- a/src/core/ngx_inet.c
+++ b/src/core/ngx_inet.c
@@ -745,7 +745,14 @@ ngx_parse_unix_domain_url(ngx_pool_t *pool, ngx_url_t *u)
     saun = (struct sockaddr_un *) &u->sockaddr;
     saun->sun_family = AF_UNIX;
     memcpy(saun->sun_path, path, len - 1);
-    saun->sun_path[len] = '\0';
+    switch (path[0]) {
+    case '\0':
+        u->socklen--;
+        break;
+    default:
+        saun->sun_path[len] = '\0';
+        break;
+    }
 
     u->addrs = ngx_pcalloc(pool, sizeof(ngx_addr_t));
     if (u->addrs == NULL) {
@@ -760,15 +767,22 @@ ngx_parse_unix_domain_url(ngx_pool_t *pool, ngx_url_t *u)
     u->family = AF_UNIX;
     u->naddrs = 1;
 
-    saun->sun_family = AF_UNIX;
-    memcpy(saun->sun_path, path, len - 1);
-    saun->sun_path[len] = '\0';
-
     u->addrs[0].sockaddr = (struct sockaddr *) saun;
     u->addrs[0].socklen = offsetof(struct sockaddr_un, sun_path) + len;
     u->addrs[0].name.len = len + 4;
     u->addrs[0].name.data = u->url.data;
 
+    saun->sun_family = AF_UNIX;
+    memcpy(saun->sun_path, path, len - 1);
+    switch (path[0]) {
+    case '\0':
+        u->addrs[0].socklen--;
+        break;
+    default:
+        saun->sun_path[len] = '\0';
+        break;
+    }
+
     return NGX_OK;
 
 #else
-- 
2.37.2


From alx.manpages at gmail.com  Tue Aug 23 23:02:37 2022
From: alx.manpages at gmail.com (Alejandro Colomar)
Date: Wed, 24 Aug 2022 01:02:37 +0200
Subject: [PATCH v4 RESEND 4/5] Log abstract Unix domain sockets with a leading '@'.
In-Reply-To: <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
References: <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
Message-ID: <20220823230237.7194-4-alx.manpages@gmail.com>

Translate the leading NUL (aka ^@, '\0') of abstract Unix sockets
into @ for user visible strings, since @ is readable by all tools.

Co-developed-by: Andrew Clayton <a.clayton at f5.com>
Signed-off-by: Andrew Clayton <a.clayton at f5.com>
Signed-off-by: Alejandro Colomar <alx.manpages at gmail.com>
Cc: Bjornar Ness <bjornar.ness at gmail.com>
---
 src/core/ngx_inet.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/src/core/ngx_inet.c b/src/core/ngx_inet.c
index 4a455e5b..bf1799d4 100644
--- a/src/core/ngx_inet.c
+++ b/src/core/ngx_inet.c
@@ -244,6 +244,11 @@ ngx_sock_ntop(struct sockaddr *sa, socklen_t socklen, u_char *text, size_t len,
         if (socklen <= (socklen_t) offsetof(struct sockaddr_un, sun_path)) {
             p = ngx_snprintf(text, len, "unix:%Z");
 
+        } else if (saun->sun_path[0] == '\0') {
+            n = strnlen(&saun->sun_path[1],
+                        socklen - offsetof(struct sockaddr_un, sun_path) - 1);
+            p = ngx_snprintf(text, len, "unix:@%*s%Z", n, &saun->sun_path[1]);
+
         } else {
             n = ngx_strnlen((u_char *) saun->sun_path,
                             socklen - offsetof(struct sockaddr_un, sun_path));
-- 
2.37.2


From alx.manpages at gmail.com  Tue Aug 23 23:02:38 2022
From: alx.manpages at gmail.com (Alejandro Colomar)
Date: Wed, 24 Aug 2022 01:02:38 +0200
Subject: [PATCH v4 RESEND 5/5] Don't try to chmod(2) abstract Unix domain sockets.
In-Reply-To: <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
References: <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
Message-ID: <20220823230237.7194-5-alx.manpages@gmail.com>

Don't try to handle Unix sockets starting with a '\0' as if they
were files, since they represent abstract sockets.

As we're using the "pretty" name generated by ngx_sock_ntop(),
which translates the '\0' into '@', test for '@'.

Co-developed-by: Andrew Clayton <a.clayton at f5.com>
Signed-off-by: Andrew Clayton <a.clayton at f5.com>
Signed-off-by: Alejandro Colomar <alx.manpages at gmail.com>
Cc: Bjornar Ness <bjornar.ness at gmail.com>
---
 src/core/ngx_connection.c | 28 ++++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/src/core/ngx_connection.c b/src/core/ngx_connection.c
index fe729a78..1d041704 100644
--- a/src/core/ngx_connection.c
+++ b/src/core/ngx_connection.c
@@ -633,17 +633,19 @@ ngx_open_listening_sockets(ngx_cycle_t *cycle)
                 u_char  *name;
 
                 name = ls[i].addr_text.data + sizeof("unix:") - 1;
-                mode = (S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH);
+                if (name[0] != '@') {
+                    mode = (S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH);
 
-                if (chmod((char *) name, mode) == -1) {
-                    ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_errno,
-                                  "chmod() \"%s\" failed", name);
-                }
-
-                if (ngx_test_config) {
-                    if (ngx_delete_file(name) == NGX_FILE_ERROR) {
+                    if (chmod((char *) name, mode) == -1) {
                         ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_errno,
-                                      ngx_delete_file_n " %s failed", name);
+                                      "chmod() \"%s\" failed", name);
+                    }
+
+                    if (ngx_test_config) {
+                        if (ngx_delete_file(name) == NGX_FILE_ERROR) {
+                            ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_errno,
+                                          ngx_delete_file_n " %s failed", name);
+                        }
                     }
                 }
             }
@@ -1078,9 +1080,11 @@ ngx_close_listening_sockets(ngx_cycle_t *cycle)
         {
             u_char *name = ls[i].addr_text.data + sizeof("unix:") - 1;
 
-            if (ngx_delete_file(name) == NGX_FILE_ERROR) {
-                ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_socket_errno,
-                              ngx_delete_file_n " %s failed", name);
+            if (name[0] != '@') {
+                if (ngx_delete_file(name) == NGX_FILE_ERROR) {
+                    ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_socket_errno,
+                                  ngx_delete_file_n " %s failed", name);
+                }
             }
         }
 
-- 
2.37.2


From alx.manpages at gmail.com  Tue Aug 23 23:04:45 2022
From: alx.manpages at gmail.com (Alejandro Colomar)
Date: Wed, 24 Aug 2022 01:04:45 +0200
Subject: [WIP/RFC v4 0/5] Support abstract Unix sockets
In-Reply-To: <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
References: <20220819142144.13915-1-alx.manpages@gmail.com>
 <20220823211805.4509-1-alx.manpages@gmail.com>
 <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
Message-ID: <cdb7c06d-0e4b-67bf-cbe1-5ee0ff81785a@gmail.com>

Also, this is missing some context from v2, which I sent a week ago and 
also didn't reach the list:

On 8/19/22 16:21, Alejandro Colomar wrote:
 > Hi,
 >
 > This patch set intends to add support for abstract Unix domain
 > sockets for both 'listen' and 'proxy_pass'.
 >
 > In 'listen' I still get a chmod(2) error, which I'll fix soon.
 > That's not a problem.
 >
 > But 'proxy_pass' already works, so I'll send this initial patch
 > set for some discussion.
 >
 > Previous to this patch set, trying to use abstract sockets in
 > 'proxy_pass' could theoretically work.  One just needs to find a
 > way to embed a null character in <nginx.conf>.  That's trivial
 > with vim(1): Ctrl+v Ctrl+j
 >
 > That would make nginx listen to an abstract socket with a name
 > starting with the string stated in the config file, but with a lot
 > of trailing null characters, which made it unusable for most use
 > cases.  But it was already possible theoretically.
 >
 > This patch set makes it more usable, by removing those trailing
 > null characters.  I also pretend to makes it even simpler, by
 > allowing one to write a @ in the config file to mean ^@.  That's
 > how most other projects do it, to avoid using an escape sequence
 > for the null character.  But I haven't implemented that yet.
 >
 > To test it, I run an instance of nginx Unit that listens on 3
 > sockets.  One IP socket, one normal (fs) Unix socket, and one
 > abstract Unix socket.  Then nginx is configured to proxy_pass to
 > those sockets.
 >
 >
 > The Unit configuration for the tests is:
 >
 > $ cat abs_nginx.json
 >   {
 > 	"listeners": {
 > 		"unix:@unitd.static.sock": { "pass": "routes" },
 > 		"unix:/run/unit.static.sock": { "pass": "routes" },
 > 		"*:8080": { "pass": "routes" }
 > 	},
 > 	"routes": [{ "action": { "share": "/home/alx/srv/www/" } }]
 >   }
 >
 > The nginx configuration is the default one that comes with nginx
 > built from source, with minimal changes:
 >
 > $ cat nginx.conf
 >
 > worker_processes  1;
 >
 > events {
 >      worker_connections  1024;
 >   }
 >
 > http {
 >      include       mime.types;
 >      default_type  application/octet-stream;
 >
 >      sendfile        on;
 >
 >      keepalive_timeout  65;
 >
 >      server {
 >          #listen       unix:nginx-in;
 >          listen       80;
 >
 >          location /abs {
 >              proxy_pass   http://unix:^@unitd.static.sock;
 >              # replace that ^@ by the real NUL character.
 >          }
 >
 >          location /sock {
 >              proxy_pass   http://unix:/run/unit.static.sock;
 >          }
 >
 >          location /ip {
 >              proxy_pass   http://localhost:8080/;
 >          }
 >      }
 >
 >   }
 >
 >
 > And then the tests:
 >
 > $ curl localhost/ip
 > idx
 > $ curl localhost/sock
 > idx
 > $ curl localhost/abs
 > idx
 >
 >
 >
 > Cheers,
 >
 > Alex
 >
 >
 > Alejandro Colomar (5):
 >    Optimize string copy in Unix sockets.
 >    Use a minimal socklen.
 >    Support abstract Unix domain sockets.
 >    Log abstract Unix domain sockets with a leading '@'.
 >    Don't add a trailing '\0' for abstract Unix domain sockets.
 >
 >   src/core/ngx_connection.c | 28 ++++++++++++++++------------
 >   src/core/ngx_inet.c       | 33 +++++++++++++++++++++++++++------
 >   2 files changed, 43 insertions(+), 18 deletions(-)
 >
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220824/10b25a06/attachment.bin>

From xeioex at nginx.com  Wed Aug 24 03:17:37 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 24 Aug 2022 03:17:37 +0000
Subject: [njs] Stream: improved s.send() with async callbacks.
Message-ID: <hg.b33aae5e8dc6.1661311057.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/b33aae5e8dc6
branches:  
changeset: 1933:b33aae5e8dc6
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Aug 23 19:36:16 2022 -0700
description:
Stream: improved s.send() with async callbacks.

Previously, s.send() was a context dependant method because the
direction it was sending data to was determined by a callback (upstream
or downstream) it was called from.  This works for synchronous
callbacks it was originally designed, but fails with async functions
(e.g. ngx.fetch()).

The fix is to store the direction data was going to as a separate flag
which can be used by s.send().

diffstat:

 nginx/ngx_stream_js_module.c |  91 ++++++++++++++++++++++++++++++++++++++-----
 1 files changed, 79 insertions(+), 12 deletions(-)

diffs (186 lines):

diff -r 2f0056631e75 -r b33aae5e8dc6 nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c	Mon Aug 22 22:03:15 2022 -0700
+++ b/nginx/ngx_stream_js_module.c	Tue Aug 23 19:36:16 2022 -0700
@@ -83,6 +83,8 @@ static ngx_int_t ngx_stream_js_phase_han
     ngx_str_t *name);
 static ngx_int_t ngx_stream_js_body_filter(ngx_stream_session_t *s,
     ngx_chain_t *in, ngx_uint_t from_upstream);
+static ngx_int_t ngx_stream_js_next_filter(ngx_stream_session_t *s,
+    ngx_stream_js_ctx_t *ctx, ngx_chain_t *out, ngx_uint_t from_upstream);
 static ngx_int_t ngx_stream_js_variable_set(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_stream_js_variable_var(ngx_stream_session_t *s,
@@ -92,7 +94,8 @@ static void ngx_stream_js_drop_events(ng
 static void ngx_stream_js_cleanup(void *data);
 static void ngx_stream_js_cleanup_vm(void *data);
 static njs_int_t ngx_stream_js_run_event(ngx_stream_session_t *s,
-    ngx_stream_js_ctx_t *ctx, ngx_stream_js_ev_t *event);
+    ngx_stream_js_ctx_t *ctx, ngx_stream_js_ev_t *event,
+    ngx_uint_t from_upstream);
 static njs_vm_event_t *ngx_stream_js_event(ngx_stream_session_t *s,
     njs_str_t *event);
 
@@ -514,6 +517,17 @@ static njs_external_t  ngx_stream_js_ext
         }
     },
 
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("from_upstream"),
+        .enumerable = 1,
+        .u.property = {
+            .handler = ngx_js_ext_flags,
+            .magic16 = NGX_JS_BOOLEAN,
+            .magic32 = 0x00000002,
+        }
+    },
+
 };
 
 
@@ -620,7 +634,7 @@ ngx_stream_js_phase_handler(ngx_stream_s
         }
     }
 
-    ret = ngx_stream_js_run_event(s, ctx, &ctx->events[NGX_JS_EVENT_UPLOAD]);
+    ret = ngx_stream_js_run_event(s, ctx, &ctx->events[NGX_JS_EVENT_UPLOAD], 0);
     if (ret != NJS_OK) {
         ngx_js_retval(ctx->vm, NULL, &exception);
 
@@ -655,11 +669,11 @@ static ngx_int_t
 ngx_stream_js_body_filter(ngx_stream_session_t *s, ngx_chain_t *in,
     ngx_uint_t from_upstream)
 {
+    ngx_int_t                  rc;
     ngx_str_t                  exception;
     njs_int_t                  ret;
-    ngx_int_t                  rc;
-    ngx_chain_t               *out, *cl, **busy;
-    ngx_connection_t          *c, *dst;
+    ngx_chain_t               *out, *cl;
+    ngx_connection_t          *c;
     ngx_stream_js_ev_t        *event;
     ngx_stream_js_ctx_t       *ctx;
     ngx_stream_js_srv_conf_t  *jscf;
@@ -704,7 +718,7 @@ ngx_stream_js_body_filter(ngx_stream_ses
         event = ngx_stream_event(from_upstream);
 
         if (event->ev != NULL) {
-            ret = ngx_stream_js_run_event(s, ctx, event);
+            ret = ngx_stream_js_run_event(s, ctx, event, from_upstream);
             if (ret != NJS_OK) {
                 ngx_js_retval(ctx->vm, NULL, &exception);
 
@@ -731,8 +745,23 @@ ngx_stream_js_body_filter(ngx_stream_ses
         in = in->next;
     }
 
+    ctx->buf = NULL;
     *ctx->last_out = NULL;
 
+    return ngx_stream_js_next_filter(s, ctx, out, from_upstream);
+}
+
+
+static ngx_int_t
+ngx_stream_js_next_filter(ngx_stream_session_t *s, ngx_stream_js_ctx_t *ctx,
+     ngx_chain_t *out, ngx_uint_t from_upstream)
+{
+    ngx_int_t           rc;
+    ngx_chain_t       **busy;
+    ngx_connection_t   *c, *dst;
+
+    c = s->connection;
+
     if (from_upstream) {
         dst = c;
         busy = &ctx->downstream_busy;
@@ -946,7 +975,7 @@ ngx_stream_js_cleanup_vm(void *data)
 
 static njs_int_t
 ngx_stream_js_run_event(ngx_stream_session_t *s, ngx_stream_js_ctx_t *ctx,
-    ngx_stream_js_ev_t *event)
+    ngx_stream_js_ev_t *event, ngx_uint_t from_upstream)
 {
     size_t             len;
     u_char            *p;
@@ -980,7 +1009,7 @@ ngx_stream_js_run_event(ngx_stream_sessi
         return ret;
     }
 
-    flags = b && b->last_buf;
+    flags = from_upstream << 1 | (uintptr_t) (b && b->last_buf);
 
     ret = njs_vm_external_create(ctx->vm, njs_value_arg(&ctx->args[2]),
                        ngx_stream_js_session_flags_proto_id, (void *) flags, 0);
@@ -1249,6 +1278,7 @@ ngx_stream_js_ext_send(njs_vm_t *vm, njs
 
     static const njs_str_t last_key = njs_str("last");
     static const njs_str_t flush_key = njs_str("flush");
+    static const njs_str_t from_key = njs_str("from_upstream");
 
     s = njs_vm_external(vm, ngx_stream_js_session_proto_id,
                         njs_argument(args, 0));
@@ -1271,8 +1301,19 @@ ngx_stream_js_ext_send(njs_vm_t *vm, njs
         return NJS_ERROR;
     }
 
-    flush = ctx->buf->flush;
-    last_buf = ctx->buf->last_buf;
+    /*
+     * ctx->buf != NULL when s.send() is called while processing incoming
+     * data chunks, otherwise s.send() is called asynchronously
+     */
+
+    if (ctx->buf != NULL) {
+        flush = ctx->buf->flush;
+        last_buf = ctx->buf->last_buf;
+
+    } else {
+        flush = 0;
+        last_buf = 0;
+    }
 
     flags = njs_arg(args, nargs, 2);
 
@@ -1308,12 +1349,38 @@ ngx_stream_js_ext_send(njs_vm_t *vm, njs
     b->pos = b->start;
     b->last = b->end;
 
-    *ctx->last_out = cl;
-    ctx->last_out = &cl->next;
+    if (ctx->buf != NULL) {
+        *ctx->last_out = cl;
+        ctx->last_out = &cl->next;
+
+    } else {
+        if (!njs_value_is_object(flags)) {
+            goto exception;
+        }
+
+        value = njs_vm_object_prop(vm, flags, &from_key, &lvalue);
+        if (value == NULL) {
+            goto exception;
+        }
+
+        if (ngx_stream_js_next_filter(s, ctx, cl, njs_value_bool(value))
+            == NGX_ERROR)
+        {
+            njs_vm_error(vm, "ngx_stream_js_next_filter() failed");
+            return NJS_ERROR;
+        }
+    }
 
     njs_value_undefined_set(njs_vm_retval(vm));
 
     return NJS_OK;
+
+exception:
+
+    njs_vm_error(vm, "\"from_upstream\" flag is expected when"
+                "called asynchronously");
+
+    return NJS_ERROR;
 }
 
 


From alx.manpages at gmail.com  Wed Aug 24 09:57:35 2022
From: alx.manpages at gmail.com (Alejandro Colomar)
Date: Wed, 24 Aug 2022 11:57:35 +0200
Subject: [WIP/RFC v4 0/5] Support abstract Unix sockets
In-Reply-To: <cdb7c06d-0e4b-67bf-cbe1-5ee0ff81785a@gmail.com>
References: <20220819142144.13915-1-alx.manpages@gmail.com>
 <20220823211805.4509-1-alx.manpages@gmail.com>
 <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
 <cdb7c06d-0e4b-67bf-cbe1-5ee0ff81785a@gmail.com>
Message-ID: <cfde8b00-78d3-fa5d-648d-33e30e233d70@gmail.com>

And of course this one didn't make it to the list either.  See this mail 
for knowing how it works.

Cheers,

Alex

On 8/23/22 23:56, Alejandro Colomar wrote:
 > Hi,
 >
 > On 8/23/22 23:18, Alejandro Colomar wrote:
 >> Alejandro Colomar (5):
 >>    Optimize string copy in Unix sockets.
 >>    Use a minimal socklen.
 >>    Don't add a trailing '\0' for abstract Unix domain sockets.
 >>    Log abstract Unix domain sockets with a leading '@'.
 >>    Don't try to chmod(2) abstract Unix domain sockets.
 >>
 >>   src/core/ngx_connection.c | 28 ++++++++++++++++------------
 >>   src/core/ngx_inet.c       | 33 +++++++++++++++++++++++++++------
 >>   2 files changed, 43 insertions(+), 18 deletions(-)
 >>
 >
 > Here's how I tested the feature.  It might help you understand how to
 > use it, or help review it.
 >
 >
 > I put a program listening on the sockets: IPv4, Unix, and abstract Unix.
 >   It was NGINX Unit built from source (since the patch set for
 > supporting abstract sockets there was merged a few days ago).
 >
 > $ cat ~/etc/unitd/abs_nginx.json
 >   {
 >      "listeners": {
 >          "unix:@unitd.static.sock": { "pass": "routes" },
 >          "unix:/run/unit.static.sock": { "pass": "routes" },
 >          "*:8080": { "pass": "routes" }
 >      },
 >      "routes": [{
 >          "action": {
 >              "share": "/home/alx/srv/www/"
 >          }
 >      }]
 >   }
 > $ sudo curl -X PUT -d @~/etc/unitd/abs_nginx.json --unix-sock
 > /opt/local/unit/control.unit.sock localhost/config/
 >   {
 >      "success": "Reconfiguration done."
 >   }
 >
 >
 > $ # Testing unitd:
 > $
 > $ curl localhost:8080
 > idx
 > $ curl --unix-sock /run/unit.static.sock localhost
 > idx
 > $ curl --abstract-unix-sock unitd.static.sock localhost
 > idx
 >
 >
 > The I configure nginx to talk to unit through the sockets, and also have
 > an abstract socket for listening:
 >
 >
 > $ cat /usr/local/nginx/conf/nginx.conf
 >   worker_processes  1;
 >
 >   events {
 >      worker_connections  1024;
 >   }
 >
 >   http {
 >      default_type  application/octet-stream;
 >      sendfile        on;
 >      keepalive_timeout  65;
 >
 >      server {
 >          # replace NUL by the real null character with Ctrl+v Ctrl+j
 >          listen       unix:NULnginx-in;
 >          listen       80;
 >
 >          location /abs {
 >              # replace NUL by the real null character with Ctrl+v Ctrl+j
 >              proxy_pass   http://unix:NULunitd.static.sock;
 >          }
 >
 >          location /sock {
 >              proxy_pass   http://unix:/run/unit.static.sock;
 >          }
 >
 >          location /ip {
 >              proxy_pass   http://localhost:8080/;
 >          }
 >      }
 >   }
 > $ sudo /usr/local/nginx/sbin/nginx
 >
 >
 > $ # Testing nginx:
 > $
 > $ ss -l | grep nginx-in
 > u_str LISTEN 0      511                                       @nginx-in
 > 23314                           * 0
 > $ curl --abstract-unix-sock nginx-in localhost/non-sock
 > idx
 > $ curl --abstract-unix-sock nginx-in localhost/sock
 > idx
 > $ curl --abstract-unix-sock nginx-in localhost/abs
 > idx
 >
 >
 >
 > Cheers,
 >
 > Alex
 >
 >


-- 
Alejandro Colomar
<http://www.alejandro-colomar.es/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220824/d5d07feb/attachment.bin>

From alx.manpages at gmail.com  Wed Aug 24 11:33:20 2022
From: alx.manpages at gmail.com (Alejandro Colomar)
Date: Wed, 24 Aug 2022 13:33:20 +0200
Subject: [WIP/RFC v4 0/5] Support abstract Unix sockets
In-Reply-To: <cfde8b00-78d3-fa5d-648d-33e30e233d70@gmail.com>
References: <20220819142144.13915-1-alx.manpages@gmail.com>
 <20220823211805.4509-1-alx.manpages@gmail.com>
 <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
 <cdb7c06d-0e4b-67bf-cbe1-5ee0ff81785a@gmail.com>
 <cfde8b00-78d3-fa5d-648d-33e30e233d70@gmail.com>
Message-ID: <df7abff6-20e5-c228-9113-67c8c518aa97@gmail.com>



On 8/24/22 11:57, Alejandro Colomar wrote:
> And of course this one didn't make it to the list either.  See this mail 
> for knowing how it works.
> 
> Cheers,
> 
> Alex
> 
> On 8/23/22 23:56, Alejandro Colomar wrote:
>  > Hi,
>  >
>  > On 8/23/22 23:18, Alejandro Colomar wrote:
>  >> Alejandro Colomar (5):
>  >>    Optimize string copy in Unix sockets.
>  >>    Use a minimal socklen.
>  >>    Don't add a trailing '\0' for abstract Unix domain sockets.
>  >>    Log abstract Unix domain sockets with a leading '@'.
>  >>    Don't try to chmod(2) abstract Unix domain sockets.
>  >>
>  >>   src/core/ngx_connection.c | 28 ++++++++++++++++------------
>  >>   src/core/ngx_inet.c       | 33 +++++++++++++++++++++++++++------
>  >>   2 files changed, 43 insertions(+), 18 deletions(-)
>  >>
>  >
>  > Here's how I tested the feature.  It might help you understand how to
>  > use it, or help review it.
>  >
>  >
>  > I put a program listening on the sockets: IPv4, Unix, and abstract Unix.
>  >   It was NGINX Unit built from source (since the patch set for
>  > supporting abstract sockets there was merged a few days ago).
>  >
>  > $ cat ~/etc/unitd/abs_nginx.json
>  >   {
>  >      "listeners": {
>  >          "unix:@unitd.static.sock": { "pass": "routes" },
>  >          "unix:/run/unit.static.sock": { "pass": "routes" },
>  >          "*:8080": { "pass": "routes" }
>  >      },
>  >      "routes": [{
>  >          "action": {
>  >              "share": "/home/alx/srv/www/"
>  >          }
>  >      }]
>  >   }
>  > $ sudo curl -X PUT -d @~/etc/unitd/abs_nginx.json --unix-sock
>  > /opt/local/unit/control.unit.sock localhost/config/
>  >   {
>  >      "success": "Reconfiguration done."
>  >   }
>  >
>  >
>  > $ # Testing unitd:
>  > $
>  > $ curl localhost:8080
>  > idx
>  > $ curl --unix-sock /run/unit.static.sock localhost
>  > idx
>  > $ curl --abstract-unix-sock unitd.static.sock localhost
>  > idx
>  >
>  >
>  > The I configure nginx to talk to unit through the sockets, and also have
>  > an abstract socket for listening:
>  >
>  >
>  > $ cat /usr/local/nginx/conf/nginx.conf
>  >   worker_processes  1;
>  >
>  >   events {
>  >      worker_connections  1024;
>  >   }
>  >
>  >   http {
>  >      default_type  application/octet-stream;
>  >      sendfile        on;
>  >      keepalive_timeout  65;
>  >
>  >      server {
>  >          # replace NUL by the real null character with Ctrl+v Ctrl+j
>  >          listen       unix:NULnginx-in;
>  >          listen       80;
>  >
>  >          location /abs {
>  >              # replace NUL by the real null character with Ctrl+v Ctrl+j
>  >              proxy_pass   http://unix:NULunitd.static.sock;
>  >          }
>  >
>  >          location /sock {
>  >              proxy_pass   http://unix:/run/unit.static.sock;
>  >          }
>  >
>  >          location /ip {
>  >              proxy_pass   http://localhost:8080/;
>  >          }
>  >      }
>  >   }
>  > $ sudo /usr/local/nginx/sbin/nginx
>  >
>  >
>  > $ # Testing nginx:
>  > $
>  > $ ss -l | grep nginx-in
>  > u_str LISTEN 0      511                                       @nginx-in
>  > 23314                           * 0
>  > $ curl --abstract-unix-sock nginx-in localhost/non-sock

s/non-sock/ip/

>  > idx
>  > $ curl --abstract-unix-sock nginx-in localhost/sock
>  > idx
>  > $ curl --abstract-unix-sock nginx-in localhost/abs
>  > idx
>  >
>  >
>  >
>  > Cheers,
>  >
>  > Alex
>  >
>  >
> 
> 

-- 
Alejandro Colomar
<http://www.alejandro-colomar.es/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220824/a63e9a61/attachment.bin>

From mdounin at mdounin.ru  Wed Aug 24 14:58:29 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 24 Aug 2022 17:58:29 +0300
Subject: [WIP/RFC v4 0/5] Support abstract Unix sockets
In-Reply-To: <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
References: <20220819142144.13915-1-alx.manpages@gmail.com>
 <20220823211805.4509-1-alx.manpages@gmail.com>
 <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
Message-ID: <YwY8lSobLyraTzvr@mdounin.ru>

Hello!

On Wed, Aug 24, 2022 at 01:00:26AM +0200, Alejandro Colomar wrote:

> I'll resend v4 in a moment, since it didn't arrive to the list.
> 
> On 8/23/22 23:18, Alejandro Colomar wrote:
> > Hi,
> > 
> > Patch set ready for review.
> > Both 'listen' and 'proxy_pass' work fine, with the only caveat
> > that writing a NUL character in the nginx.conf file is not the
> > friendliest interface.  I will add a few more patches to make '@'
> > a synonym for NUL, as many other projects do.
> > 
> > However, this patch set can be applied on its own if you prefer
> > to make them separate, since this one adds the feature in its
> > simplest way, and the upcoming patches are just making it more
> > user friendly.  Please advice on your preferred way of proceeding.

A while ago a working implementation of Linux abstract namespace 
sockets was already posted in this mailing list, see here:

https://mailman.nginx.org/pipermail/nginx-devel/2016-October/008878.html

It received zero interest and no tests/reviews, hence there are 
no further work on this.

If you think that Linux abstract namespace support is needed, you 
may want to start with basic things, notably: 

> Try to make it clear why the suggested change is needed, and 
> provide a use case, if possible.

(Quote from http://nginx.org/en/docs/contributing_changes.html)

That is, please clarify how do you expect to use it, and why 
existing options does not work for you.

-- 
Maxim Dounin
http://mdounin.ru/


From alx.manpages at gmail.com  Wed Aug 24 15:21:48 2022
From: alx.manpages at gmail.com (Alejandro Colomar)
Date: Wed, 24 Aug 2022 17:21:48 +0200
Subject: [WIP/RFC v4 0/5] Support abstract Unix sockets
In-Reply-To: <YwY8lSobLyraTzvr@mdounin.ru>
References: <20220819142144.13915-1-alx.manpages@gmail.com>
 <20220823211805.4509-1-alx.manpages@gmail.com>
 <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
 <YwY8lSobLyraTzvr@mdounin.ru>
Message-ID: <bbe31ffd-8cdb-f30c-5a4f-32bdf89b940e@gmail.com>

Hi Maxim,

On 8/24/22 16:58, Maxim Dounin wrote:
> Hello!
> 
> On Wed, Aug 24, 2022 at 01:00:26AM +0200, Alejandro Colomar wrote:
> 
>> I'll resend v4 in a moment, since it didn't arrive to the list.
>>
>> On 8/23/22 23:18, Alejandro Colomar wrote:
>>> Hi,
>>>
>>> Patch set ready for review.
>>> Both 'listen' and 'proxy_pass' work fine, with the only caveat
>>> that writing a NUL character in the nginx.conf file is not the
>>> friendliest interface.  I will add a few more patches to make '@'
>>> a synonym for NUL, as many other projects do.
>>>
>>> However, this patch set can be applied on its own if you prefer
>>> to make them separate, since this one adds the feature in its
>>> simplest way, and the upcoming patches are just making it more
>>> user friendly.  Please advice on your preferred way of proceeding.
> 
> A while ago a working implementation of Linux abstract namespace
> sockets was already posted in this mailing list, see here:
> 
> https://mailman.nginx.org/pipermail/nginx-devel/2016-October/008878.html
> 
> It received zero interest and no tests/reviews, hence there are
> no further work on this.

I didn't know about that one, but found two others (one from 2012 and 
another from 2019).  I based my v1 on the 2019 one, but then decided 
that I didn't like it and rewrote it from scratch.

> 
> If you think that Linux abstract namespace support is needed, you
> may want to start with basic things, notably:
> 
>> Try to make it clear why the suggested change is needed, and
>> provide a use case, if possible.

Yeah, that's the most important thing.  I mentioned it in v1, which also 
didn't reach the mailing list :(.  I'll restate here.

For having nginx as a reverse proxy in front of a server, such as unit, 
it's faster to communicate through Unix sockets (UDS), rather than TCP 
(localhost).  Nginx already supports UDS.  But UDS has a problem: the 
kernel creates a file in the fs, and it's not always trivial to clean up 
those files.  Then, if the application is restarted, there's no 
SO_REUSEADDR to allow reusing the socket file, so the application will 
just fail.

This happens in nginx Unit, which creates listener sockets from a 
privileged thread, and then uses them from unprivileged threads.  When 
the unprivileged thread stops using the socket, it can't remove the 
file, and doing so would require huge complexity to implement.  It's 
easier to just tell the kernel we want an abstract UDS (AUDS), so that 
there's no file at all.  Then if the user restarts Unit, it'll be able 
to recreate the AUDS.

A user reported this problem with normal UDS and we concluded that the 
easiest solution would be to add support for AUDS.  His set-up is a 
kubernetes pod, where a container uses nginx and another container uses 
Unit.  Communicating through an AUDS would be trivial and fast.

> 
> (Quote from http://nginx.org/en/docs/contributing_changes.html)
> 
> That is, please clarify how do you expect to use it, and why
> existing options does not work for you.

About how I expect to use it, sorry for the mess of emails; I sent one 
email with a complete example, but the list rejected it, and so I 
replied to it so that the list received it, but now this whole email 
thread is a bit messy.  Maybe I should send a clean v5 that is 
self-contained.

To simplify, the usage I want to support is:

listen unix:@foo

and

proxy_pass http://unix:@bar


But for now in v4 I have an intermediate solution that is exactly as 
above but replacing the @ by the NUL character.  When I add support for 
@, NUL will continue working, but I wouldn't document that; it's just an 
implementation detail.

Cheers,

Alex

> 

-- 
Alejandro Colomar
<http://www.alejandro-colomar.es/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220824/63d1fa49/attachment.bin>

From mdounin at mdounin.ru  Wed Aug 24 16:00:44 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 24 Aug 2022 19:00:44 +0300
Subject: [WIP/RFC v4 0/5] Support abstract Unix sockets
In-Reply-To: <bbe31ffd-8cdb-f30c-5a4f-32bdf89b940e@gmail.com>
References: <20220819142144.13915-1-alx.manpages@gmail.com>
 <20220823211805.4509-1-alx.manpages@gmail.com>
 <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
 <YwY8lSobLyraTzvr@mdounin.ru>
 <bbe31ffd-8cdb-f30c-5a4f-32bdf89b940e@gmail.com>
Message-ID: <YwZLLEPd6T9Npzt8@mdounin.ru>

Hello!

On Wed, Aug 24, 2022 at 05:21:48PM +0200, Alejandro Colomar wrote:

> Hi Maxim,
> 
> On 8/24/22 16:58, Maxim Dounin wrote:
> > Hello!
> > 
> > On Wed, Aug 24, 2022 at 01:00:26AM +0200, Alejandro Colomar wrote:
> > 
> >> I'll resend v4 in a moment, since it didn't arrive to the list.
> >>
> >> On 8/23/22 23:18, Alejandro Colomar wrote:
> >>> Hi,
> >>>
> >>> Patch set ready for review.
> >>> Both 'listen' and 'proxy_pass' work fine, with the only caveat
> >>> that writing a NUL character in the nginx.conf file is not the
> >>> friendliest interface.  I will add a few more patches to make '@'
> >>> a synonym for NUL, as many other projects do.
> >>>
> >>> However, this patch set can be applied on its own if you prefer
> >>> to make them separate, since this one adds the feature in its
> >>> simplest way, and the upcoming patches are just making it more
> >>> user friendly.  Please advice on your preferred way of proceeding.
> > 
> > A while ago a working implementation of Linux abstract namespace
> > sockets was already posted in this mailing list, see here:
> > 
> > https://mailman.nginx.org/pipermail/nginx-devel/2016-October/008878.html
> > 
> > It received zero interest and no tests/reviews, hence there are
> > no further work on this.
> 
> I didn't know about that one, but found two others (one from 2012 and 
> another from 2019).  I based my v1 on the 2019 one, but then decided 
> that I didn't like it and rewrote it from scratch.
> 
> > 
> > If you think that Linux abstract namespace support is needed, you
> > may want to start with basic things, notably:
> > 
> >> Try to make it clear why the suggested change is needed, and
> >> provide a use case, if possible.
> 
> Yeah, that's the most important thing.  I mentioned it in v1, which also 
> didn't reach the mailing list :(.  I'll restate here.
> 
> For having nginx as a reverse proxy in front of a server, such as unit, 
> it's faster to communicate through Unix sockets (UDS), rather than TCP 
> (localhost).

I would rather say "it's a common misconception that it's faster 
to communicate through Unix sockets".  While it Unix sockets can 
be beneficial for some microbenchmarks, in most production setups 
it makes no difference, yet used to introduce various issues.

> Nginx already supports UDS.  But UDS has a problem: the 
> kernel creates a file in the fs, and it's not always trivial to clean up 
> those files.  Then, if the application is restarted, there's no 
> SO_REUSEADDR to allow reusing the socket file, so the application will 
> just fail.
> 
> This happens in nginx Unit, which creates listener sockets from a 
> privileged thread, and then uses them from unprivileged threads.  When 
> the unprivileged thread stops using the socket, it can't remove the 
> file, and doing so would require huge complexity to implement.  It's 
> easier to just tell the kernel we want an abstract UDS (AUDS), so that 
> there's no file at all.  Then if the user restarts Unit, it'll be able 
> to recreate the AUDS.
> 
> A user reported this problem with normal UDS and we concluded that the 
> easiest solution would be to add support for AUDS.  His set-up is a 
> kubernetes pod, where a container uses nginx and another container uses 
> Unit.  Communicating through an AUDS would be trivial and fast.

So, you are trying to implement abstract namespace sockets as a 
bandaid for Unit bug, which is not able to properly remove Unix 
sockets in some cases and fails to restart, correct?  This does 
not look like a valid reason to me, especially given that a) 
abstract namespace sockets are Linux-only, and b) there are 
multiple approaches to filing the abstract socket address.

-- 
Maxim Dounin
http://mdounin.ru/


From serg.brester at sebres.de  Wed Aug 24 17:38:02 2022
From: serg.brester at sebres.de (Dipl. Ing. Sergey Brester)
Date: Wed, 24 Aug 2022 19:38:02 +0200
Subject: Precedence return directive and nested locations
Message-ID: <1bf5c8eb861a070e673fe2d71ec5b3dd@sebres.de>

 

Hi, 

it seems that the question of precedence of non-conditional _return_
directive vs nested _location_s is not really clear,
or rather some constellations (like fallback) are impossible or else the
configuration may look weird.

For instance: 

server {
 server_name ...;

 location ~ ^/(some-uri|other-uri) {
 return 200 ...;
 }

 # fallback for any not matched uri:
 return 444;
} 

will ALWAYS return with 444 for that server no matter whether it matches
the location(s) or doesn't.
Basically the locations are totally superfluous here (despite
specified), considering the return is non-conditional. 

Sure, the documentation says "Stops processing and returns the specified
_code_ to a client.",
but normally the nested locations (if matched) have always higher
precedence over anything else
in the parent location. 

Furthermore the docu of ngx_http_rewrite_module [1] says at begin: 

 	* the directives of this module specified on the server [2] level are
executed sequentially;

	* repeatedly: 

 	* a location [3] is searched based on a request URI;
 	* the directives of this module specified inside the found location
are executed sequentially;

What is a bit ambiguous. But if it is to understand directly - it is
clear that a return directive at server level
bypasses all other locations (sequence in current level before its
locations).
Just it makes hardly possible (up to impossible depending on location
tree) to use return directive for a fallback case. 

To implement the fallback return, one can surely pack this into a
location like: 

location ~ .* { return 444; } 
Just it is a bit ugly in my opinion, let alone it would quasi disallow
the usage of longest match prefix locations, 
because they work only if no match with a regular expression is found
(then the configuration of the 
prefix location remembered earlier is used). 

So I assume:

	* either it is a lack of documentation (and it must get a hint about
the precedence) 
and/or still better a description how one could achieve such a
"fallback" return (location or whatever).
(but do we have at all a possibility to specify such proper fallback
location matching at end, 
if nothing else matched?)

	* or (improbably) this is a flaw and must be "fixed" or enhanced rather
for a non-conditional return case
(unsure it wouldn't introduce some compat-issue); 

Any thoughts? 

Regards,
Sergey. 

Links:
------
[1] http://nginx.org/en/docs/http/ngx_http_rewrite_module.html
[2] http://nginx.org/en/docs/http/ngx_http_core_module.html#server
[3] http://nginx.org/en/docs/http/ngx_http_core_module.html#location
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220824/bce1577c/attachment.htm>

From serg.brester at sebres.de  Wed Aug 24 17:57:50 2022
From: serg.brester at sebres.de (Dipl. Ing. Sergey Brester)
Date: Wed, 24 Aug 2022 19:57:50 +0200
Subject: Precedence return directive and nested locations
In-Reply-To: <1bf5c8eb861a070e673fe2d71ec5b3dd@sebres.de>
References: <1bf5c8eb861a070e673fe2d71ec5b3dd@sebres.de>
Message-ID: <e83a13463e141878e683552215b13ec5@sebres.de>

 

OK, 

regarding the "fallback" location, this one can be used (empty -
shortest match): 

location "" {
 return 444;
}

Regards,
Serg. 

24.08.2022 19:38, Sergey Brester via nginx-devel wrote: 

> Hi, 
> 
> it seems that the question of precedence of non-conditional _return_ directive vs nested _location_s is not really clear,
> or rather some constellations (like fallback) are impossible or else the configuration may look weird.
> 
> For instance: 
> 
> server {
> server_name ...;
> 
> location ~ ^/(some-uri|other-uri) {
> return 200 ...;
> }
> 
> # fallback for any not matched uri:
> return 444;
> } 
> 
> will ALWAYS return with 444 for that server no matter whether it matches the location(s) or doesn't.
> Basically the locations are totally superfluous here (despite specified), considering the return is non-conditional. 
> 
> Sure, the documentation says "Stops processing and returns the specified _code_ to a client.",
> but normally the nested locations (if matched) have always higher precedence over anything else
> in the parent location. 
> 
> Furthermore the docu of ngx_http_rewrite_module [1] says at begin: 
> 
> * the directives of this module specified on the server [2] level are executed sequentially;
> 
> * repeatedly: 
> 
> * a location [3] is searched based on a request URI;
> * the directives of this module specified inside the found location are executed sequentially;
> 
> What is a bit ambiguous. But if it is to understand directly - it is clear that a return directive at server level
> bypasses all other locations (sequence in current level before its locations).
> Just it makes hardly possible (up to impossible depending on location tree) to use return directive for a fallback case. 
> 
> To implement the fallback return, one can surely pack this into a location like: 
> 
> location ~ .* { return 444; } 
> Just it is a bit ugly in my opinion, let alone it would quasi disallow the usage of longest match prefix locations, 
> because they work only if no match with a regular expression is found (then the configuration of the 
> prefix location remembered earlier is used). 
> 
> So I assume:
> 
> * either it is a lack of documentation (and it must get a hint about the precedence) 
> and/or still better a description how one could achieve such a "fallback" return (location or whatever).
> (but do we have at all a possibility to specify such proper fallback location matching at end, 
> if nothing else matched?)
> 
> * or (improbably) this is a flaw and must be "fixed" or enhanced rather for a non-conditional return case
> (unsure it wouldn't introduce some compat-issue); 
> 
> Any thoughts? 
> 
> Regards,
> Sergey. 
> 
> _______________________________________________
> nginx-devel mailing list -- nginx-devel at nginx.org
> To unsubscribe send an email to nginx-devel-leave at nginx.org
 

Links:
------
[1] http://nginx.org/en/docs/http/ngx_http_rewrite_module.html
[2] http://nginx.org/en/docs/http/ngx_http_core_module.html#server
[3] http://nginx.org/en/docs/http/ngx_http_core_module.html#location
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220824/8b596371/attachment.htm>

From alx.manpages at gmail.com  Wed Aug 24 18:15:40 2022
From: alx.manpages at gmail.com (Alejandro Colomar)
Date: Wed, 24 Aug 2022 20:15:40 +0200
Subject: [WIP/RFC v4 0/5] Support abstract Unix sockets
In-Reply-To: <YwZLLEPd6T9Npzt8@mdounin.ru>
References: <20220819142144.13915-1-alx.manpages@gmail.com>
 <20220823211805.4509-1-alx.manpages@gmail.com>
 <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
 <YwY8lSobLyraTzvr@mdounin.ru>
 <bbe31ffd-8cdb-f30c-5a4f-32bdf89b940e@gmail.com>
 <YwZLLEPd6T9Npzt8@mdounin.ru>
Message-ID: <e1f537e7-6fde-eb6e-739f-4c0402df5167@gmail.com>

Hello Maxim!

On 8/24/22 18:00, Maxim Dounin wrote:
[...]
>> For having nginx as a reverse proxy in front of a server, such as unit,
>> it's faster to communicate through Unix sockets (UDS), rather than TCP
>> (localhost).
> 
> I would rather say "it's a common misconception that it's faster
> to communicate through Unix sockets".  While it Unix sockets can
> be beneficial for some microbenchmarks,

Yeah, I don't know how much of an improvement it will really bring.  I'm 
going to develop some benchmark soon, as a side effect of another task, 
but it will still be a bit artificial, and not a real load.  But it will 
help see how much of an improvement UDS have in the context of Unit.

> in most production setups
> it makes no difference, yet used to introduce various issues.

The only one I can think of is security, since you don't have file 
permissions, but one could also argue that the security can't be lower 
than that of TCP sockets.

> 
>> Nginx already supports UDS.  But UDS has a problem: the
>> kernel creates a file in the fs, and it's not always trivial to clean up
>> those files.  Then, if the application is restarted, there's no
>> SO_REUSEADDR to allow reusing the socket file, so the application will
>> just fail.
>>
>> This happens in nginx Unit, which creates listener sockets from a
>> privileged thread, and then uses them from unprivileged threads.  When
>> the unprivileged thread stops using the socket, it can't remove the
>> file, and doing so would require huge complexity to implement.  It's
>> easier to just tell the kernel we want an abstract UDS (AUDS), so that
>> there's no file at all.  Then if the user restarts Unit, it'll be able
>> to recreate the AUDS.
>>
>> A user reported this problem with normal UDS and we concluded that the
>> easiest solution would be to add support for AUDS.  His set-up is a
>> kubernetes pod, where a container uses nginx and another container uses
>> Unit.  Communicating through an AUDS would be trivial and fast.
> 
> So, you are trying to implement abstract namespace sockets as a
> bandaid for Unit bug, which is not able to properly remove Unix
> sockets in some cases and fails to restart, correct? 

Yeah, it helps avoid fixing that bug.  But that's why the Linux kernel 
implemented that feature in the first place: so that programs don't need 
to cleanup the filesystem in these cases, recognizing that it may not be 
always trivial.

> This does
> not look like a valid reason to me, especially given that a)
> abstract namespace sockets are Linux-only, and

Since other systems can use localhost, I don't think it's so important 
to support abstract sockets in them (it's not losing a big feature, but 
rather a minor improvement).  What Unit (and some other web servers do) 
is just report an error in the configuration.

> b) there are
> multiple approaches to filing the abstract socket address.


I don't understand this last point.  Do you mean the NUL and @ 
alternatives?  NUL is the one supported by the kernel, and so the most 
obvious one.  Since NUL is not usable by most tools, most programs use 
the alternative syntax @.  Some programs only allow @; but since it 
doesn't hurt to allow NUL, and it makes for a very easy implementation, 
I'd do it that way.

Cheers,

Alex

-- 
Alejandro Colomar
<http://www.alejandro-colomar.es/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220824/511f0e93/attachment.bin>

From mdounin at mdounin.ru  Wed Aug 24 20:37:32 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 24 Aug 2022 23:37:32 +0300
Subject: Precedence return directive and nested locations
In-Reply-To: <1bf5c8eb861a070e673fe2d71ec5b3dd@sebres.de>
References: <1bf5c8eb861a070e673fe2d71ec5b3dd@sebres.de>
Message-ID: <YwaMDKNJQL/t0Nqm@mdounin.ru>

Hello!

On Wed, Aug 24, 2022 at 07:38:02PM +0200, Dipl. Ing. Sergey Brester via nginx-devel wrote:

> it seems that the question of precedence of non-conditional _return_
> directive vs nested _location_s is not really clear,
> or rather some constellations (like fallback) are impossible or else the
> configuration may look weird.
> 
> For instance: 
> 
> server {
>  server_name ...;
> 
>  location ~ ^/(some-uri|other-uri) {
>  return 200 ...;
>  }
> 
>  # fallback for any not matched uri:
>  return 444;
> } 
> 
> will ALWAYS return with 444 for that server no matter whether it matches
> the location(s) or doesn't.
> Basically the locations are totally superfluous here (despite
> specified), considering the return is non-conditional. 
> 
> Sure, the documentation says "Stops processing and returns the specified
> _code_ to a client.",
> but normally the nested locations (if matched) have always higher
> precedence over anything else
> in the parent location. 
> 
> Furthermore the docu of ngx_http_rewrite_module [1] says at begin: 
> 
>  	* the directives of this module specified on the server [2] level are
> executed sequentially;
> 
> 	* repeatedly: 
> 
>  	* a location [3] is searched based on a request URI;
>  	* the directives of this module specified inside the found location
> are executed sequentially;
> 
> What is a bit ambiguous. But if it is to understand directly - it is
> clear that a return directive at server level
> bypasses all other locations (sequence in current level before its
> locations).
> Just it makes hardly possible (up to impossible depending on location
> tree) to use return directive for a fallback case. 
> 
> To implement the fallback return, one can surely pack this into a
> location like: 
> 
> location ~ .* { return 444; } 
> Just it is a bit ugly in my opinion, let alone it would quasi disallow
> the usage of longest match prefix locations, 
> because they work only if no match with a regular expression is found
> (then the configuration of the 
> prefix location remembered earlier is used). 
> 
> So I assume:
> 
> 	* either it is a lack of documentation (and it must get a hint about
> the precedence) 
> and/or still better a description how one could achieve such a
> "fallback" return (location or whatever).
> (but do we have at all a possibility to specify such proper fallback
> location matching at end, 
> if nothing else matched?)
> 
> 	* or (improbably) this is a flaw and must be "fixed" or enhanced rather
> for a non-conditional return case
> (unsure it wouldn't introduce some compat-issue); 
> 
> Any thoughts? 

As of now, handling of the rewrite module directives, including 
"return", is explicitly defined and documented.  In particular, 
rewrite module directives at the server level are handled 
differently than location-level rewrite directives, and this 
happens before any location matching.  This is to make it possible 
to change URI (or reject requests) at the very start of the 
request processing, before any location matching.  And I believe 
the documentation is clear enough on this.

For fallback actions, consider using a generic location with 
appropriate action.  Usually something like

location / {
    return 444;
}

is a good choice, since it is matched only if there are no other 
matching locations, neither prefix nor defined by a regular 
expression.

What can be confusing here is that rewrite directives at the server 
level in a server block with locations behave differently from 
rewrite directives in a location block with nested locations.  For 
example:

server {
    rewrite /foo(/.*) /bar$1 break;
    location /bar/ { return 200 ...; }
}

is quite different from:

server {
    location / {
        rewrite /foo(/.*) /bar$1 break;
        location /bar/ { return 200 ...; }
    }
}

since rewrite in "location /" will be executed after the location 
matching, and only if the request is not matched by "location 
/bar/" inside "location /".  Similarly, "return" instead of 
"rewrite" will unconditionally terminate processing of all request 
in the server block:

server {
    return 403;
    location /bar/ { return 200 ...; }
}

but will not affect requests matched by inner locations if 
specified in "location /":

server {
    location / {
        return 403;
        location /bar/ { return 200 ...; }
    }
}

I don't there is anything to fix here though.

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Wed Aug 24 22:14:23 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 25 Aug 2022 01:14:23 +0300
Subject: [WIP/RFC v4 0/5] Support abstract Unix sockets
In-Reply-To: <e1f537e7-6fde-eb6e-739f-4c0402df5167@gmail.com>
References: <20220819142144.13915-1-alx.manpages@gmail.com>
 <20220823211805.4509-1-alx.manpages@gmail.com>
 <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
 <YwY8lSobLyraTzvr@mdounin.ru>
 <bbe31ffd-8cdb-f30c-5a4f-32bdf89b940e@gmail.com>
 <YwZLLEPd6T9Npzt8@mdounin.ru>
 <e1f537e7-6fde-eb6e-739f-4c0402df5167@gmail.com>
Message-ID: <YwaivwwJwHwlkaLb@mdounin.ru>

Hello!

On Wed, Aug 24, 2022 at 08:15:40PM +0200, Alejandro Colomar wrote:

> Hello Maxim!
> 
> On 8/24/22 18:00, Maxim Dounin wrote:
> [...]
> >> For having nginx as a reverse proxy in front of a server, such as unit,
> >> it's faster to communicate through Unix sockets (UDS), rather than TCP
> >> (localhost).
> > 
> > I would rather say "it's a common misconception that it's faster
> > to communicate through Unix sockets".  While it Unix sockets can
> > be beneficial for some microbenchmarks,
> 
> Yeah, I don't know how much of an improvement it will really bring.  I'm 
> going to develop some benchmark soon, as a side effect of another task, 
> but it will still be a bit artificial, and not a real load.  But it will 
> help see how much of an improvement UDS have in the context of Unit.
> 
> > in most production setups
> > it makes no difference, yet used to introduce various issues.
> 
> The only one I can think of is security, since you don't have file 
> permissions, but one could also argue that the security can't be lower 
> than that of TCP sockets.

Apart from being non-scalable to multiple hosts and various issues 
with socket files, Unix sockets used to trigger various kernel 
bugs in different OSes.  In particular, there were multiple 
sendfile() issues, at least on FreeBSD and IIRC on Linux.

> >> Nginx already supports UDS.  But UDS has a problem: the
> >> kernel creates a file in the fs, and it's not always trivial to clean up
> >> those files.  Then, if the application is restarted, there's no
> >> SO_REUSEADDR to allow reusing the socket file, so the application will
> >> just fail.
> >>
> >> This happens in nginx Unit, which creates listener sockets from a
> >> privileged thread, and then uses them from unprivileged threads.  When
> >> the unprivileged thread stops using the socket, it can't remove the
> >> file, and doing so would require huge complexity to implement.  It's
> >> easier to just tell the kernel we want an abstract UDS (AUDS), so that
> >> there's no file at all.  Then if the user restarts Unit, it'll be able
> >> to recreate the AUDS.
> >>
> >> A user reported this problem with normal UDS and we concluded that the
> >> easiest solution would be to add support for AUDS.  His set-up is a
> >> kubernetes pod, where a container uses nginx and another container uses
> >> Unit.  Communicating through an AUDS would be trivial and fast.
> > 
> > So, you are trying to implement abstract namespace sockets as a
> > bandaid for Unit bug, which is not able to properly remove Unix
> > sockets in some cases and fails to restart, correct? 
> 
> Yeah, it helps avoid fixing that bug.  But that's why the Linux kernel 
> implemented that feature in the first place: so that programs don't need 
> to cleanup the filesystem in these cases, recognizing that it may not be 
> always trivial.
> 
> > This does
> > not look like a valid reason to me, especially given that a)
> > abstract namespace sockets are Linux-only, and
> 
> Since other systems can use localhost, I don't think it's so important 
> to support abstract sockets in them (it's not losing a big feature, but 
> rather a minor improvement).  What Unit (and some other web servers do) 
> is just report an error in the configuration.

The point is that you'll have to fix the bug anyway, since the bug 
is there and, even assuming the workaround is good enough, it's 
Linux-specific.

If you are looking for a workaround, a readily available one is 
using TCP sockets.

> > b) there are
> > multiple approaches to filing the abstract socket address.
> 
> I don't understand this last point.  Do you mean the NUL and @ 
> alternatives?  NUL is the one supported by the kernel, and so the most 
> obvious one.  Since NUL is not usable by most tools, most programs use 
> the alternative syntax @.  Some programs only allow @; but since it 
> doesn't hurt to allow NUL, and it makes for a very easy implementation, 
> I'd do it that way.

Since null bytes in the socket address are not special, there is 
more than one way to fill the sockaddr structure.  The patch I've 
referenced fills the struct sockaddr_un's sun_path with null 
bytes, and uses socklen set to sizeof(struct sockaddr_un): this 
approach is compatible with HAProxy's abns@ and socat with 
unix-tightsocklen=0.  In contrast, curl uses different approach 
and instead uses socklen set to exact length of the name provided 
with leading null bytes and no trailing null bytes (and this is 
what you've tried in your patches).

-- 
Maxim Dounin
http://mdounin.ru/


From alx.manpages at gmail.com  Wed Aug 24 23:14:48 2022
From: alx.manpages at gmail.com (Alejandro Colomar)
Date: Thu, 25 Aug 2022 01:14:48 +0200
Subject: [WIP/RFC v4 0/5] Support abstract Unix sockets
In-Reply-To: <YwaivwwJwHwlkaLb@mdounin.ru>
References: <20220819142144.13915-1-alx.manpages@gmail.com>
 <20220823211805.4509-1-alx.manpages@gmail.com>
 <6940ef35-c3b6-8cc2-2eb6-19471808e93a@gmail.com>
 <YwY8lSobLyraTzvr@mdounin.ru>
 <bbe31ffd-8cdb-f30c-5a4f-32bdf89b940e@gmail.com>
 <YwZLLEPd6T9Npzt8@mdounin.ru>
 <e1f537e7-6fde-eb6e-739f-4c0402df5167@gmail.com>
 <YwaivwwJwHwlkaLb@mdounin.ru>
Message-ID: <17bdd610-0912-7baf-5231-8bf63f8e7456@gmail.com>

Hi Maxim,

On 8/25/22 00:14, Maxim Dounin wrote:
[...]

>>> in most production setups
>>> it makes no difference, yet used to introduce various issues.
>>
>> The only one I can think of is security, since you don't have file
>> permissions, but one could also argue that the security can't be lower
>> than that of TCP sockets.
> 
> Apart from being non-scalable to multiple hosts and various issues

Yeah, that one is obvious.  But it should be trivial to change the 
configuration to use a TCP one.

> with socket files, Unix sockets used to trigger various kernel
> bugs in different OSes.  In particular, there were multiple
> sendfile() issues, at least on FreeBSD and IIRC on Linux.

Never heard of those.  I would guess that they are old bugs not present 
in current kernels.  Of course, if there have been bugs, there might be 
other bugs in the future, but I wouldn't go so far as to not recommend 
UDS at all, and instead go always for TCP.

> 
>>>> Nginx already supports UDS.  But UDS has a problem: the
>>>> kernel creates a file in the fs, and it's not always trivial to clean up
>>>> those files.  Then, if the application is restarted, there's no
>>>> SO_REUSEADDR to allow reusing the socket file, so the application will
>>>> just fail.
>>>>
>>>> This happens in nginx Unit, which creates listener sockets from a
>>>> privileged thread, and then uses them from unprivileged threads.  When
>>>> the unprivileged thread stops using the socket, it can't remove the
>>>> file, and doing so would require huge complexity to implement.  It's
>>>> easier to just tell the kernel we want an abstract UDS (AUDS), so that
>>>> there's no file at all.  Then if the user restarts Unit, it'll be able
>>>> to recreate the AUDS.
>>>>
>>>> A user reported this problem with normal UDS and we concluded that the
>>>> easiest solution would be to add support for AUDS.  His set-up is a
>>>> kubernetes pod, where a container uses nginx and another container uses
>>>> Unit.  Communicating through an AUDS would be trivial and fast.
>>>
>>> So, you are trying to implement abstract namespace sockets as a
>>> bandaid for Unit bug, which is not able to properly remove Unix
>>> sockets in some cases and fails to restart, correct?
>>
>> Yeah, it helps avoid fixing that bug.  But that's why the Linux kernel
>> implemented that feature in the first place: so that programs don't need
>> to cleanup the filesystem in these cases, recognizing that it may not be
>> always trivial.
>>
>>> This does
>>> not look like a valid reason to me, especially given that a)
>>> abstract namespace sockets are Linux-only, and
>>
>> Since other systems can use localhost, I don't think it's so important
>> to support abstract sockets in them (it's not losing a big feature, but
>> rather a minor improvement).  What Unit (and some other web servers do)
>> is just report an error in the configuration.
> 
> The point is that you'll have to fix the bug anyway, since the bug
> is there and, even assuming the workaround is good enough, it's
> Linux-specific.

I would say we don't need to fix it, even if AUDS can't be used. 
localhost is as good as them in terms of usability.  AUDS only helps in 
performance, compared to localhost.

If a user really wants to use file sockets, they can clean up /run with 
a script in between runs of the program.  Unit should very rarely need 
to restart if the OS doesn't.  So for this very rare case, 
localhost/AUDS are the simple solution.  Since support for AUDS is not 
so hard to implement, I thought we could go for it.

In fact, we had some buggy implementation of AUDS in Unit, written a 
long time ago by Igor, and I just fixed the bugs to make it work, but I 
didn't write it from scratch in Unit.  (BTW, he used the curl(1) approach.)

> 
> If you are looking for a workaround, a readily available one is
> using TCP sockets.

Yep.

In fact, nginx really supports abstract sockets, or I should say socket 
(singular).  If you try to proxy_pass to any address starting with a NUL 
character, it will try to connect to an abstract socket with a name 
consisting of all 0s.  This patch set only makes it more useful, but for 
a single socket, it already works.

> 
>>> b) there are
>>> multiple approaches to filing the abstract socket address.
>>
>> I don't understand this last point.  Do you mean the NUL and @
>> alternatives?  NUL is the one supported by the kernel, and so the most
>> obvious one.  Since NUL is not usable by most tools, most programs use
>> the alternative syntax @.  Some programs only allow @; but since it
>> doesn't hurt to allow NUL, and it makes for a very easy implementation,
>> I'd do it that way.
> 
> Since null bytes in the socket address are not special, there is
> more than one way to fill the sockaddr structure.  The patch I've
> referenced fills the struct sockaddr_un's sun_path with null
> bytes, and uses socklen set to sizeof(struct sockaddr_un): this
> approach is compatible with HAProxy's abns@ and socat with
> unix-tightsocklen=0.  In contrast, curl uses different approach
> and instead uses socklen set to exact length of the name provided
> with leading null bytes and no trailing null bytes (and this is
> what you've tried in your patches).

Ahhh, I didn't know those used a different syntax.  I'd go for the 
curl(1) approach, since it is more flexible, and one can always specify 
a socket with the full length by specifying all of the characters in the 
name.  Not very friendly, but it's possible.

Anyway, if you're not convinced by the feature, let me come back to it 
after I do some performance testing in Unit to see how significant are 
the numbers.

Thanks for the review!

Cheers,
Alex


-- 
Alejandro Colomar
<http://www.alejandro-colomar.es/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220825/06d7f8ad/attachment.bin>

From xeioex at nginx.com  Thu Aug 25 01:09:58 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Thu, 25 Aug 2022 01:09:58 +0000
Subject: [njs] Modules: sorting external object property descriptors
 alphabetically.
Message-ID: <hg.e023db05004f.1661389798.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/e023db05004f
branches:  
changeset: 1934:e023db05004f
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Aug 24 17:04:58 2022 -0700
description:
Modules: sorting external object property descriptors alphabetically.

diffstat:

 nginx/ngx_http_js_module.c   |  357 +++++++++++++++++++++---------------------
 nginx/ngx_js.c               |   63 +++---
 nginx/ngx_stream_js_module.c |  166 +++++++++----------
 3 files changed, 292 insertions(+), 294 deletions(-)

diffs (764 lines):

diff -r b33aae5e8dc6 -r e023db05004f nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c	Tue Aug 23 19:36:16 2022 -0700
+++ b/nginx/ngx_http_js_module.c	Wed Aug 24 17:04:58 2022 -0700
@@ -459,11 +459,100 @@ static njs_external_t  ngx_http_js_ext_r
 
     {
         .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("uri"),
+        .name.string = njs_str("args"),
         .enumerable = 1,
         .u.property = {
-            .handler = ngx_js_ext_string,
-            .magic32 = offsetof(ngx_http_request_t, uri),
+            .handler = ngx_http_js_ext_get_args,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("done"),
+        .writable = 1,
+        .configurable = 1,
+        .enumerable = 1,
+        .u.method = {
+            .native = ngx_http_js_ext_done,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("error"),
+        .writable = 1,
+        .configurable = 1,
+        .enumerable = 1,
+        .u.method = {
+            .native = ngx_js_ext_log,
+            .magic8 = NGX_LOG_ERR,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("finish"),
+        .writable = 1,
+        .configurable = 1,
+        .enumerable = 1,
+        .u.method = {
+            .native = ngx_http_js_ext_finish,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_OBJECT,
+        .name.string = njs_str("headersIn"),
+        .enumerable = 1,
+        .u.object = {
+            .enumerable = 1,
+            .prop_handler = ngx_http_js_ext_header_in,
+            .keys = ngx_http_js_ext_keys_header_in,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_OBJECT,
+        .name.string = njs_str("headersOut"),
+        .enumerable = 1,
+        .u.object = {
+            .writable = 1,
+            .configurable = 1,
+            .enumerable = 1,
+            .prop_handler = ngx_http_js_ext_header_out,
+            .keys = ngx_http_js_ext_keys_header_out,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("httpVersion"),
+        .enumerable = 1,
+        .u.property = {
+            .handler = ngx_http_js_ext_get_http_version,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("internalRedirect"),
+        .writable = 1,
+        .configurable = 1,
+        .enumerable = 1,
+        .u.method = {
+            .native = ngx_http_js_ext_internal_redirect,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("log"),
+        .writable = 1,
+        .configurable = 1,
+        .enumerable = 1,
+        .u.method = {
+            .native = ngx_js_ext_log,
+            .magic8 = NGX_LOG_INFO,
         }
     },
 
@@ -479,10 +568,37 @@ static njs_external_t  ngx_http_js_ext_r
 
     {
         .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("httpVersion"),
-        .enumerable = 1,
+        .name.string = njs_str("parent"),
+        .u.property = {
+            .handler = ngx_http_js_ext_get_parent,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("rawHeadersIn"),
         .u.property = {
-            .handler = ngx_http_js_ext_get_http_version,
+            .handler = ngx_http_js_ext_raw_header,
+            .magic32 = 0,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("rawHeadersOut"),
+        .u.property = {
+            .handler = ngx_http_js_ext_raw_header,
+            .magic32 = 1,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_OBJECT,
+        .name.string = njs_str("rawVariables"),
+        .u.object = {
+            .writable = 1,
+            .prop_handler = ngx_http_js_ext_variables,
+            .magic32 = NGX_JS_BUFFER,
         }
     },
 
@@ -506,6 +622,15 @@ static njs_external_t  ngx_http_js_ext_r
 
     {
         .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("requestBuffer"),
+        .u.property = {
+            .handler = ngx_http_js_ext_get_request_body,
+            .magic32 = NGX_JS_BUFFER,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
         .name.string = njs_str("requestText"),
         .enumerable = 1,
         .u.property = {
@@ -516,27 +641,19 @@ static njs_external_t  ngx_http_js_ext_r
 
     {
         .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("requestBuffer"),
+        .name.string = njs_str("responseBody"),
         .u.property = {
-            .handler = ngx_http_js_ext_get_request_body,
-            .magic32 = NGX_JS_BUFFER,
+            .handler = ngx_http_js_ext_get_response_body,
+            .magic32 = NGX_JS_STRING | NGX_JS_DEPRECATED,
         }
     },
 
     {
         .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("parent"),
-        .u.property = {
-            .handler = ngx_http_js_ext_get_parent,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("responseBody"),
+        .name.string = njs_str("responseBuffer"),
         .u.property = {
             .handler = ngx_http_js_ext_get_response_body,
-            .magic32 = NGX_JS_STRING | NGX_JS_DEPRECATED,
+            .magic32 = NGX_JS_BUFFER,
         }
     },
 
@@ -551,150 +668,13 @@ static njs_external_t  ngx_http_js_ext_r
     },
 
     {
-        .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("responseBuffer"),
-        .u.property = {
-            .handler = ngx_http_js_ext_get_response_body,
-            .magic32 = NGX_JS_BUFFER,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_OBJECT,
-        .name.string = njs_str("headersIn"),
-        .enumerable = 1,
-        .u.object = {
-            .enumerable = 1,
-            .prop_handler = ngx_http_js_ext_header_in,
-            .keys = ngx_http_js_ext_keys_header_in,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("rawHeadersIn"),
-        .u.property = {
-            .handler = ngx_http_js_ext_raw_header,
-            .magic32 = 0,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("args"),
-        .enumerable = 1,
-        .u.property = {
-            .handler = ngx_http_js_ext_get_args,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_OBJECT,
-        .name.string = njs_str("variables"),
-        .u.object = {
-            .writable = 1,
-            .prop_handler = ngx_http_js_ext_variables,
-            .magic32 = NGX_JS_STRING,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_OBJECT,
-        .name.string = njs_str("rawVariables"),
-        .u.object = {
-            .writable = 1,
-            .prop_handler = ngx_http_js_ext_variables,
-            .magic32 = NGX_JS_BUFFER,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("status"),
-        .writable = 1,
-        .enumerable = 1,
-        .u.property = {
-            .handler = ngx_http_js_ext_status,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_OBJECT,
-        .name.string = njs_str("headersOut"),
-        .enumerable = 1,
-        .u.object = {
-            .writable = 1,
-            .configurable = 1,
-            .enumerable = 1,
-            .prop_handler = ngx_http_js_ext_header_out,
-            .keys = ngx_http_js_ext_keys_header_out,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("rawHeadersOut"),
-        .u.property = {
-            .handler = ngx_http_js_ext_raw_header,
-            .magic32 = 1,
-        }
-    },
-
-    {
         .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("subrequest"),
+        .name.string = njs_str("return"),
         .writable = 1,
         .configurable = 1,
         .enumerable = 1,
         .u.method = {
-            .native = ngx_http_js_ext_subrequest,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("log"),
-        .writable = 1,
-        .configurable = 1,
-        .enumerable = 1,
-        .u.method = {
-            .native = ngx_js_ext_log,
-            .magic8 = NGX_LOG_INFO,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("warn"),
-        .writable = 1,
-        .configurable = 1,
-        .enumerable = 1,
-        .u.method = {
-            .native = ngx_js_ext_log,
-            .magic8 = NGX_LOG_WARN,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("error"),
-        .writable = 1,
-        .configurable = 1,
-        .enumerable = 1,
-        .u.method = {
-            .native = ngx_js_ext_log,
-            .magic8 = NGX_LOG_ERR,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("sendHeader"),
-        .writable = 1,
-        .configurable = 1,
-        .enumerable = 1,
-        .u.method = {
-            .native = ngx_http_js_ext_send_header,
+            .native = ngx_http_js_ext_return,
         }
     },
 
@@ -722,6 +702,17 @@ static njs_external_t  ngx_http_js_ext_r
 
     {
         .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("sendHeader"),
+        .writable = 1,
+        .configurable = 1,
+        .enumerable = 1,
+        .u.method = {
+            .native = ngx_http_js_ext_send_header,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
         .name.string = njs_str("setReturnValue"),
         .writable = 1,
         .configurable = 1,
@@ -732,49 +723,57 @@ static njs_external_t  ngx_http_js_ext_r
     },
 
     {
-        .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("done"),
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("status"),
         .writable = 1,
-        .configurable = 1,
         .enumerable = 1,
-        .u.method = {
-            .native = ngx_http_js_ext_done,
+        .u.property = {
+            .handler = ngx_http_js_ext_status,
         }
     },
 
     {
         .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("finish"),
+        .name.string = njs_str("subrequest"),
         .writable = 1,
         .configurable = 1,
         .enumerable = 1,
         .u.method = {
-            .native = ngx_http_js_ext_finish,
+            .native = ngx_http_js_ext_subrequest,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("uri"),
+        .enumerable = 1,
+        .u.property = {
+            .handler = ngx_js_ext_string,
+            .magic32 = offsetof(ngx_http_request_t, uri),
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_OBJECT,
+        .name.string = njs_str("variables"),
+        .u.object = {
+            .writable = 1,
+            .prop_handler = ngx_http_js_ext_variables,
+            .magic32 = NGX_JS_STRING,
         }
     },
 
     {
         .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("return"),
+        .name.string = njs_str("warn"),
         .writable = 1,
         .configurable = 1,
         .enumerable = 1,
         .u.method = {
-            .native = ngx_http_js_ext_return,
+            .native = ngx_js_ext_log,
+            .magic8 = NGX_LOG_WARN,
         }
     },
-
-    {
-        .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("internalRedirect"),
-        .writable = 1,
-        .configurable = 1,
-        .enumerable = 1,
-        .u.method = {
-            .native = ngx_http_js_ext_internal_redirect,
-        }
-    },
-
 };
 
 
diff -r b33aae5e8dc6 -r e023db05004f nginx/ngx_js.c
--- a/nginx/ngx_js.c	Tue Aug 23 19:36:16 2022 -0700
+++ b/nginx/ngx_js.c	Wed Aug 24 17:04:58 2022 -0700
@@ -18,37 +18,6 @@ extern njs_module_t  njs_webcrypto_modul
 static njs_external_t  ngx_js_ext_core[] = {
 
     {
-        .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("log"),
-        .writable = 1,
-        .configurable = 1,
-        .enumerable = 1,
-        .u.method = {
-            .native = ngx_js_ext_log,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("INFO"),
-        .u.property = {
-            .handler = ngx_js_ext_constant,
-            .magic32 = NGX_LOG_INFO,
-            .magic16 = NGX_JS_NUMBER,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("WARN"),
-        .u.property = {
-            .handler = ngx_js_ext_constant,
-            .magic32 = NGX_LOG_WARN,
-            .magic16 = NGX_JS_NUMBER,
-        }
-    },
-
-    {
         .flags = NJS_EXTERN_PROPERTY,
         .name.string = njs_str("ERR"),
         .u.property = {
@@ -68,6 +37,38 @@ static njs_external_t  ngx_js_ext_core[]
             .native = ngx_js_ext_fetch,
         }
     },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("INFO"),
+        .u.property = {
+            .handler = ngx_js_ext_constant,
+            .magic32 = NGX_LOG_INFO,
+            .magic16 = NGX_JS_NUMBER,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("log"),
+        .writable = 1,
+        .configurable = 1,
+        .enumerable = 1,
+        .u.method = {
+            .native = ngx_js_ext_log,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("WARN"),
+        .u.property = {
+            .handler = ngx_js_ext_constant,
+            .magic32 = NGX_LOG_WARN,
+            .magic16 = NGX_JS_NUMBER,
+        }
+    },
+
 };
 
 
diff -r b33aae5e8dc6 -r e023db05004f nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c	Tue Aug 23 19:36:16 2022 -0700
+++ b/nginx/ngx_stream_js_module.c	Wed Aug 24 17:04:58 2022 -0700
@@ -326,45 +326,6 @@ static njs_external_t  ngx_stream_js_ext
     },
 
     {
-        .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("status"),
-        .enumerable = 1,
-        .u.property = {
-            .handler = ngx_js_ext_uint,
-            .magic32 = offsetof(ngx_stream_session_t, status),
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("remoteAddress"),
-        .enumerable = 1,
-        .u.property = {
-            .handler = ngx_stream_js_ext_get_remote_address,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_OBJECT,
-        .name.string = njs_str("variables"),
-        .u.object = {
-            .writable = 1,
-            .prop_handler = ngx_stream_js_ext_variables,
-            .magic32 = NGX_JS_STRING,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_OBJECT,
-        .name.string = njs_str("rawVariables"),
-        .u.object = {
-            .writable = 1,
-            .prop_handler = ngx_stream_js_ext_variables,
-            .magic32 = NGX_JS_BUFFER,
-        }
-    },
-
-    {
         .flags = NJS_EXTERN_METHOD,
         .name.string = njs_str("allow"),
         .writable = 1,
@@ -378,6 +339,18 @@ static njs_external_t  ngx_stream_js_ext
 
     {
         .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("decline"),
+        .writable = 1,
+        .configurable = 1,
+        .enumerable = 1,
+        .u.method = {
+            .native = ngx_stream_js_ext_done,
+            .magic8 = -NGX_DECLINED,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
         .name.string = njs_str("deny"),
         .writable = 1,
         .configurable = 1,
@@ -390,18 +363,6 @@ static njs_external_t  ngx_stream_js_ext
 
     {
         .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("decline"),
-        .writable = 1,
-        .configurable = 1,
-        .enumerable = 1,
-        .u.method = {
-            .native = ngx_stream_js_ext_done,
-            .magic8 = -NGX_DECLINED,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_METHOD,
         .name.string = njs_str("done"),
         .writable = 1,
         .configurable = 1,
@@ -415,6 +376,18 @@ static njs_external_t  ngx_stream_js_ext
 
     {
         .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("error"),
+        .writable = 1,
+        .configurable = 1,
+        .enumerable = 1,
+        .u.method = {
+            .native = ngx_js_ext_log,
+            .magic8 = NGX_LOG_ERR,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
         .name.string = njs_str("log"),
         .writable = 1,
         .configurable = 1,
@@ -427,25 +400,12 @@ static njs_external_t  ngx_stream_js_ext
 
     {
         .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("warn"),
+        .name.string = njs_str("off"),
         .writable = 1,
         .configurable = 1,
         .enumerable = 1,
         .u.method = {
-            .native = ngx_js_ext_log,
-            .magic8 = NGX_LOG_WARN,
-        }
-    },
-
-    {
-        .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("error"),
-        .writable = 1,
-        .configurable = 1,
-        .enumerable = 1,
-        .u.method = {
-            .native = ngx_js_ext_log,
-            .magic8 = NGX_LOG_ERR,
+            .native = ngx_stream_js_ext_off,
         }
     },
 
@@ -461,13 +421,21 @@ static njs_external_t  ngx_stream_js_ext
     },
 
     {
-        .flags = NJS_EXTERN_METHOD,
-        .name.string = njs_str("off"),
-        .writable = 1,
-        .configurable = 1,
+        .flags = NJS_EXTERN_OBJECT,
+        .name.string = njs_str("rawVariables"),
+        .u.object = {
+            .writable = 1,
+            .prop_handler = ngx_stream_js_ext_variables,
+            .magic32 = NGX_JS_BUFFER,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("remoteAddress"),
         .enumerable = 1,
-        .u.method = {
-            .native = ngx_stream_js_ext_off,
+        .u.property = {
+            .handler = ngx_stream_js_ext_get_remote_address,
         }
     },
 
@@ -493,6 +461,37 @@ static njs_external_t  ngx_stream_js_ext
         }
     },
 
+    {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("status"),
+        .enumerable = 1,
+        .u.property = {
+            .handler = ngx_js_ext_uint,
+            .magic32 = offsetof(ngx_stream_session_t, status),
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_OBJECT,
+        .name.string = njs_str("variables"),
+        .u.object = {
+            .writable = 1,
+            .prop_handler = ngx_stream_js_ext_variables,
+            .magic32 = NGX_JS_STRING,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_METHOD,
+        .name.string = njs_str("warn"),
+        .writable = 1,
+        .configurable = 1,
+        .enumerable = 1,
+        .u.method = {
+            .native = ngx_js_ext_log,
+            .magic8 = NGX_LOG_WARN,
+        }
+    },
 };
 
 
@@ -508,6 +507,17 @@ static njs_external_t  ngx_stream_js_ext
 
     {
         .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("from_upstream"),
+        .enumerable = 1,
+        .u.property = {
+            .handler = ngx_js_ext_flags,
+            .magic16 = NGX_JS_BOOLEAN,
+            .magic32 = 0x00000002,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
         .name.string = njs_str("last"),
         .enumerable = 1,
         .u.property = {
@@ -516,18 +526,6 @@ static njs_external_t  ngx_stream_js_ext
             .magic32 = 0x00000001,
         }
     },
-
-    {
-        .flags = NJS_EXTERN_PROPERTY,
-        .name.string = njs_str("from_upstream"),
-        .enumerable = 1,
-        .u.property = {
-            .handler = ngx_js_ext_flags,
-            .magic16 = NGX_JS_BOOLEAN,
-            .magic32 = 0x00000002,
-        }
-    },
-
 };
 
 


From xeioex at nginx.com  Fri Aug 26 00:49:12 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 26 Aug 2022 00:49:12 +0000
Subject: [njs] HTTP: added r.internal property.
Message-ID: <hg.43b31a943c08.1661474952.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/43b31a943c08
branches:  
changeset: 1935:43b31a943c08
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Thu Aug 25 16:57:28 2022 -0700
description:
HTTP: added r.internal property.

diffstat:

 nginx/ngx_http_js_module.c |  30 ++++++++++++++++++++++++++++++
 1 files changed, 30 insertions(+), 0 deletions(-)

diffs (54 lines):

diff -r e023db05004f -r 43b31a943c08 nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c	Wed Aug 24 17:04:58 2022 -0700
+++ b/nginx/ngx_http_js_module.c	Thu Aug 25 16:57:28 2022 -0700
@@ -174,6 +174,9 @@ static njs_int_t ngx_http_js_ext_interna
 static njs_int_t ngx_http_js_ext_get_http_version(njs_vm_t *vm,
     njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval,
     njs_value_t *retval);
+static njs_int_t ngx_http_js_ext_internal(njs_vm_t *vm,
+    njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval,
+    njs_value_t *retval);
 static njs_int_t ngx_http_js_ext_get_remote_address(njs_vm_t *vm,
     njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval,
     njs_value_t *retval);
@@ -534,6 +537,15 @@ static njs_external_t  ngx_http_js_ext_r
     },
 
     {
+        .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("internal"),
+        .enumerable = 1,
+        .u.property = {
+            .handler = ngx_http_js_ext_internal,
+        }
+    },
+
+    {
         .flags = NJS_EXTERN_METHOD,
         .name.string = njs_str("internalRedirect"),
         .writable = 1,
@@ -2504,6 +2516,24 @@ ngx_http_js_ext_get_http_version(njs_vm_
 
 
 static njs_int_t
+ngx_http_js_ext_internal(njs_vm_t *vm, njs_object_prop_t *prop,
+    njs_value_t *value, njs_value_t *setval, njs_value_t *retval)
+{
+    ngx_http_request_t  *r;
+
+    r = njs_vm_external(vm, ngx_http_js_request_proto_id, value);
+    if (r == NULL) {
+        njs_value_undefined_set(retval);
+        return NJS_DECLINED;
+    }
+
+    njs_value_boolean_set(retval, r->internal);
+
+    return NJS_OK;
+}
+
+
+static njs_int_t
 ngx_http_js_ext_get_remote_address(njs_vm_t *vm, njs_object_prop_t *prop,
     njs_value_t *value, njs_value_t *setval, njs_value_t *retval)
 {


From mdounin at mdounin.ru  Fri Aug 26 01:17:41 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Fri, 26 Aug 2022 04:17:41 +0300
Subject: [PATCH] Events: fixed style and wrong error handling in the iocp
 module
Message-ID: <3296b3825fa9b8cd4d0a.1661476661@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1661476617 -10800
#      Fri Aug 26 04:16:57 2022 +0300
# Node ID 3296b3825fa9b8cd4d0aafb39bb5ea1fac97c6db
# Parent  069a4813e8d6d7ec662d282a10f5f7062ebd817f
Events: fixed style and wrong error handling in the iocp module.

diff --git a/src/event/modules/ngx_iocp_module.c b/src/event/modules/ngx_iocp_module.c
--- a/src/event/modules/ngx_iocp_module.c
+++ b/src/event/modules/ngx_iocp_module.c
@@ -231,9 +231,8 @@ ngx_iocp_del_connection(ngx_connection_t
 }
 
 
-static
-ngx_int_t ngx_iocp_process_events(ngx_cycle_t *cycle, ngx_msec_t timer,
-    ngx_uint_t flags)
+static ngx_int_t
+ngx_iocp_process_events(ngx_cycle_t *cycle, ngx_msec_t timer, ngx_uint_t flags)
 {
     int                rc;
     u_int              key;
@@ -356,7 +355,7 @@ ngx_iocp_create_conf(ngx_cycle_t *cycle)
 
     cf = ngx_palloc(cycle->pool, sizeof(ngx_iocp_conf_t));
     if (cf == NULL) {
-        return NGX_CONF_ERROR;
+        return NULL;
     }
 
     cf->threads = NGX_CONF_UNSET;


From mdounin at mdounin.ru  Fri Aug 26 03:01:07 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Fri, 26 Aug 2022 06:01:07 +0300
Subject: [PATCH 00 of 11] SSL session handling patches
Message-ID: <patchbomb.1661482867@vm-bsd.mdounin.ru>

Hello!

Below is a patch series to address various issues with SSL session
caching and session tickets, notably:

- session cache trashing with OpenSSL and TLSv1.3
- excessive logging of session cache allocation failures

It also introduces automatic rotation of session ticket keys (as long
as session cache in the shared memory is configured).

-- 
Maxim Dounin
http://mdounin.ru/


From mdounin at mdounin.ru  Fri Aug 26 03:01:08 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Fri, 26 Aug 2022 06:01:08 +0300
Subject: [PATCH 01 of 11] SSL: disabled saving tickets to session cache
In-Reply-To: <patchbomb.1661482867@vm-bsd.mdounin.ru>
References: <patchbomb.1661482867@vm-bsd.mdounin.ru>
Message-ID: <2cd8fbeb4edc5a99b725.1661482868@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1661481945 -10800
#      Fri Aug 26 05:45:45 2022 +0300
# Node ID 2cd8fbeb4edc5a99b725585edc02a16a8a0c503e
# Parent  069a4813e8d6d7ec662d282a10f5f7062ebd817f
SSL: disabled saving tickets to session cache.

OpenSSL for TLSv1.3 tries to save tickets into session cache "because some
applications just want to know about the creation of a session".  To avoid
trashing session cache with useless data, we do not save such sessions now.

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -3815,6 +3815,22 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_
     ngx_ssl_session_cache_t  *cache;
     u_char                    buf[NGX_SSL_MAX_SESSION_SIZE];
 
+#ifdef TLS1_3_VERSION
+
+    /*
+     * OpenSSL for TLSv1.3 tries to save tickets into session cache
+     * "because some applications just want to know about the creation
+     * of a session"; do not cache such sessions
+     */
+
+    if (SSL_version(ssl_conn) == TLS1_3_VERSION
+        && (SSL_get_options(ssl_conn) & SSL_OP_NO_TICKET) == 0)
+    {
+        return 0;
+    }
+
+#endif
+
     len = i2d_SSL_SESSION(sess, NULL);
 
     /* do not cache too big session */


From mdounin at mdounin.ru  Fri Aug 26 03:01:09 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Fri, 26 Aug 2022 06:01:09 +0300
Subject: [PATCH 02 of 11] SSL: reduced logging of session cache failures
 (ticket #621)
In-Reply-To: <patchbomb.1661482867@vm-bsd.mdounin.ru>
References: <patchbomb.1661482867@vm-bsd.mdounin.ru>
Message-ID: <5b137f110e84af974ef2.1661482869@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1661481947 -10800
#      Fri Aug 26 05:45:47 2022 +0300
# Node ID 5b137f110e84af974ef2b9efcf35bec2d883c187
# Parent  2cd8fbeb4edc5a99b725585edc02a16a8a0c503e
SSL: reduced logging of session cache failures (ticket #621).

Session cache allocations might fail as long as the new session is different
in size from the one least recently used (and freed when the first allocation
fails).  In particular, it might not be possible to allocate space for
sessions with client certificates, since they are noticeably bigger than
normal sessions.

To ensure such allocation failures won't clutter logs, logging level changed
to "warn", and logging is now limited to at most one warning per second.

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -3949,8 +3949,11 @@ failed:
 
     ngx_shmtx_unlock(&shpool->mutex);
 
-    ngx_log_error(NGX_LOG_ALERT, c->log, 0,
-                  "could not allocate new session%s", shpool->log_ctx);
+    if (cache->fail_time != ngx_time()) {
+        cache->fail_time = ngx_time();
+        ngx_log_error(NGX_LOG_WARN, c->log, 0,
+                      "could not allocate new session%s", shpool->log_ctx);
+    }
 
     return 0;
 }
diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h
--- a/src/event/ngx_event_openssl.h
+++ b/src/event/ngx_event_openssl.h
@@ -150,6 +150,7 @@ typedef struct {
     ngx_rbtree_t                session_rbtree;
     ngx_rbtree_node_t           sentinel;
     ngx_queue_t                 expire_queue;
+    time_t                      fail_time;
 } ngx_ssl_session_cache_t;
 
 


From mdounin at mdounin.ru  Fri Aug 26 03:01:10 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Fri, 26 Aug 2022 06:01:10 +0300
Subject: [PATCH 03 of 11] SSL: updated comment about session sizes
In-Reply-To: <patchbomb.1661482867@vm-bsd.mdounin.ru>
References: <patchbomb.1661482867@vm-bsd.mdounin.ru>
Message-ID: <86d827338fdd13ea899d.1661482870@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1661481948 -10800
#      Fri Aug 26 05:45:48 2022 +0300
# Node ID 86d827338fdd13ea899d618b0bcb2be23469cbac
# Parent  5b137f110e84af974ef2b9efcf35bec2d883c187
SSL: updated comment about session sizes.

Previous numbers are somewhat outdated, typical ASN1 representations of
sessions are slightly bigger now.

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -3785,16 +3785,16 @@ ngx_ssl_session_cache_init(ngx_shm_zone_
 
 /*
  * The length of the session id is 16 bytes for SSLv2 sessions and
- * between 1 and 32 bytes for SSLv3/TLSv1, typically 32 bytes.
- * It seems that the typical length of the external ASN1 representation
- * of a session is 118 or 119 bytes for SSLv3/TSLv1.
+ * between 1 and 32 bytes for SSLv3 and TLS, typically 32 bytes.
+ * Typical length of the external ASN1 representation of a session
+ * is about 150 bytes plus SNI server name.
  *
- * Thus on 32-bit platforms we allocate separately an rbtree node,
+ * On 32-bit platforms we allocate separately an rbtree node,
  * a session id, and an ASN1 representation, they take accordingly
- * 64, 32, and 128 bytes.
+ * 64, 32, and 256 bytes.
  *
  * On 64-bit platforms we allocate separately an rbtree node + session_id,
- * and an ASN1 representation, they take accordingly 128 and 128 bytes.
+ * and an ASN1 representation, they take accordingly 128 and 256 bytes.
  *
  * OpenSSL's i2d_SSL_SESSION() and d2i_SSL_SESSION are slow,
  * so they are outside the code locked by shared pool mutex


From mdounin at mdounin.ru  Fri Aug 26 03:01:11 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Fri, 26 Aug 2022 06:01:11 +0300
Subject: [PATCH 04 of 11] SSL: explicit session id length checking
In-Reply-To: <patchbomb.1661482867@vm-bsd.mdounin.ru>
References: <patchbomb.1661482867@vm-bsd.mdounin.ru>
Message-ID: <f4ae0f4ee928cf203465.1661482871@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1661481949 -10800
#      Fri Aug 26 05:45:49 2022 +0300
# Node ID f4ae0f4ee928cf20346530e96f1431314ecd0171
# Parent  86d827338fdd13ea899d618b0bcb2be23469cbac
SSL: explicit session id length checking.

Session ids are not expected to be longer than 32 bytes, but this is
theoretically possible with TLSv1.3, where session ids are essentially
arbitrary and sent as session tickets.  Since on 64-bit platforms we
use fixed 32-byte buffer for session ids, added an explicit length check
to make sure the buffer is large enough.

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -3842,6 +3842,14 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_
     p = buf;
     i2d_SSL_SESSION(sess, &p);
 
+    session_id = (u_char *) SSL_SESSION_get_id(sess, &session_id_length);
+
+    /* do not cache sessions with too long session id */
+
+    if (session_id_length > 32) {
+        return 0;
+    }
+
     c = ngx_ssl_get_connection(ssl_conn);
 
     ssl_ctx = c->ssl->session_ctx;
@@ -3886,8 +3894,6 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_
         }
     }
 
-    session_id = (u_char *) SSL_SESSION_get_id(sess, &session_id_length);
-
 #if (NGX_PTR_SIZE == 8)
 
     id = sess_id->sess_id;


From mdounin at mdounin.ru  Fri Aug 26 03:01:16 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Fri, 26 Aug 2022 06:01:16 +0300
Subject: [PATCH 09 of 11] SSL: renamed session ticket key functions and data
 index
In-Reply-To: <patchbomb.1661482867@vm-bsd.mdounin.ru>
References: <patchbomb.1661482867@vm-bsd.mdounin.ru>
Message-ID: <97158a99745be75aeea0.1661482876@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1661481955 -10800
#      Fri Aug 26 05:45:55 2022 +0300
# Node ID 97158a99745be75aeea01d8d1b8cd8666faab2b2
# Parent  92c96bf72bc64b49bf8825a8c1d6159af4cb3d32
SSL: renamed session ticket key functions and data index.

Previously used names are way too long, renamed to simplify writing code.

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -71,10 +71,10 @@ static void ngx_ssl_session_rbtree_inser
     ngx_rbtree_node_t *node, ngx_rbtree_node_t *sentinel);
 
 #ifdef SSL_CTRL_SET_TLSEXT_TICKET_KEY_CB
-static int ngx_ssl_session_ticket_key_callback(ngx_ssl_conn_t *ssl_conn,
+static int ngx_ssl_ticket_key_callback(ngx_ssl_conn_t *ssl_conn,
     unsigned char *name, unsigned char *iv, EVP_CIPHER_CTX *ectx,
     HMAC_CTX *hctx, int enc);
-static void ngx_ssl_session_ticket_keys_cleanup(void *data);
+static void ngx_ssl_ticket_keys_cleanup(void *data);
 #endif
 
 #ifndef X509_CHECK_FLAG_ALWAYS_CHECK_SUBJECT
@@ -131,7 +131,7 @@ ngx_module_t  ngx_openssl_module = {
 int  ngx_ssl_connection_index;
 int  ngx_ssl_server_conf_index;
 int  ngx_ssl_session_cache_index;
-int  ngx_ssl_session_ticket_keys_index;
+int  ngx_ssl_ticket_keys_index;
 int  ngx_ssl_ocsp_index;
 int  ngx_ssl_certificate_index;
 int  ngx_ssl_next_certificate_index;
@@ -208,9 +208,9 @@ ngx_ssl_init(ngx_log_t *log)
         return NGX_ERROR;
     }
 
-    ngx_ssl_session_ticket_keys_index = SSL_CTX_get_ex_new_index(0, NULL, NULL,
-                                                                 NULL, NULL);
-    if (ngx_ssl_session_ticket_keys_index == -1) {
+    ngx_ssl_ticket_keys_index = SSL_CTX_get_ex_new_index(0, NULL, NULL, NULL,
+                                                         NULL);
+    if (ngx_ssl_ticket_keys_index == -1) {
         ngx_ssl_error(NGX_LOG_ALERT, log, 0,
                       "SSL_CTX_get_ex_new_index() failed");
         return NGX_ERROR;
@@ -4249,7 +4249,7 @@ ngx_ssl_session_ticket_keys(ngx_conf_t *
         return NGX_ERROR;
     }
 
-    cln->handler = ngx_ssl_session_ticket_keys_cleanup;
+    cln->handler = ngx_ssl_ticket_keys_cleanup;
     cln->data = keys;
 
     path = paths->elts;
@@ -4327,16 +4327,13 @@ ngx_ssl_session_ticket_keys(ngx_conf_t *
         ngx_explicit_memzero(&buf, 80);
     }
 
-    if (SSL_CTX_set_ex_data(ssl->ctx, ngx_ssl_session_ticket_keys_index, keys)
-        == 0)
-    {
+    if (SSL_CTX_set_ex_data(ssl->ctx, ngx_ssl_ticket_keys_index, keys) == 0) {
         ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0,
                       "SSL_CTX_set_ex_data() failed");
         return NGX_ERROR;
     }
 
-    if (SSL_CTX_set_tlsext_ticket_key_cb(ssl->ctx,
-                                         ngx_ssl_session_ticket_key_callback)
+    if (SSL_CTX_set_tlsext_ticket_key_cb(ssl->ctx, ngx_ssl_ticket_key_callback)
         == 0)
     {
         ngx_log_error(NGX_LOG_WARN, cf->log, 0,
@@ -4362,7 +4359,7 @@ failed:
 
 
 static int
-ngx_ssl_session_ticket_key_callback(ngx_ssl_conn_t *ssl_conn,
+ngx_ssl_ticket_key_callback(ngx_ssl_conn_t *ssl_conn,
     unsigned char *name, unsigned char *iv, EVP_CIPHER_CTX *ectx,
     HMAC_CTX *hctx, int enc)
 {
@@ -4384,7 +4381,7 @@ ngx_ssl_session_ticket_key_callback(ngx_
     digest = EVP_sha256();
 #endif
 
-    keys = SSL_CTX_get_ex_data(ssl_ctx, ngx_ssl_session_ticket_keys_index);
+    keys = SSL_CTX_get_ex_data(ssl_ctx, ngx_ssl_ticket_keys_index);
     if (keys == NULL) {
         return -1;
     }
@@ -4497,7 +4494,7 @@ ngx_ssl_session_ticket_key_callback(ngx_
 
 
 static void
-ngx_ssl_session_ticket_keys_cleanup(void *data)
+ngx_ssl_ticket_keys_cleanup(void *data)
 {
     ngx_array_t  *keys = data;
 
diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h
--- a/src/event/ngx_event_openssl.h
+++ b/src/event/ngx_event_openssl.h
@@ -317,7 +317,7 @@ void ngx_ssl_cleanup_ctx(void *data);
 extern int  ngx_ssl_connection_index;
 extern int  ngx_ssl_server_conf_index;
 extern int  ngx_ssl_session_cache_index;
-extern int  ngx_ssl_session_ticket_keys_index;
+extern int  ngx_ssl_ticket_keys_index;
 extern int  ngx_ssl_ocsp_index;
 extern int  ngx_ssl_certificate_index;
 extern int  ngx_ssl_next_certificate_index;


From mdounin at mdounin.ru  Fri Aug 26 03:01:15 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Fri, 26 Aug 2022 06:01:15 +0300
Subject: [PATCH 08 of 11] SSL: renamed session ticket key type
In-Reply-To: <patchbomb.1661482867@vm-bsd.mdounin.ru>
References: <patchbomb.1661482867@vm-bsd.mdounin.ru>
Message-ID: <92c96bf72bc64b49bf88.1661482875@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1661481954 -10800
#      Fri Aug 26 05:45:54 2022 +0300
# Node ID 92c96bf72bc64b49bf8825a8c1d6159af4cb3d32
# Parent  84919c2ee8173f704649a8cb4901887e1bf79588
SSL: renamed session ticket key type.

The ngx_ssl_session_ticket_key_t is way too long, renamed to
ngx_ssl_ticket_key_t to simplify writing code.

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -4223,23 +4223,23 @@ ngx_ssl_session_rbtree_insert_value(ngx_
 ngx_int_t
 ngx_ssl_session_ticket_keys(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_array_t *paths)
 {
-    u_char                         buf[80];
-    size_t                         size;
-    ssize_t                        n;
-    ngx_str_t                     *path;
-    ngx_file_t                     file;
-    ngx_uint_t                     i;
-    ngx_array_t                   *keys;
-    ngx_file_info_t                fi;
-    ngx_pool_cleanup_t            *cln;
-    ngx_ssl_session_ticket_key_t  *key;
+    u_char                 buf[80];
+    size_t                 size;
+    ssize_t                n;
+    ngx_str_t             *path;
+    ngx_file_t             file;
+    ngx_uint_t             i;
+    ngx_array_t           *keys;
+    ngx_file_info_t        fi;
+    ngx_pool_cleanup_t    *cln;
+    ngx_ssl_ticket_key_t  *key;
 
     if (paths == NULL) {
         return NGX_OK;
     }
 
     keys = ngx_array_create(cf->pool, paths->nelts,
-                            sizeof(ngx_ssl_session_ticket_key_t));
+                            sizeof(ngx_ssl_ticket_key_t));
     if (keys == NULL) {
         return NGX_ERROR;
     }
@@ -4366,14 +4366,14 @@ ngx_ssl_session_ticket_key_callback(ngx_
     unsigned char *name, unsigned char *iv, EVP_CIPHER_CTX *ectx,
     HMAC_CTX *hctx, int enc)
 {
-    size_t                         size;
-    SSL_CTX                       *ssl_ctx;
-    ngx_uint_t                     i;
-    ngx_array_t                   *keys;
-    ngx_connection_t              *c;
-    ngx_ssl_session_ticket_key_t  *key;
-    const EVP_MD                  *digest;
-    const EVP_CIPHER              *cipher;
+    size_t                 size;
+    SSL_CTX               *ssl_ctx;
+    ngx_uint_t             i;
+    ngx_array_t           *keys;
+    ngx_connection_t      *c;
+    ngx_ssl_ticket_key_t  *key;
+    const EVP_MD          *digest;
+    const EVP_CIPHER      *cipher;
 
     c = ngx_ssl_get_connection(ssl_conn);
     ssl_ctx = c->ssl->session_ctx;
@@ -4502,7 +4502,7 @@ ngx_ssl_session_ticket_keys_cleanup(void
     ngx_array_t  *keys = data;
 
     ngx_explicit_memzero(keys->elts,
-                         keys->nelts * sizeof(ngx_ssl_session_ticket_key_t));
+                         keys->nelts * sizeof(ngx_ssl_ticket_key_t));
 }
 
 #else
diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h
--- a/src/event/ngx_event_openssl.h
+++ b/src/event/ngx_event_openssl.h
@@ -161,7 +161,7 @@ typedef struct {
     u_char                      name[16];
     u_char                      hmac_key[32];
     u_char                      aes_key[32];
-} ngx_ssl_session_ticket_key_t;
+} ngx_ssl_ticket_key_t;
 
 #endif
 


From mdounin at mdounin.ru  Fri Aug 26 03:01:14 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Fri, 26 Aug 2022 06:01:14 +0300
Subject: [PATCH 07 of 11] SSL: style
In-Reply-To: <patchbomb.1661482867@vm-bsd.mdounin.ru>
References: <patchbomb.1661482867@vm-bsd.mdounin.ru>
Message-ID: <84919c2ee8173f704649.1661482874@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1661481953 -10800
#      Fri Aug 26 05:45:53 2022 +0300
# Node ID 84919c2ee8173f704649a8cb4901887e1bf79588
# Parent  d5c6eae914325fb6a9b19105fe09aecd04da21e2
SSL: style.

Runtime OCSP functions separated from configuration ones.

diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h
--- a/src/event/ngx_event_openssl.h
+++ b/src/event/ngx_event_openssl.h
@@ -205,10 +205,12 @@ ngx_int_t ngx_ssl_ocsp(ngx_conf_t *cf, n
     ngx_uint_t depth, ngx_shm_zone_t *shm_zone);
 ngx_int_t ngx_ssl_ocsp_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl,
     ngx_resolver_t *resolver, ngx_msec_t resolver_timeout);
+
 ngx_int_t ngx_ssl_ocsp_validate(ngx_connection_t *c);
 ngx_int_t ngx_ssl_ocsp_get_status(ngx_connection_t *c, const char **s);
 void ngx_ssl_ocsp_cleanup(ngx_connection_t *c);
 ngx_int_t ngx_ssl_ocsp_cache_init(ngx_shm_zone_t *shm_zone, void *data);
+
 ngx_array_t *ngx_ssl_read_password_file(ngx_conf_t *cf, ngx_str_t *file);
 ngx_array_t *ngx_ssl_preserve_passwords(ngx_conf_t *cf,
     ngx_array_t *passwords);


From mdounin at mdounin.ru  Fri Aug 26 03:01:17 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Fri, 26 Aug 2022 06:01:17 +0300
Subject: [PATCH 10 of 11] SSL: shorter debug messages about session tickets
In-Reply-To: <patchbomb.1661482867@vm-bsd.mdounin.ru>
References: <patchbomb.1661482867@vm-bsd.mdounin.ru>
Message-ID: <2487bf5766f79c813b33.1661482877@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1661481957 -10800
#      Fri Aug 26 05:45:57 2022 +0300
# Node ID 2487bf5766f79c813b3397b3bb897424c3590445
# Parent  97158a99745be75aeea01d8d1b8cd8666faab2b2
SSL: shorter debug messages about session tickets.

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -4392,7 +4392,7 @@ ngx_ssl_ticket_key_callback(ngx_ssl_conn
         /* encrypt session ticket */
 
         ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0,
-                       "ssl session ticket encrypt, key: \"%*xs\" (%s session)",
+                       "ssl ticket encrypt, key: \"%*xs\" (%s session)",
                        (size_t) 16, key[0].name,
                        SSL_session_reused(ssl_conn) ? "reused" : "new");
 
@@ -4439,7 +4439,7 @@ ngx_ssl_ticket_key_callback(ngx_ssl_conn
         }
 
         ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0,
-                       "ssl session ticket decrypt, key: \"%*xs\" not found",
+                       "ssl ticket decrypt, key: \"%*xs\" not found",
                        (size_t) 16, name);
 
         return 0;
@@ -4447,7 +4447,7 @@ ngx_ssl_ticket_key_callback(ngx_ssl_conn
     found:
 
         ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0,
-                       "ssl session ticket decrypt, key: \"%*xs\"%s",
+                       "ssl ticket decrypt, key: \"%*xs\"%s",
                        (size_t) 16, key[i].name, (i == 0) ? " (default)" : "");
 
         if (key[i].size == 48) {


From mdounin at mdounin.ru  Fri Aug 26 03:01:12 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Fri, 26 Aug 2022 06:01:12 +0300
Subject: [PATCH 05 of 11] SSL: single allocation in session cache on 32-bit
 platforms
In-Reply-To: <patchbomb.1661482867@vm-bsd.mdounin.ru>
References: <patchbomb.1661482867@vm-bsd.mdounin.ru>
Message-ID: <e88baee178eed529c617.1661482872@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1661481950 -10800
#      Fri Aug 26 05:45:50 2022 +0300
# Node ID e88baee178eed529c6170678e373f5e2e0883c37
# Parent  f4ae0f4ee928cf20346530e96f1431314ecd0171
SSL: single allocation in session cache on 32-bit platforms.

Given the present typical SSL session sizes, on 32-bit platforms it is
now beneficial to store all data in a single allocation, since rbtree
node + session id + ASN1 representation of a session takes 256 bytes of
shared memory (36 + 32 + 150 = about 218 bytes plus SNI server name).

Storing all data in a single allocation is beneficial for SNI names up to
about 40 characters long and makes it possible to store about 4000 sessions
in one megabyte (instead of about 3000 sessions now).  This also slightly
simplifies the code.

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -3789,9 +3789,9 @@ ngx_ssl_session_cache_init(ngx_shm_zone_
  * Typical length of the external ASN1 representation of a session
  * is about 150 bytes plus SNI server name.
  *
- * On 32-bit platforms we allocate separately an rbtree node,
- * a session id, and an ASN1 representation, they take accordingly
- * 64, 32, and 256 bytes.
+ * On 32-bit platforms we allocate an rbtree node, a session id, and
+ * an ASN1 representation in a single allocation, it typically takes
+ * 256 bytes.
  *
  * On 64-bit platforms we allocate separately an rbtree node + session_id,
  * and an ASN1 representation, they take accordingly 128 and 256 bytes.
@@ -3804,7 +3804,8 @@ static int
 ngx_ssl_new_session(ngx_ssl_conn_t *ssl_conn, ngx_ssl_session_t *sess)
 {
     int                       len;
-    u_char                   *p, *id, *cached_sess, *session_id;
+    u_char                   *p, *session_id;
+    size_t                    n;
     uint32_t                  hash;
     SSL_CTX                  *ssl_ctx;
     unsigned int              session_id_length;
@@ -3863,23 +3864,13 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_
     /* drop one or two expired sessions */
     ngx_ssl_expire_sessions(cache, shpool, 1);
 
-    cached_sess = ngx_slab_alloc_locked(shpool, len);
-
-    if (cached_sess == NULL) {
-
-        /* drop the oldest non-expired session and try once more */
-
-        ngx_ssl_expire_sessions(cache, shpool, 0);
-
-        cached_sess = ngx_slab_alloc_locked(shpool, len);
-
-        if (cached_sess == NULL) {
-            sess_id = NULL;
-            goto failed;
-        }
-    }
-
-    sess_id = ngx_slab_alloc_locked(shpool, sizeof(ngx_ssl_sess_id_t));
+#if (NGX_PTR_SIZE == 8)
+    n = sizeof(ngx_ssl_sess_id_t);
+#else
+    n = offsetof(ngx_ssl_sess_id_t, session) + len;
+#endif
+
+    sess_id = ngx_slab_alloc_locked(shpool, n);
 
     if (sess_id == NULL) {
 
@@ -3887,7 +3878,7 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_
 
         ngx_ssl_expire_sessions(cache, shpool, 0);
 
-        sess_id = ngx_slab_alloc_locked(shpool, sizeof(ngx_ssl_sess_id_t));
+        sess_id = ngx_slab_alloc_locked(shpool, n);
 
         if (sess_id == NULL) {
             goto failed;
@@ -3896,30 +3887,25 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_
 
 #if (NGX_PTR_SIZE == 8)
 
-    id = sess_id->sess_id;
-
-#else
-
-    id = ngx_slab_alloc_locked(shpool, session_id_length);
-
-    if (id == NULL) {
+    sess_id->session = ngx_slab_alloc_locked(shpool, len);
+
+    if (sess_id->session == NULL) {
 
         /* drop the oldest non-expired session and try once more */
 
         ngx_ssl_expire_sessions(cache, shpool, 0);
 
-        id = ngx_slab_alloc_locked(shpool, session_id_length);
-
-        if (id == NULL) {
+        sess_id->session = ngx_slab_alloc_locked(shpool, len);
+
+        if (sess_id->session == NULL) {
             goto failed;
         }
     }
 
 #endif
 
-    ngx_memcpy(cached_sess, buf, len);
-
-    ngx_memcpy(id, session_id, session_id_length);
+    ngx_memcpy(sess_id->session, buf, len);
+    ngx_memcpy(sess_id->id, session_id, session_id_length);
 
     hash = ngx_crc32_short(session_id, session_id_length);
 
@@ -3929,9 +3915,7 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_
 
     sess_id->node.key = hash;
     sess_id->node.data = (u_char) session_id_length;
-    sess_id->id = id;
     sess_id->len = len;
-    sess_id->session = cached_sess;
 
     sess_id->expire = ngx_time() + SSL_CTX_get_timeout(ssl_ctx);
 
@@ -3945,10 +3929,6 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_
 
 failed:
 
-    if (cached_sess) {
-        ngx_slab_free_locked(shpool, cached_sess);
-    }
-
     if (sess_id) {
         ngx_slab_free_locked(shpool, sess_id);
     }
@@ -4045,9 +4025,8 @@ ngx_ssl_get_cached_session(ngx_ssl_conn_
 
             ngx_rbtree_delete(&cache->session_rbtree, node);
 
+#if (NGX_PTR_SIZE == 8)
             ngx_slab_free_locked(shpool, sess_id->session);
-#if (NGX_PTR_SIZE == 4)
-            ngx_slab_free_locked(shpool, sess_id->id);
 #endif
             ngx_slab_free_locked(shpool, sess_id);
 
@@ -4135,9 +4114,8 @@ ngx_ssl_remove_session(SSL_CTX *ssl, ngx
 
             ngx_rbtree_delete(&cache->session_rbtree, node);
 
+#if (NGX_PTR_SIZE == 8)
             ngx_slab_free_locked(shpool, sess_id->session);
-#if (NGX_PTR_SIZE == 4)
-            ngx_slab_free_locked(shpool, sess_id->id);
 #endif
             ngx_slab_free_locked(shpool, sess_id);
 
@@ -4184,9 +4162,8 @@ ngx_ssl_expire_sessions(ngx_ssl_session_
 
         ngx_rbtree_delete(&cache->session_rbtree, &sess_id->node);
 
+#if (NGX_PTR_SIZE == 8)
         ngx_slab_free_locked(shpool, sess_id->session);
-#if (NGX_PTR_SIZE == 4)
-        ngx_slab_free_locked(shpool, sess_id->id);
 #endif
         ngx_slab_free_locked(shpool, sess_id);
     }
diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h
--- a/src/event/ngx_event_openssl.h
+++ b/src/event/ngx_event_openssl.h
@@ -134,14 +134,14 @@ typedef struct ngx_ssl_sess_id_s  ngx_ss
 
 struct ngx_ssl_sess_id_s {
     ngx_rbtree_node_t           node;
-    u_char                     *id;
     size_t                      len;
-    u_char                     *session;
     ngx_queue_t                 queue;
     time_t                      expire;
+    u_char                      id[32];
 #if (NGX_PTR_SIZE == 8)
-    void                       *stub;
-    u_char                      sess_id[32];
+    u_char                     *session;
+#else
+    u_char                      session[1];
 #endif
 };
 


From mdounin at mdounin.ru  Fri Aug 26 03:01:13 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Fri, 26 Aug 2022 06:01:13 +0300
Subject: [PATCH 06 of 11] SSL: explicit clearing of expired sessions
In-Reply-To: <patchbomb.1661482867@vm-bsd.mdounin.ru>
References: <patchbomb.1661482867@vm-bsd.mdounin.ru>
Message-ID: <d5c6eae914325fb6a9b1.1661482873@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1661481952 -10800
#      Fri Aug 26 05:45:52 2022 +0300
# Node ID d5c6eae914325fb6a9b19105fe09aecd04da21e2
# Parent  e88baee178eed529c6170678e373f5e2e0883c37
SSL: explicit clearing of expired sessions.

This reduces lifetime of session keying material in server's memory, and
therefore can be beneficial from forward secrecy point of view.

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -4025,6 +4025,8 @@ ngx_ssl_get_cached_session(ngx_ssl_conn_
 
             ngx_rbtree_delete(&cache->session_rbtree, node);
 
+            ngx_explicit_memzero(sess_id->session, sess_id->len);
+
 #if (NGX_PTR_SIZE == 8)
             ngx_slab_free_locked(shpool, sess_id->session);
 #endif
@@ -4114,6 +4116,8 @@ ngx_ssl_remove_session(SSL_CTX *ssl, ngx
 
             ngx_rbtree_delete(&cache->session_rbtree, node);
 
+            ngx_explicit_memzero(sess_id->session, sess_id->len);
+
 #if (NGX_PTR_SIZE == 8)
             ngx_slab_free_locked(shpool, sess_id->session);
 #endif
@@ -4162,6 +4166,8 @@ ngx_ssl_expire_sessions(ngx_ssl_session_
 
         ngx_rbtree_delete(&cache->session_rbtree, &sess_id->node);
 
+        ngx_explicit_memzero(sess_id->session, sess_id->len);
+
 #if (NGX_PTR_SIZE == 8)
         ngx_slab_free_locked(shpool, sess_id->session);
 #endif


From mdounin at mdounin.ru  Fri Aug 26 03:01:18 2022
From: mdounin at mdounin.ru (=?utf-8?q?Maxim_Dounin?=)
Date: Fri, 26 Aug 2022 06:01:18 +0300
Subject: [PATCH 11 of 11] SSL: automatic rotation of session ticket keys
In-Reply-To: <patchbomb.1661482867@vm-bsd.mdounin.ru>
References: <patchbomb.1661482867@vm-bsd.mdounin.ru>
Message-ID: <5c26fe5f6ab0bf4c0d18.1661482878@vm-bsd.mdounin.ru>

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1661481958 -10800
#      Fri Aug 26 05:45:58 2022 +0300
# Node ID 5c26fe5f6ab0bf4c0d18cae8f6f6483348243d4b
# Parent  2487bf5766f79c813b3397b3bb897424c3590445
SSL: automatic rotation of session ticket keys.

As long as ssl_session_cache in shared memory is configured, session ticket
keys are now automatically generated in shared memory, and rotated
periodically.  This can be beneficial from forward secrecy point of view,
and also avoids increased CPU usage after configuration reloads.

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -74,6 +74,7 @@ static void ngx_ssl_session_rbtree_inser
 static int ngx_ssl_ticket_key_callback(ngx_ssl_conn_t *ssl_conn,
     unsigned char *name, unsigned char *iv, EVP_CIPHER_CTX *ectx,
     HMAC_CTX *hctx, int enc);
+static ngx_int_t ngx_ssl_rotate_ticket_keys(SSL_CTX *ssl_ctx, ngx_log_t *log);
 static void ngx_ssl_ticket_keys_cleanup(void *data);
 #endif
 
@@ -3767,6 +3768,9 @@ ngx_ssl_session_cache_init(ngx_shm_zone_
 
     ngx_queue_init(&cache->expire_queue);
 
+    cache->ticket_keys[0].expire = 0;
+    cache->ticket_keys[1].expire = 0;
+
     len = sizeof(" in SSL session shared cache \"\"") + shm_zone->shm.name.len;
 
     shpool->log_ctx = ngx_slab_alloc(shpool, len);
@@ -4234,11 +4238,13 @@ ngx_ssl_session_ticket_keys(ngx_conf_t *
     ngx_pool_cleanup_t    *cln;
     ngx_ssl_ticket_key_t  *key;
 
-    if (paths == NULL) {
+    if (paths == NULL
+        && SSL_CTX_get_ex_data(ssl->ctx, ngx_ssl_session_cache_index) == NULL)
+    {
         return NGX_OK;
     }
 
-    keys = ngx_array_create(cf->pool, paths->nelts,
+    keys = ngx_array_create(cf->pool, paths ? paths->nelts : 2,
                             sizeof(ngx_ssl_ticket_key_t));
     if (keys == NULL) {
         return NGX_ERROR;
@@ -4252,6 +4258,34 @@ ngx_ssl_session_ticket_keys(ngx_conf_t *
     cln->handler = ngx_ssl_ticket_keys_cleanup;
     cln->data = keys;
 
+    if (SSL_CTX_set_ex_data(ssl->ctx, ngx_ssl_ticket_keys_index, keys) == 0) {
+        ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0,
+                      "SSL_CTX_set_ex_data() failed");
+        return NGX_ERROR;
+    }
+
+    if (SSL_CTX_set_tlsext_ticket_key_cb(ssl->ctx, ngx_ssl_ticket_key_callback)
+        == 0)
+    {
+        ngx_log_error(NGX_LOG_WARN, cf->log, 0,
+                      "nginx was built with Session Tickets support, however, "
+                      "now it is linked dynamically to an OpenSSL library "
+                      "which has no tlsext support, therefore Session Tickets "
+                      "are not available");
+        return NGX_OK;
+    }
+
+    if (paths == NULL) {
+
+        /* placeholder for keys in shared memory */
+
+        key = ngx_array_push_n(keys, 2);
+        key[0].shared = 1;
+        key[1].shared = 1;
+
+        return NGX_OK;
+    }
+
     path = paths->elts;
     for (i = 0; i < paths->nelts; i++) {
 
@@ -4306,6 +4340,8 @@ ngx_ssl_session_ticket_keys(ngx_conf_t *
             goto failed;
         }
 
+        key->shared = 0;
+
         if (size == 48) {
             key->size = 48;
             ngx_memcpy(key->name, buf, 16);
@@ -4327,22 +4363,6 @@ ngx_ssl_session_ticket_keys(ngx_conf_t *
         ngx_explicit_memzero(&buf, 80);
     }
 
-    if (SSL_CTX_set_ex_data(ssl->ctx, ngx_ssl_ticket_keys_index, keys) == 0) {
-        ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0,
-                      "SSL_CTX_set_ex_data() failed");
-        return NGX_ERROR;
-    }
-
-    if (SSL_CTX_set_tlsext_ticket_key_cb(ssl->ctx, ngx_ssl_ticket_key_callback)
-        == 0)
-    {
-        ngx_log_error(NGX_LOG_WARN, cf->log, 0,
-                      "nginx was built with Session Tickets support, however, "
-                      "now it is linked dynamically to an OpenSSL library "
-                      "which has no tlsext support, therefore Session Tickets "
-                      "are not available");
-    }
-
     return NGX_OK;
 
 failed:
@@ -4375,6 +4395,10 @@ ngx_ssl_ticket_key_callback(ngx_ssl_conn
     c = ngx_ssl_get_connection(ssl_conn);
     ssl_ctx = c->ssl->session_ctx;
 
+    if (ngx_ssl_rotate_ticket_keys(ssl_ctx, c->log) != NGX_OK) {
+        return -1;
+    }
+
 #ifdef OPENSSL_NO_SHA256
     digest = EVP_sha1();
 #else
@@ -4493,6 +4517,113 @@ ngx_ssl_ticket_key_callback(ngx_ssl_conn
 }
 
 
+static ngx_int_t
+ngx_ssl_rotate_ticket_keys(SSL_CTX *ssl_ctx, ngx_log_t *log)
+{
+    time_t                    now, expire;
+    ngx_array_t              *keys;
+    ngx_shm_zone_t           *shm_zone;
+    ngx_slab_pool_t          *shpool;
+    ngx_ssl_ticket_key_t     *key;
+    ngx_ssl_session_cache_t  *cache;
+    u_char                    buf[80];
+
+    keys = SSL_CTX_get_ex_data(ssl_ctx, ngx_ssl_ticket_keys_index);
+    if (keys == NULL) {
+        return NGX_OK;
+    }
+
+    key = keys->elts;
+
+    if (!key[0].shared) {
+        return NGX_OK;
+    }
+
+    now = ngx_time();
+    expire = now + SSL_CTX_get_timeout(ssl_ctx);
+
+    shm_zone = SSL_CTX_get_ex_data(ssl_ctx, ngx_ssl_session_cache_index);
+
+    cache = shm_zone->data;
+    shpool = (ngx_slab_pool_t *) shm_zone->shm.addr;
+
+    ngx_shmtx_lock(&shpool->mutex);
+
+    key = cache->ticket_keys;
+
+    if (key[0].expire == 0) {
+
+        /* initialize the current key */
+
+        if (RAND_bytes(buf, 80) != 1) {
+            ngx_ssl_error(NGX_LOG_ALERT, log, 0, "RAND_bytes() failed");
+            ngx_shmtx_unlock(&shpool->mutex);
+            return NGX_ERROR;
+        }
+
+        key->shared = 1;
+        key->expire = expire;
+        key->size = 80;
+        ngx_memcpy(key->name, buf, 16);
+        ngx_memcpy(key->hmac_key, buf + 16, 32);
+        ngx_memcpy(key->aes_key, buf + 48, 32);
+
+        ngx_explicit_memzero(&buf, 80);
+
+        ngx_log_debug2(NGX_LOG_DEBUG_EVENT, log, 0,
+                       "ssl ticket key: \"%*xs\"",
+                       (size_t) 16, key->name);
+    }
+
+    if (key[1].expire < now) {
+
+        /*
+         * if the previous key is no longer needed (or not initialized),
+         * replace it with the current key and generate new current key
+         */
+
+        key[1] = key[0];
+
+        if (RAND_bytes(buf, 80) != 1) {
+            ngx_ssl_error(NGX_LOG_ALERT, log, 0, "RAND_bytes() failed");
+            ngx_shmtx_unlock(&shpool->mutex);
+            return NGX_ERROR;
+        }
+
+        key->shared = 1;
+        key->expire = expire;
+        key->size = 80;
+        ngx_memcpy(key->name, buf, 16);
+        ngx_memcpy(key->hmac_key, buf + 16, 32);
+        ngx_memcpy(key->aes_key, buf + 48, 32);
+
+        ngx_explicit_memzero(&buf, 80);
+
+        ngx_log_debug2(NGX_LOG_DEBUG_EVENT, log, 0,
+                       "ssl ticket key: \"%*xs\"",
+                       (size_t) 16, key->name);
+    }
+
+    /*
+     * update expiration of the current key: it is going to be needed
+     * at least till the session being created expires
+     */
+
+    if (expire > key[0].expire) {
+        key[0].expire = expire;
+    }
+
+    /* sync keys to the worker process memory */
+
+    ngx_memcpy(keys->elts, cache->ticket_keys,
+               2 * sizeof(ngx_ssl_ticket_key_t));
+
+    ngx_shmtx_unlock(&shpool->mutex);
+
+    return NGX_OK;
+}
+
+
 static void
 ngx_ssl_ticket_keys_cleanup(void *data)
 {
diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h
--- a/src/event/ngx_event_openssl.h
+++ b/src/event/ngx_event_openssl.h
@@ -147,25 +147,24 @@ struct ngx_ssl_sess_id_s {
 
 
 typedef struct {
+    u_char                      name[16];
+    u_char                      hmac_key[32];
+    u_char                      aes_key[32];
+    time_t                      expire;
+    unsigned                    size:8;
+    unsigned                    shared:1;
+} ngx_ssl_ticket_key_t;
+
+
+typedef struct {
     ngx_rbtree_t                session_rbtree;
     ngx_rbtree_node_t           sentinel;
     ngx_queue_t                 expire_queue;
+    ngx_ssl_ticket_key_t        ticket_keys[2];
     time_t                      fail_time;
 } ngx_ssl_session_cache_t;
 
 
-#ifdef SSL_CTRL_SET_TLSEXT_TICKET_KEY_CB
-
-typedef struct {
-    size_t                      size;
-    u_char                      name[16];
-    u_char                      hmac_key[32];
-    u_char                      aes_key[32];
-} ngx_ssl_ticket_key_t;
-
-#endif
-
-
 #define NGX_SSL_SSLv2    0x0002
 #define NGX_SSL_SSLv3    0x0004
 #define NGX_SSL_TLSv1    0x0008


From mdounin at mdounin.ru  Mon Aug 29 23:06:36 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 30 Aug 2022 02:06:36 +0300
Subject: [PATCH] SSL: logging level of client issue during TLSv1.3
In-Reply-To: <9B8DF086-1BAA-4F3D-BE16-392DBC81E887@gmail.com>
References: <2BCAB58C-861D-46FC-8750-AD8137116F27@gmail.com>
 <9B8DF086-1BAA-4F3D-BE16-392DBC81E887@gmail.com>
Message-ID: <Yw1GfGNjrnIXyr4Z@mdounin.ru>

Hello!

On Wed, Aug 10, 2022 at 07:15:29PM -0300, Murilo Andrade wrote:

> > Are you seeing these errors in practice?
> Yes, it's happening 310 times an hour. We are seeing in log like this:
>     2022/07/26 13:49:51 [crit] 1316#1316: *147702846 SSL_read() failed =
> (SSL: error:1408F1BB:SSL routines:ssl3_get_record:bad record type) while =
> waiting for request

Thanks for the details, I was able to reproduce this.

Pushed to http://mdounin.ru/hg/nginx/ with slightly cleaned up 
commit log.

-- 
Maxim Dounin
http://mdounin.ru/


From xeioex at nginx.com  Mon Aug 29 23:42:53 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Mon, 29 Aug 2022 23:42:53 +0000
Subject: [njs] WebCrypto: fixed sign and verify methods for ECDSA.
Message-ID: <hg.f70929728e86.1661816573.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/f70929728e86
branches:  
changeset: 1936:f70929728e86
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Aug 26 21:49:14 2022 -0700
description:
WebCrypto: fixed sign and verify methods for ECDSA.

diffstat:

 external/njs_openssl.h                      |    2 +
 external/njs_webcrypto_module.c             |  277 ++++++++++++++++++++++++++++
 test/webcrypto/README.rst                   |   12 +-
 test/webcrypto/asn12ieeep1336.c             |   49 ++++
 test/webcrypto/text.base64.sha1.ecdsa.sig   |    4 +-
 test/webcrypto/text.base64.sha256.ecdsa.sig |    4 +-
 6 files changed, 342 insertions(+), 6 deletions(-)

diffs (403 lines):

diff -r 43b31a943c08 -r f70929728e86 external/njs_openssl.h
--- a/external/njs_openssl.h	Thu Aug 25 16:57:28 2022 -0700
+++ b/external/njs_openssl.h	Fri Aug 26 21:49:14 2022 -0700
@@ -43,6 +43,8 @@
 #else
 #define njs_evp_md_ctx_new()  EVP_MD_CTX_create()
 #define njs_evp_md_ctx_free(_ctx)  EVP_MD_CTX_destroy(_ctx)
+#define ECDSA_SIG_get0_s(sig) (sig)->s
+#define ECDSA_SIG_get0_r(sig) (sig)->r
 #endif
 
 
diff -r 43b31a943c08 -r f70929728e86 external/njs_webcrypto_module.c
--- a/external/njs_webcrypto_module.c	Thu Aug 25 16:57:28 2022 -0700
+++ b/external/njs_webcrypto_module.c	Fri Aug 26 21:49:14 2022 -0700
@@ -1935,6 +1935,266 @@ njs_set_rsa_padding(njs_vm_t *vm, njs_va
 }
 
 
+static const EC_KEY *
+njs_pkey_get_ec_key(EVP_PKEY *pkey)
+{
+#if (OPENSSL_VERSION_NUMBER >= 0x10100000L)
+    return EVP_PKEY_get0_EC_KEY(pkey);
+#else
+    if (pkey->type != EVP_PKEY_EC) {
+        return NULL;
+    }
+
+    return pkey->pkey.ec;
+#endif
+}
+
+
+static int
+njs_ec_group_order_bits(const EC_GROUP *group)
+{
+#if (OPENSSL_VERSION_NUMBER >= 0x10100000L)
+    return EC_GROUP_order_bits(group);
+#else
+    int     bits;
+    BIGNUM  *order;
+
+    order = BN_new();
+    if (order == NULL) {
+        return 0;
+    }
+
+    if (EC_GROUP_get_order(group, order, NULL) == 0) {
+        return 0;
+    }
+
+    bits = BN_num_bits(order);
+
+    BN_free(order);
+
+    return bits;
+#endif
+}
+
+
+static unsigned int
+njs_ec_rs_size(EVP_PKEY *pkey)
+{
+    int             bits;
+    const EC_KEY    *ec_key;
+    const EC_GROUP  *ec_group;
+
+    ec_key = njs_pkey_get_ec_key(pkey);
+    if (ec_key == NULL) {
+        return 0;
+    }
+
+    ec_group = EC_KEY_get0_group(ec_key);
+    if (ec_group == NULL) {
+        return 0;
+    }
+
+    bits = njs_ec_group_order_bits(ec_group);
+    if (bits == 0) {
+        return 0;
+    }
+
+    return (bits + 7) / 8;
+}
+
+
+static int
+njs_bn_bn2binpad(const BIGNUM *bn, unsigned char *to, int tolen)
+{
+#if (OPENSSL_VERSION_NUMBER >= 0x10100000L)
+    return BN_bn2binpad(bn, to, tolen);
+#else
+    return BN_bn2bin(bn, &to[tolen - BN_num_bytes(bn)]);
+#endif
+}
+
+
+static njs_int_t
+njs_convert_der_to_p1363(njs_vm_t *vm, EVP_PKEY *pkey, const u_char *der,
+    size_t der_len, u_char **pout, size_t *out_len)
+{
+    u_char     *data;
+    unsigned   n;
+    njs_int_t  ret;
+    ECDSA_SIG  *ec_sig;
+
+    ret = NJS_OK;
+    ec_sig = NULL;
+
+    n = njs_ec_rs_size(pkey);
+    if (n == 0) {
+        goto fail;
+    }
+
+    data = njs_mp_alloc(njs_vm_memory_pool(vm), 2 * n);
+    if (njs_slow_path(data == NULL)) {
+        goto memory_error;
+    }
+
+    ec_sig = d2i_ECDSA_SIG(NULL, &der, der_len);
+    if (ec_sig == NULL) {
+        goto fail;
+    }
+
+#if (OPENSSL_VERSION_NUMBER < 0x10100000L)
+    memset(data, 0, 2 * n);
+#endif
+
+    if (njs_bn_bn2binpad(ECDSA_SIG_get0_r(ec_sig), data, n) <= 0) {
+        goto fail;
+    }
+
+    if (njs_bn_bn2binpad(ECDSA_SIG_get0_s(ec_sig), &data[n], n) <= 0) {
+        goto fail;
+    }
+
+    *pout = data;
+    *out_len = 2 * n;
+
+    goto done;
+
+fail:
+
+    *out_len = 0;
+
+done:
+
+    if (ec_sig != NULL) {
+        ECDSA_SIG_free(ec_sig);
+    }
+
+    return ret;
+
+memory_error:
+
+    njs_vm_memory_pool(vm);
+
+    return NJS_ERROR;
+}
+
+
+static int
+njs_ecdsa_sig_set0(ECDSA_SIG *sig, BIGNUM *r, BIGNUM *s)
+{
+#if (OPENSSL_VERSION_NUMBER >= 0x10100000L)
+    return ECDSA_SIG_set0(sig, r, s);
+#else
+    if (r == NULL || s == NULL) {
+        return 0;
+    }
+
+    BN_clear_free(sig->r);
+    BN_clear_free(sig->s);
+
+    sig->r = r;
+    sig->s = s;
+
+    return 1;
+#endif
+}
+
+
+static njs_int_t
+njs_convert_p1363_to_der(njs_vm_t *vm, EVP_PKEY *pkey, u_char *p1363,
+    size_t p1363_len, u_char **pout, size_t *out_len)
+{
+    int        len;
+    BIGNUM     *r, *s;
+    u_char     *data;
+    unsigned   n;
+    njs_int_t  ret;
+    ECDSA_SIG  *ec_sig;
+
+    ret = NJS_OK;
+    ec_sig = NULL;
+    r = NULL;
+    s = NULL;
+
+    n = njs_ec_rs_size(pkey);
+
+    if (njs_slow_path(n == 0 || p1363_len != 2 * n)) {
+        goto fail;
+    }
+
+    ec_sig = ECDSA_SIG_new();
+    if (njs_slow_path(ec_sig == NULL)) {
+        goto memory_error;
+    }
+
+    r = BN_new();
+    if (njs_slow_path(r == NULL)) {
+        goto memory_error;
+    }
+
+    s = BN_new();
+    if (njs_slow_path(s == NULL)) {
+        goto memory_error;
+    }
+
+    if (r != BN_bin2bn(p1363, n, r)) {
+        goto fail;
+    }
+
+    if (s != BN_bin2bn(&p1363[n], n, s)) {
+        goto fail;
+    }
+
+    if (njs_ecdsa_sig_set0(ec_sig, r, s) != 1) {
+        njs_webcrypto_error(vm, "njs_ecdsa_sig_set0() failed");
+        ret = NJS_ERROR;
+        goto fail;
+    }
+
+    data = njs_mp_alloc(njs_vm_memory_pool(vm), 2 * n + 16);
+    if (njs_slow_path(data == NULL)) {
+        goto memory_error;
+    }
+
+    *pout = data;
+    len = i2d_ECDSA_SIG(ec_sig, &data);
+
+    if (len < 0) {
+        goto fail;
+    }
+
+    *out_len = len;
+
+    goto done;
+
+fail:
+
+    *out_len = 0;
+
+done:
+
+    if (ec_sig != NULL) {
+        ECDSA_SIG_free(ec_sig);
+
+    } else {
+        if (s != NULL) {
+            BN_free(s);
+        }
+
+        if (r != NULL) {
+            BN_free(r);
+        }
+    }
+
+    return ret;
+
+memory_error:
+
+    njs_vm_memory_pool(vm);
+
+    return NJS_ERROR;
+}
+
+
 static njs_int_t
 njs_ext_sign(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
     njs_index_t verify)
@@ -2121,7 +2381,24 @@ njs_ext_sign(njs_vm_t *vm, njs_value_t *
                 goto fail;
             }
 
+            if (alg->type == NJS_ALGORITHM_ECDSA) {
+                ret = njs_convert_der_to_p1363(vm, key->pkey, dst, outlen,
+                                               &dst, &outlen);
+                if (njs_slow_path(ret != NJS_OK)) {
+                    goto fail;
+                }
+            }
+
         } else {
+            if (alg->type == NJS_ALGORITHM_ECDSA) {
+                ret = njs_convert_p1363_to_der(vm, key->pkey, sig.start,
+                                               sig.length, &sig.start,
+                                               &sig.length);
+                if (njs_slow_path(ret != NJS_OK)) {
+                    goto fail;
+                }
+            }
+
             ret = EVP_PKEY_verify(pctx, sig.start, sig.length, m, m_len);
             if (njs_slow_path(ret < 0)) {
                 njs_webcrypto_error(vm, "EVP_PKEY_verify() failed");
diff -r 43b31a943c08 -r f70929728e86 test/webcrypto/README.rst
--- a/test/webcrypto/README.rst	Thu Aug 25 16:57:28 2022 -0700
+++ b/test/webcrypto/README.rst	Fri Aug 26 21:49:14 2022 -0700
@@ -124,13 +124,21 @@ Signing data using RSA-PSS
 Signing data using ECDSA
 ------------------------
 
+Note: there are two types of ECDSA signatures: ASN.1 and IEEE P1363
+Webcrypto requires IEEE P1363, but OpenSSL outputs only ASN.1 variety.
+To create P1363, we build an auxilary program asn12IEEEP1336
+
 .. code-block:: shell
 
     echo -n "SigneD-TExt" > text.txt
     openssl dgst -sha256 -binary text.txt > text.sha256
     openssl pkeyutl -sign -in text.sha256 -inkey test/webcrypto/ec.pkcs8 | \
-        base64 > test/webcrypto/text.base64.sha256.ecdsa.sig
-    base64 -d test/webcrypto/text.base64.sha256.ecdsa.sig > text.sha256.ecdsa.sig
+        base64 > test/webcrypto/text.base64.sha256.ecdsa.asn1.sig
+    base64 -d test/webcrypto/text.base64.sha256.ecdsa.asn1.sig > text.sha256.ecdsa.sig
     openssl pkeyutl -verify -in text.sha256 -pubin -inkey test/webcrypto/ec.spki  -sigfile text.sha256.ecdsa.sig
     Signature Verified Successfully
 
+    # convert to IEEE P1363
+    gcc test/webcrypto/asn12ieeep1336.c  -lcrypto -o test/webcrypto/asn12ieeep1336
+    base64 -d test/webcrypto/text.base64.sha256.ecdsa.asn1.sig | ./test/webcrypto/asn12IEEEP1336 | \
+        base64 > test/webcrypto/text.base64.sha256.ecdsa.sig
diff -r 43b31a943c08 -r f70929728e86 test/webcrypto/asn12ieeep1336.c
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/test/webcrypto/asn12ieeep1336.c	Fri Aug 26 21:49:14 2022 -0700
@@ -0,0 +1,49 @@
+#include <openssl/ecdsa.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+#include <stdlib.h>
+
+int main(int argc, char * argv[]) {
+    int                  rbytes, sbytes, len, n;
+    ECDSA_SIG            *ecSig;
+    unsigned char        *p, *end;
+    const unsigned char  *start;
+    unsigned char        der[512];
+    unsigned char        out[64];
+
+    p = der;
+    end = &der[sizeof(der)];
+
+    for ( ;; ) {
+        n = read(STDIN_FILENO, der, end - p);
+
+        if (n == 0) {
+            break;
+        }
+
+        if ((end - p) == 0) {
+            printf("too large (> 512) der length in stdin");
+            return EXIT_FAILURE;
+        }
+
+        p += n;
+    }
+
+    start = der;
+    ecSig = d2i_ECDSA_SIG(NULL, &start, p - der);
+    if (ecSig == NULL) {
+        printf("d2i_ECDSA_SIG() failed");
+        return EXIT_FAILURE;
+    }
+
+    rbytes = BN_num_bytes(ECDSA_SIG_get0_r(ecSig));
+    sbytes = BN_num_bytes(ECDSA_SIG_get0_s(ecSig));
+
+    BN_bn2binpad(ECDSA_SIG_get0_r(ecSig), out, rbytes);
+    BN_bn2binpad(ECDSA_SIG_get0_s(ecSig), &out[32], sbytes);
+
+    write(STDOUT_FILENO, out, sizeof(out));
+
+    return EXIT_SUCCESS;
+}
diff -r 43b31a943c08 -r f70929728e86 test/webcrypto/text.base64.sha1.ecdsa.sig
--- a/test/webcrypto/text.base64.sha1.ecdsa.sig	Thu Aug 25 16:57:28 2022 -0700
+++ b/test/webcrypto/text.base64.sha1.ecdsa.sig	Fri Aug 26 21:49:14 2022 -0700
@@ -1,2 +1,2 @@
-MEQCIAZ/sGPfuYivvm5UsqZgiR2jtT88d2moIgnAh6h1jKdVAiALKiu3myhI046rhEThSLyReuTu
-eIEgeCPBa2xGZnFXEg==
+Bn+wY9+5iK++blSypmCJHaO1Pzx3aagiCcCHqHWMp1ULKiu3myhI046rhEThSLyReuTueIEgeCPB
+a2xGZnFXEg==
diff -r 43b31a943c08 -r f70929728e86 test/webcrypto/text.base64.sha256.ecdsa.sig
--- a/test/webcrypto/text.base64.sha256.ecdsa.sig	Thu Aug 25 16:57:28 2022 -0700
+++ b/test/webcrypto/text.base64.sha256.ecdsa.sig	Fri Aug 26 21:49:14 2022 -0700
@@ -1,2 +1,2 @@
-MEUCIFEw11evEWohKswRe3Za0P0u7mvGj4kSnHix/EOKhxApAiEAq2QtwNvFg8RdY6t01ff8mUTP
-nT1lEfMSRZmtuVxQuQA=
+UTDXV68RaiEqzBF7dlrQ/S7ua8aPiRKceLH8Q4qHECmrZC3A28WDxF1jq3TV9/yZRM+dPWUR8xJF
+ma25XFC5AA==


From xeioex at nginx.com  Tue Aug 30 04:13:32 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 30 Aug 2022 04:13:32 +0000
Subject: [njs] Fixed typo introduced in f70929728e86.
Message-ID: <hg.b48963b98d97.1661832812.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/b48963b98d97
branches:  
changeset: 1937:b48963b98d97
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Aug 29 21:08:47 2022 -0700
description:
Fixed typo introduced in f70929728e86.

Found by Coverity (CID 1512553, 1512555).

diffstat:

 external/njs_webcrypto_module.c |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (21 lines):

diff -r f70929728e86 -r b48963b98d97 external/njs_webcrypto_module.c
--- a/external/njs_webcrypto_module.c	Fri Aug 26 21:49:14 2022 -0700
+++ b/external/njs_webcrypto_module.c	Mon Aug 29 21:08:47 2022 -0700
@@ -2072,7 +2072,7 @@ done:
 
 memory_error:
 
-    njs_vm_memory_pool(vm);
+    njs_vm_memory_error(vm);
 
     return NJS_ERROR;
 }
@@ -2189,7 +2189,7 @@ done:
 
 memory_error:
 
-    njs_vm_memory_pool(vm);
+    njs_vm_memory_error(vm);
 
     return NJS_ERROR;
 }


From xeioex at nginx.com  Tue Aug 30 04:13:34 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 30 Aug 2022 04:13:34 +0000
Subject: [njs] Removed dead code introduced in f70929728e86.
Message-ID: <hg.907ec4824fa0.1661832814.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/907ec4824fa0
branches:  
changeset: 1938:907ec4824fa0
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Aug 29 21:09:11 2022 -0700
description:
Removed dead code introduced in f70929728e86.

Found by Coverity (CID 1512554, 1512556).

diffstat:

 external/njs_webcrypto_module.c |  11 -----------
 1 files changed, 0 insertions(+), 11 deletions(-)

diffs (28 lines):

diff -r b48963b98d97 -r 907ec4824fa0 external/njs_webcrypto_module.c
--- a/external/njs_webcrypto_module.c	Mon Aug 29 21:08:47 2022 -0700
+++ b/external/njs_webcrypto_module.c	Mon Aug 29 21:09:11 2022 -0700
@@ -2112,8 +2112,6 @@ njs_convert_p1363_to_der(njs_vm_t *vm, E
 
     ret = NJS_OK;
     ec_sig = NULL;
-    r = NULL;
-    s = NULL;
 
     n = njs_ec_rs_size(pkey);
 
@@ -2174,15 +2172,6 @@ done:
 
     if (ec_sig != NULL) {
         ECDSA_SIG_free(ec_sig);
-
-    } else {
-        if (s != NULL) {
-            BN_free(s);
-        }
-
-        if (r != NULL) {
-            BN_free(r);
-        }
     }
 
     return ret;


From xeioex at nginx.com  Tue Aug 30 04:13:36 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 30 Aug 2022 04:13:36 +0000
Subject: [njs] Fixed njs_value_to_string() with non-writable dst argument.
Message-ID: <hg.456ac02d226e.1661832816.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/456ac02d226e
branches:  
changeset: 1939:456ac02d226e
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Aug 29 21:09:12 2022 -0700
description:
Fixed njs_value_to_string() with non-writable dst argument.

njs_arg(args, nargs, N) returns a pointer to Nth argument OR a pointer
to undefined constant value njs_value_undefined if N >= nargs.
njs_value_to_string() writes to a dst argument its result.
This means that it is incorrect to use value of njs_arg() directly
as a second argument to njs_value_to_string().

This closes #570 issue on Github.

diffstat:

 external/njs_webcrypto_module.c |  9 +++++----
 src/njs_number.c                |  8 ++++----
 src/njs_symbol.c                |  4 ++--
 src/test/njs_unit_test.c        |  9 +++++++++
 4 files changed, 20 insertions(+), 10 deletions(-)

diffs (111 lines):

diff -r 907ec4824fa0 -r 456ac02d226e external/njs_webcrypto_module.c
--- a/external/njs_webcrypto_module.c	Mon Aug 29 21:09:11 2022 -0700
+++ b/external/njs_webcrypto_module.c	Mon Aug 29 21:09:12 2022 -0700
@@ -2486,8 +2486,9 @@ njs_webcrypto_cleanup_pkey(void *data)
 static njs_webcrypto_key_format_t
 njs_key_format(njs_vm_t *vm, njs_value_t *value, njs_str_t *format)
 {
-    njs_int_t   ret;
-    njs_uint_t  fmt;
+    njs_int_t    ret;
+    njs_uint_t   fmt;
+    njs_value_t  string;
 
     static const struct {
         njs_str_t   name;
@@ -2499,12 +2500,12 @@ njs_key_format(njs_vm_t *vm, njs_value_t
         { njs_str("jwk"), NJS_KEY_FORMAT_JWK },
     };
 
-    ret = njs_value_to_string(vm, value, value);
+    ret = njs_value_to_string(vm, &string, value);
     if (njs_slow_path(ret != NJS_OK)) {
         return NJS_ERROR;
     }
 
-    njs_string_get(value, format);
+    njs_string_get(&string, format);
 
     fmt = 0;
 
diff -r 907ec4824fa0 -r 456ac02d226e src/njs_number.c
--- a/src/njs_number.c	Mon Aug 29 21:09:11 2022 -0700
+++ b/src/njs_number.c	Mon Aug 29 21:09:12 2022 -0700
@@ -1068,13 +1068,13 @@ njs_number_parse_int(njs_vm_t *vm, njs_v
     int32_t            radix;
     njs_int_t          ret;
     njs_bool_t         minus, test_prefix;
-    njs_value_t        *value;
+    njs_value_t        *value, lvalue;
     const u_char       *p, *end;
     njs_string_prop_t  string;
 
     num = NAN;
 
-    value = njs_arg(args, nargs, 1);
+    value = njs_lvalue_arg(&lvalue, args, nargs, 1);
 
     ret = njs_value_to_string(vm, value, value);
     if (njs_slow_path(ret != NJS_OK)) {
@@ -1146,9 +1146,9 @@ njs_number_parse_float(njs_vm_t *vm, njs
     njs_index_t unused)
 {
     njs_int_t    ret;
-    njs_value_t  *value;
+    njs_value_t  *value, lvalue;
 
-    value = njs_arg(args, nargs, 1);
+    value = njs_lvalue_arg(&lvalue, args, nargs, 1);
 
     ret = njs_value_to_string(vm, value, value);
     if (njs_slow_path(ret != NJS_OK)) {
diff -r 907ec4824fa0 -r 456ac02d226e src/njs_symbol.c
--- a/src/njs_symbol.c	Mon Aug 29 21:09:11 2022 -0700
+++ b/src/njs_symbol.c	Mon Aug 29 21:09:12 2022 -0700
@@ -151,11 +151,11 @@ njs_symbol_for(njs_vm_t *vm, njs_value_t
 {
     uint64_t              key;
     njs_int_t             ret;
-    njs_value_t           *value;
+    njs_value_t           *value, lvalue;
     njs_rbtree_node_t     *rb_node;
     njs_rb_symbol_node_t  *node;
 
-    value = njs_arg(args, nargs, 1);
+    value = njs_lvalue_arg(&lvalue, args, nargs, 1);
 
     if (njs_slow_path(!njs_is_string(value))) {
         ret = njs_value_to_string(vm, value, value);
diff -r 907ec4824fa0 -r 456ac02d226e src/test/njs_unit_test.c
--- a/src/test/njs_unit_test.c	Mon Aug 29 21:09:11 2022 -0700
+++ b/src/test/njs_unit_test.c	Mon Aug 29 21:09:12 2022 -0700
@@ -13062,6 +13062,9 @@ static njs_unit_test_t  njs_test[] =
     { njs_str("Symbol.for({toString: () => 'desc'}).description"),
       njs_str("desc") },
 
+    { njs_str("Symbol.for().toString()"),
+      njs_str("Symbol(undefined)") },
+
     { njs_str("Symbol.for('desc') === Symbol.for('desc')"),
       njs_str("true") },
 
@@ -16910,6 +16913,9 @@ static njs_unit_test_t  njs_test[] =
     { njs_str("parseInt.length"),
       njs_str("2") },
 
+    { njs_str("parseInt()"),
+      njs_str("NaN") },
+
     { njs_str("parseInt('12345abc')"),
       njs_str("12345") },
 
@@ -16994,6 +17000,9 @@ static njs_unit_test_t  njs_test[] =
     { njs_str("parseFloat('12345abc')"),
       njs_str("12345") },
 
+    { njs_str("parseFloat()"),
+      njs_str("NaN") },
+
     { njs_str("parseFloat('')"),
       njs_str("NaN") },
 


From xeioex at nginx.com  Tue Aug 30 04:34:01 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 30 Aug 2022 04:34:01 +0000
Subject: [njs] Version 0.7.7.
Message-ID: <hg.1592d46d9076.1661834041.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/1592d46d9076
branches:  
changeset: 1940:1592d46d9076
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Aug 29 21:18:20 2022 -0700
description:
Version 0.7.7.

diffstat:

 CHANGES |  34 ++++++++++++++++++++++++++++++++++
 1 files changed, 34 insertions(+), 0 deletions(-)

diffs (41 lines):

diff -r 456ac02d226e -r 1592d46d9076 CHANGES
--- a/CHANGES	Mon Aug 29 21:09:12 2022 -0700
+++ b/CHANGES	Mon Aug 29 21:18:20 2022 -0700
@@ -1,3 +1,37 @@
+Changes with njs 0.7.7                                        30 Aug 2022
+
+    nginx modules:
+
+    *) Feature: the number of nginx configuration contexts where
+       js directives can be specified is extended.
+
+       HTTP: js_import, js_path, js_set and js_var are allowed
+       in server and location contexts. js_content, js_body_filter
+       and js_header_filter are allowed in 'if' context.
+
+       Stream: js_import, js_path, js_set and js_var are allowed
+       in server context.
+
+    *) Feature: added r.internal property.
+
+    *) Bugfix: fixed reading response body in fetch API.
+
+    *) Bugfix: fixed "js_fetch_timeout" in stream module.
+
+    *) Bugfix: fixed socket leak with 0 fetch timeout.
+
+    Core:
+
+    *) Feature: extended "fs" module.  Added fs.openSync(),
+       fs.promises.open(), fs.fstatSync(), fs.readSync(),
+       fs.writeSync().
+
+       The following properties of FileHandle are implemented:
+       fd, read(), stat(), write(), close().
+
+    *) Bugfix: fixed parseInt(), parseFloat(), Symbol.for()
+       with no arguments.
+
 Changes with njs 0.7.6                                        19 Jul 2022
 
     nginx modules:


From xeioex at nginx.com  Tue Aug 30 04:34:03 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 30 Aug 2022 04:34:03 +0000
Subject: [njs] Added tag 0.7.7 for changeset 1592d46d9076
Message-ID: <hg.215ab6e04407.1661834043.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/215ab6e04407
branches:  
changeset: 1941:215ab6e04407
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Aug 29 21:33:29 2022 -0700
description:
Added tag 0.7.7 for changeset 1592d46d9076

diffstat:

 .hgtags |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (8 lines):

diff -r 1592d46d9076 -r 215ab6e04407 .hgtags
--- a/.hgtags	Mon Aug 29 21:18:20 2022 -0700
+++ b/.hgtags	Mon Aug 29 21:33:29 2022 -0700
@@ -52,3 +52,4 @@ f15d039cf625fb92e061f21c9f28a788032a0faa
 b5198f7f11a3b5e174f9e75a7bd50394fa354fb0 0.7.4
 63c258c456ca018385b13f352faefdf25c7bd3bb 0.7.5
 461dfb0bb60e531d361319f30993f29860c19f55 0.7.6
+1592d46d9076aa832b2d37d50b90f5edfca67030 0.7.7


From l.crilly at f5.com  Tue Aug 30 12:47:39 2022
From: l.crilly at f5.com (Liam Crilly)
Date: Tue, 30 Aug 2022 12:47:39 +0000
Subject: [PATCH 0 of 2] Enhanced website navigation with 2-level menu and
 directive search
Message-ID:  <BY3PR01MB66745E14EE9371A04AED2EB1A5799@BY3PR01MB6674.prod.exchangelabs.com>

Hello! First-time posting here...

This patch series applies to the nginx.org repo. A video preview of the
changes can be seen at https://youtu.be/-XcBIfTqdxc 

Patch 1 adds a search* box below the logo with the goal of helping users
jump to the documentation for a specific directive or variable. It simply
constructs a nginx.org/r/... URI (there is no dynamic content or JavaScript
here). A corresponding location block is required in nginx.conf:
location = /s {
    rewrite ^ /r/$arg_q;
}

* it is not really a search :)

Patch 2 modifies the layout of the menu navigation with the goal of highlighting
the njs and Unit projects. These are significant NGINX projects which are too easy
to overlook through natural site navigation. Summary of changes:
 - A two-level heirarchy organises content into sections for nginx, other
   nginx projects, and information
 - The "projects" of njs and unit are moved up, to a more prominent position
 - News is moved lower down, to the "info" section. A third menu level
   contains the individual years so that the top of the menu is always the
   same, regardless of which page you are on.
 - Links to external sites are indicated by a right-arrow character


Cheers,
Liam.


From l.crilly at f5.com  Tue Aug 30 12:52:33 2022
From: l.crilly at f5.com (Liam Crilly)
Date: Tue, 30 Aug 2022 12:52:33 +0000
Subject: [PATCH 1 of 2] Enhanced website navigation with 2-level menu and
 directive search
In-Reply-To:  <BY3PR01MB66745E14EE9371A04AED2EB1A5799@BY3PR01MB6674.prod.exchangelabs.com>
References:  <BY3PR01MB66745E14EE9371A04AED2EB1A5799@BY3PR01MB6674.prod.exchangelabs.com>
Message-ID:  <BY3PR01MB6674E4BC59B6616444ED8D90A5799@BY3PR01MB6674.prod.exchangelabs.com>

# HG changeset patch
# User Liam Crilly <liam.crilly at nginx.com>
# Date 1659696328 -3600
#      Fri Aug 05 11:45:28 2022 +0100
# Node ID 1087570ec093271d10de8e901122c99003a79b6d
# Parent  60851da88db689f3bfaa54453977ec9803b00509
Added search box to navigate directly to directive or variable docs.

This patch adds a search box below the logo as a shortcut to constructing a
nginx.org/r/... URI. Directive names and variables can be entered here.
A corresponding location block is required in nginx.conf:
location = /s {
    rewrite ^ /r/$arg_q;
}

diff -r 60851da88db6 -r 1087570ec093 dtd/content.dtd
--- a/dtd/content.dtd	Tue Jul 19 16:53:15 2022 +0100
+++ b/dtd/content.dtd	Fri Aug 05 11:45:28 2022 +0100
@@ -147,6 +147,21 @@
            versions     CDATA #IMPLIED
 >
 
+<!ELEMENT  form        (form+) >
+<!ATTLIST  form
+           action       CDATA #IMPLIED
+           method       CDATA #IMPLIED
+           id           CDATA #IMPLIED
+>
+
+<!ELEMENT  input       (input+) >
+<!ATTLIST  input
+           type        CDATA #IMPLIED
+           name        CDATA #IMPLIED
+           size        CDATA #IMPLIED
+           value       CDATA #IMPLIED
+>
+
 <!ENTITY   nbsp         "&#160;">
 <!ENTITY   times        "&#215;">
 <!ENTITY   lsquo        "&#8216;">
diff -r 60851da88db6 -r 1087570ec093 xsls/body.xsls
--- a/xsls/body.xsls	Tue Jul 19 16:53:15 2022 +0100
+++ b/xsls/body.xsls	Fri Aug 05 11:45:28 2022 +0100
@@ -29,6 +29,14 @@
             </a>
         </h1>
         <div>
+            directive/variable search
+            <form action="/s" method="GET">
+                <input type="text" name="q" size="14"></input>
+                <input type="submit" value="go"></input>
+            </form>
+            <br/>
+        </div>
+        <div>
         !! "document(concat($XML, '/menu.xml'))
                         /menus/menu[@lang = $lang]/item";
         </div>


From l.crilly at f5.com  Tue Aug 30 12:55:39 2022
From: l.crilly at f5.com (Liam Crilly)
Date: Tue, 30 Aug 2022 12:55:39 +0000
Subject: [PATCH 2 of 2] Enhanced website navigation with 2-level menu and
 directive search
In-Reply-To:  <BY3PR01MB66745E14EE9371A04AED2EB1A5799@BY3PR01MB6674.prod.exchangelabs.com>
References:  <BY3PR01MB66745E14EE9371A04AED2EB1A5799@BY3PR01MB6674.prod.exchangelabs.com>
Message-ID:  <BY3PR01MB66744DA46EA617B898D99C48A5799@BY3PR01MB6674.prod.exchangelabs.com>

# HG changeset patch
# User Liam Crilly <liam.crilly at nginx.com>
# Date 1659696358 -3600
#      Fri Aug 05 11:45:58 2022 +0100
# Node ID 8781046c34459c05b4d6f296bbbadc3733d51b17
# Parent  1087570ec093271d10de8e901122c99003a79b6d
Modified layout of navigation menu.

This patch modifies the layout of the menu navigation with several changes:
 - A two-level heirarchy organises content into sections for nginx, other
   nginx projects, and information
 - The "projects" of njs and unit are moved up, to a more prominent position
 - News is moved lower down, to the "info" section. A third menu level
   contains the individual years so that the top of the menu is always the
   same, regardless of which page you are on.
 - Links to external sites are indicated by a right-arrow character

diff -r 1087570ec093 -r 8781046c3445 xml/menu.xml
--- a/xml/menu.xml	Fri Aug 05 11:45:28 2022 +0100
+++ b/xml/menu.xml	Fri Aug 05 11:45:58 2022 +0100
@@ -22,6 +22,7 @@
 <item />
 -->
 
+<item> nginx </item>
 <item href="/" lang="en"> ���� </item>
 <item href="/cn/"> nginx ���� </item>
 <item href="/en/download.html" lang="en"> ���� </item>
@@ -32,14 +33,16 @@
 <item href="/en/support.html" lang="en"> ֧�� </item>
 <item />
 
+<item> projects </item>
+<item href="https://unit.nginx.org/"> unit </item>
+<item href="/en/docs/njs/"> njs </item>
+<item />
+
+<item> info </item>
 <item href="http://trac.nginx.org/nginx"> trac </item>
 <item href="http://twitter.com/nginxorg"> twitter </item>
 <item href="http://nginx.com/blog/"> blog </item>
 <item />
-
-<item href="https://unit.nginx.org/"> unit </item>
-<item href="/en/docs/njs/"> njs </item>
-
 </menu>
 
 
@@ -58,6 +61,22 @@
 <item />
 -->
 
+<item> nginx </item>
+<item href="/en/"> about </item>
+<item href="/en/download.html"> download </item>
+<item href="/en/security_advisories.html"> security </item>
+<item href="/en/docs/"> documentation </item>
+<item href="/en/docs/faq.html"> faq </item>
+<item href="/en/books.html"> books </item>
+<item href="/en/support.html"> support </item>
+<item />
+
+<item> projects </item>
+<item href="/en/docs/njs/"> njs </item>
+<item href="https://unit.nginx.org/"> unit </item>
+<item />
+
+<item> info </item>
 <item href="/"> news </item>
 <item href="2021.html" year="2021" />
 <item href="2020.html" year="2020" />
@@ -74,23 +93,11 @@
 <item href="2009.html" year="2009" />
 <item year="" />
 
-<item href="/en/"> about </item>
-<item href="/en/download.html"> download </item>
-<item href="/en/security_advisories.html"> security </item>
-<item href="/en/docs/"> documentation </item>
-<item href="/en/docs/faq.html"> faq </item>
-<item href="/en/books.html"> books </item>
-<item href="/en/support.html"> support </item>
-<item />
-
 <item href="http://trac.nginx.org/nginx"> trac </item>
 <item href="http://twitter.com/nginxorg"> twitter </item>
 <item href="https://www.nginx.com/blog/"> blog </item>
 <item />
 
-<item href="https://unit.nginx.org/"> unit </item>
-<item href="/en/docs/njs/"> njs </item>
-
 </menu>
 
 
@@ -144,6 +151,7 @@
 <item />
 -->
 
+<item> nginx </item>
 <item href="/" lang="en"> �˥�`�� </item>
 <item href="/ja/"> nginx �ˤĤ��� </item>
 <item href="/en/download.html" lang="en"> �������`�� </item>
@@ -154,14 +162,16 @@
 <item href="/en/support.html" lang="en"> ���ݩ`�� </item>
 <item />
 
+<item> projects </item>
+<item href="https://unit.nginx.org/"> unit </item>
+<item href="/en/docs/njs/"> njs </item>
+<item />
+
 <item href="http://trac.nginx.org/nginx"> trac </item>
 <item href="http://twitter.com/nginxorg"> twitter </item>
 <item href="http://nginx.com/blog/"> blog </item>
 <item />
 
-<item href="https://unit.nginx.org/"> unit </item>
-<item href="/en/docs/njs/"> njs </item>
-
 </menu>
 
 
@@ -180,6 +190,7 @@
 <item />
 -->
 
+<item> nginx </item>
 <item href="/" lang="en"> �ߧ�ӧ���� </item>
 <item href="/ru/"> ��� nginx </item>
 <item href="/ru/download.html"> ��ܧѧ�ѧ�� </item>
@@ -190,14 +201,17 @@
 <item href="/ru/support.html"> ���էէ֧�اܧ� </item>
 <item />
 
+<item> ����֧ܧ�� </item>
+<item href="https://unit.nginx.org/" lang="en"> unit </item>
+<item href="/ru/docs/njs/"> njs </item>
+<item />
+
+<item> �ڧߧ���ާѧ�ڧ� </item>
 <item href="http://trac.nginx.org/nginx"> trac </item>
 <item href="http://twitter.com/nginxorg"> twitter </item>
 <item href="https://www.nginx.com/blog/"> blog </item>
 <item />
 
-<item href="https://unit.nginx.org/"> unit </item>
-<item href="/ru/docs/njs/"> njs </item>
-
 </menu>
 
 
diff -r 1087570ec093 -r 8781046c3445 xsls/menu.xsls
--- a/xsls/menu.xsls	Fri Aug 05 11:45:28 2022 +0100
+++ b/xsls/menu.xsls	Fri Aug 05 11:45:58 2022 +0100
@@ -15,9 +15,9 @@
 
     X:if "@href = $LINK" {
         X:if "$YEAR and @href='/'" {
-            <a href="./"> news </a> <br/>
+            &#160; <a href="./"> news </a> <br/>
         } else {
-            !{ normalize-space(text()) }<br/>
+            &#160; !{ normalize-space(text()) }<br/>
         }
 
     } else {
@@ -47,7 +47,7 @@
             }
         } else {
 
-            <a>
+            &#160; <a>
             X:attribute "href" {
 
                 X:if "starts-with(@href, $DIRNAME)" {
@@ -75,9 +75,9 @@
 X:template = "menu/item[@year]" {
     X:if "$YEAR or $LINK='/'" {
         X:if "$YEAR=@year" {
-            !{@year} <br/>
+            &#160; &#160; !{@year} <br/>
         } else {
-            X:if "@href" { <a href="{@href}"> !{@year} </a> }
+            X:if "@href" { &#160; &#160; <a href="{@href}"> !{@year} </a> }
             <br/>
         }
     }
@@ -85,7 +85,7 @@
 
 
 X:template = "menu/item[starts-with(@href, 'http://') or starts-with(@href, 'https://')]" {
-    <a href="{@href}"> !{ normalize-space(text()) } </a>
+    &#160; <a href="{@href}"> !{ normalize-space(text()) } </a>&#8594;
     X:if "@lang" { X:text { [} !{@lang} X:text {]}}
     <br/>
 }

From geoff.bache at gmail.com  Wed Aug 31 14:59:40 2022
From: geoff.bache at gmail.com (Geoff Bache)
Date: Wed, 31 Aug 2022 16:59:40 +0200
Subject: Trying to build nginx with njs module on Windows
Message-ID: <CA+Jhifu6FBqeQ2xREy2HBgiZd4Nrpr7BZwVgwh21bWtBH+6ZNA@mail.gmail.com>

Hi all,

I'm having trouble building from source with the njs module included (which
seems to be the only way to get this module on Windows, right?)

I am following the advice on
https://nginx.org/en/docs/njs/install.html

and
http://nginx.org/en/docs/howto_build_on_win32.html

All seems to go fine with the build in msys2, using nmake from VS Build
Tools as indicated above, until it gets around to the njs module, when I get

cd /home/SEGEBAC1/src/njs/nginx/..  && if [ -f build/Makefile ]; then
"C:\Program Files (x86)\Microsoft Visual
Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\bin\HostX64\x64\nmake.exe"
clean; fi  && CFLAGS="-O2  -W4 -WX -nologo -MT -Zi -Fdobjs/nginx.pdb
-DFD_SETSIZE=1024 -DNO_SYS_TYPES_H" CC="cl" ./configure --no-openssl
--no-pcre  && "C:\Program Files (x86)\Microsoft Visual
Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\bin\HostX64\x64\nmake.exe"
libnjs
-f was unexpected at this time.
NMAKE : fatal error U1077: 'cd' : return code '0x1'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual
Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\bin\HostX64\x64\nmake.exe"'
: return code '0x2'
Stop.

Any ideas? I tried executing these command separately, but then I get

MSYS ~/src/njs
$  CFLAGS="-O2  -W4 -WX -nologo -MT -Zi -Fdobjs/nginx.pdb -DFD_SETSIZE=1024
-DNO_SYS_TYPES_H" CC="cl" ./configure --no-openssl --no-pcre
configuring for MSYS_NT-10.0-19042 3.3.5-341.x86_64 unknown
checking for C compiler: cl
auto/cc: line 16: [: too many arguments
auto/cc: line 62: NJS_CC_NAME: unbound variable

which seems to indicate that the njs module does not handle the Visual
Studio compiler, which seems odd in combination with the suggestions above
to use nmake. Providing gcc does not help either, as then I get

MSYS ~/src/njs
$  CFLAGS="-O2  -W4 -WX -nologo -MT -Zi -Fdobjs/nginx.pdb -DFD_SETSIZE=1024
-DNO_SYS_TYPES_H" CC="gcc" ./configure --no-openssl --no-pcre
configuring for MSYS_NT-10.0-19042 3.3.5-341.x86_64 unknown
checking for C compiler: gcc
 + using GNU C compiler
 + gcc version 8.3.0 (x86_64-posix-seh, Built by strawberryperl.com project)
checking for sizeof(int) ... not found
checking for sizeof(u_int) ... not found
checking for sizeof(void *) ... not found
checking for sizeof(uintptr_t) ... not found
checking for sizeof(size_t) ... not found
checking for sizeof(off_t) ... not found
checking for sizeof(time_t) ... not found
checking for system byte ordering ... not found

./configure: error: cannot detect system byte ordering

Grateful for any help and suggestions.

Regards,
Geoff Bache
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20220831/5e9865f3/attachment.htm>

From mdounin at mdounin.ru  Wed Aug 31 15:25:28 2022
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 31 Aug 2022 18:25:28 +0300
Subject: [PATCH 1 of 2] Enhanced website navigation with 2-level menu and
 directive search
In-Reply-To: <BY3PR01MB6674E4BC59B6616444ED8D90A5799@BY3PR01MB6674.prod.exchangelabs.com>
References: <BY3PR01MB66745E14EE9371A04AED2EB1A5799@BY3PR01MB6674.prod.exchangelabs.com>
 <BY3PR01MB6674E4BC59B6616444ED8D90A5799@BY3PR01MB6674.prod.exchangelabs.com>
Message-ID: <Yw99aEmZ8nJxoVN/@mdounin.ru>

Hello!

On Tue, Aug 30, 2022 at 12:52:33PM +0000, Liam Crilly via nginx-devel wrote:

> # HG changeset patch
> # User Liam Crilly <liam.crilly at nginx.com>
> # Date 1659696328 -3600
> #      Fri Aug 05 11:45:28 2022 +0100
> # Node ID 1087570ec093271d10de8e901122c99003a79b6d
> # Parent  60851da88db689f3bfaa54453977ec9803b00509
> Added search box to navigate directly to directive or variable docs.
> 
> This patch adds a search box below the logo as a shortcut to constructing a
> nginx.org/r/... URI. Directive names and variables can be entered here.
> A corresponding location block is required in nginx.conf:
> location = /s {
>     rewrite ^ /r/$arg_q;
> }

I don't think this is a good idea.  The /r/ redirector is a 
service to provide short links, and not a search service.  User 
expectations for search are quite different.

-- 
Maxim Dounin
http://mdounin.ru/


From pluknet at nginx.com  Wed Aug 31 15:38:24 2022
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Wed, 31 Aug 2022 19:38:24 +0400
Subject: nginx-quic tests
Message-ID: <88BF2599-1471-47FC-A99F-975EB48793DE@nginx.com>

Hello.

Initial HTTP/3 tests are committed as part of nginx-quic project.
It is in early development and subject to heavy changes,
to be merged with the common test suite when it's done.

You can track development here:
https://hg.nginx.org/nginx-tests-quic/

-- 
Sergey Kandaurov


From arut at nginx.com  Wed Aug 31 15:52:15 2022
From: arut at nginx.com (=?iso-8859-1?q?Roman_Arutyunyan?=)
Date: Wed, 31 Aug 2022 19:52:15 +0400
Subject: [PATCH] Core: support for reading PROXY protocol v2 TLVs
Message-ID: <4b856f1dff939e4eb9c1.1661961135@arut-laptop>

# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1661436099 -14400
#      Thu Aug 25 18:01:39 2022 +0400
# Node ID 4b856f1dff939e4eb9c131e17af061cf2c38cfac
# Parent  069a4813e8d6d7ec662d282a10f5f7062ebd817f
Core: support for reading PROXY protocol v2 TLVs.

The TLV values are available in HTTP and Stream variables
$proxy_protocol_tlv_0xN, where N is a hexadecimal TLV type number with no
leading zeroes.

diff --git a/src/core/ngx_proxy_protocol.c b/src/core/ngx_proxy_protocol.c
--- a/src/core/ngx_proxy_protocol.c
+++ b/src/core/ngx_proxy_protocol.c
@@ -40,6 +40,12 @@ typedef struct {
 } ngx_proxy_protocol_inet6_addrs_t;
 
 
+typedef struct {
+    u_char                                  type;
+    u_char                                  len[2];
+} ngx_proxy_protocol_tlv_t;
+
+
 static u_char *ngx_proxy_protocol_read_addr(ngx_connection_t *c, u_char *p,
     u_char *last, ngx_str_t *addr);
 static u_char *ngx_proxy_protocol_read_port(u_char *p, u_char *last,
@@ -273,8 +279,11 @@ ngx_proxy_protocol_v2_read(ngx_connectio
     size_t                              len;
     socklen_t                           socklen;
     ngx_uint_t                          version, command, family, transport;
+    ngx_list_t                         *pp_tlv;
     ngx_sockaddr_t                      src_sockaddr, dst_sockaddr;
+    ngx_table_elt_t                    *t;
     ngx_proxy_protocol_t               *pp;
+    ngx_proxy_protocol_tlv_t           *tlv;
     ngx_proxy_protocol_header_t        *header;
     ngx_proxy_protocol_inet_addrs_t    *in;
 #if (NGX_HAVE_INET6)
@@ -412,8 +421,63 @@ ngx_proxy_protocol_v2_read(ngx_connectio
                    &pp->src_addr, pp->src_port, &pp->dst_addr, pp->dst_port);
 
     if (buf < end) {
-        ngx_log_debug1(NGX_LOG_DEBUG_CORE, c->log, 0,
-                       "PROXY protocol v2 %z bytes of tlv ignored", end - buf);
+        pp_tlv = ngx_list_create(c->pool, 1, sizeof(ngx_table_elt_t));
+        if (pp_tlv == NULL) {
+            return NULL;
+        }
+
+        while (buf < end) {
+            if ((size_t) (end - buf) < sizeof(ngx_proxy_protocol_tlv_t)) {
+                ngx_log_error(NGX_LOG_ERR, c->log, 0,
+                              "broken PROXY protocol TLV");
+                return NULL;
+            }
+
+            tlv = (ngx_proxy_protocol_tlv_t *) buf;
+
+            buf += sizeof(ngx_proxy_protocol_tlv_t);
+
+            len = ngx_proxy_protocol_parse_uint16(tlv->len);
+
+            if ((size_t) (end - buf) < len) {
+                ngx_log_error(NGX_LOG_ERR, c->log, 0,
+                              "broken PROXY protocol TLV");
+                return NULL;
+            }
+
+            ngx_log_debug2(NGX_LOG_DEBUG_CORE, c->log, 0,
+                           "PROXY protocol v2 TLV type:0x%0xd len:%uz",
+                           tlv->type, len);
+
+            t = ngx_list_push(pp_tlv);
+            if (t == NULL) {
+                return NULL;
+            }
+
+            t->hash = 1;
+            t->lowcase_key = NULL;
+
+            t->key.data = ngx_pnalloc(c->pool, 2);
+            if (t->key.data == NULL) {
+                return NULL;
+            }
+
+            t->key.len = ngx_sprintf(t->key.data, "%0xd", tlv->type)
+                         - t->key.data;
+
+            t->value.data = ngx_pnalloc(c->pool, len);
+            if (t->value.data == NULL) {
+                return NULL;
+            }
+
+            ngx_memcpy(t->value.data, buf, len);
+
+            t->value.len = len;
+
+            buf += len;
+        }
+
+        pp->tlv = pp_tlv;
     }
 
     c->proxy_protocol = pp;
diff --git a/src/core/ngx_proxy_protocol.h b/src/core/ngx_proxy_protocol.h
--- a/src/core/ngx_proxy_protocol.h
+++ b/src/core/ngx_proxy_protocol.h
@@ -21,6 +21,7 @@ struct ngx_proxy_protocol_s {
     ngx_str_t           dst_addr;
     in_port_t           src_port;
     in_port_t           dst_port;
+    ngx_list_t         *tlv;
 };
 
 
diff --git a/src/http/ngx_http_variables.c b/src/http/ngx_http_variables.c
--- a/src/http/ngx_http_variables.c
+++ b/src/http/ngx_http_variables.c
@@ -61,6 +61,8 @@ static ngx_int_t ngx_http_variable_proxy
     ngx_http_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_http_variable_proxy_protocol_port(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_http_variable_proxy_protocol_tlv(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_http_variable_server_addr(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_http_variable_server_port(ngx_http_request_t *r,
@@ -214,6 +216,10 @@ static ngx_http_variable_t  ngx_http_cor
       ngx_http_variable_proxy_protocol_port,
       offsetof(ngx_proxy_protocol_t, dst_port), 0, 0 },
 
+    { ngx_string("proxy_protocol_tlv_0x"), NULL,
+      ngx_http_variable_proxy_protocol_tlv,
+      0, NGX_HTTP_VAR_PREFIX, 0 },
+
     { ngx_string("server_addr"), NULL, ngx_http_variable_server_addr, 0, 0, 0 },
 
     { ngx_string("server_port"), NULL, ngx_http_variable_server_port, 0, 0, 0 },
@@ -1387,6 +1393,24 @@ ngx_http_variable_proxy_protocol_port(ng
 
 
 static ngx_int_t
+ngx_http_variable_proxy_protocol_tlv(ngx_http_request_t *r,
+    ngx_http_variable_value_t *v, uintptr_t data)
+{
+    ngx_proxy_protocol_t  *pp;
+
+    pp = r->connection->proxy_protocol;
+    if (pp == NULL || pp->tlv == NULL) {
+        v->not_found = 1;
+        return NGX_OK;
+    }
+
+    return ngx_http_variable_unknown_header(r, v, (ngx_str_t *) data,
+                                          &pp->tlv->part,
+                                          sizeof("proxy_protocol_tlv_0x") - 1);
+}
+
+
+static ngx_int_t
 ngx_http_variable_server_addr(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data)
 {
diff --git a/src/stream/ngx_stream_variables.c b/src/stream/ngx_stream_variables.c
--- a/src/stream/ngx_stream_variables.c
+++ b/src/stream/ngx_stream_variables.c
@@ -23,6 +23,8 @@ static ngx_int_t ngx_stream_variable_pro
     ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_stream_variable_proxy_protocol_port(
     ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_stream_variable_proxy_protocol_tlv(
+    ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_stream_variable_server_addr(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_stream_variable_server_port(ngx_stream_session_t *s,
@@ -51,6 +53,10 @@ static ngx_int_t ngx_stream_variable_tim
 static ngx_int_t ngx_stream_variable_protocol(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
 
+static ngx_int_t ngx_stream_variable_unknown_header(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, ngx_str_t *var, ngx_list_part_t *part,
+    size_t prefix);
+
 
 static ngx_stream_variable_t  ngx_stream_core_variables[] = {
 
@@ -79,6 +85,10 @@ static ngx_stream_variable_t  ngx_stream
       ngx_stream_variable_proxy_protocol_port,
       offsetof(ngx_proxy_protocol_t, dst_port), 0, 0 },
 
+    { ngx_string("proxy_protocol_tlv_0x"), NULL,
+      ngx_stream_variable_proxy_protocol_tlv,
+      0, NGX_STREAM_VAR_PREFIX, 0 },
+
     { ngx_string("server_addr"), NULL,
       ngx_stream_variable_server_addr, 0, 0, 0 },
 
@@ -622,6 +632,24 @@ ngx_stream_variable_proxy_protocol_port(
 
 
 static ngx_int_t
+ngx_stream_variable_proxy_protocol_tlv(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data)
+{
+    ngx_proxy_protocol_t  *pp;
+
+    pp = s->connection->proxy_protocol;
+    if (pp == NULL || pp->tlv == NULL) {
+        v->not_found = 1;
+        return NGX_OK;
+    }
+
+    return ngx_stream_variable_unknown_header(s, v, (ngx_str_t *) data,
+                                          &pp->tlv->part,
+                                          sizeof("proxy_protocol_tlv_0x") - 1);
+}
+
+
+static ngx_int_t
 ngx_stream_variable_server_addr(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data)
 {
@@ -911,6 +939,116 @@ ngx_stream_variable_protocol(ngx_stream_
 }
 
 
+static ngx_int_t
+ngx_stream_variable_unknown_header(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, ngx_str_t *var,
+    ngx_list_part_t *part, size_t prefix)
+{
+    u_char           *p, ch;
+    size_t            len;
+    ngx_uint_t        i, n;
+    ngx_table_elt_t  *header, *h, **ph;
+
+    ph = &h;
+#if (NGX_SUPPRESS_WARN)
+    len = 0;
+#endif
+
+    header = part->elts;
+
+    for (i = 0; /* void */ ; i++) {
+
+        if (i >= part->nelts) {
+            if (part->next == NULL) {
+                break;
+            }
+
+            part = part->next;
+            header = part->elts;
+            i = 0;
+        }
+
+        if (header[i].hash == 0) {
+            continue;
+        }
+
+        if (header[i].key.len != var->len - prefix) {
+            continue;
+        }
+
+        for (n = 0; n < var->len - prefix; n++) {
+            ch = header[i].key.data[n];
+
+            if (ch >= 'A' && ch <= 'Z') {
+                ch |= 0x20;
+
+            } else if (ch == '-') {
+                ch = '_';
+            }
+
+            if (var->data[n + prefix] != ch) {
+                break;
+            }
+        }
+
+        if (n != var->len - prefix) {
+            continue;
+        }
+
+        len += header[i].value.len + 2;
+
+        *ph = &header[i];
+        ph = &header[i].next;
+    }
+
+    *ph = NULL;
+
+    if (h == NULL) {
+        v->not_found = 1;
+        return NGX_OK;
+    }
+
+    len -= 2;
+
+    if (h->next == NULL) {
+
+        v->len = h->value.len;
+        v->valid = 1;
+        v->no_cacheable = 0;
+        v->not_found = 0;
+        v->data = h->value.data;
+
+        return NGX_OK;
+    }
+
+    p = ngx_pnalloc(s->connection->pool, len);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
+
+    v->len = len;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+    v->data = p;
+
+    for ( ;; ) {
+
+        p = ngx_copy(p, h->value.data, h->value.len);
+
+        if (h->next == NULL) {
+            break;
+        }
+
+        *p++ = ','; *p++ = ' ';
+
+        h = h->next;
+    }
+
+    return NGX_OK;
+}
+
+
 void *
 ngx_stream_map_find(ngx_stream_session_t *s, ngx_stream_map_t *map,
     ngx_str_t *match)


From xeioex at nginx.com  Wed Aug 31 23:53:14 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 31 Aug 2022 23:53:14 +0000
Subject: [njs] Version bump.
Message-ID: <hg.dfbd55660d40.1661989994.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/dfbd55660d40
branches:  
changeset: 1942:dfbd55660d40
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Aug 31 16:52:10 2022 -0700
description:
Version bump.

diffstat:

 src/njs.h |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff -r 215ab6e04407 -r dfbd55660d40 src/njs.h
--- a/src/njs.h	Mon Aug 29 21:33:29 2022 -0700
+++ b/src/njs.h	Wed Aug 31 16:52:10 2022 -0700
@@ -11,8 +11,8 @@
 
 #include <njs_auto_config.h>
 
-#define NJS_VERSION                 "0.7.7"
-#define NJS_VERSION_NUMBER          0x000707
+#define NJS_VERSION                 "0.7.8"
+#define NJS_VERSION_NUMBER          0x000708
 
 
 #include <unistd.h>                 /* STDOUT_FILENO, STDERR_FILENO */


From xeioex at nginx.com  Wed Aug 31 23:53:16 2022
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 31 Aug 2022 23:53:16 +0000
Subject: [njs] Added ngx.conf_prefix.
Message-ID: <hg.26b8f0c2ef94.1661989996.5965299922797593991@dev.nginx>

details:   https://hg.nginx.org/njs/rev/26b8f0c2ef94
branches:  
changeset: 1943:26b8f0c2ef94
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Aug 31 16:52:16 2022 -0700
description:
Added ngx.conf_prefix.

diffstat:

 nginx/ngx_js.c |  21 +++++++++++++++++++++
 1 files changed, 21 insertions(+), 0 deletions(-)

diffs (45 lines):

diff -r dfbd55660d40 -r 26b8f0c2ef94 nginx/ngx_js.c
--- a/nginx/ngx_js.c	Wed Aug 31 16:52:10 2022 -0700
+++ b/nginx/ngx_js.c	Wed Aug 31 16:52:16 2022 -0700
@@ -12,6 +12,10 @@
 #include "ngx_js_fetch.h"
 
 
+njs_int_t ngx_js_ext_conf_prefix(njs_vm_t *vm, njs_object_prop_t *prop,
+    njs_value_t *value, njs_value_t *setval, njs_value_t *retval);
+
+
 extern njs_module_t  njs_webcrypto_module;
 
 
@@ -19,6 +23,14 @@ static njs_external_t  ngx_js_ext_core[]
 
     {
         .flags = NJS_EXTERN_PROPERTY,
+        .name.string = njs_str("conf_prefix"),
+        .u.property = {
+            .handler = ngx_js_ext_conf_prefix,
+        }
+    },
+
+    {
+        .flags = NJS_EXTERN_PROPERTY,
         .name.string = njs_str("ERR"),
         .u.property = {
             .handler = ngx_js_ext_constant,
@@ -305,6 +317,15 @@ ngx_js_ext_flags(njs_vm_t *vm, njs_objec
 
 
 njs_int_t
+ngx_js_ext_conf_prefix(njs_vm_t *vm, njs_object_prop_t *prop,
+    njs_value_t *value, njs_value_t *setval, njs_value_t *retval)
+{
+    return njs_vm_value_string_set(vm, retval, ngx_cycle->conf_prefix.data,
+                                   ngx_cycle->conf_prefix.len);
+}
+
+
+njs_int_t
 ngx_js_ext_log(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs,
     njs_index_t level)
 {