From yongguang.bai at quantil.com  Sat Dec  1 01:16:06 2018
From: yongguang.bai at quantil.com (Yongguang Bai)
Date: Fri, 30 Nov 2018 17:16:06 -0800
Subject: [PATCH] Mail: nginx_error:cache file has too long header
In-Reply-To: <20181128124543.GX99070@mdounin.ru>
References: <4d48e1f3ebf6f0634e8e.1543284693@puppetagent>
 <20181128124543.GX99070@mdounin.ru>
Message-ID: <CAGZP941Fxkp4TW1iYqHA=xoR+iqXTNU1oFsRD-N8rmEtsW2jdQ@mail.gmail.com>

Hi Maxim,

Sorry for my late reply.

We use Nginx as CDN Cache. And it is normal case that the responses are
different from different origin servers. For instance: the origin server
could be
the customer server or some other CDN provider Cache server. The response
header could be different.

In race condition, I found the inconsistency between disk file and metadata
in
shared memory.
When (c->uniq != of.uniq) happens, I saw " too long header" in error log.

To fix this, I picked the ngx_pagesize to initiate c->body_start again.
then the "too long header"
is gone from error log.

I will try your patches next week. Thanks!


On Wed, Nov 28, 2018 at 4:45 AM Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Mon, Nov 26, 2018 at 06:11:33PM -0800, Yongguang Bai wrote:
>
> > # HG changeset patch
> > # User Yongguang Bai <yongguang.bai at quantil.com>
> > # Date 1543276278 28800
> > #      Mon Nov 26 15:51:18 2018 -0800
> > # Node ID 4d48e1f3ebf6f0634e8e2cbd057d9fcfbea9d988
> > # Parent  a7ff19afbb14795fef14f599a304d0ad21052b70
> > Mail: nginx_error:cache file has too long header
> >
> > This error is printed when cached file is expired and the
> > response is changed in proxied server, and Nginx is under
> > heavy load.
> >
> > diff -r a7ff19afbb14 -r 4d48e1f3ebf6 src/http/ngx_http_file_cache.c
> > --- a/src/http/ngx_http_file_cache.c  Mon Nov 26 18:29:56 2018 +0300
> > +++ b/src/http/ngx_http_file_cache.c  Mon Nov 26 15:51:18 2018 -0800
> > @@ -376,6 +376,10 @@
> >      ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
> >                     "http file cache fd: %d", of.fd);
> >
> > +    if(c->uniq != of.uniq){
> > +        c->body_start = ngx_pagesize;
> > +    }
> > +
> >      c->file.fd = of.fd;
> >      c->file.log = r->connection->log;
> >      c->uniq = of.uniq;
>
> This change looks wrong.  In particular, ngx_pagesize, which is
> usually 4k, can be smaller than existing c->body_start, and this
> change will make things worse instead of fixing anything.
>
> Also, could you please elaborate what exactly you are trying to
> fix?
>
> There is a known race which may lead to "cache file has too long
> header" error - and, actually the error was added to ensure that
> this race is properly handled if happens, see here:
>
> http://hg.nginx.org/nginx/rev/6f97afc238de
> http://mailman.nginx.org/pipermail/nginx-devel/2011-September/001287.html
>
> Note that this race can only result in problems if for some reason
> the backend returns responses with different headers nearly at the
> same time.  This may indicate that there is something wrong with
> the backend.
>
> Also, there is a known bug with multiple variants which also can
> lead to the same error.  The patch can be found here:
>
> http://mailman.nginx.org/pipermail/nginx-devel/2018-January/010774.html
>
> If you are hitting this bug, please test if the patch works for
> you.
>
> --
> Maxim Dounin
> http://mdounin.ru/
>


-- 

Best Regards,

*Yongguang Bai*

Engineering Manager
[image: Quantil]

Connecting users with content...it's that simple.

Direct: +1-408-421-6265
Office: +1-888-847-9851

[image: Tweeter] <https://twitter.com/Team_Quantil>  [image: Google Plus]
<https://plus.google.com/+Quantil_team/>  [image: Linked In]
<https://www.linkedin.com/company/quantil>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20181130/2b6ee3a1/attachment.html>

From vbart at nginx.com  Sat Dec  1 19:32:14 2018
From: vbart at nginx.com (Valentin Bartenev)
Date: Sat, 01 Dec 2018 19:32:14 +0000
Subject: [njs] Slightly optimized njs_object_keys_array().
Message-ID: <hg.4a543ed58c95.1543692734.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/4a543ed58c95
branches:  
changeset: 681:4a543ed58c95
user:      Valentin Bartenev <vbart at nginx.com>
date:      Sat Dec 01 22:32:33 2018 +0300
description:
Slightly optimized njs_object_keys_array().

There is no need to iterate over the object properties second time if after
the first attempt we know that it contains no enumerable properties.

Also removed surplus conditions.

diffstat:

 njs/njs_object.c |  19 ++++++++++---------
 1 files changed, 10 insertions(+), 9 deletions(-)

diffs (61 lines):

diff -r e713f648ef71 -r 4a543ed58c95 njs/njs_object.c
--- a/njs/njs_object.c	Fri Nov 30 17:52:02 2018 +0300
+++ b/njs/njs_object.c	Sat Dec 01 22:32:33 2018 +0300
@@ -888,7 +888,7 @@ njs_object_keys(njs_vm_t *vm, njs_value_
 njs_array_t *
 njs_object_keys_array(njs_vm_t *vm, const njs_value_t *value)
 {
-    uint32_t           i, n, length, keys_length;
+    uint32_t           i, n, length, keys_length, properties;
     njs_value_t        *string;
     njs_array_t        *keys, *array;
     nxt_lvlhsh_t       *hash;
@@ -930,15 +930,14 @@ njs_object_keys_array(njs_vm_t *vm, cons
         break;
     }
 
+    /* GCC 4 and Clang 3 complain about uninitialized hash. */
+    hash = NULL;
+    properties = 0;
+
     if (nxt_fast_path(njs_is_object(value))) {
         nxt_lvlhsh_each_init(&lhe, &njs_object_hash_proto);
         hash = &value->data.u.object->hash;
 
-    } else {
-        hash = NULL;
-    }
-
-    if (nxt_fast_path(hash != NULL)) {
         for ( ;; ) {
             prop = nxt_lvlhsh_each(hash, &lhe);
 
@@ -947,9 +946,11 @@ njs_object_keys_array(njs_vm_t *vm, cons
             }
 
             if (prop->type != NJS_WHITEOUT && prop->enumerable) {
-                keys_length++;
+                properties++;
             }
         }
+
+        keys_length += properties;
     }
 
     keys = njs_array_alloc(vm, keys_length, NJS_ARRAY_SPARE);
@@ -966,13 +967,13 @@ njs_object_keys_array(njs_vm_t *vm, cons
             }
         }
 
-    } else if (length != 0) {
+    } else {
         for (i = 0; i < length; i++) {
             njs_uint32_to_string(&keys->start[n++], i);
         }
     }
 
-    if (nxt_fast_path(hash != NULL)) {
+    if (nxt_fast_path(properties != 0)) {
         nxt_lvlhsh_each_init(&lhe, &njs_object_hash_proto);
 
         for ( ;; ) {

From mdounin at mdounin.ru  Sun Dec  2 14:32:03 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sun, 2 Dec 2018 17:32:03 +0300
Subject: [PATCH] HTTP/2: fixed handling of fully preread request bodies
In-Reply-To: <466c154c5c53b9566602.1543314856@piotrsikora.c.googlers.com>
References: <466c154c5c53b9566602.1543314856@piotrsikora.c.googlers.com>
Message-ID: <20181202143200.GV99070@mdounin.ru>

Hello!

On Tue, Nov 27, 2018 at 02:34:16AM -0800, Piotr Sikora via nginx-devel wrote:

> # HG changeset patch
> # User Piotr Sikora <piotrsikora at google.com>
> # Date 1540435636 25200
> #      Wed Oct 24 19:47:16 2018 -0700
> # Node ID 466c154c5c53b956660211df96331b3c25669485
> # Parent  be5cb9c67c05ccaf22dab7abba78aa4c1545a8ee
> HTTP/2: fixed handling of fully preread request bodies.
> 
> Previously, fully preread request body of a request without the
> "Content-Length" header was always written to a temporary file.
> 
> Reported by Wayne Zhang.
> 
> Signed-off-by: Piotr Sikora <piotrsikora at google.com>
> 
> diff -r be5cb9c67c05 -r 466c154c5c53 src/http/v2/ngx_http_v2.c
> --- a/src/http/v2/ngx_http_v2.c
> +++ b/src/http/v2/ngx_http_v2.c
> @@ -3863,6 +3863,12 @@ ngx_http_v2_read_request_body(ngx_http_r
>      {
>          rb->buf = ngx_create_temp_buf(r->pool, (size_t) len);
>  
> +    } else if (len < 0 && stream->in_closed && stream->preread
> +               && !r->request_body_in_file_only)
> +    {
> +        rb->buf = ngx_create_temp_buf(r->pool,
> +                                      (size_t) ngx_buf_size(stream->preread));
> +
>      } else {
>          rb->buf = ngx_calloc_buf(r->pool);
>  

I believe the problem is more than that.  HTTP/2 code in general 
does not work well with request bodies if Content-Length is not 
known, and always writes them to disk.

In particular, I've seen this happening when working on gRPC 
proxying - as gRPC clients does not seem to indicate 
Content-Length even if it is known in advance.  (In case of gRPC 
proxying this is not currently important though, as it does not 
happen when using unbuffered proxying.)

I think that if we want to address this, this needs to be 
addressed for all cases, not only a particular case of fully 
preread request body.

-- 
Maxim Dounin
http://mdounin.ru/

From Elliot.Thomas at bbc.co.uk  Mon Dec  3 12:11:15 2018
From: Elliot.Thomas at bbc.co.uk (Elliot Thomas)
Date: Mon, 3 Dec 2018 12:11:15 +0000
Subject: [PATCH] Apply cache locking behaviour to stale cache entries. 
Message-ID: <D82ACDE2.2781E%Elliot.Thomas@bbc.co.uk>

Hello,

This patch applies cache locking behaviour to stale cache entries, so
 in the case where the *_cache_lock directives are used, the same
 locking behaviour is used for stale content as it is for new entries.

Previously, this was only done for new cache entries.
(see: http://mailman.nginx.org/pipermail/nginx/2016-January/049734.html)

This is useful when serving stale content is not permissable but sending
 many requests upstream is undesriable.


This patch exposes the ngx_http_file_cache_lock function; this function
 is then called in ngx_http_upstream if a cache entry has expired.

I have attached two versions of this patch, a "minimal" one that avoids
 changes where not strictly necessary, and a "cleaner" version which is
 more invasive but (in my opinion) cleaner.

Both patches cause no (additional) failures in the nginx test suite.
I tested with as many modules as I could reasonably enable.

Additionally, a colleague will add tests for this new behaviour in a
 follow-up patch.


Regards, Elliot.



-----------------------------
http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and
may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in
error, please delete it from your system.
Do not use, copy or disclose the
information in any way nor act in reliance on it and notify the sender
immediately.
Please note that the BBC monitors e-mails
sent or received.
Further communication will signify your consent to
this.
-----------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: proxy-cache-lock-on-stale-cleaner.patch
Type: application/octet-stream
Size: 2709 bytes
Desc: proxy-cache-lock-on-stale-cleaner.patch
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20181203/202c3ce7/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: proxy-cache-lock-on-stale-minimal.patch
Type: application/octet-stream
Size: 2207 bytes
Desc: proxy-cache-lock-on-stale-minimal.patch
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20181203/202c3ce7/attachment-0001.obj>

From vbart at nginx.com  Mon Dec  3 19:07:22 2018
From: vbart at nginx.com (Valentin Bartenev)
Date: Mon, 03 Dec 2018 19:07:22 +0000
Subject: [njs] Object.values() method.
Message-ID: <hg.1af8f40573f2.1543864042.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/1af8f40573f2
branches:  
changeset: 682:1af8f40573f2
user:      Valentin Bartenev <vbart at nginx.com>
date:      Mon Dec 03 19:20:58 2018 +0300
description:
Object.values() method.

diffstat:

 njs/njs_json.c           |    4 +-
 njs/njs_object.c         |  169 +++++++++++++++++++++++++++++++++++++++-------
 njs/njs_object.h         |    9 ++-
 njs/test/njs_unit_test.c |   38 ++++++++++
 4 files changed, 191 insertions(+), 29 deletions(-)

diffs (347 lines):

diff -r 4a543ed58c95 -r 1af8f40573f2 njs/njs_json.c
--- a/njs/njs_json.c	Sat Dec 01 22:32:33 2018 +0300
+++ b/njs/njs_json.c	Mon Dec 03 19:20:58 2018 +0300
@@ -1095,7 +1095,7 @@ njs_json_push_parse_state(njs_vm_t *vm, 
         } else {
             state->type = NJS_JSON_OBJECT_START;
             state->prop_value = NULL;
-            state->keys = njs_object_keys_array(vm, value);
+            state->keys = njs_object_enumerate(vm, value, NJS_ENUM_KEYS);
             if (state->keys == NULL) {
                 return NULL;
             }
@@ -1659,7 +1659,7 @@ njs_json_push_stringify_state(njs_vm_t *
                 state->keys = njs_extern_keys_array(vm, value->external.proto);
 
             } else {
-                state->keys = njs_object_keys_array(vm, value);
+                state->keys = njs_object_enumerate(vm, value, NJS_ENUM_KEYS);
             }
 
             if (state->keys == NULL) {
diff -r 4a543ed58c95 -r 1af8f40573f2 njs/njs_object.c
--- a/njs/njs_object.c	Sat Dec 01 22:32:33 2018 +0300
+++ b/njs/njs_object.c	Mon Dec 03 19:20:58 2018 +0300
@@ -872,7 +872,7 @@ njs_object_keys(njs_vm_t *vm, njs_value_
         return NXT_ERROR;
     }
 
-    keys = njs_object_keys_array(vm, value);
+    keys = njs_object_enumerate(vm, value, NJS_ENUM_KEYS);
     if (keys == NULL) {
         return NXT_ERROR;
     }
@@ -885,20 +885,52 @@ njs_object_keys(njs_vm_t *vm, njs_value_
 }
 
 
+static njs_ret_t
+njs_object_values(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
+    njs_index_t unused)
+ {
+    njs_array_t        *array;
+    const njs_value_t  *value;
+
+    value = njs_arg(args, nargs, 1);
+
+    if (njs_is_null_or_void(value)) {
+        njs_type_error(vm, "cannot convert %s argument to object",
+                       njs_type_string(value->type));
+
+        return NXT_ERROR;
+    }
+
+    array = njs_object_enumerate(vm, value, NJS_ENUM_VALUES);
+    if (array == NULL) {
+        return NXT_ERROR;
+    }
+
+    vm->retval.data.u.array = array;
+    vm->retval.type = NJS_ARRAY;
+    vm->retval.data.truth = 1;
+
+    return NXT_OK;
+}
+
+
 njs_array_t *
-njs_object_keys_array(njs_vm_t *vm, const njs_value_t *value)
+njs_object_enumerate(njs_vm_t *vm, const njs_value_t *value,
+    njs_object_enum_t kind)
 {
-    uint32_t           i, n, length, keys_length, properties;
-    njs_value_t        *string;
-    njs_array_t        *keys, *array;
+    u_char             *dst;
+    uint32_t           i, length, size, items_length, properties;
+    njs_value_t        *string, *item;
+    njs_array_t        *items, *array;
     nxt_lvlhsh_t       *hash;
+    const u_char       *src, *end;
     njs_object_prop_t  *prop;
     njs_string_prop_t  string_prop;
     nxt_lvlhsh_each_t  lhe;
 
     array = NULL;
     length = 0;
-    keys_length = 0;
+    items_length = 0;
 
     switch (value->type) {
     case NJS_ARRAY:
@@ -907,7 +939,7 @@ njs_object_keys_array(njs_vm_t *vm, cons
 
         for (i = 0; i < length; i++) {
             if (njs_is_valid(&array->start[i])) {
-                keys_length++;
+                items_length++;
             }
         }
 
@@ -923,7 +955,7 @@ njs_object_keys_array(njs_vm_t *vm, cons
         }
 
         length = njs_string_prop(&string_prop, string);
-        keys_length += length;
+        items_length += length;
         break;
 
     default:
@@ -950,46 +982,123 @@ njs_object_keys_array(njs_vm_t *vm, cons
             }
         }
 
-        keys_length += properties;
+        items_length += properties;
     }
 
-    keys = njs_array_alloc(vm, keys_length, NJS_ARRAY_SPARE);
-    if (nxt_slow_path(keys == NULL)) {
+    items = njs_array_alloc(vm, items_length, NJS_ARRAY_SPARE);
+    if (nxt_slow_path(items == NULL)) {
         return NULL;
     }
 
-    n = 0;
+    item = items->start;
 
     if (array != NULL) {
-        for (i = 0; i < length; i++) {
-            if (njs_is_valid(&array->start[i])) {
-                njs_uint32_to_string(&keys->start[n++], i);
+
+        switch (kind) {
+        case NJS_ENUM_KEYS:
+            for (i = 0; i < length; i++) {
+                if (njs_is_valid(&array->start[i])) {
+                    njs_uint32_to_string(item++, i);
+                }
             }
+
+            break;
+
+        case NJS_ENUM_VALUES:
+            for (i = 0; i < length; i++) {
+                if (njs_is_valid(&array->start[i])) {
+                    /* GC: retain. */
+                    *item++ = array->start[i];
+                }
+            }
+
+            break;
         }
 
-    } else {
-        for (i = 0; i < length; i++) {
-            njs_uint32_to_string(&keys->start[n++], i);
+    } else if (length != 0) {
+
+        switch (kind) {
+        case NJS_ENUM_KEYS:
+            for (i = 0; i < length; i++) {
+                njs_uint32_to_string(item++, i);
+            }
+
+            break;
+
+        case NJS_ENUM_VALUES:
+            if (string_prop.size == (size_t) length) {
+                /* Byte or ASCII string. */
+
+                for (i = 0; i < length; i++) {
+                    dst = njs_string_short_start(item);
+                    dst[0] = string_prop.start[i];
+
+                    njs_string_short_set(item, 1, 1);
+
+                    item++;
+                }
+
+            } else {
+                /* UTF-8 string. */
+
+                src = string_prop.start;
+                end = src + string_prop.size;
+
+                do {
+                    dst = njs_string_short_start(item);
+                    dst = nxt_utf8_copy(dst, &src, end);
+                    size = dst - njs_string_short_start(value);
+
+                    njs_string_short_set(item, size, 1);
+
+                    item++;
+
+                } while (src != end);
+            }
+
+            break;
         }
     }
 
     if (nxt_fast_path(properties != 0)) {
         nxt_lvlhsh_each_init(&lhe, &njs_object_hash_proto);
 
-        for ( ;; ) {
-            prop = nxt_lvlhsh_each(hash, &lhe);
-
-            if (prop == NULL) {
-                break;
+        switch (kind) {
+
+        case NJS_ENUM_KEYS:
+            for ( ;; ) {
+                prop = nxt_lvlhsh_each(hash, &lhe);
+
+                if (prop == NULL) {
+                    break;
+                }
+
+                if (prop->type != NJS_WHITEOUT && prop->enumerable) {
+                    njs_string_copy(item++, &prop->name);
+                }
             }
 
-            if (prop->type != NJS_WHITEOUT && prop->enumerable) {
-                njs_string_copy(&keys->start[n++], &prop->name);
+            break;
+
+        case NJS_ENUM_VALUES:
+            for ( ;; ) {
+                prop = nxt_lvlhsh_each(hash, &lhe);
+
+                if (prop == NULL) {
+                    break;
+                }
+
+                if (prop->type != NJS_WHITEOUT && prop->enumerable) {
+                    /* GC: retain. */
+                    *item++ = prop->value;
+                }
             }
+
+            break;
         }
     }
 
-    return keys;
+    return items;
 }
 
 
@@ -1853,6 +1962,14 @@ static const njs_object_prop_t  njs_obje
                                      NJS_SKIP_ARG, NJS_OBJECT_ARG),
     },
 
+    /* ES8: Object.values(). */
+    {
+        .type = NJS_METHOD,
+        .name = njs_string("values"),
+        .value = njs_native_function(njs_object_values, 0,
+                                     NJS_SKIP_ARG, NJS_OBJECT_ARG),
+    },
+
     /* Object.defineProperty(). */
     {
         .type = NJS_METHOD,
diff -r 4a543ed58c95 -r 1af8f40573f2 njs/njs_object.h
--- a/njs/njs_object.h	Sat Dec 01 22:32:33 2018 +0300
+++ b/njs/njs_object.h	Mon Dec 03 19:20:58 2018 +0300
@@ -17,6 +17,12 @@ typedef enum {
 } njs_object_property_type_t;
 
 
+typedef enum {
+    NJS_ENUM_KEYS = 0,
+    NJS_ENUM_VALUES,
+} njs_object_enum_t;
+
+
 /*
  * Attributes are generally used as Boolean values.
  * The UNSET value is used internally only by njs_define_property().
@@ -79,7 +85,8 @@ njs_object_t *njs_object_alloc(njs_vm_t 
 njs_object_t *njs_object_value_copy(njs_vm_t *vm, njs_value_t *value);
 njs_object_t *njs_object_value_alloc(njs_vm_t *vm, const njs_value_t *value,
     nxt_uint_t type);
-njs_array_t *njs_object_keys_array(njs_vm_t *vm, const njs_value_t *value);
+njs_array_t *njs_object_enumerate(njs_vm_t *vm, const njs_value_t *value,
+    njs_object_enum_t kind);
 njs_ret_t njs_value_property(njs_vm_t *vm, njs_value_t *value,
     const njs_value_t *property, njs_value_t *retval);
 njs_object_prop_t *njs_object_property(njs_vm_t *vm, const njs_object_t *obj,
diff -r 4a543ed58c95 -r 1af8f40573f2 njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Sat Dec 01 22:32:33 2018 +0300
+++ b/njs/test/njs_unit_test.c	Mon Dec 03 19:20:58 2018 +0300
@@ -7421,6 +7421,30 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("Object.keys(true)"),
       nxt_string("") },
 
+    { nxt_string("var o = {a:3, b:2, c:1}; Object.values(o)"),
+      nxt_string("3,2,1") },
+
+    { nxt_string("Object.values('s')"),
+      nxt_string("s") },
+
+    { nxt_string("Object.values('??? abc')"),
+      nxt_string("?,?,?, ,a,b,c") },
+
+    { nxt_string("var s = new String('abc'); s.three = 3; Object.values(s)"),
+      nxt_string("a,b,c,3") },
+
+    { nxt_string("var a = [,,5,,4,,,3,2]; a.one = 1; Object.values(a)"),
+      nxt_string("5,4,3,2,1") },
+
+    { nxt_string("Object.values([{}, null, false, NaN, function() {}])"),
+      nxt_string("[object Object],,false,NaN,[object Function]") },
+
+    { nxt_string("Object.values(1)"),
+      nxt_string("") },
+
+    { nxt_string("Object.values()"),
+      nxt_string("TypeError: cannot convert undefined argument to object") },
+
     { nxt_string("var o = {}; Object.defineProperty(o, 'a', {}); o.a"),
       nxt_string("undefined") },
 
@@ -7444,6 +7468,20 @@ static njs_unit_test_t  njs_test[] =
                  "Object.keys(o)"),
       nxt_string("a,c,b") },
 
+    { nxt_string("var o = {a:1, c:2}; Object.defineProperty(o, 'b', {});"
+                 "Object.values(o)"),
+      nxt_string("1,2") },
+
+    { nxt_string("var o = {a:1, c:2};"
+                 "Object.defineProperty(o, 'b', {enumerable:false, value:3});"
+                 "Object.values(o)"),
+      nxt_string("1,2") },
+
+    { nxt_string("var o = {a:1, c:3};"
+                 "Object.defineProperty(o, 'b', {enumerable:true, value:2});"
+                 "Object.values(o)"),
+      nxt_string("1,3,2") },
+
     { nxt_string("var o = {}; Object.defineProperty(o, 'a', {}); o.a = 1"),
       nxt_string("TypeError: Cannot assign to read-only property 'a' of object") },
 

From vbart at nginx.com  Mon Dec  3 19:07:22 2018
From: vbart at nginx.com (Valentin Bartenev)
Date: Mon, 03 Dec 2018 19:07:22 +0000
Subject: [njs] Object.entries() method.
Message-ID: <hg.ce3a943df9c4.1543864042.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/ce3a943df9c4
branches:  
changeset: 683:ce3a943df9c4
user:      Valentin Bartenev <vbart at nginx.com>
date:      Mon Dec 03 19:23:27 2018 +0300
description:
Object.entries() method.

diffstat:

 njs/njs_object.c         |  151 ++++++++++++++++++++++++++++++++++++++++++++++-
 njs/njs_object.h         |    1 +
 njs/test/njs_unit_test.c |   37 +++++++++++
 3 files changed, 188 insertions(+), 1 deletions(-)

diffs (261 lines):

diff -r 1af8f40573f2 -r ce3a943df9c4 njs/njs_object.c
--- a/njs/njs_object.c	Mon Dec 03 19:20:58 2018 +0300
+++ b/njs/njs_object.c	Mon Dec 03 19:23:27 2018 +0300
@@ -914,6 +914,35 @@ njs_object_values(njs_vm_t *vm, njs_valu
 }
 
 
+static njs_ret_t
+njs_object_entries(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
+    njs_index_t unused)
+ {
+    njs_array_t        *array;
+    const njs_value_t  *value;
+
+    value = njs_arg(args, nargs, 1);
+
+    if (njs_is_null_or_void(value)) {
+        njs_type_error(vm, "cannot convert %s argument to object",
+                       njs_type_string(value->type));
+
+        return NXT_ERROR;
+    }
+
+    array = njs_object_enumerate(vm, value, NJS_ENUM_BOTH);
+    if (array == NULL) {
+        return NXT_ERROR;
+    }
+
+    vm->retval.data.u.array = array;
+    vm->retval.type = NJS_ARRAY;
+    vm->retval.data.truth = 1;
+
+    return NXT_OK;
+}
+
+
 njs_array_t *
 njs_object_enumerate(njs_vm_t *vm, const njs_value_t *value,
     njs_object_enum_t kind)
@@ -921,7 +950,7 @@ njs_object_enumerate(njs_vm_t *vm, const
     u_char             *dst;
     uint32_t           i, length, size, items_length, properties;
     njs_value_t        *string, *item;
-    njs_array_t        *items, *array;
+    njs_array_t        *items, *array, *entry;
     nxt_lvlhsh_t       *hash;
     const u_char       *src, *end;
     njs_object_prop_t  *prop;
@@ -1013,6 +1042,29 @@ njs_object_enumerate(njs_vm_t *vm, const
             }
 
             break;
+
+        case NJS_ENUM_BOTH:
+            for (i = 0; i < length; i++) {
+                if (njs_is_valid(&array->start[i])) {
+                    entry = njs_array_alloc(vm, 2, 0);
+                    if (nxt_slow_path(entry == NULL)) {
+                        return NULL;
+                    }
+
+                    njs_uint32_to_string(&entry->start[0], i);
+
+                    /* GC: retain. */
+                    entry->start[1] = array->start[i];
+
+                    item->data.u.array = entry;
+                    item->type = NJS_ARRAY;
+                    item->data.truth = 1;
+
+                    item++;
+                }
+            }
+
+            break;
         }
 
     } else if (length != 0) {
@@ -1057,6 +1109,66 @@ njs_object_enumerate(njs_vm_t *vm, const
             }
 
             break;
+
+        case NJS_ENUM_BOTH:
+            if (string_prop.size == (size_t) length) {
+                /* Byte or ASCII string. */
+
+                for (i = 0; i < length; i++) {
+                    entry = njs_array_alloc(vm, 2, 0);
+                    if (nxt_slow_path(entry == NULL)) {
+                        return NULL;
+                    }
+
+                    njs_uint32_to_string(&entry->start[0], i);
+
+                    string = &entry->start[1];
+
+                    dst = njs_string_short_start(string);
+                    dst[0] = string_prop.start[i];
+
+                    njs_string_short_set(string, 1, 1);
+
+                    item->data.u.array = entry;
+                    item->type = NJS_ARRAY;
+                    item->data.truth = 1;
+
+                    item++;
+                }
+
+            } else {
+                /* UTF-8 string. */
+
+                src = string_prop.start;
+                end = src + string_prop.size;
+                i = 0;
+
+                do {
+                    entry = njs_array_alloc(vm, 2, 0);
+                    if (nxt_slow_path(entry == NULL)) {
+                        return NULL;
+                    }
+
+                    njs_uint32_to_string(&entry->start[0], i++);
+
+                    string = &entry->start[1];
+
+                    dst = njs_string_short_start(string);
+                    dst = nxt_utf8_copy(dst, &src, end);
+                    size = dst - njs_string_short_start(value);
+
+                    njs_string_short_set(string, size, 1);
+
+                    item->data.u.array = entry;
+                    item->type = NJS_ARRAY;
+                    item->data.truth = 1;
+
+                    item++;
+
+                } while (src != end);
+            }
+
+            break;
         }
     }
 
@@ -1095,6 +1207,35 @@ njs_object_enumerate(njs_vm_t *vm, const
             }
 
             break;
+
+        case NJS_ENUM_BOTH:
+            for ( ;; ) {
+                prop = nxt_lvlhsh_each(hash, &lhe);
+
+                if (prop == NULL) {
+                    break;
+                }
+
+                if (prop->type != NJS_WHITEOUT && prop->enumerable) {
+                    entry = njs_array_alloc(vm, 2, 0);
+                    if (nxt_slow_path(entry == NULL)) {
+                        return NULL;
+                    }
+
+                    njs_string_copy(&entry->start[0], &prop->name);
+
+                    /* GC: retain. */
+                    entry->start[1] = prop->value;
+
+                    item->data.u.array = entry;
+                    item->type = NJS_ARRAY;
+                    item->data.truth = 1;
+
+                    item++;
+                }
+            }
+
+            break;
         }
     }
 
@@ -1970,6 +2111,14 @@ static const njs_object_prop_t  njs_obje
                                      NJS_SKIP_ARG, NJS_OBJECT_ARG),
     },
 
+    /* ES8: Object.entries(). */
+    {
+        .type = NJS_METHOD,
+        .name = njs_string("entries"),
+        .value = njs_native_function(njs_object_entries, 0,
+                                     NJS_SKIP_ARG, NJS_OBJECT_ARG),
+    },
+
     /* Object.defineProperty(). */
     {
         .type = NJS_METHOD,
diff -r 1af8f40573f2 -r ce3a943df9c4 njs/njs_object.h
--- a/njs/njs_object.h	Mon Dec 03 19:20:58 2018 +0300
+++ b/njs/njs_object.h	Mon Dec 03 19:23:27 2018 +0300
@@ -20,6 +20,7 @@ typedef enum {
 typedef enum {
     NJS_ENUM_KEYS = 0,
     NJS_ENUM_VALUES,
+    NJS_ENUM_BOTH,
 } njs_object_enum_t;
 
 
diff -r 1af8f40573f2 -r ce3a943df9c4 njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Mon Dec 03 19:20:58 2018 +0300
+++ b/njs/test/njs_unit_test.c	Mon Dec 03 19:23:27 2018 +0300
@@ -7445,6 +7445,29 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("Object.values()"),
       nxt_string("TypeError: cannot convert undefined argument to object") },
 
+    { nxt_string("Object.entries('abc')"),
+      nxt_string("0,a,1,b,2,c") },
+
+    { nxt_string("JSON.stringify(Object.entries('???'))"),
+      nxt_string("[[\"0\",\"?\"],[\"1\",\"?\"],[\"2\",\"?\"]]") },
+
+    { nxt_string("var o = {a:\"?\", b:\"?\", c:\"?\"};"
+                 "JSON.stringify(Object.entries(o))"),
+      nxt_string("[[\"a\",\"?\"],[\"b\",\"?\"],[\"c\",\"?\"]]") },
+
+    { nxt_string("JSON.stringify(Object.entries([0]))"),
+      nxt_string("[[\"0\",0]]") },
+
+    { nxt_string("var s = new String('??'); s.two = null; s[3] = true;"
+                 "Object.entries(s)"),
+      nxt_string("0,?,1,?,two,,3,true") },
+
+    { nxt_string("Object.entries(true)"),
+      nxt_string("") },
+
+    { nxt_string("Object.entries()"),
+      nxt_string("TypeError: cannot convert undefined argument to object") },
+
     { nxt_string("var o = {}; Object.defineProperty(o, 'a', {}); o.a"),
       nxt_string("undefined") },
 
@@ -7482,6 +7505,20 @@ static njs_unit_test_t  njs_test[] =
                  "Object.values(o)"),
       nxt_string("1,3,2") },
 
+    { nxt_string("var o = {a:1, c:2}; Object.defineProperty(o, 'b', {});"
+                 "Object.entries(o)"),
+      nxt_string("a,1,c,2") },
+
+    { nxt_string("var o = {a:1, c:2};"
+                 "Object.defineProperty(o, 'b', {enumerable:false, value:3});"
+                 "Object.entries(o)"),
+      nxt_string("a,1,c,2") },
+
+    { nxt_string("var o = {a:1, c:3};"
+                 "Object.defineProperty(o, 'b', {enumerable:true, value:2});"
+                 "Object.entries(o)"),
+      nxt_string("a,1,c,3,b,2") },
+
     { nxt_string("var o = {}; Object.defineProperty(o, 'a', {}); o.a = 1"),
       nxt_string("TypeError: Cannot assign to read-only property 'a' of object") },
 

From andrew.wheat at bbc.co.uk  Tue Dec  4 11:27:07 2018
From: andrew.wheat at bbc.co.uk (Andrew Wheat)
Date: Tue, 4 Dec 2018 11:27:07 +0000
Subject: [PATCH] Tests for proxy_cache_lock without serving stale.
In-Reply-To: <D82AEC3F.1330E%andrew.wheat@bbc.co.uk>
References: <D82AEC3F.1330E%andrew.wheat@bbc.co.uk>
Message-ID: <D82C1445.13327%andrew.wheat@bbc.co.uk>


As mentioned in a previous patch [0], here is a patch against
http://hg.nginx.org/nginx-tests supplying the tests that check that
proxy_cache_lock can be used when proxy_cache_use_stale is set to off.

There are two cases:
Firstly when the upstream sets a 'Cache-Control' header, the response is
cached for the specified duration.
Secondly when proxy_cache_valid is used to define the cache duration.


[0]
http://mailman.nginx.org/pipermail/nginx-devel/2018-December/011658.html



-----------------------------
http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and
may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in
error, please delete it from your system.
Do not use, copy or disclose the
information in any way nor act in reliance on it and notify the sender
immediately.
Please note that the BBC monitors e-mails
sent or received.
Further communication will signify your consent to
this.
-----------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: proxy_cache_lock-and-proxy_cache_use_stale.patch
Type: application/octet-stream
Size: 10179 bytes
Desc: proxy_cache_lock-and-proxy_cache_use_stale.patch
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20181204/8ec1b53d/attachment.obj>

From mdounin at mdounin.ru  Tue Dec  4 13:36:43 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:36:43 +0000
Subject: [nginx] Version bump.
Message-ID: <hg.d1525e45255a.1543930603.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/d1525e45255a
branches:  stable-1.14
changeset: 7405:d1525e45255a
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Dec 03 20:07:36 2018 +0300
description:
Version bump.

diffstat:

 src/core/nginx.h |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff --git a/src/core/nginx.h b/src/core/nginx.h
--- a/src/core/nginx.h
+++ b/src/core/nginx.h
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1014001
-#define NGINX_VERSION      "1.14.1"
+#define nginx_version      1014002
+#define NGINX_VERSION      "1.14.2"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #ifdef NGX_BUILD

From mdounin at mdounin.ru  Tue Dec  4 13:36:45 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:36:45 +0000
Subject: [nginx] Silenced -Wcast-function-type warnings (closes #1546).
Message-ID: <hg.ed5b3c4c1284.1543930605.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/ed5b3c4c1284
branches:  stable-1.14
changeset: 7406:ed5b3c4c1284
user:      Sergey Kandaurov <pluknet at nginx.com>
date:      Mon May 07 09:54:37 2018 +0000
description:
Silenced -Wcast-function-type warnings (closes #1546).

Cast to intermediate "void *" to lose compiler knowledge about the original
type and pass the warning.  This is not a real fix but rather a workaround.

Found by gcc8.

diffstat:

 src/http/modules/ngx_http_fastcgi_module.c |   6 ++++--
 src/http/modules/ngx_http_grpc_module.c    |   3 ++-
 src/http/modules/ngx_http_proxy_module.c   |   3 ++-
 src/http/modules/ngx_http_scgi_module.c    |   6 ++++--
 src/http/modules/ngx_http_uwsgi_module.c   |   6 ++++--
 src/http/ngx_http_script.c                 |  13 ++++++++-----
 src/stream/ngx_stream_script.c             |  13 +++++++------
 7 files changed, 31 insertions(+), 19 deletions(-)

diffs (183 lines):

diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c
--- a/src/http/modules/ngx_http_fastcgi_module.c
+++ b/src/http/modules/ngx_http_fastcgi_module.c
@@ -3264,7 +3264,8 @@ ngx_http_fastcgi_init_params(ngx_conf_t 
             return NGX_ERROR;
         }
 
-        copy->code = (ngx_http_script_code_pt) ngx_http_script_copy_len_code;
+        copy->code = (ngx_http_script_code_pt) (void *)
+                                                 ngx_http_script_copy_len_code;
         copy->len = src[i].key.len;
 
         copy = ngx_array_push_n(params->lengths,
@@ -3273,7 +3274,8 @@ ngx_http_fastcgi_init_params(ngx_conf_t 
             return NGX_ERROR;
         }
 
-        copy->code = (ngx_http_script_code_pt) ngx_http_script_copy_len_code;
+        copy->code = (ngx_http_script_code_pt) (void *)
+                                                 ngx_http_script_copy_len_code;
         copy->len = src[i].skip_empty;
 
 
diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c
--- a/src/http/modules/ngx_http_grpc_module.c
+++ b/src/http/modules/ngx_http_grpc_module.c
@@ -4404,7 +4404,8 @@ ngx_http_grpc_init_headers(ngx_conf_t *c
             return NGX_ERROR;
         }
 
-        copy->code = (ngx_http_script_code_pt) ngx_http_script_copy_len_code;
+        copy->code = (ngx_http_script_code_pt) (void *)
+                                                 ngx_http_script_copy_len_code;
         copy->len = src[i].key.len;
 
         size = (sizeof(ngx_http_script_copy_code_t)
diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c
--- a/src/http/modules/ngx_http_proxy_module.c
+++ b/src/http/modules/ngx_http_proxy_module.c
@@ -3493,7 +3493,8 @@ ngx_http_proxy_init_headers(ngx_conf_t *
             return NGX_ERROR;
         }
 
-        copy->code = (ngx_http_script_code_pt) ngx_http_script_copy_len_code;
+        copy->code = (ngx_http_script_code_pt) (void *)
+                                                 ngx_http_script_copy_len_code;
         copy->len = src[i].key.len;
 
         size = (sizeof(ngx_http_script_copy_code_t)
diff --git a/src/http/modules/ngx_http_scgi_module.c b/src/http/modules/ngx_http_scgi_module.c
--- a/src/http/modules/ngx_http_scgi_module.c
+++ b/src/http/modules/ngx_http_scgi_module.c
@@ -1724,7 +1724,8 @@ ngx_http_scgi_init_params(ngx_conf_t *cf
             return NGX_ERROR;
         }
 
-        copy->code = (ngx_http_script_code_pt) ngx_http_script_copy_len_code;
+        copy->code = (ngx_http_script_code_pt) (void *)
+                                                 ngx_http_script_copy_len_code;
         copy->len = src[i].key.len + 1;
 
         copy = ngx_array_push_n(params->lengths,
@@ -1733,7 +1734,8 @@ ngx_http_scgi_init_params(ngx_conf_t *cf
             return NGX_ERROR;
         }
 
-        copy->code = (ngx_http_script_code_pt) ngx_http_script_copy_len_code;
+        copy->code = (ngx_http_script_code_pt) (void *)
+                                                 ngx_http_script_copy_len_code;
         copy->len = src[i].skip_empty;
 
 
diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c
--- a/src/http/modules/ngx_http_uwsgi_module.c
+++ b/src/http/modules/ngx_http_uwsgi_module.c
@@ -1987,7 +1987,8 @@ ngx_http_uwsgi_init_params(ngx_conf_t *c
             return NGX_ERROR;
         }
 
-        copy->code = (ngx_http_script_code_pt) ngx_http_script_copy_len_code;
+        copy->code = (ngx_http_script_code_pt) (void *)
+                                                 ngx_http_script_copy_len_code;
         copy->len = src[i].key.len;
 
         copy = ngx_array_push_n(params->lengths,
@@ -1996,7 +1997,8 @@ ngx_http_uwsgi_init_params(ngx_conf_t *c
             return NGX_ERROR;
         }
 
-        copy->code = (ngx_http_script_code_pt) ngx_http_script_copy_len_code;
+        copy->code = (ngx_http_script_code_pt) (void *)
+                                                 ngx_http_script_copy_len_code;
         copy->len = src[i].skip_empty;
 
 
diff --git a/src/http/ngx_http_script.c b/src/http/ngx_http_script.c
--- a/src/http/ngx_http_script.c
+++ b/src/http/ngx_http_script.c
@@ -695,7 +695,8 @@ ngx_http_script_add_copy_code(ngx_http_s
         return NGX_ERROR;
     }
 
-    code->code = (ngx_http_script_code_pt) ngx_http_script_copy_len_code;
+    code->code = (ngx_http_script_code_pt) (void *)
+                                                 ngx_http_script_copy_len_code;
     code->len = len;
 
     size = (sizeof(ngx_http_script_copy_code_t) + len + sizeof(uintptr_t) - 1)
@@ -784,7 +785,8 @@ ngx_http_script_add_var_code(ngx_http_sc
         return NGX_ERROR;
     }
 
-    code->code = (ngx_http_script_code_pt) ngx_http_script_copy_var_len_code;
+    code->code = (ngx_http_script_code_pt) (void *)
+                                             ngx_http_script_copy_var_len_code;
     code->index = (uintptr_t) index;
 
     code = ngx_http_script_add_code(*sc->values,
@@ -1178,8 +1180,8 @@ ngx_http_script_add_capture_code(ngx_htt
         return NGX_ERROR;
     }
 
-    code->code = (ngx_http_script_code_pt)
-                      ngx_http_script_copy_capture_len_code;
+    code->code = (ngx_http_script_code_pt) (void *)
+                                         ngx_http_script_copy_capture_len_code;
     code->n = 2 * n;
 
 
@@ -1293,7 +1295,8 @@ ngx_http_script_add_full_name_code(ngx_h
         return NGX_ERROR;
     }
 
-    code->code = (ngx_http_script_code_pt) ngx_http_script_full_name_len_code;
+    code->code = (ngx_http_script_code_pt) (void *)
+                                            ngx_http_script_full_name_len_code;
     code->conf_prefix = sc->conf_prefix;
 
     code = ngx_http_script_add_code(*sc->values,
diff --git a/src/stream/ngx_stream_script.c b/src/stream/ngx_stream_script.c
--- a/src/stream/ngx_stream_script.c
+++ b/src/stream/ngx_stream_script.c
@@ -587,7 +587,8 @@ ngx_stream_script_add_copy_code(ngx_stre
         return NGX_ERROR;
     }
 
-    code->code = (ngx_stream_script_code_pt) ngx_stream_script_copy_len_code;
+    code->code = (ngx_stream_script_code_pt) (void *)
+                                               ngx_stream_script_copy_len_code;
     code->len = len;
 
     size = (sizeof(ngx_stream_script_copy_code_t) + len + sizeof(uintptr_t) - 1)
@@ -677,8 +678,8 @@ ngx_stream_script_add_var_code(ngx_strea
         return NGX_ERROR;
     }
 
-    code->code = (ngx_stream_script_code_pt)
-                      ngx_stream_script_copy_var_len_code;
+    code->code = (ngx_stream_script_code_pt) (void *)
+                                           ngx_stream_script_copy_var_len_code;
     code->index = (uintptr_t) index;
 
     code = ngx_stream_script_add_code(*sc->values,
@@ -767,8 +768,8 @@ ngx_stream_script_add_capture_code(ngx_s
         return NGX_ERROR;
     }
 
-    code->code = (ngx_stream_script_code_pt)
-                      ngx_stream_script_copy_capture_len_code;
+    code->code = (ngx_stream_script_code_pt) (void *)
+                                       ngx_stream_script_copy_capture_len_code;
     code->n = 2 * n;
 
 
@@ -859,7 +860,7 @@ ngx_stream_script_add_full_name_code(ngx
         return NGX_ERROR;
     }
 
-    code->code = (ngx_stream_script_code_pt)
+    code->code = (ngx_stream_script_code_pt) (void *)
                                           ngx_stream_script_full_name_len_code;
     code->conf_prefix = sc->conf_prefix;
 

From mdounin at mdounin.ru  Tue Dec  4 13:36:46 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:36:46 +0000
Subject: [nginx] Removed glibc crypt_r() bug workaround (ticket #1469).
Message-ID: <hg.b1a166ab7f04.1543930606.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/b1a166ab7f04
branches:  stable-1.14
changeset: 7407:b1a166ab7f04
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Wed May 23 16:38:16 2018 +0300
description:
Removed glibc crypt_r() bug workaround (ticket #1469).

The bug in question was fixed in glibc 2.3.2 and is no longer expected
to manifest itself on real servers.  On the other hand, the workaround
causes compilation problems on various systems.  Previously, we've
already fixed the code to compile with musl libc (fd6fd02f6a4d), and
now it is broken on Fedora 28 where glibc's crypt library was replaced
by libxcrypt.  So the workaround was removed.

diffstat:

 src/os/unix/ngx_user.c |  4 ----
 1 files changed, 0 insertions(+), 4 deletions(-)

diffs (14 lines):

diff --git a/src/os/unix/ngx_user.c b/src/os/unix/ngx_user.c
--- a/src/os/unix/ngx_user.c
+++ b/src/os/unix/ngx_user.c
@@ -21,10 +21,6 @@ ngx_libc_crypt(ngx_pool_t *pool, u_char 
     struct crypt_data   cd;
 
     cd.initialized = 0;
-#ifdef __GLIBC__
-    /* work around the glibc bug */
-    cd.current_salt[0] = ~salt[0];
-#endif
 
     value = crypt_r((char *) key, (char *) salt, &cd);
 

From mdounin at mdounin.ru  Tue Dec  4 13:36:50 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:36:50 +0000
Subject: [nginx] Upstream: disable body cleanup with preserve_output (ticket
 #1565).
Message-ID: <hg.17ee239ae2e6.1543930610.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/17ee239ae2e6
branches:  stable-1.14
changeset: 7409:17ee239ae2e6
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Wed Jun 13 15:28:11 2018 +0300
description:
Upstream: disable body cleanup with preserve_output (ticket #1565).

With u->conf->preserve_output set the request body file might be used
after the response header is sent, so avoid cleaning it.  (Normally
this is not a problem as u->conf->preserve_output is only set with
r->request_body_no_buffering, but the request body might be already
written to a file in a different context.)

diffstat:

 src/http/ngx_http_upstream.c |  3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diffs (13 lines):

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -2901,7 +2901,8 @@ ngx_http_upstream_send_response(ngx_http
     }
 
     if (r->request_body && r->request_body->temp_file
-        && r == r->main && !r->preserve_body)
+        && r == r->main && !r->preserve_body
+        && !u->conf->preserve_output)
     {
         ngx_pool_run_cleanup_file(r->pool, r->request_body->temp_file->file.fd);
         r->request_body->temp_file->file.fd = NGX_INVALID_FILE;

From mdounin at mdounin.ru  Tue Dec  4 13:36:48 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:36:48 +0000
Subject: [nginx] Events: fixed handling zero-length client address.
Message-ID: <hg.8af6dceeb648.1543930608.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/8af6dceeb648
branches:  stable-1.14
changeset: 7408:8af6dceeb648
user:      Roman Arutyunyan <arut at nginx.com>
date:      Fri Jun 01 16:53:02 2018 +0300
description:
Events: fixed handling zero-length client address.

On Linux recvmsg() syscall may return a zero-length client address when
receiving a datagram from an unbound unix datagram socket.  It is usually
assumed that socket address has at least the sa_family member.  Zero-length
socket address caused buffer over-read in functions which receive socket
address, for example ngx_sock_ntop().  Typically the over-read resulted in
unexpected socket family followed by session close.  Now a fake socket address
is allocated instead of a zero-length client address.

diffstat:

 src/event/ngx_event_accept.c |  12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

diffs (22 lines):

diff --git a/src/event/ngx_event_accept.c b/src/event/ngx_event_accept.c
--- a/src/event/ngx_event_accept.c
+++ b/src/event/ngx_event_accept.c
@@ -448,6 +448,18 @@ ngx_event_recvmsg(ngx_event_t *ev)
             c->socklen = sizeof(ngx_sockaddr_t);
         }
 
+        if (c->socklen == 0) {
+
+            /*
+             * on Linux recvmsg() returns zero msg_namelen
+             * when receiving packets from unbound AF_UNIX sockets
+             */
+
+            c->socklen = sizeof(struct sockaddr);
+            ngx_memzero(&sa, sizeof(struct sockaddr));
+            sa.sockaddr.sa_family = ls->sockaddr->sa_family;
+        }
+
 #if (NGX_STAT_STUB)
         (void) ngx_atomic_fetch_add(ngx_stat_active, 1);
 #endif

From mdounin at mdounin.ru  Tue Dec  4 13:36:51 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:36:51 +0000
Subject: [nginx] gRPC: clearing buffers in ngx_http_grpc_get_buf().
Message-ID: <hg.9ac0e8b9aced.1543930611.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/9ac0e8b9aced
branches:  stable-1.14
changeset: 7410:9ac0e8b9aced
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Jul 02 19:02:08 2018 +0300
description:
gRPC: clearing buffers in ngx_http_grpc_get_buf().

We copy input buffers to our buffers, so various flags might be
unexpectedly set in buffers returned by ngx_chain_get_free_buf().

In particular, the b->in_file flag might be set when the body was
written to a file in a different context.  With sendfile enabled this
in turn might result in protocol corruption if such a buffer was reused
for a control frame.

Make sure to clear buffers and set only fields we really need to be set.

diffstat:

 src/http/modules/ngx_http_grpc_module.c |  29 +++++++++++++++++------------
 1 files changed, 17 insertions(+), 12 deletions(-)

diffs (57 lines):

diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c
--- a/src/http/modules/ngx_http_grpc_module.c
+++ b/src/http/modules/ngx_http_grpc_module.c
@@ -3883,6 +3883,7 @@ ngx_http_grpc_send_window_update(ngx_htt
 static ngx_chain_t *
 ngx_http_grpc_get_buf(ngx_http_request_t *r, ngx_http_grpc_ctx_t *ctx)
 {
+    u_char       *start;
     ngx_buf_t    *b;
     ngx_chain_t  *cl;
 
@@ -3892,29 +3893,33 @@ ngx_http_grpc_get_buf(ngx_http_request_t
     }
 
     b = cl->buf;
-
-    b->tag = (ngx_buf_tag_t) &ngx_http_grpc_body_output_filter;
-    b->temporary = 1;
-    b->flush = 1;
-
-    if (b->start == NULL) {
+    start = b->start;
+
+    if (start == NULL) {
 
         /*
          * each buffer is large enough to hold two window update
          * frames in a row
          */
 
-        b->start = ngx_palloc(r->pool, 2 * sizeof(ngx_http_grpc_frame_t) + 8);
-        if (b->start == NULL) {
+        start = ngx_palloc(r->pool, 2 * sizeof(ngx_http_grpc_frame_t) + 8);
+        if (start == NULL) {
             return NULL;
         }
 
-        b->pos = b->start;
-        b->last = b->start;
-
-        b->end = b->start + 2 * sizeof(ngx_http_grpc_frame_t) + 8;
     }
 
+    ngx_memzero(b, sizeof(ngx_buf_t));
+
+    b->start = start;
+    b->pos = start;
+    b->last = start;
+    b->end = start + 2 * sizeof(ngx_http_grpc_frame_t) + 8;
+
+    b->tag = (ngx_buf_tag_t) &ngx_http_grpc_body_output_filter;
+    b->temporary = 1;
+    b->flush = 1;
+
     return cl;
 }
 

From mdounin at mdounin.ru  Tue Dec  4 13:36:53 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:36:53 +0000
Subject: [nginx] Upstream: fixed tcp_nopush with gRPC.
Message-ID: <hg.abe1b60c4213.1543930613.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/abe1b60c4213
branches:  stable-1.14
changeset: 7411:abe1b60c4213
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Jul 02 19:03:04 2018 +0300
description:
Upstream: fixed tcp_nopush with gRPC.

With gRPC it is possible that a request sending is blocked due to flow
control.  Moreover, further sending might be only allowed once the
backend sees all the data we've already sent.  With such a backend
it is required to clear the TCP_NOPUSH socket option to make sure all
the data we've sent are actually delivered to the backend.

As such, we now clear TCP_NOPUSH in ngx_http_upstream_send_request()
also on NGX_AGAIN if c->write->ready is set.  This fixes a test (which
waits for all the 64k bytes as per initial window before allowing more
bytes) with sendfile enabled when the body was written to a file
in a different context.

diffstat:

 src/http/ngx_http_upstream.c |  12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

diffs (22 lines):

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -2008,6 +2008,18 @@ ngx_http_upstream_send_request(ngx_http_
             return;
         }
 
+        if (c->write->ready && c->tcp_nopush == NGX_TCP_NOPUSH_SET) {
+            if (ngx_tcp_push(c->fd) == -1) {
+                ngx_log_error(NGX_LOG_CRIT, c->log, ngx_socket_errno,
+                              ngx_tcp_push_n " failed");
+                ngx_http_upstream_finalize_request(r, u,
+                                               NGX_HTTP_INTERNAL_SERVER_ERROR);
+                return;
+            }
+
+            c->tcp_nopush = NGX_TCP_NOPUSH_UNSET;
+        }
+
         return;
     }
 

From mdounin at mdounin.ru  Tue Dec  4 13:36:54 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:36:54 +0000
Subject: [nginx] SSL: logging level of "https proxy request" errors.
Message-ID: <hg.6c52c99c475e.1543930614.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/6c52c99c475e
branches:  stable-1.14
changeset: 7412:6c52c99c475e
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Jul 05 20:45:29 2018 +0300
description:
SSL: logging level of "https proxy request" errors.

The "http request" and "https proxy request" errors cannot happen
with HTTP due to pre-handshake checks in ngx_http_ssl_handshake(),
but can happen when SSL is used in stream and mail modules.

diffstat:

 src/event/ngx_event_openssl.c |  2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diffs (12 lines):

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -2063,6 +2063,8 @@ ngx_ssl_connection_error(ngx_connection_
             || n == SSL_R_DIGEST_CHECK_FAILED                        /*  149 */
             || n == SSL_R_ERROR_IN_RECEIVED_CIPHER_LIST              /*  151 */
             || n == SSL_R_EXCESSIVE_MESSAGE_SIZE                     /*  152 */
+            || n == SSL_R_HTTPS_PROXY_REQUEST                        /*  155 */
+            || n == SSL_R_HTTP_REQUEST                               /*  156 */
             || n == SSL_R_LENGTH_MISMATCH                            /*  159 */
 #ifdef SSL_R_NO_CIPHERS_PASSED
             || n == SSL_R_NO_CIPHERS_PASSED                          /*  182 */

From mdounin at mdounin.ru  Tue Dec  4 13:36:56 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:36:56 +0000
Subject: [nginx] SSL: logging levels of "unsupported protocol",
 "version too low".
Message-ID: <hg.7c00d8dbdb3a.1543930616.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/7c00d8dbdb3a
branches:  stable-1.14
changeset: 7413:7c00d8dbdb3a
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Jul 16 17:47:18 2018 +0300
description:
SSL: logging levels of "unsupported protocol", "version too low".

Starting with OpenSSL 1.1.0, SSL_R_UNSUPPORTED_PROTOCOL instead of
SSL_R_UNKNOWN_PROTOCOL is reported when a protocol is disabled via
an SSL_OP_NO_* option.

Additionally, SSL_R_VERSION_TOO_LOW is reported when using MinProtocol
or when seclevel checks (as set by @SECLEVEL=n in the cipher string)
rejects a protocol, and this is what happens with SSLv3 and @SECLEVEL=1,
which is the default.

There is also the SSL_R_VERSION_TOO_HIGH error code, but it looks like
it is not possible to trigger it.

diffstat:

 src/event/ngx_event_openssl.c |  4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diffs (21 lines):

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -2080,6 +2080,7 @@ ngx_ssl_connection_error(ngx_connection_
             || n == SSL_R_UNEXPECTED_RECORD                          /*  245 */
             || n == SSL_R_UNKNOWN_ALERT_TYPE                         /*  246 */
             || n == SSL_R_UNKNOWN_PROTOCOL                           /*  252 */
+            || n == SSL_R_UNSUPPORTED_PROTOCOL                       /*  258 */
             || n == SSL_R_WRONG_VERSION_NUMBER                       /*  267 */
             || n == SSL_R_DECRYPTION_FAILED_OR_BAD_RECORD_MAC        /*  281 */
 #ifdef SSL_R_RENEGOTIATE_EXT_TOO_LONG
@@ -2096,6 +2097,9 @@ ngx_ssl_connection_error(ngx_connection_
 #ifdef SSL_R_INAPPROPRIATE_FALLBACK
             || n == SSL_R_INAPPROPRIATE_FALLBACK                     /*  373 */
 #endif
+#ifdef SSL_R_VERSION_TOO_LOW
+            || n == SSL_R_VERSION_TOO_LOW                            /*  396 */
+#endif
             || n == 1000 /* SSL_R_SSLV3_ALERT_CLOSE_NOTIFY */
 #ifdef SSL_R_SSLV3_ALERT_UNEXPECTED_MESSAGE
             || n == SSL_R_SSLV3_ALERT_UNEXPECTED_MESSAGE             /* 1010 */

From mdounin at mdounin.ru  Tue Dec  4 13:36:57 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:36:57 +0000
Subject: [nginx] SSL: fixed SSL_clear_options() usage with OpenSSL 1.1.0+.
Message-ID: <hg.ea3f1f9af72c.1543930617.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/ea3f1f9af72c
branches:  stable-1.14
changeset: 7414:ea3f1f9af72c
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Jul 16 17:47:20 2018 +0300
description:
SSL: fixed SSL_clear_options() usage with OpenSSL 1.1.0+.

In OpenSSL 1.1.0 the SSL_CTRL_CLEAR_OPTIONS macro was removed, so
conditional compilation test on it results in SSL_clear_options()
and SSL_CTX_clear_options() not being used.  Notably, this caused
"ssl_prefer_server_ciphers off" to not work in SNI-based virtual
servers if server preference was switched on in the default server.

It looks like the only possible fix is to test OPENSSL_VERSION_NUMBER
explicitly.

diffstat:

 src/event/ngx_event_openssl.c |  2 +-
 src/http/ngx_http_request.c   |  2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diffs (24 lines):

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -296,7 +296,7 @@ ngx_ssl_create(ngx_ssl_t *ssl, ngx_uint_
 
     SSL_CTX_set_options(ssl->ctx, SSL_OP_SINGLE_DH_USE);
 
-#ifdef SSL_CTRL_CLEAR_OPTIONS
+#if OPENSSL_VERSION_NUMBER >= 0x009080dfL
     /* only in 0.9.8m+ */
     SSL_CTX_clear_options(ssl->ctx,
                           SSL_OP_NO_SSLv2|SSL_OP_NO_SSLv3|SSL_OP_NO_TLSv1);
diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c
+++ b/src/http/ngx_http_request.c
@@ -923,7 +923,7 @@ ngx_http_ssl_servername(ngx_ssl_conn_t *
 
         SSL_set_verify_depth(ssl_conn, SSL_CTX_get_verify_depth(sscf->ssl.ctx));
 
-#ifdef SSL_CTRL_CLEAR_OPTIONS
+#if OPENSSL_VERSION_NUMBER >= 0x009080dfL
         /* only in 0.9.8m+ */
         SSL_clear_options(ssl_conn, SSL_get_options(ssl_conn) &
                                     ~SSL_CTX_get_options(sscf->ssl.ctx));

From mdounin at mdounin.ru  Tue Dec  4 13:36:59 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:36:59 +0000
Subject: [nginx] SSL: fixed build with LibreSSL 2.8.0 (ticket #1605).
Message-ID: <hg.2cf1d945bbb3.1543930619.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/2cf1d945bbb3
branches:  stable-1.14
changeset: 7415:2cf1d945bbb3
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Fri Aug 10 20:49:06 2018 +0300
description:
SSL: fixed build with LibreSSL 2.8.0 (ticket #1605).

LibreSSL 2.8.0 "added const annotations to many existing APIs from OpenSSL,
making interoperability easier for downstream applications".  This includes
the const change in the SSL_CTX_sess_set_get_cb() callback function (see
9dd43f4ef67e), which breaks compilation.

To fix this, added a condition on how we redefine OPENSSL_VERSION_NUMBER
when working with LibreSSL (see 382fc7069e3a).  With LibreSSL 2.8.0,
we now set OPENSSL_VERSION_NUMBER to 0x1010000fL (OpenSSL 1.1.0), so the
appropriate conditions in the code will use "const" as it happens with
OpenSSL 1.1.0 and later versions.

diffstat:

 src/event/ngx_event_openssl.h |  4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diffs (16 lines):

diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h
--- a/src/event/ngx_event_openssl.h
+++ b/src/event/ngx_event_openssl.h
@@ -36,8 +36,12 @@
 
 #if (defined LIBRESSL_VERSION_NUMBER && OPENSSL_VERSION_NUMBER == 0x20000000L)
 #undef OPENSSL_VERSION_NUMBER
+#if (LIBRESSL_VERSION_NUMBER >= 0x2080000fL)
+#define OPENSSL_VERSION_NUMBER  0x1010000fL
+#else
 #define OPENSSL_VERSION_NUMBER  0x1000107fL
 #endif
+#endif
 
 
 #if (OPENSSL_VERSION_NUMBER >= 0x10100001L)

From mdounin at mdounin.ru  Tue Dec  4 13:37:01 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:37:01 +0000
Subject: [nginx] gRPC: improved keepalive handling.
Message-ID: <hg.c948804cd628.1543930621.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/c948804cd628
branches:  stable-1.14
changeset: 7416:c948804cd628
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Sep 03 19:34:01 2018 +0300
description:
gRPC: improved keepalive handling.

The code is now able to parse additional control frames after
the response is received, and can send control frames as well.
This fixes keepalive problems as observed with grpc-c, which can
send window update and ping frames after the response, see
http://mailman.nginx.org/pipermail/nginx/2018-August/056620.html.

diffstat:

 src/http/modules/ngx_http_grpc_module.c |  100 +++++++++++++++++++++----------
 1 files changed, 67 insertions(+), 33 deletions(-)

diffs (173 lines):

diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c
--- a/src/http/modules/ngx_http_grpc_module.c
+++ b/src/http/modules/ngx_http_grpc_module.c
@@ -114,6 +114,7 @@ typedef struct {
     unsigned                   output_closed:1;
     unsigned                   parsing_headers:1;
     unsigned                   end_stream:1;
+    unsigned                   done:1;
     unsigned                   status:1;
 
     ngx_http_request_t        *request;
@@ -1077,6 +1078,7 @@ ngx_http_grpc_reinit_request(ngx_http_re
     ctx->output_closed = 0;
     ctx->parsing_headers = 0;
     ctx->end_stream = 0;
+    ctx->done = 0;
     ctx->status = 0;
     ctx->connection = NULL;
 
@@ -1096,6 +1098,7 @@ ngx_http_grpc_body_output_filter(void *d
     ngx_int_t               rc;
     ngx_uint_t              next, last;
     ngx_chain_t            *cl, *out, **ll;
+    ngx_http_upstream_t    *u;
     ngx_http_grpc_ctx_t    *ctx;
     ngx_http_grpc_frame_t  *f;
 
@@ -1410,6 +1413,28 @@ ngx_http_grpc_body_output_filter(void *d
         rc = NGX_AGAIN;
     }
 
+    if (ctx->done) {
+
+        /*
+         * We have already got the response and were sending some additional
+         * control frames.  Even if there is still something unsent, stop
+         * here anyway.
+         */
+
+        u = r->upstream;
+        u->length = 0;
+
+        if (ctx->in == NULL
+            && ctx->out == NULL
+            && ctx->output_closed
+            && ctx->state == ngx_http_grpc_st_start)
+        {
+            u->keepalive = 1;
+        }
+
+        ngx_post_event(u->peer.connection->read, &ngx_posted_events);
+    }
+
     return rc;
 }
 
@@ -1835,6 +1860,33 @@ ngx_http_grpc_filter(void *data, ssize_t
             rc = ngx_http_grpc_parse_frame(r, ctx, b);
 
             if (rc == NGX_AGAIN) {
+
+                if (ctx->done) {
+
+                    /*
+                     * We have finished parsing the response and the
+                     * remaining control frames.  If there are unsent
+                     * control frames, post a write event to send them.
+                     */
+
+                    if (ctx->out) {
+                        ngx_post_event(u->peer.connection->write,
+                                       &ngx_posted_events);
+                        return NGX_AGAIN;
+                    }
+
+                    u->length = 0;
+
+                    if (ctx->in == NULL
+                        && ctx->output_closed
+                        && ctx->state == ngx_http_grpc_st_start)
+                    {
+                        u->keepalive = 1;
+                    }
+
+                    break;
+                }
+
                 return NGX_AGAIN;
             }
 
@@ -1901,6 +1953,13 @@ ngx_http_grpc_filter(void *data, ssize_t
                 return NGX_ERROR;
             }
 
+            if (ctx->stream_id && ctx->done) {
+                ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                              "upstream sent frame for closed stream %ui",
+                              ctx->stream_id);
+                return NGX_ERROR;
+            }
+
             ctx->padding = 0;
         }
 
@@ -1917,17 +1976,7 @@ ngx_http_grpc_filter(void *data, ssize_t
             ctx->state = ngx_http_grpc_st_start;
 
             if (ctx->flags & NGX_HTTP_V2_END_STREAM_FLAG) {
-                u->length = 0;
-
-                if (ctx->in == NULL
-                    && ctx->out == NULL
-                    && ctx->output_closed
-                    && b->last == b->pos)
-                {
-                    u->keepalive = 1;
-                }
-
-                break;
+                ctx->done = 1;
             }
 
             continue;
@@ -2097,17 +2146,8 @@ ngx_http_grpc_filter(void *data, ssize_t
                                    "grpc trailer done");
 
                     if (ctx->end_stream) {
-                        u->length = 0;
-
-                        if (ctx->in == NULL
-                            && ctx->out == NULL
-                            && ctx->output_closed
-                            && b->last == b->pos)
-                        {
-                            u->keepalive = 1;
-                        }
-
-                        return NGX_OK;
+                        ctx->done = 1;
+                        break;
                     }
 
                     ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
@@ -2124,6 +2164,10 @@ ngx_http_grpc_filter(void *data, ssize_t
                 return NGX_ERROR;
             }
 
+            if (rc == NGX_HTTP_PARSE_HEADER_DONE) {
+                continue;
+            }
+
             /* rc == NGX_AGAIN */
 
             if (ctx->rest == 0) {
@@ -2240,17 +2284,7 @@ ngx_http_grpc_filter(void *data, ssize_t
         ctx->state = ngx_http_grpc_st_start;
 
         if (ctx->flags & NGX_HTTP_V2_END_STREAM_FLAG) {
-            u->length = 0;
-
-            if (ctx->in == NULL
-                && ctx->out == NULL
-                && ctx->output_closed
-                && b->last == b->pos)
-            {
-                u->keepalive = 1;
-            }
-
-            break;
+            ctx->done = 1;
         }
     }
 

From mdounin at mdounin.ru  Tue Dec  4 13:37:02 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:37:02 +0000
Subject: [nginx] gRPC: disabled keepalive when sending control frames was
 blocked.
Message-ID: <hg.dc69f7aa6ca6.1543930622.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/dc69f7aa6ca6
branches:  stable-1.14
changeset: 7417:dc69f7aa6ca6
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Sep 03 19:34:02 2018 +0300
description:
gRPC: disabled keepalive when sending control frames was blocked.

If sending request body was not completed (u->request_body_sent is not set),
the upstream keepalive module won't save such a connection.  However, it
is theoretically possible (though highly unlikely) that sending of some
control frames can be blocked after the request body was sent.  The
ctx->output_blocked flag introduced to disable keepalive in such cases.

diffstat:

 src/http/modules/ngx_http_grpc_module.c |  12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

diffs (57 lines):

diff --git a/src/http/modules/ngx_http_grpc_module.c b/src/http/modules/ngx_http_grpc_module.c
--- a/src/http/modules/ngx_http_grpc_module.c
+++ b/src/http/modules/ngx_http_grpc_module.c
@@ -112,6 +112,7 @@ typedef struct {
 
     unsigned                   header_sent:1;
     unsigned                   output_closed:1;
+    unsigned                   output_blocked:1;
     unsigned                   parsing_headers:1;
     unsigned                   end_stream:1;
     unsigned                   done:1;
@@ -1076,6 +1077,7 @@ ngx_http_grpc_reinit_request(ngx_http_re
     ctx->state = 0;
     ctx->header_sent = 0;
     ctx->output_closed = 0;
+    ctx->output_blocked = 0;
     ctx->parsing_headers = 0;
     ctx->end_stream = 0;
     ctx->done = 0;
@@ -1413,6 +1415,13 @@ ngx_http_grpc_body_output_filter(void *d
         rc = NGX_AGAIN;
     }
 
+    if (rc == NGX_AGAIN) {
+        ctx->output_blocked = 1;
+
+    } else {
+        ctx->output_blocked = 0;
+    }
+
     if (ctx->done) {
 
         /*
@@ -1427,6 +1436,7 @@ ngx_http_grpc_body_output_filter(void *d
         if (ctx->in == NULL
             && ctx->out == NULL
             && ctx->output_closed
+            && !ctx->output_blocked
             && ctx->state == ngx_http_grpc_st_start)
         {
             u->keepalive = 1;
@@ -1777,6 +1787,7 @@ ngx_http_grpc_process_header(ngx_http_re
                     if (ctx->in == NULL
                         && ctx->out == NULL
                         && ctx->output_closed
+                        && !ctx->output_blocked
                         && b->last == b->pos)
                     {
                         u->keepalive = 1;
@@ -1879,6 +1890,7 @@ ngx_http_grpc_filter(void *data, ssize_t
 
                     if (ctx->in == NULL
                         && ctx->output_closed
+                        && !ctx->output_blocked
                         && ctx->state == ngx_http_grpc_st_start)
                     {
                         u->keepalive = 1;

From mdounin at mdounin.ru  Tue Dec  4 13:37:04 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:37:04 +0000
Subject: [nginx] SSL: logging level of "no suitable key share".
Message-ID: <hg.ce5e87e98772.1543930624.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/ce5e87e98772
branches:  stable-1.14
changeset: 7418:ce5e87e98772
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Sep 25 13:59:53 2018 +0300
description:
SSL: logging level of "no suitable key share".

The "no suitable key share" errors are reported by OpenSSL 1.1.1 when
using TLSv1.3 if there are no shared groups (that is, elliptic curves).
In particular, it is easy enough to trigger by using only a single
curve in ssl_ecdh_curve:

    ssl_ecdh_curve secp384r1;

and using a different curve in the client:

    openssl s_client -connect 127.0.0.1:443 -curves prime256v1

On the client side it is seen as "sslv3 alert handshake failure",
"SSL alert number 40":

0:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:ssl/record/rec_layer_s3.c:1528:SSL alert number 40

It can be also triggered with default ssl_ecdh_curve by using a curve
which is not in the default list (X25519, prime256v1, X448, secp521r1,
secp384r1):

    openssl s_client -connect 127.0.0.1:8443 -curves brainpoolP512r1

Given that many clients hardcode prime256v1, these errors might become
a common problem with TLSv1.3 if ssl_ecdh_curve is redefined.  Previously
this resulted in not using ECDH with such clients, but with TLSv1.3 it
is no longer possible and will result in a handshake failure.

The SSL_R_NO_SHARED_GROUP error is what BoringSSL returns in the same
situation.

Seen at:

https://serverfault.com/questions/932102/nginx-ssl-handshake-error-no-suitable-key-share

diffstat:

 src/event/ngx_event_openssl.c |  6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diffs (23 lines):

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -2059,6 +2059,9 @@ ngx_ssl_connection_error(ngx_connection_
 
             /* handshake failures */
         if (n == SSL_R_BAD_CHANGE_CIPHER_SPEC                        /*  103 */
+#ifdef SSL_R_NO_SUITABLE_KEY_SHARE
+            || n == SSL_R_NO_SUITABLE_KEY_SHARE                      /*  101 */
+#endif
             || n == SSL_R_BLOCK_CIPHER_PAD_IS_WRONG                  /*  129 */
             || n == SSL_R_DIGEST_CHECK_FAILED                        /*  149 */
             || n == SSL_R_ERROR_IN_RECEIVED_CIPHER_LIST              /*  151 */
@@ -2081,6 +2084,9 @@ ngx_ssl_connection_error(ngx_connection_
             || n == SSL_R_UNKNOWN_ALERT_TYPE                         /*  246 */
             || n == SSL_R_UNKNOWN_PROTOCOL                           /*  252 */
             || n == SSL_R_UNSUPPORTED_PROTOCOL                       /*  258 */
+#ifdef SSL_R_NO_SHARED_GROUP
+            || n == SSL_R_NO_SHARED_GROUP                            /*  266 */
+#endif
             || n == SSL_R_WRONG_VERSION_NUMBER                       /*  267 */
             || n == SSL_R_DECRYPTION_FAILED_OR_BAD_RECORD_MAC        /*  281 */
 #ifdef SSL_R_RENEGOTIATE_EXT_TOO_LONG

From mdounin at mdounin.ru  Tue Dec  4 13:37:05 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:37:05 +0000
Subject: [nginx] SSL: logging level of "no suitable signature algorithm".
Message-ID: <hg.c5d7a72abadc.1543930625.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/c5d7a72abadc
branches:  stable-1.14
changeset: 7419:c5d7a72abadc
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Sep 25 14:00:04 2018 +0300
description:
SSL: logging level of "no suitable signature algorithm".

The "no suitable signature algorithm" errors are reported by OpenSSL 1.1.1
when using TLSv1.3 if there are no shared signature algorithms.  In
particular, this can happen if the client limits available signature
algorithms to something we don't have a certificate for, or to an empty
list.  For example, the following command:

    openssl s_client -connect 127.0.0.1:8443 -sigalgs rsa_pkcs1_sha1

will always result in the "no suitable signature algorithm" error
as the "rsa_pkcs1_sha1" algorithm refers solely to signatures which
appear in certificates and not defined for use in TLS 1.3 handshake
messages.

The SSL_R_NO_COMMON_SIGNATURE_ALGORITHMS error is what BoringSSL returns
in the same situation.

diffstat:

 src/event/ngx_event_openssl.c |  6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diffs (23 lines):

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -2062,6 +2062,9 @@ ngx_ssl_connection_error(ngx_connection_
 #ifdef SSL_R_NO_SUITABLE_KEY_SHARE
             || n == SSL_R_NO_SUITABLE_KEY_SHARE                      /*  101 */
 #endif
+#ifdef SSL_R_NO_SUITABLE_SIGNATURE_ALGORITHM
+            || n == SSL_R_NO_SUITABLE_SIGNATURE_ALGORITHM            /*  118 */
+#endif
             || n == SSL_R_BLOCK_CIPHER_PAD_IS_WRONG                  /*  129 */
             || n == SSL_R_DIGEST_CHECK_FAILED                        /*  149 */
             || n == SSL_R_ERROR_IN_RECEIVED_CIPHER_LIST              /*  151 */
@@ -2083,6 +2086,9 @@ ngx_ssl_connection_error(ngx_connection_
             || n == SSL_R_UNEXPECTED_RECORD                          /*  245 */
             || n == SSL_R_UNKNOWN_ALERT_TYPE                         /*  246 */
             || n == SSL_R_UNKNOWN_PROTOCOL                           /*  252 */
+#ifdef SSL_R_NO_COMMON_SIGNATURE_ALGORITHMS
+            || n == SSL_R_NO_COMMON_SIGNATURE_ALGORITHMS             /*  253 */
+#endif
             || n == SSL_R_UNSUPPORTED_PROTOCOL                       /*  258 */
 #ifdef SSL_R_NO_SHARED_GROUP
             || n == SSL_R_NO_SHARED_GROUP                            /*  266 */

From mdounin at mdounin.ru  Tue Dec  4 13:37:07 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:37:07 +0000
Subject: [nginx] SSL: enabled TLSv1.3 with BoringSSL.
Message-ID: <hg.b3a4f6d23e82.1543930627.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/b3a4f6d23e82
branches:  stable-1.14
changeset: 7420:b3a4f6d23e82
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Aug 07 02:15:28 2018 +0300
description:
SSL: enabled TLSv1.3 with BoringSSL.

BoringSSL currently requires SSL_CTX_set_max_proto_version(TLS1_3_VERSION)
to be able to enable TLS 1.3.  This is because by default max protocol
version is set to TLS 1.2, and the SSL_OP_NO_* options are merely used
as a blacklist within the version range specified using the
SSL_CTX_set_min_proto_version() and SSL_CTX_set_max_proto_version()
functions.

With this change, we now call SSL_CTX_set_max_proto_version() with an
explicit maximum version set.  This enables TLS 1.3 with BoringSSL.
As a side effect, this change also limits maximum protocol version to
the newest protocol we know about, TLS 1.3.  This seems to be a good
change, as enabling unknown protocols might have unexpected results.

Additionally, we now explicitly call SSL_CTX_set_min_proto_version()
with 0.  This is expected to help with Debian system-wide default
of MinProtocol set to TLSv1.2, see
http://mailman.nginx.org/pipermail/nginx-ru/2017-October/060411.html.

Note that there is no SSL_CTX_set_min_proto_version macro in BoringSSL,
so we call SSL_CTX_set_min_proto_version() and SSL_CTX_set_max_proto_version()
as long as the TLS1_3_VERSION macro is defined.

diffstat:

 src/event/ngx_event_openssl.c |  5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diffs (15 lines):

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -330,6 +330,11 @@ ngx_ssl_create(ngx_ssl_t *ssl, ngx_uint_
     }
 #endif
 
+#ifdef TLS1_3_VERSION
+    SSL_CTX_set_min_proto_version(ssl->ctx, 0);
+    SSL_CTX_set_max_proto_version(ssl->ctx, TLS1_3_VERSION);
+#endif
+
 #ifdef SSL_OP_NO_COMPRESSION
     SSL_CTX_set_options(ssl->ctx, SSL_OP_NO_COMPRESSION);
 #endif

From mdounin at mdounin.ru  Tue Dec  4 13:37:10 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:37:10 +0000
Subject: [nginx] Mp4: fixed possible pointer overflow on 32-bit platforms.
Message-ID: <hg.2dad54c2b8ed.1543930630.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/2dad54c2b8ed
branches:  stable-1.14
changeset: 7422:2dad54c2b8ed
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Wed Nov 21 20:23:16 2018 +0300
description:
Mp4: fixed possible pointer overflow on 32-bit platforms.

On 32-bit platforms mp4->buffer_pos might overflow when a large
enough (close to 4 gigabytes) atom is being skipped, resulting in
incorrect memory addesses being read further in the code.  In most
cases this results in harmless errors being logged, though may also
result in a segmentation fault if hitting unmapped pages.

To address this, ngx_mp4_atom_next() now only increments mp4->buffer_pos
up to mp4->buffer_end.  This ensures that overflow cannot happen.

diffstat:

 src/http/modules/ngx_http_mp4_module.c |  9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diffs (19 lines):

diff --git a/src/http/modules/ngx_http_mp4_module.c b/src/http/modules/ngx_http_mp4_module.c
--- a/src/http/modules/ngx_http_mp4_module.c
+++ b/src/http/modules/ngx_http_mp4_module.c
@@ -169,7 +169,14 @@ typedef struct {
 
 
 #define ngx_mp4_atom_next(mp4, n)                                             \
-    mp4->buffer_pos += (size_t) n;                                            \
+                                                                              \
+    if (n > (size_t) (mp4->buffer_end - mp4->buffer_pos)) {                   \
+        mp4->buffer_pos = mp4->buffer_end;                                    \
+                                                                              \
+    } else {                                                                  \
+        mp4->buffer_pos += (size_t) n;                                        \
+    }                                                                         \
+                                                                              \
     mp4->offset += n
 
 

From mdounin at mdounin.ru  Tue Dec  4 13:37:09 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:37:09 +0000
Subject: [nginx] SSL: explicitly set maximum version (ticket #1654).
Message-ID: <hg.11be3c0723bd.1543930629.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/11be3c0723bd
branches:  stable-1.14
changeset: 7421:11be3c0723bd
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Oct 23 22:11:48 2018 +0300
description:
SSL: explicitly set maximum version (ticket #1654).

With maximum version explicitly set, TLSv1.3 will not be unexpectedly
enabled if nginx compiled with OpenSSL 1.1.0 (without TLSv1.3 support)
will be run with OpenSSL 1.1.1 (with TLSv1.3 support).

diffstat:

 src/event/ngx_event_openssl.c |  5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diffs (15 lines):

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -330,6 +330,11 @@ ngx_ssl_create(ngx_ssl_t *ssl, ngx_uint_
     }
 #endif
 
+#ifdef SSL_CTX_set_min_proto_version
+    SSL_CTX_set_min_proto_version(ssl->ctx, 0);
+    SSL_CTX_set_max_proto_version(ssl->ctx, TLS1_2_VERSION);
+#endif
+
 #ifdef TLS1_3_VERSION
     SSL_CTX_set_min_proto_version(ssl->ctx, 0);
     SSL_CTX_set_max_proto_version(ssl->ctx, TLS1_3_VERSION);

From mdounin at mdounin.ru  Tue Dec  4 13:37:12 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 13:37:12 +0000
Subject: [nginx] Updated OpenSSL used for win32 builds.
Message-ID: <hg.931f9d2abf28.1543930632.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/931f9d2abf28
branches:  stable-1.14
changeset: 7423:931f9d2abf28
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Nov 27 17:02:56 2018 +0300
description:
Updated OpenSSL used for win32 builds.

diffstat:

 misc/GNUmakefile |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff --git a/misc/GNUmakefile b/misc/GNUmakefile
--- a/misc/GNUmakefile
+++ b/misc/GNUmakefile
@@ -6,7 +6,7 @@ TEMP =		tmp
 
 CC =		cl
 OBJS =		objs.msvc8
-OPENSSL =	openssl-1.0.2p
+OPENSSL =	openssl-1.0.2q
 ZLIB =		zlib-1.2.11
 PCRE =		pcre-8.42
 

From mdounin at mdounin.ru  Tue Dec  4 15:00:51 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 15:00:51 +0000
Subject: [nginx] nginx-1.14.2-RELEASE
Message-ID: <hg.72e39ad2427d.1543935651.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/72e39ad2427d
branches:  stable-1.14
changeset: 7424:72e39ad2427d
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Dec 04 17:52:24 2018 +0300
description:
nginx-1.14.2-RELEASE

diffstat:

 docs/xml/nginx/changes.xml |  115 +++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 115 insertions(+), 0 deletions(-)

diffs (125 lines):

diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml
--- a/docs/xml/nginx/changes.xml
+++ b/docs/xml/nginx/changes.xml
@@ -5,6 +5,121 @@
 <change_log title="nginx">
 
 
+<changes ver="1.14.2" date="2018-12-04">
+
+<change type="bugfix">
+<para lang="ru">
+nginx ?? ????????? gcc 8.1.
+</para>
+<para lang="en">
+nginx could not be built by gcc 8.1.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+nginx ?? ????????? ?? Fedora 28 Linux.
+</para>
+<para lang="en">
+nginx could not be built on Fedora 28 Linux.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ????????? ??????? ???????? ??? ????????????? unix domain listen-???????
+??? ?????? ? ???????????? ?? Linux.
+</para>
+<para lang="en">
+in handling of client addresses when using unix domain listen sockets
+to work with datagrams on Linux.
+</para>
+</change>
+
+<change type="change">
+<para lang="ru">
+??????? ???????????? ?????? SSL "http request", "https proxy request",
+"unsupported protocol", "version too low", "no suitable key share" ?
+"no suitable signature algorithm"
+??????? ? ?????? crit ?? info.
+</para>
+<para lang="en">
+the logging level of the "http request", "https proxy request",
+"unsupported protocol", "version too low", "no suitable key share", and
+"no suitable signature algorithm" SSL errors
+has been lowered from "crit" to "info".
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ????????????? OpenSSL 1.1.0 ? ?????
+????????? ssl_prefer_server_ciphers ?????? ???? ?????????
+? ??????????? ???????, ???? ??? ???? ???????? ? ??????? ?? ?????????.
+</para>
+<para lang="en">
+when using OpenSSL 1.1.0 or newer
+it was not possible to switch off "ssl_prefer_server_ciphers" in
+a virtual server if it was switched on in the default server.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+nginx ?? ????????? ? LibreSSL 2.8.0.
+</para>
+<para lang="en">
+nginx could not be built with LibreSSL 2.8.0.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+???? nginx ??? ?????? ? OpenSSL 1.1.0, ? ????????????? ? OpenSSL 1.1.1,
+???????? TLS 1.3 ?????? ??? ????????.
+</para>
+<para lang="en">
+if nginx was built with OpenSSL 1.1.0 and used with OpenSSL 1.1.1,
+the TLS 1.3 protocol was always enabled.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ???????? ???????????? ?? ???? ???? ??????? ?? gRPC-??????
+????? ????????? ??????.
+</para>
+<para lang="en">
+sending a disk-buffered request body to a gRPC backend
+might fail.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+?????????? ? ????????? gRPC-???????? ????? ?? ????????????
+??? ????????????? ????????? keepalive.
+</para>
+<para lang="en">
+connections with some gRPC backends might not be cached
+when using the "keepalive" directive.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ??????? ???????? ??? ????????? segmentation fault,
+???? ????????????? ?????? ngx_http_mp4_module ?? 32-?????? ??????????.
+</para>
+<para lang="en">
+a segmentation fault might occur in a worker process
+if the ngx_http_mp4_module was used on 32-bit platforms.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.14.1" date="2018-11-06">
 
 <change type="security">

From mdounin at mdounin.ru  Tue Dec  4 15:00:53 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 04 Dec 2018 15:00:53 +0000
Subject: [nginx] release-1.14.2 tag
Message-ID: <hg.ebf8c9686b8c.1543935653.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/ebf8c9686b8c
branches:  stable-1.14
changeset: 7425:ebf8c9686b8c
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Dec 04 17:52:24 2018 +0300
description:
release-1.14.2 tag

diffstat:

 .hgtags |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (8 lines):

diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -427,3 +427,4 @@ 64179f242cb55fc206bca59de9bfdc4cf5ebcec7
 051e5fa03b92b8a564f6b12debd483d267391e82 release-1.13.12
 588054867fef65d3dc38df1ffefcade02a88f140 release-1.14.0
 fe8e8322d9c13597f06b80f2015124446f0a900c release-1.14.1
+72e39ad2427d02fa29e892715298013f043f2b80 release-1.14.2

From jackdev at mailbox.org  Tue Dec  4 16:00:10 2018
From: jackdev at mailbox.org (Jack Henschel)
Date: Tue, 4 Dec 2018 17:00:10 +0100
Subject: [PATCH] Clarify proxy modules $proxy_host variable content
Message-ID: <2f1f0736-df17-172f-a215-fffa527734ff@mailbox.org>

Hello,

>From the current documentation it was not immediately obvious (at least
to me and my colleague) that nginx does not set the $proxy_host variable
to the hosts specified in an upstream block, but literally uses the name
used in the proxy_pass directive.

Maxim Dounin has explained to me the reason for this [1], nevertheless I
think it is worthwhile explicitly adding this behavior to the documentation.

Regards
Jack

[1] http://mailman.nginx.org/pipermail/nginx/2018-November/057184.html

-------------- next part --------------
A non-text attachment was scrubbed...
Name: docs-clarify-proxy-host-variable-content.patch
Type: text/x-patch
Size: 1970 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20181204/84d291d7/attachment-0001.bin>

From xeioex at nginx.com  Tue Dec  4 18:13:38 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 04 Dec 2018 18:13:38 +0000
Subject: [njs] Fixed type of iteration variable in for-in with array values.
Message-ID: <hg.1dbef01a6d6c.1543947218.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/1dbef01a6d6c
branches:  
changeset: 684:1dbef01a6d6c
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Dec 04 21:13:13 2018 +0300
description:
Fixed type of iteration variable in for-in with array values.

diffstat:

 njs/njs_vm.c             |  2 +-
 njs/test/njs_unit_test.c |  3 +++
 2 files changed, 4 insertions(+), 1 deletions(-)

diffs (25 lines):

diff -r ce3a943df9c4 -r 1dbef01a6d6c njs/njs_vm.c
--- a/njs/njs_vm.c	Mon Dec 03 19:23:27 2018 +0300
+++ b/njs/njs_vm.c	Tue Dec 04 21:13:13 2018 +0300
@@ -829,7 +829,7 @@ njs_vmcode_property_next(njs_vm_t *vm, n
                 n = next->index++;
 
                 if (njs_is_valid(&array->start[n])) {
-                    njs_value_number_set(retval, n);
+                    njs_uint32_to_string(retval, n);
 
                     return code->offset;
                 }
diff -r ce3a943df9c4 -r 1dbef01a6d6c njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Mon Dec 03 19:23:27 2018 +0300
+++ b/njs/test/njs_unit_test.c	Tue Dec 04 21:13:13 2018 +0300
@@ -2244,6 +2244,9 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("var s = ''; for (var p in [1,2]) {s += p}; s"),
       nxt_string("01") },
 
+    { nxt_string("var s; for (var p in [1]) {s = typeof(p)}; s"),
+      nxt_string("string") },
+
     { nxt_string("var s = ''; for (var p in {a:1, b:2}) {s += p}; s"),
       nxt_string("ab") },
 

From maxim at nginx.com  Wed Dec  5 09:43:42 2018
From: maxim at nginx.com (Maxim Konovalov)
Date: Wed, 5 Dec 2018 12:43:42 +0300
Subject: cache: move open to thread pool
In-Reply-To: <4ba7414a-0eba-6bd3-5a50-2eca591ac25e@nginx.com>
References: <CAEZnf2HjMqXPDbzYfHXgKvUvdA9b7fqZ71sPo0LxFbG38fdfJw@mail.gmail.com>
 <20180808181653.GX56558@mdounin.ru>
 <CAEZnf2FmrA0Fg6F5qNE3FhOcu4kJZErkHYr6J6qG1a9XMCgf+A@mail.gmail.com>
 <20180810113946.GG56558@mdounin.ru>
 <CAEZnf2E-e4m1kUbnOoVNG4boxPJ+_o7_Sx7x8bfU6gVM_moYWw@mail.gmail.com>
 <20180903160903.GI56558@mdounin.ru>
 <CAEZnf2EOEecQYBbq3mcaxtyWJVsS079_nPE=r-WhCyji-=kpbA@mail.gmail.com>
 <b3b1e80c-1c47-5f9a-f5a2-e28e48bd9fd0@nginx.com>
 <CAEZnf2HcYQxC1RXOWvW4NYkbUDw4oxK4HFXsExbch_ZfYVNALw@mail.gmail.com>
 <4ba7414a-0eba-6bd3-5a50-2eca591ac25e@nginx.com>
Message-ID: <b518712c-0dc6-0b47-2b2b-61fe55882efa@nginx.com>

Hello,

just a reminder that we are looking for a tester for these patches.

Thanks,

Maxim

On 16/11/2018 12:10, Maxim Konovalov wrote:
> Thanks, Ka-Hing, we'll wait for your feedback.
> 
> On 16/11/2018 01:53, Ka-Hing Cheung wrote:
>> Hi,
>>
>> I didn't forget about this. I am pretty swamped at the moment and
>> there's a holiday freeze coming up. Will get to his in December.
>>
>> - Ka-Hing
>>
>> On Thu, Nov 8, 2018 at 6:19 AM Maxim Konovalov <maxim at nginx.com> wrote:
>>>
>>> Hi Ka-Hing,
>>>
>>> would you mind to test Roman's most recent patches that add
>>> "aio_open" directive?
>>>
>>> http://mailman.nginx.org/pipermail/nginx-devel/2018-November/011538.html
>>>
>>> We are looking for overall performance and stability metrics and
>>> feedback.
>>>
>>> Much appreciated,
>>>
>>> Maxim
>>>
>>> --
>>> Maxim Konovalov
> 
> 


-- 
Maxim Konovalov

From igor at sysoev.ru  Wed Dec  5 14:03:08 2018
From: igor at sysoev.ru (Igor Sysoev)
Date: Wed, 05 Dec 2018 14:03:08 +0000
Subject: [njs] Fixed building on paltforms without librt.
Message-ID: <hg.23fecf3fcc60.1544018588.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/23fecf3fcc60
branches:  
changeset: 685:23fecf3fcc60
user:      Igor Sysoev <igor at sysoev.ru>
date:      Wed Dec 05 15:38:33 2018 +0300
description:
Fixed building on paltforms without librt.

diffstat:

 nxt/auto/time |  14 ++++----------
 1 files changed, 4 insertions(+), 10 deletions(-)

diffs (28 lines):

diff -r 1dbef01a6d6c -r 23fecf3fcc60 nxt/auto/time
--- a/nxt/auto/time	Tue Dec 04 21:13:13 2018 +0300
+++ b/nxt/auto/time	Wed Dec 05 15:38:33 2018 +0300
@@ -27,20 +27,14 @@ if [ $nxt_found = no ]; then
     nxt_feature="clock_gettime(CLOCK_MONOTONIC) in librt"
     nxt_feature_libs="-lrt"
     . ${NXT_AUTO}feature
-fi
 
-if [ $nxt_found = yes ]; then
-    cat << END >> $NXT_MAKEFILE_CONF
+    if [ $nxt_found = yes ]; then
+        cat << END >> $NXT_MAKEFILE_CONF
 
-NXT_LIBRT = -lrt
+NXT_LIBRT =	-lrt
 END
 
-else
-    cat << END >> $NXT_MAKEFILE_CONF
-
-NXT_LIBRT =
-END
-
+    fi
 fi
 
 # Linux, FreeBSD, MacOSX.

From maxim at nginx.com  Wed Dec  5 16:15:45 2018
From: maxim at nginx.com (Maxim Konovalov)
Date: Wed, 5 Dec 2018 19:15:45 +0300
Subject: [PATCH] Upstream: added $upstream_bytes_sent variable
In-Reply-To: <20181130153932.GB50234@lo0.su>
References: <fafbb3ee41e5bb03bcfb.1543314850@piotrsikora.c.googlers.com>
 <20181129150051.GK99070@mdounin.ru> <20181130153932.GB50234@lo0.su>
Message-ID: <70641dfb-0e70-84ef-3f89-694c0ac8d58b@nginx.com>

Hi Piotr.

On 30/11/2018 18:39, Ruslan Ermilov wrote:
> On Thu, Nov 29, 2018 at 06:00:51PM +0300, Maxim Dounin wrote:
>> Hello!
>>
>> On Tue, Nov 27, 2018 at 02:34:10AM -0800, Piotr Sikora via nginx-devel wrote:
>>
>>> # HG changeset patch
>>> # User Piotr Sikora <piotrsikora at google.com>
>>> # Date 1494129075 25200
>>> #      Sat May 06 20:51:15 2017 -0700
>>> # Node ID fafbb3ee41e5bb03bcfba73f7d4367b8ab7d36cc
>>> # Parent  be5cb9c67c05ccaf22dab7abba78aa4c1545a8ee
>>> Upstream: added $upstream_bytes_sent variable.
>>
>> [...]
>>
>> Ruslan made a similar patch a while ago.  It wasn't committed 
>> since there were questions if such a variable is actually needed - 
>> I think we are aware of at most one feature request for this:
>>
>> http://mailman.nginx.org/pipermail/nginx/2018-March/055940.html
>>
>> I've asked Ruslan to post his version of the patch (or, rather, a 
>> patch series), please review. 
> 
> # HG changeset patch
> # User Ruslan Ermilov <ru at nginx.com>
> # Date 1543592116 -10800
> #      Fri Nov 30 18:35:16 2018 +0300
> # Node ID 79c7b169816cdc63044838b03084c631c0d2f0a3
> # Parent  5cff15dd07cd298e4eff44c04c2833066c217318
> Upstream: style.
> 
> Introduced local variable "c" in ngx_http_upstream_next() and
> ngx_http_upstream_finalize_request().
> 
> No functional changes.
> 
[...]
> # HG changeset patch
> # User Ruslan Ermilov <ru at nginx.com>
> # Date 1543592133 -10800
> #      Fri Nov 30 18:35:33 2018 +0300
> # Node ID 95b0ee9297fc3b8782ee1a383e3221b935639cc3
> # Parent  79c7b169816cdc63044838b03084c631c0d2f0a3
> Upstream: implemented $upstream_bytes_sent.
> 
[...]

To make the intention explicit: it would be nice if you do a formal
review of these patches.

Thanks,

Maxim

-- 
Maxim Konovalov

From jan.prachar at gmail.com  Wed Dec  5 23:39:15 2018
From: jan.prachar at gmail.com (=?UTF-8?Q?Honza_Pracha=C5=99?=)
Date: Thu, 6 Dec 2018 00:39:15 +0100
Subject: OpenSSL and Early data
Message-ID: <CACqAjk3tSwjBpS3bQn2P6NhPE7E8jAVf=NeFYX9VW6-P4VLQ+Q@mail.gmail.com>

Hello! FYI there is an issue with TLS 1.3 Early data in OpenSSL ?
https://github.com/openssl/openssl/issues/7757

So maybe you would want to consider ignoring Early data with HTTP/2 and
OpenSSL. Or try to fix the problem on the nginx side, i.e. do not call
SSL_read_early_data() until all pending data is written with
SSL_write_early_data().

Best regards
Jan Pracha?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20181206/b649b3eb/attachment.html>

From deepbluemistake at gmail.com  Thu Dec  6 11:42:23 2018
From: deepbluemistake at gmail.com (Andras Farkas)
Date: Thu, 6 Dec 2018 06:42:23 -0500
Subject: Patch to make 404 pages etc and autoindex pages use valid HTML
In-Reply-To: <20181128163108.GB99070@mdounin.ru>
References: <CAA0nTRv3wsv8oRFNgeWMyhxwqkgHSEci9cB8fefhjGe-qm3--Q@mail.gmail.com>
 <20181128163108.GB99070@mdounin.ru>
Message-ID: <CAA0nTRvFYOf89_PPdP7SxzGGo2XhBCtf91vQtMq8PgjSHUVrJQ@mail.gmail.com>

On Wed, Nov 28, 2018 at 11:31 AM Maxim Dounin <mdounin at mdounin.ru> wrote:
> Such a change will significantly modify typical rendering of error
> pages, so I would rather refrain from it unless there are more
> practical reasons than triggering standards-complaint mode instead
> of quirks mode - which does no seem to make any difference for
> pages nginx generates.
Is having it render differently a bad thing?  The meaning of any page
changed still remains clear and easy to read.  Invalid HTML (like
<center>) is so easy to remove and to turn into valid HTML, I don't
see why not to.
But, thank you for your reply, and for reading the patches!
:D

From ru at nginx.com  Thu Dec  6 14:45:22 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 6 Dec 2018 17:45:22 +0300
Subject: limit_rate_after support variables
In-Reply-To: <CAEx3=QFVHn2-h=cj=rkVrgDNZsEGxzxsKa2j_smbW6gg-AbsLQ@mail.gmail.com>
References: <CAEx3=QEMKkoiAve-=fn86tS-3vcTNkS=Ukju7yf1zzWHh0jHJA@mail.gmail.com>
 <20180827112847.GE71000@lo0.su>
 <CAEx3=QEQGLn91h85PrQro29kcr4TQGkxmaJgQHTOWXQbTF=6tw@mail.gmail.com>
 <20180827160021.GG71000@lo0.su>
 <CAEx3=QGR8X1QJ__azWi5jYQnkM8DAd740Sn6LHNsosFjvpYtcA@mail.gmail.com>
 <20180829114230.GC43480@lo0.su>
 <CAEx3=QFxT_rngL64H+83tG3JntcVQ6Du+VyVBzVXipK5Epjj2A@mail.gmail.com>
 <CAEx3=QHDuWyVf+tw0WDNY82hbt2BPJvBsUdURe9MUoyO-C0wyw@mail.gmail.com>
 <20181120141005.GA72348@lo0.su>
 <CAEx3=QFVHn2-h=cj=rkVrgDNZsEGxzxsKa2j_smbW6gg-AbsLQ@mail.gmail.com>
Message-ID: <20181206144522.GF50234@lo0.su>

On Wed, Nov 21, 2018 at 03:50:30PM +0100, Miroslav Novy wrote:
> Hi Ruslan,
>   there is my fix to remove warning "using uninitialized variable".
> 
> I have moved the reading value of limit_rate variable from function
> ngx_http_update_location_config to function ngx_http_write_filter
> because value is initialized later.
> 
> Best regards
> Miroslav Nov?
> 
> # HG changeset patch
> # User Miroslav Nov? <miranovy at gmail.com>
> # Date 1542811768 0
> #      Wed Nov 21 14:49:28 2018 +0000
> # Node ID e2139cd62c7263dc182b11a847a6eaf2d9560a0d
> # Parent  82eed7650622fd780dcd4a86661de7b80b44199c
> Variables support in limit_rate and limit_rate_after fix
> 
> Reading value of limit_rate variable moved to function
> ngx_http_write_filter because in function
> ngx_http_update_location_config is not inicialized yet.
> 
> diff -r 82eed7650622 -r e2139cd62c72 src/http/ngx_http_core_module.c
> --- a/src/http/ngx_http_core_module.c Wed Nov 21 10:40:01 2018 +0000
> +++ b/src/http/ngx_http_core_module.c Wed Nov 21 14:49:28 2018 +0000
> @@ -1212,8 +1212,6 @@
>  void
>  ngx_http_update_location_config(ngx_http_request_t *r)
>  {
> -    size_t                     limit_rate;
> -    ngx_str_t                  val;
>      ngx_http_core_loc_conf_t  *clcf;
> 
>      clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module);
> @@ -1283,20 +1281,6 @@
>          r->connection->tcp_nopush = NGX_TCP_NOPUSH_DISABLED;
>      }
> 
> -    if (r->limit_rate == 0
> -        && clcf->limit_rate
> -        && ngx_http_complex_value_size(r, clcf->limit_rate, &val, &limit_rate)
> -           == NGX_OK)
> -    {
> -        if (limit_rate != (size_t) NGX_ERROR) {
> -            r->limit_rate = limit_rate;
> -
> -        } else if (val.len) {
> -            ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> -                          "invalid \"limit_rate\" value \"%V\"", &val);
> -        }
> -    }
> -
>      if (clcf->handler) {
>          r->content_handler = clcf->handler;
>      }
> diff -r 82eed7650622 -r e2139cd62c72 src/http/ngx_http_write_filter_module.c
> --- a/src/http/ngx_http_write_filter_module.c Wed Nov 21 10:40:01 2018 +0000
> +++ b/src/http/ngx_http_write_filter_module.c Wed Nov 21 14:49:28 2018 +0000
> @@ -48,7 +48,7 @@
>  ngx_http_write_filter(ngx_http_request_t *r, ngx_chain_t *in)
>  {
>      off_t                      size, sent, nsent, limit;
> -    size_t                     limit_rate_after;
> +    ssize_t                    limit_rate, limit_rate_after;
>      ngx_str_t                  val;
>      ngx_uint_t                 last, flush, sync;
>      ngx_msec_t                 delay;
> @@ -220,6 +220,25 @@
>          return NGX_ERROR;
>      }
> 
> +    if (r->limit_rate == 0
> +         && clcf->limit_rate
> +         && ngx_http_complex_value_size(r, clcf->limit_rate, &val,
> +                                        &limit_rate)
> +         == NGX_OK)
> +     {
> +         if (limit_rate_after != NGX_ERROR) {

You're checking the wrong variable here.

> +             r->limit_rate = limit_rate;
> +
> +         } else if (val.len) {
> +             ngx_log_error(NGX_LOG_ERR, c->log, 0,
> +                           "invalid \"limit_rate\" value \"%V\"",
> +                           &val);
> +         }
> +     }
> +
> +     ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, "http limit rate \"%z\"",
> +                           r->limit_rate);
> +

Unfortunately, moving it here from the write filter the way it's done
breaks unbuffered proxying combined with limit_rate set.

>      if (r->limit_rate) {
>          if (r->limit_rate_after == 0
>              && clcf->limit_rate_after
> @@ -227,7 +246,7 @@
>                                             &limit_rate_after)
>                 == NGX_OK)
>          {
> -            if (limit_rate_after != (size_t) NGX_ERROR) {
> +            if (limit_rate_after != NGX_ERROR) {
>                  r->limit_rate_after = limit_rate_after;
> 
>              } else if (val.len) {

Here's my take.  I didn't change size_t vs. ssize_t as I believe
it's okay the way I did it.

# HG changeset patch
# User Ruslan Ermilov <ru at nginx.com>
# Date 1544099131 -10800
#      Thu Dec 06 15:25:31 2018 +0300
# Node ID 1a8c0a959f636b5eea0a4f42711af08591c23a99
# Parent  2117637f64e981e0e14c3a4b0509252fefd8a78a
Added ngx_http_set_complex_value_size_slot().

If a complex value is expected to be size_t, and the compiled value
is constant, the constant size_t value is remembered at compile time.

The value is accessed through ngx_http_complex_value_size() which
either returns the remembered constant or evaluates the expression
and parses it as size_t.

diff --git a/src/http/ngx_http_script.c b/src/http/ngx_http_script.c
--- a/src/http/ngx_http_script.c
+++ b/src/http/ngx_http_script.c
@@ -105,6 +105,25 @@ ngx_http_complex_value(ngx_http_request_
 
 
 ngx_int_t
+ngx_http_complex_value_size(ngx_http_request_t *r,
+    ngx_http_complex_value_t *val, ngx_str_t *value, size_t *size)
+{
+    if (val->lengths == NULL) {
+        *size = val->u.size;
+        return NGX_OK;
+    }
+
+    if (ngx_http_complex_value(r, val, value) != NGX_OK) {
+        return NGX_ERROR;
+    }
+
+    *size = ngx_parse_size(value);
+
+    return NGX_OK;
+}
+
+
+ngx_int_t
 ngx_http_compile_complex_value(ngx_http_compile_complex_value_t *ccv)
 {
     ngx_str_t                  *v;
@@ -244,6 +263,36 @@ ngx_http_set_complex_value_slot(ngx_conf
 }
 
 
+char *
+ngx_http_set_complex_value_size_slot(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf)
+{
+    char  *p = conf;
+
+    char                      *rv;
+    ngx_http_complex_value_t  *cv;
+
+    rv = ngx_http_set_complex_value_slot(cf, cmd, conf);
+
+    if (rv != NGX_CONF_OK) {
+        return rv;
+    }
+
+    cv = *(ngx_http_complex_value_t **) (p + cmd->offset);
+
+    if (cv->lengths) {
+        return NGX_CONF_OK;
+    }
+
+    cv->u.size = ngx_parse_size(&cv->value);
+    if (cv->u.size == (size_t) NGX_ERROR) {
+        return "invalid value";
+    }
+
+    return NGX_CONF_OK;
+}
+
+
 ngx_int_t
 ngx_http_test_predicates(ngx_http_request_t *r, ngx_array_t *predicates)
 {
diff --git a/src/http/ngx_http_script.h b/src/http/ngx_http_script.h
--- a/src/http/ngx_http_script.h
+++ b/src/http/ngx_http_script.h
@@ -68,6 +68,10 @@ typedef struct {
     ngx_uint_t                 *flushes;
     void                       *lengths;
     void                       *values;
+
+    union {
+        size_t                  size;
+    } u;
 } ngx_http_complex_value_t;
 
 
@@ -207,9 +211,13 @@ void ngx_http_script_flush_complex_value
     ngx_http_complex_value_t *val);
 ngx_int_t ngx_http_complex_value(ngx_http_request_t *r,
     ngx_http_complex_value_t *val, ngx_str_t *value);
+ngx_int_t ngx_http_complex_value_size(ngx_http_request_t *r,
+    ngx_http_complex_value_t *val, ngx_str_t *value, size_t *size);
 ngx_int_t ngx_http_compile_complex_value(ngx_http_compile_complex_value_t *ccv);
 char *ngx_http_set_complex_value_slot(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf);
+char *ngx_http_set_complex_value_size_slot(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf);
 
 
 ngx_int_t ngx_http_test_predicates(ngx_http_request_t *r,
# HG changeset patch
# User Ruslan Ermilov <ru at nginx.com>
# Date 1544106977 -10800
#      Thu Dec 06 17:36:17 2018 +0300
# Node ID 586277aaab1be28cf53d2caa553a2bef11be2f34
# Parent  1a8c0a959f636b5eea0a4f42711af08591c23a99
Variables support in limit_rate and limit_rate_after (ticket #293).

diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c
+++ b/src/http/ngx_http_core_module.c
@@ -479,7 +479,7 @@ static ngx_command_t  ngx_http_core_comm
     { ngx_string("limit_rate"),
       NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF
                         |NGX_CONF_TAKE1,
-      ngx_conf_set_size_slot,
+      ngx_http_set_complex_value_size_slot,
       NGX_HTTP_LOC_CONF_OFFSET,
       offsetof(ngx_http_core_loc_conf_t, limit_rate),
       NULL },
@@ -487,7 +487,7 @@ static ngx_command_t  ngx_http_core_comm
     { ngx_string("limit_rate_after"),
       NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF
                         |NGX_CONF_TAKE1,
-      ngx_conf_set_size_slot,
+      ngx_http_set_complex_value_size_slot,
       NGX_HTTP_LOC_CONF_OFFSET,
       offsetof(ngx_http_core_loc_conf_t, limit_rate_after),
       NULL },
@@ -1281,10 +1281,6 @@ ngx_http_update_location_config(ngx_http
         r->connection->tcp_nopush = NGX_TCP_NOPUSH_DISABLED;
     }
 
-    if (r->limit_rate == 0) {
-        r->limit_rate = clcf->limit_rate;
-    }
-
     if (clcf->handler) {
         r->content_handler = clcf->handler;
     }
@@ -3362,6 +3358,8 @@ ngx_http_core_create_loc_conf(ngx_conf_t
      *     clcf->exact_match = 0;
      *     clcf->auto_redirect = 0;
      *     clcf->alias = 0;
+     *     clcf->limit_rate = NULL;
+     *     clcf->limit_rate_after = NULL;
      *     clcf->gzip_proxied = 0;
      *     clcf->keepalive_disable = 0;
      */
@@ -3392,8 +3390,6 @@ ngx_http_core_create_loc_conf(ngx_conf_t
     clcf->send_timeout = NGX_CONF_UNSET_MSEC;
     clcf->send_lowat = NGX_CONF_UNSET_SIZE;
     clcf->postpone_output = NGX_CONF_UNSET_SIZE;
-    clcf->limit_rate = NGX_CONF_UNSET_SIZE;
-    clcf->limit_rate_after = NGX_CONF_UNSET_SIZE;
     clcf->keepalive_timeout = NGX_CONF_UNSET_MSEC;
     clcf->keepalive_header = NGX_CONF_UNSET;
     clcf->keepalive_requests = NGX_CONF_UNSET_UINT;
@@ -3581,6 +3577,14 @@ ngx_http_core_merge_loc_conf(ngx_conf_t 
     ngx_conf_merge_msec_value(conf->client_body_timeout,
                               prev->client_body_timeout, 60000);
 
+    if (conf->limit_rate == NULL) {
+        conf->limit_rate = prev->limit_rate;
+    }
+
+    if (conf->limit_rate_after == NULL) {
+        conf->limit_rate_after = prev->limit_rate_after;
+    }
+
     ngx_conf_merge_bitmask_value(conf->keepalive_disable,
                               prev->keepalive_disable,
                               (NGX_CONF_BITMASK_SET
@@ -3622,9 +3626,6 @@ ngx_http_core_merge_loc_conf(ngx_conf_t 
     ngx_conf_merge_size_value(conf->send_lowat, prev->send_lowat, 0);
     ngx_conf_merge_size_value(conf->postpone_output, prev->postpone_output,
                               1460);
-    ngx_conf_merge_size_value(conf->limit_rate, prev->limit_rate, 0);
-    ngx_conf_merge_size_value(conf->limit_rate_after, prev->limit_rate_after,
-                              0);
     ngx_conf_merge_msec_value(conf->keepalive_timeout,
                               prev->keepalive_timeout, 75000);
     ngx_conf_merge_sec_value(conf->keepalive_header,
diff --git a/src/http/ngx_http_core_module.h b/src/http/ngx_http_core_module.h
--- a/src/http/ngx_http_core_module.h
+++ b/src/http/ngx_http_core_module.h
@@ -350,13 +350,14 @@ struct ngx_http_core_loc_conf_s {
     size_t        client_body_buffer_size; /* client_body_buffer_size */
     size_t        send_lowat;              /* send_lowat */
     size_t        postpone_output;         /* postpone_output */
-    size_t        limit_rate;              /* limit_rate */
-    size_t        limit_rate_after;        /* limit_rate_after */
     size_t        sendfile_max_chunk;      /* sendfile_max_chunk */
     size_t        read_ahead;              /* read_ahead */
     size_t        subrequest_output_buffer_size;
                                            /* subrequest_output_buffer_size */
 
+    ngx_http_complex_value_t  *limit_rate; /* limit_rate */
+    ngx_http_complex_value_t  *limit_rate_after; /* limit_rate_after */
+
     ngx_msec_t    client_body_timeout;     /* client_body_timeout */
     ngx_msec_t    send_timeout;            /* send_timeout */
     ngx_msec_t    keepalive_timeout;       /* keepalive_timeout */
diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -150,6 +150,9 @@
 #define NGX_HTTP_COPY_BUFFERED             0x04
 
 
+#define NGX_HTTP_LIMIT_RATE_OFF            NGX_MAX_SIZE_T_VALUE
+
+
 typedef enum {
     NGX_HTTP_INITING_REQUEST_STATE = 0,
     NGX_HTTP_READING_REQUEST_STATE,
diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -2974,7 +2974,7 @@ ngx_http_upstream_send_response(ngx_http
         r->write_event_handler =
                              ngx_http_upstream_process_non_buffered_downstream;
 
-        r->limit_rate = 0;
+        r->limit_rate = NGX_HTTP_LIMIT_RATE_OFF;
 
         if (u->input_filter_init(u->input_filter_ctx) == NGX_ERROR) {
             ngx_http_upstream_finalize_request(r, u, NGX_ERROR);
diff --git a/src/http/ngx_http_write_filter_module.c b/src/http/ngx_http_write_filter_module.c
--- a/src/http/ngx_http_write_filter_module.c
+++ b/src/http/ngx_http_write_filter_module.c
@@ -48,6 +48,8 @@ ngx_int_t
 ngx_http_write_filter(ngx_http_request_t *r, ngx_chain_t *in)
 {
     off_t                      size, sent, nsent, limit;
+    size_t                     limit_rate;
+    ngx_str_t                  val;
     ngx_uint_t                 last, flush, sync;
     ngx_msec_t                 delay;
     ngx_chain_t               *cl, *ln, **ll, *chain;
@@ -250,9 +252,43 @@ ngx_http_write_filter(ngx_http_request_t
         return NGX_ERROR;
     }
 
-    if (r->limit_rate) {
-        if (r->limit_rate_after == 0) {
-            r->limit_rate_after = clcf->limit_rate_after;
+    if (r->limit_rate == 0) {
+
+        if (clcf->limit_rate
+            && ngx_http_complex_value_size(r, clcf->limit_rate, &val,
+                                           &limit_rate)
+               == NGX_OK)
+        {
+            if (limit_rate != (size_t) NGX_ERROR) {
+                r->limit_rate = limit_rate;
+
+            } else if (val.len) {
+                ngx_log_error(NGX_LOG_ERR, c->log, 0,
+                              "invalid \"limit_rate\" value \"%V\"", &val);
+            }
+        }
+
+        if (r->limit_rate == 0) {
+            r->limit_rate = NGX_HTTP_LIMIT_RATE_OFF;
+        }
+    }
+
+    if (r->limit_rate != NGX_HTTP_LIMIT_RATE_OFF) {
+
+        if (r->limit_rate_after == 0
+            && clcf->limit_rate_after
+            && ngx_http_complex_value_size(r, clcf->limit_rate_after, &val,
+                                           &limit_rate)
+               == NGX_OK)
+        {
+            if (limit_rate != (size_t) NGX_ERROR) {
+                r->limit_rate_after = limit_rate;
+
+            } else if (val.len) {
+                ngx_log_error(NGX_LOG_ERR, c->log, 0,
+                              "invalid \"limit_rate_after\" value \"%V\"",
+                              &val);
+            }
         }
 
         limit = (off_t) r->limit_rate * (ngx_time() - r->start_sec + 1)
@@ -293,7 +329,7 @@ ngx_http_write_filter(ngx_http_request_t
         return NGX_ERROR;
     }
 
-    if (r->limit_rate) {
+    if (r->limit_rate != NGX_HTTP_LIMIT_RATE_OFF) {
 
         nsent = c->sent;
 

From pluknet at nginx.com  Thu Dec  6 15:13:39 2018
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Thu, 6 Dec 2018 18:13:39 +0300
Subject: OpenSSL and Early data
In-Reply-To: <CACqAjk3tSwjBpS3bQn2P6NhPE7E8jAVf=NeFYX9VW6-P4VLQ+Q@mail.gmail.com>
References: <CACqAjk3tSwjBpS3bQn2P6NhPE7E8jAVf=NeFYX9VW6-P4VLQ+Q@mail.gmail.com>
Message-ID: <005D72EE-CAD8-4EE3-B358-157F573AFFFB@nginx.com>



> On 6 Dec 2018, at 02:39, Honza Pracha? <jan.prachar at gmail.com> wrote:
> 
> Hello! FYI there is an issue with TLS 1.3 Early data in OpenSSL ? 
> https://github.com/openssl/openssl/issues/7757
> 
> So maybe you would want to consider ignoring Early data with HTTP/2 and OpenSSL. Or try to fix the problem on the nginx side, i.e. do not call SSL_read_early_data() until all pending data is written with SSL_write_early_data().

Hello.

This is not strictly related to HTTP/2.
I could reproduce it with s_client -early_data over HTTP/1.1,
where 1st request is sent in 0-RTT, and 2nd - after handshake.

This quick workaround helped me.  The idea is that we block reading
if SSL_write_early_data returned SSL_ERROR_WANT_WRITE, until one of
the next SSL_write_early_data will succeed.  In practice, we won't
read until there's also no more data to send.  For static content,
that means that we will continue to read only after the whole file
was sent.  This doesn't look perfect but seems to work.

diff -r 2117637f64e9 src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c     Tue Nov 27 17:40:21 2018 +0300
+++ b/src/event/ngx_event_openssl.c     Thu Dec 06 14:51:18 2018 +0000
@@ -2352,6 +2352,7 @@
 
     if (sslerr == SSL_ERROR_WANT_WRITE) {
 
+#if 0
         if (c->ssl->saved_read_handler) {
 
             c->read->handler = c->ssl->saved_read_handler;
@@ -2364,6 +2365,11 @@
 
             ngx_post_event(c->read, &ngx_posted_events);
         }
+#endif
+        if (c->ssl->saved_read_handler == NULL) {
+            c->ssl->saved_read_handler = c->read->handler;
+            c->read->handler = ngx_ssl_read_handler;
+        }
 
         c->write->ready = 0;
         return NGX_AGAIN;


-- 
Sergey Kandaurov


From jan.prachar at gmail.com  Thu Dec  6 16:20:23 2018
From: jan.prachar at gmail.com (Jan =?UTF-8?Q?Pracha=C5=99?=)
Date: Thu, 06 Dec 2018 17:20:23 +0100
Subject: OpenSSL and Early data
In-Reply-To: <005D72EE-CAD8-4EE3-B358-157F573AFFFB@nginx.com>
References: <CACqAjk3tSwjBpS3bQn2P6NhPE7E8jAVf=NeFYX9VW6-P4VLQ+Q@mail.gmail.com>
 <005D72EE-CAD8-4EE3-B358-157F573AFFFB@nginx.com>
Message-ID: <72f6b5db15d0c785276bf6e6f95063e970d1d5f1.camel@gmail.com>

On Thu, 2018-12-06 at 18:13 +0300, Sergey Kandaurov wrote:
> > On 6 Dec 2018, at 02:39, Honza Pracha? <jan.prachar at gmail.com>
> > wrote:
> > 
> > Hello! FYI there is an issue with TLS 1.3 Early data in OpenSSL ? 
> > https://github.com/openssl/openssl/issues/7757
> > 
> > So maybe you would want to consider ignoring Early data with HTTP/2
> > and OpenSSL. Or try to fix the problem on the nginx side, i.e. do
> > not call SSL_read_early_data() until all pending data is written
> > with SSL_write_early_data().
> 
> Hello.
> 
> This is not strictly related to HTTP/2.
> I could reproduce it with s_client -early_data over HTTP/1.1,
> where 1st request is sent in 0-RTT, and 2nd - after handshake.
> 
> This quick workaround helped me.  The idea is that we block reading
> if SSL_write_early_data returned SSL_ERROR_WANT_WRITE, until one of
> the next SSL_write_early_data will succeed.  In practice, we won't
> read until there's also no more data to send.  For static content,
> that means that we will continue to read only after the whole file
> was sent.  This doesn't look perfect but seems to work.

This patch works for me too. SSL_read_early_data waits until all
requested files are sent. Then the handshake is finished.

I am afraid there isn't better solution until OpenSSL changes things
internally. You could wait with writing aplication data until End of
early data record arrives, but this would increase initial RTT.

> 
> diff -r 2117637f64e9 src/event/ngx_event_openssl.c
> --- a/src/event/ngx_event_openssl.c     Tue Nov 27 17:40:21 2018
> +0300
> +++ b/src/event/ngx_event_openssl.c     Thu Dec 06 14:51:18 2018
> +0000
> @@ -2352,6 +2352,7 @@
>  
>      if (sslerr == SSL_ERROR_WANT_WRITE) {
>  
> +#if 0
>          if (c->ssl->saved_read_handler) {
>  
>              c->read->handler = c->ssl->saved_read_handler;
> @@ -2364,6 +2365,11 @@
>  
>              ngx_post_event(c->read, &ngx_posted_events);
>          }
> +#endif
> +        if (c->ssl->saved_read_handler == NULL) {
> +            c->ssl->saved_read_handler = c->read->handler;
> +            c->read->handler = ngx_ssl_read_handler;
> +        }
>  
>          c->write->ready = 0;
>          return NGX_AGAIN;
> 
> 


From xeioex at nginx.com  Fri Dec  7 14:16:23 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 07 Dec 2018 14:16:23 +0000
Subject: [njs] Code generator is separated from parser structure.
Message-ID: <hg.2a13aff6aa1f.1544192183.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/2a13aff6aa1f
branches:  
changeset: 687:2a13aff6aa1f
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Dec 07 17:15:21 2018 +0300
description:
Code generator is separated from parser structure.

diffstat:

 Makefile                 |     7 +-
 njs/njs.c                |    22 +-
 njs/njs_function.h       |     1 -
 njs/njs_generator.c      |  1126 ++++++++++++++++++++++++---------------------
 njs/njs_generator.h      |    22 +-
 njs/njs_parser.c         |    20 +-
 njs/njs_parser.h         |    54 --
 njs/njs_string.c         |     9 +-
 njs/njs_string.h         |     4 +-
 njs/njs_vm.h             |     1 +
 njs/test/njs_unit_test.c |     9 +
 11 files changed, 664 insertions(+), 611 deletions(-)

diffs (truncated from 3210 to 1000 lines):

diff -r 8eb112dcca79 -r 2a13aff6aa1f Makefile
--- a/Makefile	Fri Dec 07 17:11:33 2018 +0300
+++ b/Makefile	Fri Dec 07 17:15:21 2018 +0300
@@ -141,6 +141,7 @@ dist:
 	njs/njs_object.h \
 	njs/njs_function.h \
 	njs/njs_parser.h \
+	njs/njs_generator.h \
 	njs/njs.h \
 	njs/njs.c \
 
@@ -503,12 +504,8 @@ dist:
 	njs/njs.h \
 	njs/njs_core.h \
 	njs/njs_vm.h \
-	njs/njs_number.h \
-	njs/njs_string.h \
-	njs/njs_object.h \
-	njs/njs_function.h \
-	njs/njs_variable.h \
 	njs/njs_parser.h \
+	njs/njs_generator.h \
 	njs/njs_generator.c \
 
 	$(NXT_CC) -c -o $(NXT_BUILDDIR)/njs_generator.o $(NXT_CFLAGS) \
diff -r 8eb112dcca79 -r 2a13aff6aa1f njs/njs.c
--- a/njs/njs.c	Fri Dec 07 17:11:33 2018 +0300
+++ b/njs/njs.c	Fri Dec 07 17:15:21 2018 +0300
@@ -220,6 +220,7 @@ njs_vm_compile(njs_vm_t *vm, u_char **st
     nxt_int_t          ret;
     njs_lexer_t        *lexer;
     njs_parser_t       *parser, *prev;
+    njs_generator_t    *generator;
     njs_parser_node_t  *node;
 
     parser = nxt_mem_cache_zalloc(vm->mem_cache_pool, sizeof(njs_parser_t));
@@ -245,9 +246,6 @@ njs_vm_compile(njs_vm_t *vm, u_char **st
     lexer->line = 1;
     lexer->keywords_hash = vm->shared->keywords_hash;
 
-    parser->code_size = sizeof(njs_vmcode_stop_t);
-    parser->scope_offset = NJS_INDEX_GLOBAL_OFFSET;
-
     if (vm->backtrace != NULL) {
         nxt_array_reset(vm->backtrace);
     }
@@ -272,15 +270,25 @@ njs_vm_compile(njs_vm_t *vm, u_char **st
      */
     vm->code = NULL;
 
-    ret = njs_generate_scope(vm, parser, node);
+    generator = nxt_mem_cache_align(vm->mem_cache_pool, sizeof(njs_value_t),
+                                    sizeof(njs_generator_t));
+
+    if (nxt_slow_path(generator == NULL)) {
+        goto fail;
+    }
+
+    nxt_memzero(generator, sizeof(njs_generator_t));
+
+    ret = njs_generate_scope(vm, generator, node);
     if (nxt_slow_path(ret != NXT_OK)) {
         goto fail;
     }
 
-    vm->current = parser->code_start;
+    vm->current = generator->code_start;
 
-    vm->global_scope = parser->local_scope;
-    vm->scope_size = parser->scope_size;
+    vm->global_scope = generator->local_scope;
+    vm->scope_size = generator->scope_size;
+
     vm->variables_hash = parser->scope->variables;
 
     if (vm->options.init) {
diff -r 8eb112dcca79 -r 2a13aff6aa1f njs/njs_function.h
--- a/njs/njs_function.h	Fri Dec 07 17:11:33 2018 +0300
+++ b/njs/njs_function.h	Fri Dec 07 17:15:21 2018 +0300
@@ -37,7 +37,6 @@ struct njs_function_lambda_s {
     njs_value_t                    *closure_scope;
 
     u_char                         *start;
-    njs_parser_t                   *parser;
 };
 
 
diff -r 8eb112dcca79 -r 2a13aff6aa1f njs/njs_generator.c
--- a/njs/njs_generator.c	Fri Dec 07 17:11:33 2018 +0300
+++ b/njs/njs_generator.c	Fri Dec 07 17:15:21 2018 +0300
@@ -1,6 +1,7 @@
 
 /*
  * Copyright (C) Igor Sysoev
+ * Copyright (C) Dmitry Volyntsev
  * Copyright (C) NGINX, Inc.
  */
 
@@ -8,158 +9,195 @@
 #include <string.h>
 
 
-static nxt_int_t njs_generator(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node);
-static u_char *njs_generate_reserve(njs_vm_t *vm, njs_parser_t *parser,
-    size_t size);
-static nxt_int_t njs_generate_name(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node);
-static nxt_int_t njs_generate_builtin_object(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node);
-static nxt_int_t njs_generate_arguments_object(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
-static nxt_int_t njs_generate_variable(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node);
-static nxt_int_t njs_generate_var_statement(njs_vm_t *vm, njs_parser_t *parser,
+typedef struct njs_generator_patch_s   njs_generator_patch_t;
+
+struct njs_generator_patch_s {
+    /*
+     * The jump_offset field points to jump offset field which contains a small
+     * adjustment and the adjustment should be added as (njs_ret_t *) because
+     * pointer to u_char accesses only one byte so this does not work on big
+     * endian platforms.
+     */
+    njs_ret_t                       jump_offset;
+    njs_generator_patch_t           *next;
+};
+
+
+typedef enum {
+    NJS_GENERATOR_BLOCK = 0,
+    NJS_GENERATOR_LOOP,
+    NJS_GENERATOR_SWITCH,
+} njs_generator_block_type_t;
+
+
+struct njs_generator_block_s {
+    njs_generator_block_type_t      type;    /* 2 bits */
+    nxt_str_t                       label;
+    njs_generator_patch_t           *continuation;
+    njs_generator_patch_t           *exit;
+    njs_generator_block_t           *next;
+};
+
+
+static nxt_int_t njs_generator(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node);
-static nxt_int_t njs_generate_if_statement(njs_vm_t *vm, njs_parser_t *parser,
+static u_char *njs_generate_reserve(njs_vm_t *vm, njs_generator_t *generator,
+    size_t size);
+static nxt_int_t njs_generate_name(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node);
+static nxt_int_t njs_generate_builtin_object(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_int_t njs_generate_arguments_object(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_int_t njs_generate_variable(njs_vm_t *vm, njs_generator_t *generator,
+    njs_parser_node_t *node);
+static nxt_int_t njs_generate_var_statement(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_int_t njs_generate_if_statement(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_cond_expression(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_switch_statement(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_while_statement(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_do_while_statement(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
-static nxt_int_t njs_generate_for_statement(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_int_t njs_generate_for_statement(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_for_in_statement(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_noinline nxt_int_t njs_generate_start_block(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_block_type_t type, const nxt_str_t *label);
+    njs_generator_t *generator, njs_generator_block_type_t type,
+    const nxt_str_t *label);
 static nxt_noinline void njs_generate_patch_loop_continuation(njs_vm_t *vm,
-    njs_parser_t *parser);
+    njs_generator_t *generator);
 static nxt_noinline void njs_generate_patch_block_exit(njs_vm_t *vm,
-    njs_parser_t *parser);
+    njs_generator_t *generator);
 static nxt_int_t njs_generate_continue_statement(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_break_statement(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
-static nxt_int_t njs_generate_statement(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node);
-static nxt_int_t njs_generate_children(njs_vm_t *vm, njs_parser_t *parser,
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_int_t njs_generate_statement(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_int_t njs_generate_children(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node);
-static nxt_int_t njs_generate_stop_statement(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node);
+static nxt_int_t njs_generate_stop_statement(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_comma_expression(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
-static nxt_int_t njs_generate_assignment(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_int_t njs_generate_assignment(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_operation_assignment(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
-static nxt_int_t njs_generate_object(njs_vm_t *vm, njs_parser_t *parser,
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_int_t njs_generate_object(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node);
-static nxt_int_t njs_generate_array(njs_vm_t *vm, njs_parser_t *parser,
+static nxt_int_t njs_generate_array(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node);
-static nxt_int_t njs_generate_function(njs_vm_t *vm, njs_parser_t *parser,
+static nxt_int_t njs_generate_function(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node);
-static nxt_int_t njs_generate_regexp(njs_vm_t *vm, njs_parser_t *parser,
+static nxt_int_t njs_generate_regexp(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node);
 static nxt_int_t njs_generate_test_jump_expression(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_3addr_operation(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node, nxt_bool_t swap);
+    njs_generator_t *generator, njs_parser_node_t *node, nxt_bool_t swap);
 static nxt_int_t njs_generate_2addr_operation(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_typeof_operation(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_inc_dec_operation(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node, nxt_bool_t post);
+    njs_generator_t *generator, njs_parser_node_t *node, nxt_bool_t post);
 static nxt_int_t njs_generate_function_declaration(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_function_scope(njs_vm_t *vm,
     njs_function_lambda_t *lambda, njs_parser_node_t *node);
 static nxt_int_t njs_generate_argument_closures(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_return_statement(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
-static nxt_int_t njs_generate_function_call(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node);
-static nxt_int_t njs_generate_method_call(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_int_t njs_generate_function_call(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_int_t njs_generate_method_call(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_noinline nxt_int_t njs_generate_call(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
-static nxt_int_t njs_generate_try_statement(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_int_t njs_generate_try_statement(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_int_t njs_generate_throw_statement(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
-static nxt_noinline njs_index_t njs_generator_dest_index(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_noinline njs_index_t njs_generate_dest_index(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_noinline njs_index_t
-    njs_generator_object_dest_index(njs_vm_t *vm, njs_parser_t *parser,
+    njs_generate_object_dest_index(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node);
-static njs_index_t njs_generator_node_temp_index_get(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
-static nxt_noinline njs_index_t njs_generator_temp_index_get(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
+static njs_index_t njs_generate_node_temp_index_get(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_noinline njs_index_t njs_generate_temp_index_get(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
 static nxt_noinline nxt_int_t
-    njs_generator_children_indexes_release(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node);
-static nxt_noinline nxt_int_t njs_generator_node_index_release(njs_vm_t *vm,
-    njs_parser_t *parser, njs_parser_node_t *node);
-static nxt_noinline nxt_int_t njs_generator_index_release(njs_vm_t *vm,
-    njs_parser_t *parser, njs_index_t index);
+    njs_generate_children_indexes_release(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_noinline nxt_int_t njs_generate_node_index_release(njs_vm_t *vm,
+    njs_generator_t *generator, njs_parser_node_t *node);
+static nxt_noinline nxt_int_t njs_generate_index_release(njs_vm_t *vm,
+    njs_generator_t *generator, njs_index_t index);
 
 static nxt_int_t njs_generate_function_debug(njs_vm_t *vm, nxt_str_t *name,
     njs_function_lambda_t *lambda, uint32_t line);
 
 
-#define njs_generate_code(parser, type, code)                                 \
+static void njs_generate_syntax_error(njs_vm_t *vm, uint32_t token_line,
+    const char* fmt, ...);
+
+
+#define njs_generate_code(generator, type, code)                              \
     do {                                                                      \
-        code = (type *) njs_generate_reserve(vm, parser, sizeof(type));       \
+        code = (type *) njs_generate_reserve(vm, generator, sizeof(type));    \
         if (nxt_slow_path(code == NULL)) {                                    \
             return NXT_ERROR;                                                 \
         }                                                                     \
-        parser->code_end += sizeof(type);                                     \
+        generator->code_end += sizeof(type);                                  \
     } while (0)
 
 
-#define njs_code_offset(parser, code)                                         \
-    ((u_char *) code - parser->code_start)
-
-
-#define njs_code_ptr(parser, type, offset)                                    \
-    (type *) (parser->code_start + offset)
-
-
-#define njs_code_jump_ptr(parser, offset)                                     \
-    (njs_ret_t *) (parser->code_start + offset)
-
-
-#define njs_code_offset_diff(parser, offset)                                  \
-    ((parser->code_end - parser->code_start) - offset)
-
-
-#define njs_code_set_offset(parser, offset, target)                           \
-    *(njs_code_jump_ptr(parser, offset)) = njs_code_offset_diff(parser, target)
-
-
-#define njs_code_set_jump_offset(parser, type, code_offset)                   \
-    *(njs_code_jump_ptr(parser, code_offset + offsetof(type, offset)))        \
-        = njs_code_offset_diff(parser, code_offset)
-
-
-#define njs_code_update_offset(parser, patch)                                 \
-    *(njs_code_jump_ptr(parser, patch->jump_offset)) +=                       \
-        njs_code_offset_diff(parser, patch->jump_offset)
+#define njs_code_offset(generator, code)                                      \
+    ((u_char *) code - generator->code_start)
+
+
+#define njs_code_ptr(generator, type, offset)                                 \
+    (type *) (generator->code_start + offset)
+
+
+#define njs_code_jump_ptr(generator, offset)                                  \
+    (njs_ret_t *) (generator->code_start + offset)
+
+
+#define njs_code_offset_diff(generator, offset)                               \
+    ((generator->code_end - generator->code_start) - offset)
+
+
+#define njs_code_set_offset(generator, offset, target)                        \
+    *(njs_code_jump_ptr(generator, offset))                                   \
+        = njs_code_offset_diff(generator, target)
+
+
+#define njs_code_set_jump_offset(generator, type, code_offset)                \
+    *(njs_code_jump_ptr(generator, code_offset + offsetof(type, offset)))     \
+        = njs_code_offset_diff(generator, code_offset)
+
+
+#define njs_code_update_offset(generator, patch)                              \
+    *(njs_code_jump_ptr(generator, patch->jump_offset)) +=                    \
+        njs_code_offset_diff(generator, patch->jump_offset)
+
 
 
 static const nxt_str_t  no_label = { 0, NULL };
 
 
 static nxt_int_t
-njs_generator(njs_vm_t *vm, njs_parser_t *parser, njs_parser_node_t *node)
+njs_generator(njs_vm_t *vm, njs_generator_t *generator, njs_parser_node_t *node)
 {
     if (node == NULL) {
         return NXT_OK;
@@ -168,46 +206,46 @@ njs_generator(njs_vm_t *vm, njs_parser_t
     switch (node->token) {
 
     case NJS_TOKEN_VAR:
-        return njs_generate_var_statement(vm, parser, node);
+        return njs_generate_var_statement(vm, generator, node);
 
     case NJS_TOKEN_IF:
-        return njs_generate_if_statement(vm, parser, node);
+        return njs_generate_if_statement(vm, generator, node);
 
     case NJS_TOKEN_CONDITIONAL:
-        return njs_generate_cond_expression(vm, parser, node);
+        return njs_generate_cond_expression(vm, generator, node);
 
     case NJS_TOKEN_SWITCH:
-        return njs_generate_switch_statement(vm, parser, node);
+        return njs_generate_switch_statement(vm, generator, node);
 
     case NJS_TOKEN_WHILE:
-        return njs_generate_while_statement(vm, parser, node);
+        return njs_generate_while_statement(vm, generator, node);
 
     case NJS_TOKEN_DO:
-        return njs_generate_do_while_statement(vm, parser, node);
+        return njs_generate_do_while_statement(vm, generator, node);
 
     case NJS_TOKEN_FOR:
-        return njs_generate_for_statement(vm, parser, node);
+        return njs_generate_for_statement(vm, generator, node);
 
     case NJS_TOKEN_FOR_IN:
-        return njs_generate_for_in_statement(vm, parser, node);
+        return njs_generate_for_in_statement(vm, generator, node);
 
     case NJS_TOKEN_CONTINUE:
-        return njs_generate_continue_statement(vm, parser, node);
+        return njs_generate_continue_statement(vm, generator, node);
 
     case NJS_TOKEN_BREAK:
-        return njs_generate_break_statement(vm, parser, node);
+        return njs_generate_break_statement(vm, generator, node);
 
     case NJS_TOKEN_STATEMENT:
-        return njs_generate_statement(vm, parser, node);
+        return njs_generate_statement(vm, generator, node);
 
     case NJS_TOKEN_END:
-        return njs_generate_stop_statement(vm, parser, node);
+        return njs_generate_stop_statement(vm, generator, node);
 
     case NJS_TOKEN_COMMA:
-        return njs_generate_comma_expression(vm, parser, node);
+        return njs_generate_comma_expression(vm, generator, node);
 
     case NJS_TOKEN_ASSIGNMENT:
-        return njs_generate_assignment(vm, parser, node);
+        return njs_generate_assignment(vm, generator, node);
 
     case NJS_TOKEN_BITWISE_OR_ASSIGNMENT:
     case NJS_TOKEN_BITWISE_XOR_ASSIGNMENT:
@@ -221,7 +259,7 @@ njs_generator(njs_vm_t *vm, njs_parser_t
     case NJS_TOKEN_EXPONENTIATION_ASSIGNMENT:
     case NJS_TOKEN_DIVISION_ASSIGNMENT:
     case NJS_TOKEN_REMAINDER_ASSIGNMENT:
-        return njs_generate_operation_assignment(vm, parser, node);
+        return njs_generate_operation_assignment(vm, generator, node);
 
     case NJS_TOKEN_BITWISE_OR:
     case NJS_TOKEN_BITWISE_XOR:
@@ -246,7 +284,7 @@ njs_generator(njs_vm_t *vm, njs_parser_t
     case NJS_TOKEN_REMAINDER:
     case NJS_TOKEN_PROPERTY_DELETE:
     case NJS_TOKEN_PROPERTY:
-        return njs_generate_3addr_operation(vm, parser, node, 0);
+        return njs_generate_3addr_operation(vm, generator, node, 0);
 
     case NJS_TOKEN_IN:
         /*
@@ -255,11 +293,11 @@ njs_generator(njs_vm_t *vm, njs_parser_t
          * should be swapped to be uniform with other property operations
          * (get/set and delete) to use the property trap.
          */
-        return njs_generate_3addr_operation(vm, parser, node, 1);
+        return njs_generate_3addr_operation(vm, generator, node, 1);
 
     case NJS_TOKEN_LOGICAL_AND:
     case NJS_TOKEN_LOGICAL_OR:
-        return njs_generate_test_jump_expression(vm, parser, node);
+        return njs_generate_test_jump_expression(vm, generator, node);
 
     case NJS_TOKEN_DELETE:
     case NJS_TOKEN_VOID:
@@ -267,25 +305,25 @@ njs_generator(njs_vm_t *vm, njs_parser_t
     case NJS_TOKEN_UNARY_NEGATION:
     case NJS_TOKEN_LOGICAL_NOT:
     case NJS_TOKEN_BITWISE_NOT:
-        return njs_generate_2addr_operation(vm, parser, node);
+        return njs_generate_2addr_operation(vm, generator, node);
 
     case NJS_TOKEN_TYPEOF:
-        return njs_generate_typeof_operation(vm, parser, node);
+        return njs_generate_typeof_operation(vm, generator, node);
 
     case NJS_TOKEN_INCREMENT:
     case NJS_TOKEN_DECREMENT:
-        return njs_generate_inc_dec_operation(vm, parser, node, 0);
+        return njs_generate_inc_dec_operation(vm, generator, node, 0);
 
     case NJS_TOKEN_POST_INCREMENT:
     case NJS_TOKEN_POST_DECREMENT:
-        return njs_generate_inc_dec_operation(vm, parser, node, 1);
+        return njs_generate_inc_dec_operation(vm, generator, node, 1);
 
     case NJS_TOKEN_UNDEFINED:
     case NJS_TOKEN_NULL:
     case NJS_TOKEN_BOOLEAN:
     case NJS_TOKEN_NUMBER:
     case NJS_TOKEN_STRING:
-        node->index = njs_value_index(vm, parser, &node->u.value);
+        node->index = njs_value_index(vm, &node->u.value, generator->runtime);
 
         if (nxt_fast_path(node->index != NJS_INDEX_NONE)) {
             return NXT_OK;
@@ -298,16 +336,16 @@ njs_generator(njs_vm_t *vm, njs_parser_t
         return NXT_OK;
 
     case NJS_TOKEN_OBJECT:
-        return njs_generate_object(vm, parser, node);
+        return njs_generate_object(vm, generator, node);
 
     case NJS_TOKEN_ARRAY:
-        return njs_generate_array(vm, parser, node);
+        return njs_generate_array(vm, generator, node);
 
     case NJS_TOKEN_FUNCTION_EXPRESSION:
-        return njs_generate_function(vm, parser, node);
+        return njs_generate_function(vm, generator, node);
 
     case NJS_TOKEN_REGEXP:
-        return njs_generate_regexp(vm, parser, node);
+        return njs_generate_regexp(vm, generator, node);
 
     case NJS_TOKEN_THIS:
     case NJS_TOKEN_OBJECT_CONSTRUCTOR:
@@ -331,7 +369,7 @@ njs_generator(njs_vm_t *vm, njs_parser_t
         return NXT_OK;
 
     case NJS_TOKEN_NAME:
-        return njs_generate_name(vm, parser, node);
+        return njs_generate_name(vm, generator, node);
 
     case NJS_TOKEN_GLOBAL_THIS:
     case NJS_TOKEN_NJS:
@@ -350,28 +388,28 @@ njs_generator(njs_vm_t *vm, njs_parser_t
     case NJS_TOKEN_REQUIRE:
     case NJS_TOKEN_SET_TIMEOUT:
     case NJS_TOKEN_CLEAR_TIMEOUT:
-        return njs_generate_builtin_object(vm, parser, node);
+        return njs_generate_builtin_object(vm, generator, node);
 
     case NJS_TOKEN_ARGUMENTS:
-        return njs_generate_arguments_object(vm, parser, node);
+        return njs_generate_arguments_object(vm, generator, node);
 
     case NJS_TOKEN_FUNCTION:
-        return njs_generate_function_declaration(vm, parser, node);
+        return njs_generate_function_declaration(vm, generator, node);
 
     case NJS_TOKEN_FUNCTION_CALL:
-        return njs_generate_function_call(vm, parser, node);
+        return njs_generate_function_call(vm, generator, node);
 
     case NJS_TOKEN_RETURN:
-        return njs_generate_return_statement(vm, parser, node);
+        return njs_generate_return_statement(vm, generator, node);
 
     case NJS_TOKEN_METHOD_CALL:
-        return njs_generate_method_call(vm, parser, node);
+        return njs_generate_method_call(vm, generator, node);
 
     case NJS_TOKEN_TRY:
-        return njs_generate_try_statement(vm, parser, node);
+        return njs_generate_try_statement(vm, generator, node);
 
     case NJS_TOKEN_THROW:
-        return njs_generate_throw_statement(vm, parser, node);
+        return njs_generate_throw_statement(vm, generator, node);
 
     default:
         nxt_thread_log_debug("unknown token: %d", node->token);
@@ -383,16 +421,18 @@ njs_generator(njs_vm_t *vm, njs_parser_t
 
 
 static u_char *
-njs_generate_reserve(njs_vm_t *vm, njs_parser_t *parser, size_t size)
+njs_generate_reserve(njs_vm_t *vm, njs_generator_t *generator, size_t size)
 {
     u_char  *p;
 
-    if (parser->code_end + size <= parser->code_start + parser->code_size) {
-        return parser->code_end;
+    if (generator->code_end + size <=
+        generator->code_start + generator->code_size)
+    {
+        return generator->code_end;
     }
 
-    size = nxt_max(parser->code_end - parser->code_start + size,
-                   parser->code_size);
+    size = nxt_max(generator->code_end - generator->code_start + size,
+                   generator->code_size);
 
     if (size < 1024) {
         size *= 2;
@@ -407,20 +447,21 @@ njs_generate_reserve(njs_vm_t *vm, njs_p
         return NULL;
     }
 
-    parser->code_size = size;
-
-    size = parser->code_end - parser->code_start;
-    memcpy(p, parser->code_start, size);
-
-    parser->code_start = p;
-    parser->code_end = p + size;
-
-    return parser->code_end;
+    generator->code_size = size;
+
+    size = generator->code_end - generator->code_start;
+    memcpy(p, generator->code_start, size);
+
+    generator->code_start = p;
+    generator->code_end = p + size;
+
+    return generator->code_end;
 }
 
 
 static nxt_int_t
-njs_generate_name(njs_vm_t *vm, njs_parser_t *parser, njs_parser_node_t *node)
+njs_generate_name(njs_vm_t *vm, njs_generator_t *generator,
+    njs_parser_node_t *node)
 {
     njs_variable_t            *var;
     njs_vmcode_object_copy_t  *copy;
@@ -432,12 +473,12 @@ njs_generate_name(njs_vm_t *vm, njs_pars
 
     if (var->type == NJS_VARIABLE_FUNCTION) {
 
-        node->index = njs_generator_dest_index(vm, parser, node);
+        node->index = njs_generate_dest_index(vm, generator, node);
         if (nxt_slow_path(node->index == NJS_INDEX_ERROR)) {
             return node->index;
         }
 
-        njs_generate_code(parser, njs_vmcode_object_copy_t, copy);
+        njs_generate_code(generator, njs_vmcode_object_copy_t, copy);
         copy->code.operation = njs_vmcode_object_copy;
         copy->code.operands = NJS_VMCODE_2OPERANDS;
         copy->code.retval = NJS_VMCODE_RETVAL;
@@ -447,12 +488,12 @@ njs_generate_name(njs_vm_t *vm, njs_pars
         return NXT_OK;
     }
 
-    return njs_generate_variable(vm, parser, node);
+    return njs_generate_variable(vm, generator, node);
 }
 
 
 static nxt_int_t
-njs_generate_builtin_object(njs_vm_t *vm, njs_parser_t *parser,
+njs_generate_builtin_object(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node)
 {
     njs_index_t               index;
@@ -463,12 +504,12 @@ njs_generate_builtin_object(njs_vm_t *vm
         return NXT_ERROR;
     }
 
-    node->index = njs_generator_dest_index(vm, parser, node);
+    node->index = njs_generate_dest_index(vm, generator, node);
     if (nxt_slow_path(node->index == NJS_INDEX_ERROR)) {
         return NXT_ERROR;
     }
 
-    njs_generate_code(parser, njs_vmcode_object_copy_t, copy);
+    njs_generate_code(generator, njs_vmcode_object_copy_t, copy);
     copy->code.operation = njs_vmcode_object_copy;
     copy->code.operands = NJS_VMCODE_2OPERANDS;
     copy->code.retval = NJS_VMCODE_RETVAL;
@@ -480,19 +521,19 @@ njs_generate_builtin_object(njs_vm_t *vm
 
 
 static nxt_int_t
-njs_generate_arguments_object(njs_vm_t *vm, njs_parser_t *parser,
+njs_generate_arguments_object(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node)
 {
     njs_vmcode_arguments_t  *gen;
 
-    parser->arguments_object = 1;
-
-    node->index = njs_generator_object_dest_index(vm, parser, node);
+    generator->arguments_object = 1;
+
+    node->index = njs_generate_object_dest_index(vm, generator, node);
     if (nxt_slow_path(node->index == NJS_INDEX_ERROR)) {
         return NXT_ERROR;
     }
 
-    njs_generate_code(parser, njs_vmcode_arguments_t, gen);
+    njs_generate_code(generator, njs_vmcode_arguments_t, gen);
     gen->code.operation = njs_vmcode_arguments;
     gen->code.operands = NJS_VMCODE_1OPERAND;
     gen->code.retval = NJS_VMCODE_RETVAL;
@@ -503,7 +544,7 @@ njs_generate_arguments_object(njs_vm_t *
 
 
 static nxt_int_t
-njs_generate_variable(njs_vm_t *vm, njs_parser_t *parser,
+njs_generate_variable(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node)
 {
     njs_index_t  index;
@@ -520,7 +561,7 @@ njs_generate_variable(njs_vm_t *vm, njs_
 
 
 static nxt_int_t
-njs_generate_var_statement(njs_vm_t *vm, njs_parser_t *parser,
+njs_generate_var_statement(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node)
 {
     nxt_int_t          ret;
@@ -546,7 +587,7 @@ njs_generate_var_statement(njs_vm_t *vm,
 
     expr->dest = lvalue;
 
-    ret = njs_generator(vm, parser, expr);
+    ret = njs_generator(vm, generator, expr);
     if (nxt_slow_path(ret != NXT_OK)) {
         return ret;
     }
@@ -556,7 +597,7 @@ njs_generate_var_statement(njs_vm_t *vm,
      * empty object or expression result is stored directly in variable.
      */
     if (lvalue->index != expr->index) {
-        njs_generate_code(parser, njs_vmcode_move_t, move);
+        njs_generate_code(generator, njs_vmcode_move_t, move);
         move->code.operation = njs_vmcode_move;
         move->code.operands = NJS_VMCODE_2OPERANDS;
         move->code.retval = NJS_VMCODE_RETVAL;
@@ -572,7 +613,7 @@ njs_generate_var_statement(njs_vm_t *vm,
 
 
 static nxt_int_t
-njs_generate_if_statement(njs_vm_t *vm, njs_parser_t *parser,
+njs_generate_if_statement(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node)
 {
     njs_ret_t               jump_offset, label_offset;
@@ -582,23 +623,23 @@ njs_generate_if_statement(njs_vm_t *vm, 
 
     /* The condition expression. */
 
-    ret = njs_generator(vm, parser, node->left);
+    ret = njs_generator(vm, generator, node->left);
     if (nxt_slow_path(ret != NXT_OK)) {
         return ret;
     }
 
-    njs_generate_code(parser, njs_vmcode_cond_jump_t, cond_jump);
+    njs_generate_code(generator, njs_vmcode_cond_jump_t, cond_jump);
     cond_jump->code.operation = njs_vmcode_if_false_jump;
     cond_jump->code.operands = NJS_VMCODE_2OPERANDS;
     cond_jump->code.retval = NJS_VMCODE_NO_RETVAL;
     cond_jump->cond = node->left->index;
 
-    ret = njs_generator_node_index_release(vm, parser, node->left);
+    ret = njs_generate_node_index_release(vm, generator, node->left);
     if (nxt_slow_path(ret != NXT_OK)) {
         return ret;
     }
 
-    jump_offset = njs_code_offset(parser, cond_jump);
+    jump_offset = njs_code_offset(generator, cond_jump);
     label_offset = jump_offset + offsetof(njs_vmcode_cond_jump_t, offset);
 
     if (node->right != NULL && node->right->token == NJS_TOKEN_BRANCHING) {
@@ -607,24 +648,24 @@ njs_generate_if_statement(njs_vm_t *vm, 
 
         node = node->right;
 
-        ret = njs_generator(vm, parser, node->left);
+        ret = njs_generator(vm, generator, node->left);
         if (nxt_slow_path(ret != NXT_OK)) {
             return ret;
         }
 
-        ret = njs_generator_node_index_release(vm, parser, node->left);
+        ret = njs_generate_node_index_release(vm, generator, node->left);
         if (nxt_slow_path(ret != NXT_OK)) {
             return ret;
         }
 
-        njs_generate_code(parser, njs_vmcode_jump_t, jump);
+        njs_generate_code(generator, njs_vmcode_jump_t, jump);
         jump->code.operation = njs_vmcode_jump;
         jump->code.operands = NJS_VMCODE_NO_OPERAND;
         jump->code.retval = NJS_VMCODE_NO_RETVAL;
 
-        njs_code_set_offset(parser, label_offset, jump_offset);
-
-        jump_offset = njs_code_offset(parser, jump);
+        njs_code_set_offset(generator, label_offset, jump_offset);
+
+        jump_offset = njs_code_offset(generator, jump);
         label_offset = jump_offset + offsetof(njs_vmcode_jump_t, offset);
     }
 
@@ -633,24 +674,24 @@ njs_generate_if_statement(njs_vm_t *vm, 
      * or the "else" branch in a case of "if/then/else" statement.
      */
 
-    ret = njs_generator(vm, parser, node->right);
+    ret = njs_generator(vm, generator, node->right);
     if (nxt_slow_path(ret != NXT_OK)) {
         return ret;
     }
 
-    ret = njs_generator_node_index_release(vm, parser, node->right);
+    ret = njs_generate_node_index_release(vm, generator, node->right);
     if (nxt_slow_path(ret != NXT_OK)) {
         return ret;
     }
 
-    njs_code_set_offset(parser, label_offset, jump_offset);
+    njs_code_set_offset(generator, label_offset, jump_offset);
 
     return NXT_OK;
 }
 
 
 static nxt_int_t
-njs_generate_cond_expression(njs_vm_t *vm, njs_parser_t *parser,
+njs_generate_cond_expression(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node)
 {
     njs_ret_t               jump_offset, cond_jump_offset;
@@ -662,19 +703,19 @@ njs_generate_cond_expression(njs_vm_t *v
 
     /* The condition expression. */
 
-    ret = njs_generator(vm, parser, node->left);
+    ret = njs_generator(vm, generator, node->left);
     if (nxt_slow_path(ret != NXT_OK)) {
         return ret;
     }
 
-    njs_generate_code(parser, njs_vmcode_cond_jump_t, cond_jump);
-    cond_jump_offset = njs_code_offset(parser, cond_jump);
+    njs_generate_code(generator, njs_vmcode_cond_jump_t, cond_jump);
+    cond_jump_offset = njs_code_offset(generator, cond_jump);
     cond_jump->code.operation = njs_vmcode_if_false_jump;
     cond_jump->code.operands = NJS_VMCODE_2OPERANDS;
     cond_jump->code.retval = NJS_VMCODE_NO_RETVAL;
     cond_jump->cond = node->left->index;
 
-    node->index = njs_generator_dest_index(vm, parser, node);
+    node->index = njs_generate_dest_index(vm, generator, node);
     if (nxt_slow_path(node->index == NJS_INDEX_ERROR)) {
         return node->index;
     }
@@ -683,7 +724,7 @@ njs_generate_cond_expression(njs_vm_t *v
 
     /* The "true" branch. */
 
-    ret = njs_generator(vm, parser, branch->left);
+    ret = njs_generator(vm, generator, branch->left);
     if (nxt_slow_path(ret != NXT_OK)) {
         return ret;
     }
@@ -695,7 +736,7 @@ njs_generate_cond_expression(njs_vm_t *v
      */
 
     if (node->index != branch->left->index) {
-        njs_generate_code(parser, njs_vmcode_move_t, move);
+        njs_generate_code(generator, njs_vmcode_move_t, move);
         move->code.operation = njs_vmcode_move;
         move->code.operands = NJS_VMCODE_2OPERANDS;
         move->code.retval = NJS_VMCODE_RETVAL;
@@ -703,28 +744,29 @@ njs_generate_cond_expression(njs_vm_t *v
         move->src = branch->left->index;
     }
 
-    ret = njs_generator_node_index_release(vm, parser, branch->left);
+    ret = njs_generate_node_index_release(vm, generator, branch->left);
     if (nxt_slow_path(ret != NXT_OK)) {
         return ret;
     }
 
-    njs_generate_code(parser, njs_vmcode_jump_t, jump);
-    jump_offset = njs_code_offset(parser, jump);
+    njs_generate_code(generator, njs_vmcode_jump_t, jump);
+    jump_offset = njs_code_offset(generator, jump);
     jump->code.operation = njs_vmcode_jump;
     jump->code.operands = NJS_VMCODE_NO_OPERAND;
     jump->code.retval = NJS_VMCODE_NO_RETVAL;
 
-    njs_code_set_jump_offset(parser, njs_vmcode_cond_jump_t, cond_jump_offset);
+    njs_code_set_jump_offset(generator, njs_vmcode_cond_jump_t,
+                             cond_jump_offset);
 
     /* The "false" branch. */
 
-    ret = njs_generator(vm, parser, branch->right);
+    ret = njs_generator(vm, generator, branch->right);
     if (nxt_slow_path(ret != NXT_OK)) {
         return ret;
     }
 
     if (node->index != branch->right->index) {
-        njs_generate_code(parser, njs_vmcode_move_t, move);
+        njs_generate_code(generator, njs_vmcode_move_t, move);
         move->code.operation = njs_vmcode_move;
         move->code.operands = NJS_VMCODE_2OPERANDS;
         move->code.retval = NJS_VMCODE_RETVAL;
@@ -732,9 +774,9 @@ njs_generate_cond_expression(njs_vm_t *v
         move->src = branch->right->index;
     }
 
-    njs_code_set_jump_offset(parser, njs_vmcode_cond_jump_t, jump_offset);
-
-    ret = njs_generator_node_index_release(vm, parser, branch->right);
+    njs_code_set_jump_offset(generator, njs_vmcode_cond_jump_t, jump_offset);
+
+    ret = njs_generate_node_index_release(vm, generator, branch->right);
     if (nxt_slow_path(ret != NXT_OK)) {
         return ret;
     }
@@ -744,7 +786,7 @@ njs_generate_cond_expression(njs_vm_t *v
 
 
 static nxt_int_t
-njs_generate_switch_statement(njs_vm_t *vm, njs_parser_t *parser,
+njs_generate_switch_statement(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *swtch)
 {
     njs_ret_t                jump_offset;
@@ -753,14 +795,14 @@ njs_generate_switch_statement(njs_vm_t *
     njs_parser_node_t        *node, *expr, *branch;
     njs_vmcode_move_t        *move;
     njs_vmcode_jump_t        *jump;
-    njs_parser_patch_t       *patch, *next, *patches, **last;
+    njs_generator_patch_t       *patch, *next, *patches, **last;
     njs_vmcode_equal_jump_t  *equal;
 
     /* The "switch" expression. */
 
     expr = swtch->left;
 
-    ret = njs_generator(vm, parser, expr);
+    ret = njs_generator(vm, generator, expr);
     if (nxt_slow_path(ret != NXT_OK)) {
         return ret;
     }
@@ -768,12 +810,12 @@ njs_generate_switch_statement(njs_vm_t *
     index = expr->index;
 
     if (!expr->temporary) {
-        index = njs_generator_temp_index_get(vm, parser, swtch);
+        index = njs_generate_temp_index_get(vm, generator, swtch);
         if (nxt_slow_path(index == NJS_INDEX_ERROR)) {
             return NXT_ERROR;
         }
 
-        njs_generate_code(parser, njs_vmcode_move_t, move);
+        njs_generate_code(generator, njs_vmcode_move_t, move);
         move->code.operation = njs_vmcode_move;
         move->code.operands = NJS_VMCODE_2OPERANDS;
         move->code.retval = NJS_VMCODE_RETVAL;
@@ -781,7 +823,8 @@ njs_generate_switch_statement(njs_vm_t *
         move->src = expr->index;
     }
 
-    ret = njs_generate_start_block(vm, parser, NJS_PARSER_SWITCH, &no_label);
+    ret = njs_generate_start_block(vm, generator, NJS_GENERATOR_SWITCH,
+                                   &no_label);
     if (nxt_slow_path(ret != NXT_OK)) {
         return ret;
     }
@@ -797,12 +840,12 @@ njs_generate_switch_statement(njs_vm_t *
 
             node = branch->right;
 
-            ret = njs_generator(vm, parser, node->left);
+            ret = njs_generator(vm, generator, node->left);
             if (nxt_slow_path(ret != NXT_OK)) {
                 return ret;
             }
 
-            njs_generate_code(parser, njs_vmcode_equal_jump_t, equal);
+            njs_generate_code(generator, njs_vmcode_equal_jump_t, equal);
             equal->code.operation = njs_vmcode_if_equal_jump;
             equal->code.operands = NJS_VMCODE_3OPERANDS;

From xeioex at nginx.com  Fri Dec  7 14:16:22 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 07 Dec 2018 14:16:22 +0000
Subject: [njs] Code generator refactored.
Message-ID: <hg.8eb112dcca79.1544192182.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/8eb112dcca79
branches:  
changeset: 686:8eb112dcca79
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Dec 07 17:11:33 2018 +0300
description:
Code generator refactored.

Previously, the code size had to be calculated before code generation
phase because raw pointers to instruction were used to determine proper
jump offsets. It introduced unnecessary coupling between parser and
generator as well as complicated the parser because it had to know in
advance what instructions will be generated.

The solution is the dynamic reallocation of the code during generation
phase and the usage of relative offsets when calculating jump instructions.

diffstat:

 njs/njs_core.h              |    1 +
 njs/njs_generator.c         |  227 ++++++++++++++++++++++++++++++++-----------
 njs/njs_generator.h         |   15 ++
 njs/njs_parser.c            |   45 +--------
 njs/njs_parser.h            |   12 +-
 njs/njs_parser_expression.c |   77 +-------------
 6 files changed, 194 insertions(+), 183 deletions(-)

diffs (truncated from 1250 to 1000 lines):

diff -r 23fecf3fcc60 -r 8eb112dcca79 njs/njs_core.h
--- a/njs/njs_core.h	Wed Dec 05 15:38:33 2018 +0300
+++ b/njs/njs_core.h	Fri Dec 07 17:11:33 2018 +0300
@@ -32,6 +32,7 @@
 #include <njs_vm.h>
 #include <njs_variable.h>
 #include <njs_parser.h>
+#include <njs_generator.h>
 #include <njs_function.h>
 #include <njs_boolean.h>
 #include <njs_number.h>
diff -r 23fecf3fcc60 -r 8eb112dcca79 njs/njs_generator.c
--- a/njs/njs_generator.c	Wed Dec 05 15:38:33 2018 +0300
+++ b/njs/njs_generator.c	Fri Dec 07 17:11:33 2018 +0300
@@ -10,6 +10,8 @@
 
 static nxt_int_t njs_generator(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node);
+static u_char *njs_generate_reserve(njs_vm_t *vm, njs_parser_t *parser,
+    size_t size);
 static nxt_int_t njs_generate_name(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node);
 static nxt_int_t njs_generate_builtin_object(njs_vm_t *vm, njs_parser_t *parser,
@@ -78,8 +80,8 @@ static nxt_int_t njs_generate_function_d
     njs_parser_t *parser, njs_parser_node_t *node);
 static nxt_int_t njs_generate_function_scope(njs_vm_t *vm,
     njs_function_lambda_t *lambda, njs_parser_node_t *node);
-static void njs_generate_argument_closures(njs_parser_t *parser,
-    njs_parser_node_t *node);
+static nxt_int_t njs_generate_argument_closures(njs_vm_t *vm,
+    njs_parser_t *parser, njs_parser_node_t *node);
 static nxt_int_t njs_generate_return_statement(njs_vm_t *vm,
     njs_parser_t *parser, njs_parser_node_t *node);
 static nxt_int_t njs_generate_function_call(njs_vm_t *vm, njs_parser_t *parser,
@@ -113,6 +115,46 @@ static nxt_int_t njs_generate_function_d
     njs_function_lambda_t *lambda, uint32_t line);
 
 
+#define njs_generate_code(parser, type, code)                                 \
+    do {                                                                      \
+        code = (type *) njs_generate_reserve(vm, parser, sizeof(type));       \
+        if (nxt_slow_path(code == NULL)) {                                    \
+            return NXT_ERROR;                                                 \
+        }                                                                     \
+        parser->code_end += sizeof(type);                                     \
+    } while (0)
+
+
+#define njs_code_offset(parser, code)                                         \
+    ((u_char *) code - parser->code_start)
+
+
+#define njs_code_ptr(parser, type, offset)                                    \
+    (type *) (parser->code_start + offset)
+
+
+#define njs_code_jump_ptr(parser, offset)                                     \
+    (njs_ret_t *) (parser->code_start + offset)
+
+
+#define njs_code_offset_diff(parser, offset)                                  \
+    ((parser->code_end - parser->code_start) - offset)
+
+
+#define njs_code_set_offset(parser, offset, target)                           \
+    *(njs_code_jump_ptr(parser, offset)) = njs_code_offset_diff(parser, target)
+
+
+#define njs_code_set_jump_offset(parser, type, code_offset)                   \
+    *(njs_code_jump_ptr(parser, code_offset + offsetof(type, offset)))        \
+        = njs_code_offset_diff(parser, code_offset)
+
+
+#define njs_code_update_offset(parser, patch)                                 \
+    *(njs_code_jump_ptr(parser, patch->jump_offset)) +=                       \
+        njs_code_offset_diff(parser, patch->jump_offset)
+
+
 static const nxt_str_t  no_label = { 0, NULL };
 
 
@@ -340,6 +382,43 @@ njs_generator(njs_vm_t *vm, njs_parser_t
 }
 
 
+static u_char *
+njs_generate_reserve(njs_vm_t *vm, njs_parser_t *parser, size_t size)
+{
+    u_char  *p;
+
+    if (parser->code_end + size <= parser->code_start + parser->code_size) {
+        return parser->code_end;
+    }
+
+    size = nxt_max(parser->code_end - parser->code_start + size,
+                   parser->code_size);
+
+    if (size < 1024) {
+        size *= 2;
+
+    } else {
+        size += size/ 2;
+    }
+
+    p = nxt_mem_cache_alloc(vm->mem_cache_pool, size);
+    if (nxt_slow_path(p == NULL)) {
+        njs_memory_error(vm);
+        return NULL;
+    }
+
+    parser->code_size = size;
+
+    size = parser->code_end - parser->code_start;
+    memcpy(p, parser->code_start, size);
+
+    parser->code_start = p;
+    parser->code_end = p + size;
+
+    return parser->code_end;
+}
+
+
 static nxt_int_t
 njs_generate_name(njs_vm_t *vm, njs_parser_t *parser, njs_parser_node_t *node)
 {
@@ -496,9 +575,8 @@ static nxt_int_t
 njs_generate_if_statement(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node)
 {
-    u_char                  *previous;
+    njs_ret_t               jump_offset, label_offset;
     nxt_int_t               ret;
-    njs_ret_t               *label;
     njs_vmcode_jump_t       *jump;
     njs_vmcode_cond_jump_t  *cond_jump;
 
@@ -520,8 +598,8 @@ njs_generate_if_statement(njs_vm_t *vm, 
         return ret;
     }
 
-    previous = (u_char *) cond_jump;
-    label = &cond_jump->offset;
+    jump_offset = njs_code_offset(parser, cond_jump);
+    label_offset = jump_offset + offsetof(njs_vmcode_cond_jump_t, offset);
 
     if (node->right != NULL && node->right->token == NJS_TOKEN_BRANCHING) {
 
@@ -544,9 +622,10 @@ njs_generate_if_statement(njs_vm_t *vm, 
         jump->code.operands = NJS_VMCODE_NO_OPERAND;
         jump->code.retval = NJS_VMCODE_NO_RETVAL;
 
-        *label = parser->code_end - previous;
-        previous = (u_char *) jump;
-        label = &jump->offset;
+        njs_code_set_offset(parser, label_offset, jump_offset);
+
+        jump_offset = njs_code_offset(parser, jump);
+        label_offset = jump_offset + offsetof(njs_vmcode_jump_t, offset);
     }
 
     /*
@@ -564,7 +643,7 @@ njs_generate_if_statement(njs_vm_t *vm, 
         return ret;
     }
 
-    *label = parser->code_end - previous;
+    njs_code_set_offset(parser, label_offset, jump_offset);
 
     return NXT_OK;
 }
@@ -574,6 +653,7 @@ static nxt_int_t
 njs_generate_cond_expression(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node)
 {
+    njs_ret_t               jump_offset, cond_jump_offset;
     nxt_int_t               ret;
     njs_parser_node_t       *branch;
     njs_vmcode_move_t       *move;
@@ -588,6 +668,7 @@ njs_generate_cond_expression(njs_vm_t *v
     }
 
     njs_generate_code(parser, njs_vmcode_cond_jump_t, cond_jump);
+    cond_jump_offset = njs_code_offset(parser, cond_jump);
     cond_jump->code.operation = njs_vmcode_if_false_jump;
     cond_jump->code.operands = NJS_VMCODE_2OPERANDS;
     cond_jump->code.retval = NJS_VMCODE_NO_RETVAL;
@@ -628,11 +709,12 @@ njs_generate_cond_expression(njs_vm_t *v
     }
 
     njs_generate_code(parser, njs_vmcode_jump_t, jump);
+    jump_offset = njs_code_offset(parser, jump);
     jump->code.operation = njs_vmcode_jump;
     jump->code.operands = NJS_VMCODE_NO_OPERAND;
     jump->code.retval = NJS_VMCODE_NO_RETVAL;
 
-    cond_jump->offset = parser->code_end - (u_char *) cond_jump;
+    njs_code_set_jump_offset(parser, njs_vmcode_cond_jump_t, cond_jump_offset);
 
     /* The "false" branch. */
 
@@ -650,7 +732,7 @@ njs_generate_cond_expression(njs_vm_t *v
         move->src = branch->right->index;
     }
 
-    jump->offset = parser->code_end - (u_char *) jump;
+    njs_code_set_jump_offset(parser, njs_vmcode_cond_jump_t, jump_offset);
 
     ret = njs_generator_node_index_release(vm, parser, branch->right);
     if (nxt_slow_path(ret != NXT_OK)) {
@@ -665,6 +747,7 @@ static nxt_int_t
 njs_generate_switch_statement(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *swtch)
 {
+    njs_ret_t                jump_offset;
     nxt_int_t                ret;
     njs_index_t              index;
     njs_parser_node_t        *node, *expr, *branch;
@@ -703,6 +786,7 @@ njs_generate_switch_statement(njs_vm_t *
         return ret;
     }
 
+    patches = NULL;
     last = &patches;
 
     for (branch = swtch->right; branch != NULL; branch = branch->left) {
@@ -737,7 +821,8 @@ njs_generate_switch_statement(njs_vm_t *
                 return NXT_ERROR;
             }
 
-            patch->address = &equal->offset;
+            patch->jump_offset = njs_code_offset(parser, equal)
+                                 + offsetof(njs_vmcode_equal_jump_t, offset);
 
             *last = patch;
             last = &patch->next;
@@ -751,6 +836,7 @@ njs_generate_switch_statement(njs_vm_t *
     }
 
     njs_generate_code(parser, njs_vmcode_jump_t, jump);
+    jump_offset = njs_code_offset(parser, jump);
     jump->code.operation = njs_vmcode_jump;
     jump->code.operands = NJS_VMCODE_1OPERAND;
     jump->code.retval = NJS_VMCODE_NO_RETVAL;
@@ -761,12 +847,13 @@ njs_generate_switch_statement(njs_vm_t *
     for (branch = swtch->right; branch != NULL; branch = branch->left) {
 
         if (branch->token == NJS_TOKEN_DEFAULT) {
-            jump->offset = parser->code_end - (u_char *) jump;
+            njs_code_set_jump_offset(parser, njs_vmcode_jump_t, jump_offset);
             jump = NULL;
             node = branch;
 
         } else {
-            *patch->address += parser->code_end - (u_char *) patch->address;
+            njs_code_update_offset(parser, patch);
+
             next = patch->next;
 
             nxt_mem_cache_free(vm->mem_cache_pool, patch);
@@ -785,7 +872,7 @@ njs_generate_switch_statement(njs_vm_t *
 
     if (jump != NULL) {
         /* A "switch" without default case. */
-        jump->offset = parser->code_end - (u_char *) jump;
+        njs_code_set_jump_offset(parser, njs_vmcode_jump_t, jump_offset);
     }
 
     /* Patch "break" statements offsets. */
@@ -799,7 +886,7 @@ static nxt_int_t
 njs_generate_while_statement(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node)
 {
-    u_char                  *loop;
+    njs_ret_t               jump_offset, loop_offset;
     nxt_int_t               ret;
     njs_parser_node_t       *condition;
     njs_vmcode_jump_t       *jump;
@@ -812,6 +899,7 @@ njs_generate_while_statement(njs_vm_t *v
      */
 
     njs_generate_code(parser, njs_vmcode_jump_t, jump);
+    jump_offset = njs_code_offset(parser, jump);
     jump->code.operation = njs_vmcode_jump;
     jump->code.operands = NJS_VMCODE_NO_OPERAND;
     jump->code.retval = NJS_VMCODE_NO_RETVAL;
@@ -823,7 +911,7 @@ njs_generate_while_statement(njs_vm_t *v
         return ret;
     }
 
-    loop = parser->code_end;
+    loop_offset = njs_code_offset(parser, parser->code_end);
 
     ret = njs_generator(vm, parser, node->left);
     if (nxt_slow_path(ret != NXT_OK)) {
@@ -834,7 +922,7 @@ njs_generate_while_statement(njs_vm_t *v
 
     njs_generate_patch_loop_continuation(vm, parser);
 
-    jump->offset = parser->code_end - (u_char *) jump;
+    njs_code_set_jump_offset(parser, njs_vmcode_jump_t, jump_offset);
 
     condition = node->right;
 
@@ -847,7 +935,7 @@ njs_generate_while_statement(njs_vm_t *v
     cond_jump->code.operation = njs_vmcode_if_true_jump;
     cond_jump->code.operands = NJS_VMCODE_2OPERANDS;
     cond_jump->code.retval = NJS_VMCODE_NO_RETVAL;
-    cond_jump->offset = loop - (u_char *) cond_jump;
+    cond_jump->offset = loop_offset - njs_code_offset(parser, cond_jump);
     cond_jump->cond = condition->index;
 
     njs_generate_patch_block_exit(vm, parser);
@@ -860,7 +948,7 @@ static nxt_int_t
 njs_generate_do_while_statement(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node)
 {
-    u_char                  *loop;
+    njs_ret_t               loop_offset;
     nxt_int_t               ret;
     njs_parser_node_t       *condition;
     njs_vmcode_cond_jump_t  *cond_jump;
@@ -872,7 +960,7 @@ njs_generate_do_while_statement(njs_vm_t
         return ret;
     }
 
-    loop = parser->code_end;
+    loop_offset = njs_code_offset(parser, parser->code_end);
 
     ret = njs_generator(vm, parser, node->left);
     if (nxt_slow_path(ret != NXT_OK)) {
@@ -894,7 +982,7 @@ njs_generate_do_while_statement(njs_vm_t
     cond_jump->code.operation = njs_vmcode_if_true_jump;
     cond_jump->code.operands = NJS_VMCODE_2OPERANDS;
     cond_jump->code.retval = NJS_VMCODE_NO_RETVAL;
-    cond_jump->offset = loop - (u_char *) cond_jump;
+    cond_jump->offset = loop_offset - njs_code_offset(parser, cond_jump);
     cond_jump->cond = condition->index;
 
     njs_generate_patch_block_exit(vm, parser);
@@ -907,7 +995,7 @@ static nxt_int_t
 njs_generate_for_statement(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node)
 {
-    u_char                  *loop;
+    njs_ret_t               jump_offset, loop_offset;
     nxt_int_t               ret;
     njs_parser_node_t       *condition, *update;
     njs_vmcode_jump_t       *jump;
@@ -935,6 +1023,9 @@ njs_generate_for_statement(njs_vm_t *vm,
     node = node->right;
     condition = node->left;
 
+    /* GCC complains about uninitialized jump_offset. */
+    jump_offset = 0;
+
     if (condition != NULL) {
         /*
          * The loop condition presents so set a jump to it.  This jump is
@@ -942,6 +1033,7 @@ njs_generate_for_statement(njs_vm_t *vm,
          * execution of one additional jump inside the loop per each iteration.
          */
         njs_generate_code(parser, njs_vmcode_jump_t, jump);
+        jump_offset = njs_code_offset(parser, jump);
         jump->code.operation = njs_vmcode_jump;
         jump->code.operands = NJS_VMCODE_NO_OPERAND;
         jump->code.retval = NJS_VMCODE_NO_RETVAL;
@@ -949,7 +1041,7 @@ njs_generate_for_statement(njs_vm_t *vm,
 
     /* The loop body. */
 
-    loop = parser->code_end;
+    loop_offset = njs_code_offset(parser, parser->code_end);
 
     node = node->right;
 
@@ -977,7 +1069,7 @@ njs_generate_for_statement(njs_vm_t *vm,
     /* The loop condition. */
 
     if (condition != NULL) {
-        jump->offset = parser->code_end - (u_char *) jump;
+        njs_code_set_jump_offset(parser, njs_vmcode_jump_t, jump_offset);
 
         ret = njs_generator(vm, parser, condition);
         if (nxt_slow_path(ret != NXT_OK)) {
@@ -988,7 +1080,7 @@ njs_generate_for_statement(njs_vm_t *vm,
         cond_jump->code.operation = njs_vmcode_if_true_jump;
         cond_jump->code.operands = NJS_VMCODE_2OPERANDS;
         cond_jump->code.retval = NJS_VMCODE_NO_RETVAL;
-        cond_jump->offset = loop - (u_char *) cond_jump;
+        cond_jump->offset = loop_offset - njs_code_offset(parser, cond_jump);
         cond_jump->cond = condition->index;
 
         njs_generate_patch_block_exit(vm, parser);
@@ -1000,7 +1092,7 @@ njs_generate_for_statement(njs_vm_t *vm,
     jump->code.operation = njs_vmcode_jump;
     jump->code.operands = NJS_VMCODE_NO_OPERAND;
     jump->code.retval = NJS_VMCODE_NO_RETVAL;
-    jump->offset = loop - (u_char *) jump;
+    jump->offset = loop_offset - njs_code_offset(parser, jump);
 
     njs_generate_patch_block_exit(vm, parser);
 
@@ -1012,7 +1104,7 @@ static nxt_int_t
 njs_generate_for_in_statement(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node)
 {
-    u_char                     *loop;
+    njs_ret_t                  loop_offset, prop_offset;
     nxt_int_t                  ret;
     njs_index_t                index;
     njs_parser_node_t          *foreach;
@@ -1034,6 +1126,7 @@ njs_generate_for_in_statement(njs_vm_t *
     }
 
     njs_generate_code(parser, njs_vmcode_prop_foreach_t, prop_foreach);
+    prop_offset = njs_code_offset(parser, prop_foreach);
     prop_foreach->code.operation = njs_vmcode_property_foreach;
     prop_foreach->code.operands = NJS_VMCODE_2OPERANDS;
     prop_foreach->code.retval = NJS_VMCODE_RETVAL;
@@ -1048,7 +1141,7 @@ njs_generate_for_in_statement(njs_vm_t *
 
     /* The loop body. */
 
-    loop = parser->code_end;
+    loop_offset = njs_code_offset(parser, parser->code_end);
 
     ret = njs_generator(vm, parser, node->right);
     if (nxt_slow_path(ret != NXT_OK)) {
@@ -1059,7 +1152,7 @@ njs_generate_for_in_statement(njs_vm_t *
 
     njs_generate_patch_loop_continuation(vm, parser);
 
-    prop_foreach->offset = parser->code_end - (u_char *) prop_foreach;
+    njs_code_set_jump_offset(parser, njs_vmcode_prop_foreach_t, prop_offset);
 
     ret = njs_generator(vm, parser, node->left->left);
     if (nxt_slow_path(ret != NXT_OK)) {
@@ -1067,13 +1160,14 @@ njs_generate_for_in_statement(njs_vm_t *
     }
 
     njs_generate_code(parser, njs_vmcode_prop_next_t, prop_next);
+    prop_offset = njs_code_offset(parser, prop_next);
     prop_next->code.operation = njs_vmcode_property_next;
     prop_next->code.operands = NJS_VMCODE_3OPERANDS;
     prop_next->code.retval = NJS_VMCODE_NO_RETVAL;
     prop_next->retval = foreach->left->index;
     prop_next->object = foreach->right->index;
     prop_next->next = index;
-    prop_next->offset = loop - (u_char *) prop_next;
+    prop_next->offset = loop_offset - prop_offset;
 
     njs_generate_patch_block_exit(vm, parser);
 
@@ -1123,7 +1217,7 @@ njs_generate_patch_loop_continuation(njs
     block = parser->block;
 
     for (patch = block->continuation; patch != NULL; patch = next) {
-        *patch->address += parser->code_end - (u_char *) patch->address;
+        njs_code_update_offset(parser, patch);
         next = patch->next;
 
         nxt_mem_cache_free(vm->mem_cache_pool, patch);
@@ -1141,7 +1235,7 @@ njs_generate_patch_block_exit(njs_vm_t *
     parser->block = block->next;
 
     for (patch = block->exit; patch != NULL; patch = next) {
-        *patch->address += parser->code_end - (u_char *) patch->address;
+        njs_code_update_offset(parser, patch);
         next = patch->next;
 
         nxt_mem_cache_free(vm->mem_cache_pool, patch);
@@ -1185,7 +1279,8 @@ found:
         jump->code.retval = NJS_VMCODE_NO_RETVAL;
         jump->offset = offsetof(njs_vmcode_jump_t, offset);
 
-        patch->address = &jump->offset;
+        patch->jump_offset = njs_code_offset(parser, jump)
+                             + offsetof(njs_vmcode_jump_t, offset);
     }
 
     return NXT_OK;
@@ -1228,7 +1323,8 @@ found:
         jump->code.retval = NJS_VMCODE_NO_RETVAL;
         jump->offset = offsetof(njs_vmcode_jump_t, offset);
 
-        patch->address = &jump->offset;
+        patch->jump_offset = njs_code_offset(parser, jump)
+                             + offsetof(njs_vmcode_jump_t, offset);
     }
 
     return NXT_OK;
@@ -1685,6 +1781,7 @@ static nxt_int_t
 njs_generate_test_jump_expression(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node)
 {
+    njs_ret_t               jump_offset;
     nxt_int_t               ret;
     njs_vmcode_move_t       *move;
     njs_vmcode_test_jump_t  *test_jump;
@@ -1695,6 +1792,7 @@ njs_generate_test_jump_expression(njs_vm
     }
 
     njs_generate_code(parser, njs_vmcode_test_jump_t, test_jump);
+    jump_offset = njs_code_offset(parser, test_jump);
     test_jump->code.operation = node->u.operation;
     test_jump->code.operands = NJS_VMCODE_2OPERANDS;
     test_jump->code.retval = NJS_VMCODE_RETVAL;
@@ -1727,7 +1825,7 @@ njs_generate_test_jump_expression(njs_vm
         move->src = node->right->index;
     }
 
-    test_jump->offset = parser->code_end - (u_char *) test_jump;
+    njs_code_set_jump_offset(parser, njs_vmcode_test_jump_t, jump_offset);
 
     return njs_generator_children_indexes_release(vm, parser, node);
 }
@@ -2040,9 +2138,6 @@ njs_generate_function_scope(njs_vm_t *vm
     parser = lambda->parser;
     node = node->right;
 
-    parser->code_size += node->scope->argument_closures
-                         * sizeof(njs_vmcode_move_t);
-
     parser->arguments_object = 0;
     ret = njs_generate_scope(vm, parser, node);
 
@@ -2074,8 +2169,9 @@ nxt_int_t
 njs_generate_scope(njs_vm_t *vm, njs_parser_t *parser, njs_parser_node_t *node)
 {
     u_char              *p;
-    size_t              code_size, size;
+    size_t              size;
     uintptr_t           scope_size;
+    nxt_int_t           ret;
     nxt_uint_t          n;
     njs_value_t         *value;
     njs_vm_code_t       *code;
@@ -2083,6 +2179,8 @@ njs_generate_scope(njs_vm_t *vm, njs_par
 
     scope = node->scope;
 
+    parser->code_size = 128;
+
     p = nxt_mem_cache_alloc(vm->mem_cache_pool, parser->code_size);
     if (nxt_slow_path(p == NULL)) {
         return NXT_ERROR;
@@ -2091,22 +2189,16 @@ njs_generate_scope(njs_vm_t *vm, njs_par
     parser->code_start = p;
     parser->code_end = p;
 
-    njs_generate_argument_closures(parser, node);
+    ret = njs_generate_argument_closures(vm, parser, node);
+    if (nxt_slow_path(ret != NXT_OK)) {
+        return NXT_ERROR;
+    }
 
     if (nxt_slow_path(njs_generator(vm, parser, node) != NXT_OK)) {
         return NXT_ERROR;
     }
 
-    code_size = parser->code_end - parser->code_start;
-
-    nxt_thread_log_debug("SCOPE CODE SIZE: %uz %uz",
-                         parser->code_size, code_size);
-
-    if (nxt_slow_path(parser->code_size < code_size)) {
-        njs_internal_error(vm, "code size mismatch expected %uz < actual %uz",
-                           parser->code_size, code_size);
-        return NXT_ERROR;
-    }
+    parser->code_size = parser->code_end - parser->code_start;
 
     scope_size = njs_scope_offset(scope->next_index[0]);
 
@@ -2152,8 +2244,9 @@ njs_generate_scope(njs_vm_t *vm, njs_par
 }
 
 
-static void
-njs_generate_argument_closures(njs_parser_t *parser, njs_parser_node_t *node)
+static nxt_int_t
+njs_generate_argument_closures(njs_vm_t *vm, njs_parser_t *parser,
+    njs_parser_node_t *node)
 {
     nxt_uint_t         n;
     njs_index_t        index;
@@ -2164,7 +2257,7 @@ njs_generate_argument_closures(njs_parse
     n = node->scope->argument_closures;
 
     if (n == 0) {
-        return;
+        return NXT_OK;
     }
 
     nxt_lvlhsh_each_init(&lhe, &njs_variables_hash_proto);
@@ -2186,6 +2279,8 @@ njs_generate_argument_closures(njs_parse
         }
 
     } while(n != 0);
+
+    return NXT_OK;
 }
 
 
@@ -2224,6 +2319,7 @@ static nxt_int_t
 njs_generate_function_call(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node)
 {
+    njs_ret_t                    func_offset;
     njs_ret_t                    ret;
     njs_parser_node_t            *name;
     njs_vmcode_function_frame_t  *func;
@@ -2246,6 +2342,7 @@ njs_generate_function_call(njs_vm_t *vm,
     }
 
     njs_generate_code(parser, njs_vmcode_function_frame_t, func);
+    func_offset = njs_code_offset(parser, func);
     func->code.operation = njs_vmcode_function_frame;
     func->code.operands = NJS_VMCODE_2OPERANDS;
     func->code.retval = NJS_VMCODE_NO_RETVAL;
@@ -2255,6 +2352,7 @@ njs_generate_function_call(njs_vm_t *vm,
     ret = njs_generate_call(vm, parser, node);
 
     if (nxt_fast_path(ret >= 0)) {
+        func = njs_code_ptr(parser, njs_vmcode_function_frame_t, func_offset);
         func->nargs = ret;
         return NXT_OK;
     }
@@ -2267,6 +2365,7 @@ static nxt_int_t
 njs_generate_method_call(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node)
 {
+    njs_ret_t                  method_offset;
     nxt_int_t                  ret;
     njs_parser_node_t          *prop;
     njs_vmcode_method_frame_t  *method;
@@ -2288,6 +2387,7 @@ njs_generate_method_call(njs_vm_t *vm, n
     }
 
     njs_generate_code(parser, njs_vmcode_method_frame_t, method);
+    method_offset = njs_code_offset(parser, method);
     method->code.operation = njs_vmcode_method_frame;
     method->code.operands = NJS_VMCODE_3OPERANDS;
     method->code.retval = NJS_VMCODE_NO_RETVAL;
@@ -2303,6 +2403,7 @@ njs_generate_method_call(njs_vm_t *vm, n
     ret = njs_generate_call(vm, parser, node);
 
     if (nxt_fast_path(ret >= 0)) {
+        method = njs_code_ptr(parser, njs_vmcode_method_frame_t, method_offset);
         method->nargs = ret;
         return NXT_OK;
     }
@@ -2362,6 +2463,7 @@ static nxt_int_t
 njs_generate_try_statement(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node)
 {
+    njs_ret_t               try_offset, catch_offset;
     nxt_int_t               ret;
     njs_index_t             index, catch_index;
     njs_vmcode_catch_t      *catch;
@@ -2370,6 +2472,7 @@ njs_generate_try_statement(njs_vm_t *vm,
     njs_vmcode_try_start_t  *try_start;
 
     njs_generate_code(parser, njs_vmcode_try_start_t, try_start);
+    try_offset = njs_code_offset(parser, try_start);
     try_start->code.operation = njs_vmcode_try_start;
     try_start->code.operands = NJS_VMCODE_2OPERANDS;
     try_start->code.retval = NJS_VMCODE_NO_RETVAL;
@@ -2391,7 +2494,8 @@ njs_generate_try_statement(njs_vm_t *vm,
     try_end->code.operands = NJS_VMCODE_NO_OPERAND;
     try_end->code.retval = NJS_VMCODE_NO_RETVAL;
 
-    try_start->offset = parser->code_end - (u_char *) try_start;
+    njs_code_set_jump_offset(parser, njs_vmcode_try_start_t, try_offset);
+    try_offset = njs_code_offset(parser, try_end);
 
     node = node->right;
 
@@ -2415,7 +2519,7 @@ njs_generate_try_statement(njs_vm_t *vm,
             return ret;
         }
 
-        try_end->offset = parser->code_end - (u_char *) try_end;
+        njs_code_set_jump_offset(parser, njs_vmcode_try_end_t, try_offset);
 
         /* TODO: release exception variable index. */
 
@@ -2429,6 +2533,7 @@ njs_generate_try_statement(njs_vm_t *vm,
             }
 
             njs_generate_code(parser, njs_vmcode_catch_t, catch);
+            catch_offset = njs_code_offset(parser, catch);
             catch->code.operation = njs_vmcode_catch;
             catch->code.operands = NJS_VMCODE_2OPERANDS;
             catch->code.retval = NJS_VMCODE_NO_RETVAL;
@@ -2444,7 +2549,8 @@ njs_generate_try_statement(njs_vm_t *vm,
             catch_end->code.operands = NJS_VMCODE_NO_OPERAND;
             catch_end->code.retval = NJS_VMCODE_NO_RETVAL;
 
-            catch->offset = parser->code_end - (u_char *) catch;
+            njs_code_set_jump_offset(parser, njs_vmcode_catch_t, catch_offset);
+            catch_offset = njs_code_offset(parser, catch_end);
 
             /* TODO: release exception variable index. */
 
@@ -2455,7 +2561,8 @@ njs_generate_try_statement(njs_vm_t *vm,
             catch->offset = sizeof(njs_vmcode_catch_t);
             catch->exception = index;
 
-            catch_end->offset = parser->code_end - (u_char *) catch_end;
+            njs_code_set_jump_offset(parser, njs_vmcode_try_end_t,
+                                     catch_offset);
 
         } else {
             /* A try/finally case. */
@@ -2468,7 +2575,7 @@ njs_generate_try_statement(njs_vm_t *vm,
             catch->exception = index;
         }
 
-        try_end->offset = parser->code_end - (u_char *) try_end;
+        njs_code_set_jump_offset(parser, njs_vmcode_try_end_t, try_offset);
 
         ret = njs_generator(vm, parser, node->right);
         if (nxt_slow_path(ret != NXT_OK)) {
diff -r 23fecf3fcc60 -r 8eb112dcca79 njs/njs_generator.h
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/njs/njs_generator.h	Fri Dec 07 17:11:33 2018 +0300
@@ -0,0 +1,15 @@
+
+/*
+ * Copyright (C) Dmitry Volyntsev
+ * Copyright (C) NGINX, Inc.
+ */
+
+#ifndef _NJS_GENERATOR_H_INCLUDED_
+#define _NJS_GENERATOR_H_INCLUDED_
+
+
+nxt_int_t njs_generate_scope(njs_vm_t *vm, njs_parser_t *parser,
+    njs_parser_node_t *node);
+
+
+#endif /* _NJS_GENERATOR_H_INCLUDED_ */
diff -r 23fecf3fcc60 -r 8eb112dcca79 njs/njs_parser.c
--- a/njs/njs_parser.c	Wed Dec 05 15:38:33 2018 +0300
+++ b/njs/njs_parser.c	Fri Dec 07 17:11:33 2018 +0300
@@ -523,7 +523,6 @@ njs_parser_function_expression(njs_vm_t 
     node->token_line = parser->lexer->token_line;
     node->scope = parser->scope;
     parser->node = node;
-    parser->code_size += sizeof(njs_vmcode_function_t);
 
     parser = njs_parser_function_create(vm, parser);
     if (nxt_slow_path(parser == NULL)) {
@@ -732,8 +731,6 @@ njs_parser_function_lambda(njs_vm_t *vm,
         }
 
         node->right->token = NJS_TOKEN_RETURN;
-
-        parser->code_size += sizeof(njs_vmcode_return_t);
     }
 
     parser->parent->node->right = parser->node;
@@ -770,7 +767,6 @@ njs_parser_return_statement(njs_vm_t *vm
 
     node->token = NJS_TOKEN_RETURN;
     parser->node = node;
-    parser->code_size += sizeof(njs_vmcode_return_t);
 
     token = njs_lexer_token(parser->lexer);
     if (nxt_slow_path(token <= NJS_TOKEN_ILLEGAL)) {
@@ -891,7 +887,6 @@ njs_parser_var_statement(njs_vm_t *vm, n
         stmt->left = left;
         stmt->right = assign;
         parser->node = stmt;
-        parser->code_size += sizeof(njs_vmcode_2addr_t);
 
         left = stmt;
 
@@ -942,7 +937,6 @@ njs_parser_if_statement(njs_vm_t *vm, nj
         node->left = stmt;
         node->right = parser->node;
         parser->node = node;
-        parser->code_size += sizeof(njs_vmcode_jump_t);
     }
 
     node = njs_parser_node_alloc(vm);
@@ -954,7 +948,6 @@ njs_parser_if_statement(njs_vm_t *vm, nj
     node->left = cond;
     node->right = parser->node;
     parser->node = node;
-    parser->code_size += sizeof(njs_vmcode_cond_jump_t);
 
     return token;
 }
@@ -1020,8 +1013,6 @@ njs_parser_switch_statement(njs_vm_t *vm
 
                     branch->right = node;
 
-                    parser->code_size += sizeof(njs_vmcode_equal_jump_t);
-
                } else {
                     if (dflt != NULL) {
                         njs_parser_syntax_error(vm, parser,
@@ -1071,7 +1062,6 @@ njs_parser_switch_statement(njs_vm_t *vm
     }
 
     parser->node = swtch;
-    parser->code_size += sizeof(njs_vmcode_move_t) + sizeof(njs_vmcode_jump_t);
 
     return njs_parser_token(parser);
 }
@@ -1104,8 +1094,6 @@ njs_parser_while_statement(njs_vm_t *vm,
     node->left = parser->node;
     node->right = cond;
     parser->node = node;
-    parser->code_size += sizeof(njs_vmcode_jump_t)
-                         + sizeof(njs_vmcode_cond_jump_t);
 
     return token;
 }
@@ -1147,7 +1135,6 @@ njs_parser_do_while_statement(njs_vm_t *
     node->left = stmt;
     node->right = parser->node;
     parser->node = node;
-    parser->code_size += sizeof(njs_vmcode_cond_jump_t);
 
     return token;
 }
@@ -1272,8 +1259,7 @@ njs_parser_for_statement(njs_vm_t *vm, n
     body->right = update;
 
     parser->node = node;
-    parser->code_size += sizeof(njs_vmcode_jump_t)
-                         + sizeof(njs_vmcode_cond_jump_t);
+
     return token;
 }
 
@@ -1368,7 +1354,6 @@ njs_parser_for_var_statement(njs_vm_t *v
         stmt->left = left;
         stmt->right = assign;
         parser->node = stmt;
-        parser->code_size += sizeof(njs_vmcode_2addr_t);
 
         left = stmt;
 
@@ -1424,8 +1409,7 @@ njs_parser_for_var_in_statement(njs_vm_t
     foreach->right = parser->node;
 
     parser->node = foreach;
-    parser->code_size += sizeof(njs_vmcode_prop_foreach_t)
-                         + sizeof(njs_vmcode_prop_next_t);
+
     return token;
 }
 
@@ -1466,8 +1450,7 @@ njs_parser_for_in_statement(njs_vm_t *vm
 
     node->right = parser->node;
     parser->node = node;
-    parser->code_size += sizeof(njs_vmcode_prop_foreach_t)
-                         + sizeof(njs_vmcode_prop_next_t);
+
     return token;
 }
 
@@ -1486,7 +1469,6 @@ njs_parser_continue_statement(njs_vm_t *
     node->token = NJS_TOKEN_CONTINUE;
     node->token_line = parser->lexer->token_line;
     parser->node = node;
-    parser->code_size += sizeof(njs_vmcode_jump_t);
 
     token = njs_lexer_token(parser->lexer);
 
@@ -1521,7 +1503,6 @@ njs_parser_break_statement(njs_vm_t *vm,
     node->token = NJS_TOKEN_BREAK;
     node->token_line = parser->lexer->token_line;
     parser->node = node;
-    parser->code_size += sizeof(njs_vmcode_jump_t);
 
     token = njs_lexer_token(parser->lexer);
 
@@ -1563,8 +1544,6 @@ njs_parser_try_statement(njs_vm_t *vm, n
     try->token = NJS_TOKEN_TRY;
     try->scope = parser->scope;
     try->left = parser->node;
-    parser->code_size += sizeof(njs_vmcode_try_start_t)
-                         + sizeof(njs_vmcode_try_end_t);
 
     if (token == NJS_TOKEN_CATCH) {
         token = njs_parser_token(parser);
@@ -1617,8 +1596,6 @@ njs_parser_try_statement(njs_vm_t *vm, n
 
         catch->left = node;
 
-        parser->code_size += sizeof(njs_vmcode_catch_t);
-
         token = njs_parser_token(parser);
         if (nxt_slow_path(token <= NJS_TOKEN_ILLEGAL)) {
             return token;
@@ -1655,12 +1632,9 @@ njs_parser_try_statement(njs_vm_t *vm, n
 
         if (try->right != NULL) {
             node->left = try->right;
-            parser->code_size += sizeof(njs_vmcode_try_end_t);
         }
 
         try->right = node;
-        parser->code_size += sizeof(njs_vmcode_catch_t)
-                             + sizeof(njs_vmcode_finally_t);
     }
 
     if (try->right == NULL) {
@@ -1842,7 +1816,6 @@ njs_parser_terminal(njs_vm_t *vm, njs_pa
             return NJS_TOKEN_ERROR;
         }
 
-        parser->code_size += sizeof(njs_vmcode_object_copy_t);
         break;
 
     case NJS_TOKEN_OPEN_BRACE:
@@ -1861,8 +1834,6 @@ njs_parser_terminal(njs_vm_t *vm, njs_pa
             parser->node = node;
         }
 
-        parser->code_size += sizeof(njs_vmcode_1addr_t);
-
         return token;
 
     case NJS_TOKEN_OPEN_BRACKET:
@@ -1881,8 +1852,6 @@ njs_parser_terminal(njs_vm_t *vm, njs_pa
             parser->node = node;
         }
 
-        parser->code_size += sizeof(njs_vmcode_2addr_t);
-
         return token;
 
     case NJS_TOKEN_DIVISION:
@@ -1895,7 +1864,6 @@ njs_parser_terminal(njs_vm_t *vm, njs_pa
 
         node->token = NJS_TOKEN_REGEXP;
         node->scope = parser->scope;
-        parser->code_size += sizeof(njs_vmcode_regexp_t);
 
         break;
 
@@ -2003,8 +1971,6 @@ njs_parser_terminal(njs_vm_t *vm, njs_pa
 
         node->token = NJS_TOKEN_ARGUMENTS;
 
-        parser->code_size += sizeof(njs_vmcode_arguments_t);
-
         break;
 
     case NJS_TOKEN_OBJECT_CONSTRUCTOR:
@@ -2127,7 +2093,6 @@ njs_parser_builtin_object(njs_vm_t *vm, 
     node->scope = parser->scope;
 
     parser->node = node;
-    parser->code_size += sizeof(njs_vmcode_object_copy_t);
 
     return njs_parser_token(parser);
 }
@@ -2160,7 +2125,6 @@ njs_parser_builtin_function(njs_vm_t *vm
     node->scope = parser->scope;
 
     parser->node = node;
-    parser->code_size += sizeof(njs_vmcode_object_copy_t);
 
     return njs_parser_token(parser);
 }
@@ -2207,7 +2171,6 @@ njs_parser_object(njs_vm_t *vm, njs_pars
         propref->token = NJS_TOKEN_PROPERTY;
         propref->left = object;
         propref->right = parser->node;
-        parser->code_size += sizeof(njs_vmcode_3addr_t);
 
         if (nxt_slow_path(token <= NJS_TOKEN_ILLEGAL)) {
             return token;
@@ -2242,7 +2205,6 @@ njs_parser_object(njs_vm_t *vm, njs_pars
         stmt->left = left;
         stmt->right = assign;
 
-        parser->code_size += sizeof(njs_vmcode_2addr_t);
         parser->node = stmt;
 
         left = stmt;
@@ -2311,7 +2273,6 @@ njs_parser_array(njs_vm_t *vm, njs_parse
         propref->token = NJS_TOKEN_PROPERTY;
         propref->left = object;
         propref->right = node;
-        parser->code_size += sizeof(njs_vmcode_3addr_t);
 
         token = njs_parser_assignment_expression(vm, parser, token);
         if (nxt_slow_path(token <= NJS_TOKEN_ILLEGAL)) {
diff -r 23fecf3fcc60 -r 8eb112dcca79 njs/njs_parser.h
--- a/njs/njs_parser.h	Wed Dec 05 15:38:33 2018 +0300
+++ b/njs/njs_parser.h	Fri Dec 07 17:11:33 2018 +0300

From xeioex at nginx.com  Fri Dec  7 15:56:23 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 07 Dec 2018 15:56:23 +0000
Subject: [njs] Freeing code buffer after is was reallocated.
Message-ID: <hg.bc7a47f35645.1544198183.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/bc7a47f35645
branches:  
changeset: 688:bc7a47f35645
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Dec 07 18:33:52 2018 +0300
description:
Freeing code buffer after is was reallocated.

Thanks to ??? (Hong Zhi Dao).

diffstat:

 njs/njs_generator.c |  4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diffs (21 lines):

diff -r 2a13aff6aa1f -r bc7a47f35645 njs/njs_generator.c
--- a/njs/njs_generator.c	Fri Dec 07 17:15:21 2018 +0300
+++ b/njs/njs_generator.c	Fri Dec 07 18:33:52 2018 +0300
@@ -452,6 +452,8 @@ njs_generate_reserve(njs_vm_t *vm, njs_g
     size = generator->code_end - generator->code_start;
     memcpy(p, generator->code_start, size);
 
+    nxt_mem_cache_free(vm->mem_cache_pool, generator->code_start);
+
     generator->code_start = p;
     generator->code_end = p + size;
 
@@ -2223,6 +2225,8 @@ njs_generate_function_scope(njs_vm_t *vm
         lambda->start = generator->code_start;
     }
 
+    nxt_mem_cache_free(vm->mem_cache_pool, generator);
+
     return ret;
 }
 

From xeioex at nginx.com  Fri Dec  7 15:56:24 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 07 Dec 2018 15:56:24 +0000
Subject: [njs] Removing unused "size" field from njs_parser_operation_t.
Message-ID: <hg.837f82cdd3ba.1544198184.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/837f82cdd3ba
branches:  
changeset: 689:837f82cdd3ba
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Dec 07 18:54:59 2018 +0300
description:
Removing unused "size" field from njs_parser_operation_t.

Thanks to ??? (Hong Zhi Dao).

This closes #68 issue on Github.

diffstat:

 njs/njs_parser_expression.c |  72 +++++++++++++++------------------------------
 1 files changed, 24 insertions(+), 48 deletions(-)

diffs (159 lines):

diff -r bc7a47f35645 -r 837f82cdd3ba njs/njs_parser_expression.c
--- a/njs/njs_parser_expression.c	Fri Dec 07 18:33:52 2018 +0300
+++ b/njs/njs_parser_expression.c	Fri Dec 07 18:54:59 2018 +0300
@@ -11,7 +11,6 @@
 typedef struct {
     njs_token_t                    token;
     njs_vmcode_operation_t         operation;
-    size_t                         size;
 } njs_parser_operation_t;
 
 
@@ -71,12 +70,9 @@ static const njs_parser_expression_t
     njs_parser_exponential_expression,
     NULL,
     3, {
-        { NJS_TOKEN_MULTIPLICATION, njs_vmcode_multiplication,
-          sizeof(njs_vmcode_3addr_t) },
-        { NJS_TOKEN_DIVISION, njs_vmcode_division,
-          sizeof(njs_vmcode_3addr_t) },
-        { NJS_TOKEN_REMAINDER, njs_vmcode_remainder,
-          sizeof(njs_vmcode_3addr_t) },
+        { NJS_TOKEN_MULTIPLICATION, njs_vmcode_multiplication },
+        { NJS_TOKEN_DIVISION, njs_vmcode_division },
+        { NJS_TOKEN_REMAINDER, njs_vmcode_remainder },
     }
 };
 
@@ -87,10 +83,8 @@ static const njs_parser_expression_t
     njs_parser_binary_expression,
     &njs_parser_factor_expression,
     2, {
-        { NJS_TOKEN_ADDITION, njs_vmcode_addition,
-          sizeof(njs_vmcode_3addr_t) },
-        { NJS_TOKEN_SUBSTRACTION, njs_vmcode_substraction,
-          sizeof(njs_vmcode_3addr_t) },
+        { NJS_TOKEN_ADDITION, njs_vmcode_addition },
+        { NJS_TOKEN_SUBSTRACTION, njs_vmcode_substraction },
     }
 };
 
@@ -101,12 +95,9 @@ static const njs_parser_expression_t
     njs_parser_binary_expression,
     &njs_parser_addition_expression,
     3, {
-        { NJS_TOKEN_LEFT_SHIFT, njs_vmcode_left_shift,
-          sizeof(njs_vmcode_3addr_t) },
-        { NJS_TOKEN_RIGHT_SHIFT, njs_vmcode_right_shift,
-          sizeof(njs_vmcode_3addr_t) },
-        { NJS_TOKEN_UNSIGNED_RIGHT_SHIFT, njs_vmcode_unsigned_right_shift,
-          sizeof(njs_vmcode_3addr_t) },
+        { NJS_TOKEN_LEFT_SHIFT, njs_vmcode_left_shift },
+        { NJS_TOKEN_RIGHT_SHIFT, njs_vmcode_right_shift },
+        { NJS_TOKEN_UNSIGNED_RIGHT_SHIFT, njs_vmcode_unsigned_right_shift },
     }
 };
 
@@ -117,18 +108,12 @@ static const njs_parser_expression_t
     njs_parser_binary_expression,
     &njs_parser_bitwise_shift_expression,
     6, {
-        { NJS_TOKEN_LESS, njs_vmcode_less,
-          sizeof(njs_vmcode_3addr_t) },
-        { NJS_TOKEN_LESS_OR_EQUAL, njs_vmcode_less_or_equal,
-          sizeof(njs_vmcode_3addr_t) },
-        { NJS_TOKEN_GREATER, njs_vmcode_greater,
-          sizeof(njs_vmcode_3addr_t) },
-        { NJS_TOKEN_GREATER_OR_EQUAL, njs_vmcode_greater_or_equal,
-          sizeof(njs_vmcode_3addr_t) },
-        { NJS_TOKEN_IN, njs_vmcode_property_in,
-          sizeof(njs_vmcode_3addr_t) },
-        { NJS_TOKEN_INSTANCEOF, njs_vmcode_instance_of,
-          sizeof(njs_vmcode_3addr_t) },
+        { NJS_TOKEN_LESS, njs_vmcode_less },
+        { NJS_TOKEN_LESS_OR_EQUAL, njs_vmcode_less_or_equal },
+        { NJS_TOKEN_GREATER, njs_vmcode_greater },
+        { NJS_TOKEN_GREATER_OR_EQUAL, njs_vmcode_greater_or_equal },
+        { NJS_TOKEN_IN, njs_vmcode_property_in },
+        { NJS_TOKEN_INSTANCEOF, njs_vmcode_instance_of },
     }
 };
 
@@ -139,14 +124,10 @@ static const njs_parser_expression_t
     njs_parser_binary_expression,
     &njs_parser_relational_expression,
     4, {
-        { NJS_TOKEN_EQUAL, njs_vmcode_equal,
-          sizeof(njs_vmcode_3addr_t) },
-        { NJS_TOKEN_NOT_EQUAL, njs_vmcode_not_equal,
-          sizeof(njs_vmcode_3addr_t) },
-        { NJS_TOKEN_STRICT_EQUAL, njs_vmcode_strict_equal,
-          sizeof(njs_vmcode_3addr_t) },
-        { NJS_TOKEN_STRICT_NOT_EQUAL, njs_vmcode_strict_not_equal,
-          sizeof(njs_vmcode_3addr_t) },
+        { NJS_TOKEN_EQUAL, njs_vmcode_equal },
+        { NJS_TOKEN_NOT_EQUAL, njs_vmcode_not_equal },
+        { NJS_TOKEN_STRICT_EQUAL, njs_vmcode_strict_equal },
+        { NJS_TOKEN_STRICT_NOT_EQUAL, njs_vmcode_strict_not_equal },
     }
 };
 
@@ -157,8 +138,7 @@ static const njs_parser_expression_t
     njs_parser_binary_expression,
     &njs_parser_equality_expression,
     1, {
-        { NJS_TOKEN_BITWISE_AND, njs_vmcode_bitwise_and,
-          sizeof(njs_vmcode_3addr_t) },
+        { NJS_TOKEN_BITWISE_AND, njs_vmcode_bitwise_and },
     }
 };
 
@@ -169,8 +149,7 @@ static const njs_parser_expression_t
     njs_parser_binary_expression,
     &njs_parser_bitwise_and_expression,
     1, {
-        { NJS_TOKEN_BITWISE_XOR, njs_vmcode_bitwise_xor,
-          sizeof(njs_vmcode_3addr_t) },
+        { NJS_TOKEN_BITWISE_XOR, njs_vmcode_bitwise_xor },
     }
 };
 
@@ -181,8 +160,7 @@ static const njs_parser_expression_t
     njs_parser_binary_expression,
     &njs_parser_bitwise_xor_expression,
     1, {
-        { NJS_TOKEN_BITWISE_OR, njs_vmcode_bitwise_or,
-          sizeof(njs_vmcode_3addr_t) },
+        { NJS_TOKEN_BITWISE_OR, njs_vmcode_bitwise_or },
     }
 };
 
@@ -193,8 +171,7 @@ static const njs_parser_expression_t
     njs_parser_binary_expression,
     &njs_parser_bitwise_or_expression,
     1, {
-        { NJS_TOKEN_LOGICAL_AND, njs_vmcode_test_if_false,
-          sizeof(njs_vmcode_test_jump_t) + sizeof(njs_vmcode_move_t) },
+        { NJS_TOKEN_LOGICAL_AND, njs_vmcode_test_if_false },
     }
 };
 
@@ -205,8 +182,7 @@ static const njs_parser_expression_t
     njs_parser_binary_expression,
     &njs_parser_logical_and_expression,
     1, {
-        { NJS_TOKEN_LOGICAL_OR, njs_vmcode_test_if_true,
-          sizeof(njs_vmcode_test_jump_t) + sizeof(njs_vmcode_move_t) },
+        { NJS_TOKEN_LOGICAL_OR, njs_vmcode_test_if_true },
     }
 };
 
@@ -217,7 +193,7 @@ static const njs_parser_expression_t
     njs_parser_any_expression,
     NULL,
     1, {
-        { NJS_TOKEN_COMMA, NULL, 0 },
+        { NJS_TOKEN_COMMA, NULL },
     }
 };
 

From xeioex at nginx.com  Fri Dec  7 16:00:06 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 07 Dec 2018 16:00:06 +0000
Subject: [njs] Style.
Message-ID: <hg.0709c3d38212.1544198406.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/0709c3d38212
branches:  
changeset: 690:0709c3d38212
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Dec 07 18:58:27 2018 +0300
description:
Style.

diffstat:

 njs/njs_generator.c |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff -r 837f82cdd3ba -r 0709c3d38212 njs/njs_generator.c
--- a/njs/njs_generator.c	Fri Dec 07 18:54:59 2018 +0300
+++ b/njs/njs_generator.c	Fri Dec 07 18:58:27 2018 +0300
@@ -438,7 +438,7 @@ njs_generate_reserve(njs_vm_t *vm, njs_g
         size *= 2;
 
     } else {
-        size += size/ 2;
+        size += size / 2;
     }
 
     p = nxt_mem_cache_alloc(vm->mem_cache_pool, size);

From miranovy at gmail.com  Mon Dec 10 13:30:57 2018
From: miranovy at gmail.com (Miroslav Novy)
Date: Mon, 10 Dec 2018 14:30:57 +0100
Subject: limit_rate_after support variables
In-Reply-To: <20181206144522.GF50234@lo0.su>
References: <CAEx3=QEMKkoiAve-=fn86tS-3vcTNkS=Ukju7yf1zzWHh0jHJA@mail.gmail.com>
 <20180827112847.GE71000@lo0.su>
 <CAEx3=QEQGLn91h85PrQro29kcr4TQGkxmaJgQHTOWXQbTF=6tw@mail.gmail.com>
 <20180827160021.GG71000@lo0.su>
 <CAEx3=QGR8X1QJ__azWi5jYQnkM8DAd740Sn6LHNsosFjvpYtcA@mail.gmail.com>
 <20180829114230.GC43480@lo0.su>
 <CAEx3=QFxT_rngL64H+83tG3JntcVQ6Du+VyVBzVXipK5Epjj2A@mail.gmail.com>
 <CAEx3=QHDuWyVf+tw0WDNY82hbt2BPJvBsUdURe9MUoyO-C0wyw@mail.gmail.com>
 <20181120141005.GA72348@lo0.su>
 <CAEx3=QFVHn2-h=cj=rkVrgDNZsEGxzxsKa2j_smbW6gg-AbsLQ@mail.gmail.com>
 <20181206144522.GF50234@lo0.su>
Message-ID: <CAEx3=QG=ZtMfjP4kO7HVF8W07phqZYFdZQ6WAf9_nJ-AnvOe2w@mail.gmail.com>

Hello,
  your path is very good. I tested it and works fine for as.

Reading complex value to the same local variable limit_rate was conflusing
for me at first time. But it is correct :)

 if (r->limit_rate_after == 0
+            && clcf->limit_rate_after
+            && ngx_http_complex_value_size(r, clcf->limit_rate_after, &val,
+                                           &limit_rate)
<-----------------------
+               == NGX_OK)
+        {
+            if (limit_rate != (size_t) NGX_ERROR) {
+                r->limit_rate_after = limit_rate;
+
+            } else if (val.len) {

Please, merge your path to master branch.

Best regards
M. Nov?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20181210/f087f962/attachment.html>

From mdounin at mdounin.ru  Mon Dec 10 17:35:23 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 10 Dec 2018 20:35:23 +0300
Subject: Patch to make 404 pages etc and autoindex pages use valid HTML
In-Reply-To: <CAA0nTRvFYOf89_PPdP7SxzGGo2XhBCtf91vQtMq8PgjSHUVrJQ@mail.gmail.com>
References: <CAA0nTRv3wsv8oRFNgeWMyhxwqkgHSEci9cB8fefhjGe-qm3--Q@mail.gmail.com>
 <20181128163108.GB99070@mdounin.ru>
 <CAA0nTRvFYOf89_PPdP7SxzGGo2XhBCtf91vQtMq8PgjSHUVrJQ@mail.gmail.com>
Message-ID: <20181210173522.GT99070@mdounin.ru>

Hello!

On Thu, Dec 06, 2018 at 06:42:23AM -0500, Andras Farkas wrote:

> On Wed, Nov 28, 2018 at 11:31 AM Maxim Dounin <mdounin at mdounin.ru> wrote:
> > Such a change will significantly modify typical rendering of error
> > pages, so I would rather refrain from it unless there are more
> > practical reasons than triggering standards-complaint mode instead
> > of quirks mode - which does no seem to make any difference for
> > pages nginx generates.
> Is having it render differently a bad thing?  The meaning of any page
> changed still remains clear and easy to read.  Invalid HTML (like
> <center>) is so easy to remove and to turn into valid HTML, I don't
> see why not to.

Well, it renders as it is now to make reading of error pages 
easier.  While changing rendering is certainly possible, this is 
something usually justified by better rendering, not just 
by switching to standards mode instead of quirks mode.

Note well that the "<center>" tag is valid HTML 3.2 tag.  While it 
was removed in HTML 5, it does not make it "invalid HTML".  We can 
even make returned HTML perfectly valid by explicitly returning 
appropriate DOCTYPE (yet again there are no practical reasons to 
do so, and this will make error pages bigger).

-- 
Maxim Dounin
http://mdounin.ru/

From mdounin at mdounin.ru  Mon Dec 10 17:38:33 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 10 Dec 2018 20:38:33 +0300
Subject: [PATCH] Clarify proxy modules $proxy_host variable content
In-Reply-To: <2f1f0736-df17-172f-a215-fffa527734ff@mailbox.org>
References: <2f1f0736-df17-172f-a215-fffa527734ff@mailbox.org>
Message-ID: <20181210173833.GU99070@mdounin.ru>

Hello!

On Tue, Dec 04, 2018 at 05:00:10PM +0100, Jack Henschel wrote:

> From the current documentation it was not immediately obvious (at least
> to me and my colleague) that nginx does not set the $proxy_host variable
> to the hosts specified in an upstream block, but literally uses the name
> used in the proxy_pass directive.
> 
> Maxim Dounin has explained to me the reason for this [1], nevertheless I
> think it is worthwhile explicitly adding this behavior to the documentation.

Thank you for your patch.  It looks too explicit for me though.

-- 
Maxim Dounin
http://mdounin.ru/

From mdounin at mdounin.ru  Mon Dec 10 19:07:18 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 10 Dec 2018 22:07:18 +0300
Subject: [PATCH] Apply cache locking behaviour to stale cache entries.
In-Reply-To: <D82ACDE2.2781E%Elliot.Thomas@bbc.co.uk>
References: <D82ACDE2.2781E%Elliot.Thomas@bbc.co.uk>
Message-ID: <20181210190717.GV99070@mdounin.ru>

Hello!

On Mon, Dec 03, 2018 at 12:11:15PM +0000, Elliot Thomas wrote:

> This patch applies cache locking behaviour to stale cache entries, so
>  in the case where the *_cache_lock directives are used, the same
>  locking behaviour is used for stale content as it is for new entries.
> 
> Previously, this was only done for new cache entries.
> (see: http://mailman.nginx.org/pipermail/nginx/2016-January/049734.html)
> 
> This is useful when serving stale content is not permissable but sending
>  many requests upstream is undesriable.
> 
> 
> This patch exposes the ngx_http_file_cache_lock function; this function
>  is then called in ngx_http_upstream if a cache entry has expired.
> 
> I have attached two versions of this patch, a "minimal" one that avoids
>  changes where not strictly necessary, and a "cleaner" version which is
>  more invasive but (in my opinion) cleaner.
> 
> Both patches cause no (additional) failures in the nginx test suite.
> I tested with as many modules as I could reasonably enable.

Thank you for the patches.  The "cleaner" version looks better, 
though it looks like it would be a good idea to clean it even 
more.  See comments below.

[...]

> -----------------------------
> http://www.bbc.co.uk
> This e-mail (and any attachments) is confidential and
> may contain personal views which are not the views of the BBC unless specifically stated.
> If you have received it in
> error, please delete it from your system.
> Do not use, copy or disclose the
> information in any way nor act in reliance on it and notify the sender
> immediately.
> Please note that the BBC monitors e-mails
> sent or received.
> Further communication will signify your consent to
> this.
> -----------------------------

Note that posting patches to a public mailing list with such a 
disclaimer might not be a good idea.  If you cannot remove it, 
please make sure to add an explicit comment that you understand 
that you are posting to a public mailing list, and you've read the 
http://nginx.org/en/docs/contributing_changes.html article.  In 
particular, that you agree with the "License" part.

> # HG changeset patch
> # User Elliot Thomas <elliot.thomas at bbc.co.uk>
> # Date 1543836716 0
> #      Mon Dec 03 11:31:56 2018 +0000
> # Node ID 28084fdcef2873df221b13d1a70ab04242e4edc5
> # Parent  2117637f64e981e0e14c3a4b0509252fefd8a78a
> Apply cache locking behaviour to stale cache entries.
> 
> Previously, this was only done for new cache entries.
> 
> diff -r 2117637f64e9 -r 28084fdcef28 src/http/ngx_http_cache.h
> --- a/src/http/ngx_http_cache.h	Tue Nov 27 17:40:21 2018 +0300
> +++ b/src/http/ngx_http_cache.h	Mon Dec 03 11:31:56 2018 +0000
> @@ -188,6 +188,7 @@
>  ngx_int_t ngx_http_file_cache_create(ngx_http_request_t *r);
>  void ngx_http_file_cache_create_key(ngx_http_request_t *r);
>  ngx_int_t ngx_http_file_cache_open(ngx_http_request_t *r);
> +ngx_int_t ngx_http_file_cache_lock(ngx_http_request_t *r, ngx_int_t rc);
>  ngx_int_t ngx_http_file_cache_set_header(ngx_http_request_t *r, u_char *buf);
>  void ngx_http_file_cache_update(ngx_http_request_t *r, ngx_temp_file_t *tf);
>  void ngx_http_file_cache_update_header(ngx_http_request_t *r);
> diff -r 2117637f64e9 -r 28084fdcef28 src/http/ngx_http_file_cache.c
> --- a/src/http/ngx_http_file_cache.c	Tue Nov 27 17:40:21 2018 +0300
> +++ b/src/http/ngx_http_file_cache.c	Mon Dec 03 11:31:56 2018 +0000
> @@ -11,8 +11,6 @@
>  #include <ngx_md5.h>
>  
>  
> -static ngx_int_t ngx_http_file_cache_lock(ngx_http_request_t *r,
> -    ngx_http_cache_t *c);
>  static void ngx_http_file_cache_lock_wait_handler(ngx_event_t *ev);
>  static void ngx_http_file_cache_lock_wait(ngx_http_request_t *r,
>      ngx_http_cache_t *c);
> @@ -391,22 +389,21 @@
>  
>  done:
>  
> -    if (rv == NGX_DECLINED) {
> -        return ngx_http_file_cache_lock(r, c);
> -    }
> -
>      return rv;
>  }

This changes makes the "done" label useless.  It can be removed, 
and corresponding gotos in the function replaced with explicit 
returns (as it was before the introduction of cache locks in 
revision 70ba81827472, http://hg.nginx.org/nginx/rev/70ba81827472).

> -static ngx_int_t
> -ngx_http_file_cache_lock(ngx_http_request_t *r, ngx_http_cache_t *c)
> +ngx_int_t
> +ngx_http_file_cache_lock(ngx_http_request_t *r, ngx_int_t rc)
>  {
>      ngx_msec_t                 now, timer;
> +    ngx_http_cache_t          *c;
>      ngx_http_file_cache_t     *cache;
>  
> +    c = r->cache;
> +
>      if (!c->lock) {

Note that c->lock is not really needed now, as 
u->conf->cache_lock can be used directly by the caller.  It might 
be a good idea to get rid of it (not sure though).

> -        return NGX_DECLINED;
> +        return rc;
>      }
>  
>      now = ngx_current_msec;
> @@ -431,7 +428,7 @@
>                     c->updating, c->wait_time);
>  
>      if (c->updating) {
> -        return NGX_DECLINED;
> +        return rc;
>      }
>  
>      if (c->lock_timeout == 0) {

I don't like the idea of pass-through of the return code.  Rather, 
return code of the ngx_http_file_cache_lock() should be checked by 
the caller, and appropriate actions takens depending on it.

> diff -r 2117637f64e9 -r 28084fdcef28 src/http/ngx_http_upstream.c
> --- a/src/http/ngx_http_upstream.c	Tue Nov 27 17:40:21 2018 +0300
> +++ b/src/http/ngx_http_upstream.c	Mon Dec 03 11:31:56 2018 +0000
> @@ -923,6 +923,11 @@
>          u->cache_status = NGX_HTTP_CACHE_HIT;
>      }
>  
> +    if (rc == NGX_DECLINED || rc == NGX_HTTP_CACHE_STALE)
> +    {

Style: "{" should be on the same line as "if" unless there isn't 
enough room.

> +        rc = ngx_http_file_cache_lock(r, rc);
> +    }
> +
>      switch (rc) {
>  
>      case NGX_OK:

Overall, this part of the ngx_http_upstream_cache() function seems 
to be overcomplicated, and it is really not clear how this is 
expected to interact with various possible rc values and various 
possible ngx_http_file_cache_lock() outcomes.

It may be a good idea to move ngx_http_file_cache_lock() further 
down instead of trying to slip it in between these switches().  
This will require separate handling of ngx_http_file_cache_lock() 
return codes, but this actually looks like a good change, see 
comments above.

Also, moving ngx_http_file_cache_lock() further down will allow us 
to avoid cache locks when proxy_cache_max_range_offset prevents 
caching anyway.

[...]

-- 
Maxim Dounin
http://mdounin.ru/

From jan.prachar at gmail.com  Mon Dec 10 20:46:28 2018
From: jan.prachar at gmail.com (Jan =?UTF-8?Q?Pracha=C5=99?=)
Date: Mon, 10 Dec 2018 21:46:28 +0100
Subject: SSL_shutdown() return value <0
Message-ID: <7eb37d13ecfeeb33b090ee64152e2b04a2e8806c.camel@gmail.com>

Hello, I would like to ask about this piece of code from function
ngx_ssl_shutdown:

    n = SSL_shutdown(c->ssl->connection);

    sslerr = 0;

    /* before 0.9.8m SSL_shutdown() returned 0 instead of -1 on errors
*/

    if (n != 1 && ERR_peek_error()) {
        sslerr = SSL_get_error(c->ssl->connection, n);
    }



Why don't you check SSL_get_error always if n < 0, but only if also
ERR_peer_error() returns non-zero value?

According to a documentation of SSL_shutdown, you should check result
of SSL_get_error() and take appropriate action if it returns
SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE,e.g. call SSL_shutdown
again, if SSL_shutdown would block on writing to SSL connection.

If ERR_peek_error() is not zero, which mean some internal OpenSSL error
occured, SSL_get_error will return SSL_ERROR_SSL, won't it?

I have also tried to change the condition to just n < 0, and came to
antoher issue. If client closes connection prematurely, there is
usually SSL_write, that has failed with error WANT_WRITE. If then the
SSL_shutdown is called repeatedly, it causes OpenSSL error (SSL:
error:1409F07F:SSL routines:ssl3_write_pending:bad write retry),
because pending SSL_write should have been called first.

Best regards,
Jan Pracha?


From arut at nginx.com  Tue Dec 11 10:28:43 2018
From: arut at nginx.com (Roman Arutyunyan)
Date: Tue, 11 Dec 2018 10:28:43 +0000
Subject: [nginx] Version bump.
Message-ID: <hg.4722b4b8aa93.1544524123.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/4722b4b8aa93
branches:  
changeset: 7426:4722b4b8aa93
user:      Roman Arutyunyan <arut at nginx.com>
date:      Tue Dec 11 13:12:35 2018 +0300
description:
Version bump.

diffstat:

 src/core/nginx.h |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff -r 2117637f64e9 -r 4722b4b8aa93 src/core/nginx.h
--- a/src/core/nginx.h	Tue Nov 27 17:40:21 2018 +0300
+++ b/src/core/nginx.h	Tue Dec 11 13:12:35 2018 +0300
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1015007
-#define NGINX_VERSION      "1.15.7"
+#define nginx_version      1015008
+#define NGINX_VERSION      "1.15.8"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #ifdef NGX_BUILD

From mdounin at mdounin.ru  Tue Dec 11 13:33:48 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 11 Dec 2018 16:33:48 +0300
Subject: SSL_shutdown() return value <0
In-Reply-To: <7eb37d13ecfeeb33b090ee64152e2b04a2e8806c.camel@gmail.com>
References: <7eb37d13ecfeeb33b090ee64152e2b04a2e8806c.camel@gmail.com>
Message-ID: <20181211133348.GW99070@mdounin.ru>

Hello!

On Mon, Dec 10, 2018 at 09:46:28PM +0100, Jan Pracha? wrote:

> Hello, I would like to ask about this piece of code from function
> ngx_ssl_shutdown:
> 
>     n = SSL_shutdown(c->ssl->connection);
> 
>     sslerr = 0;
> 
>     /* before 0.9.8m SSL_shutdown() returned 0 instead of -1 on errors
> */
> 
>     if (n != 1 && ERR_peek_error()) {
>         sslerr = SSL_get_error(c->ssl->connection, n);
>     }
> 
> 
> 
> Why don't you check SSL_get_error always if n < 0, but only if also
> ERR_peer_error() returns non-zero value?

When this code was written, SSL_shutdown() never returned negative 
values.  Instead, it returned 0 on errors.  ERR_peek_error() was 
used to filter out if there was a real error or not - but it looks 
like it does so incorrectly, see below.

> According to a documentation of SSL_shutdown, you should check result
> of SSL_get_error() and take appropriate action if it returns
> SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE,e.g. call SSL_shutdown
> again, if SSL_shutdown would block on writing to SSL connection.
> 
> If ERR_peek_error() is not zero, which mean some internal OpenSSL error
> occured, SSL_get_error will return SSL_ERROR_SSL, won't it?

The ERR_peek_error() is used to filter out cases when no real 
error occured, and calling SSL_get_error() returned meaningless 
SSL_ERROR_SYSCALL.  See

http://mailman.nginx.org/pipermail/nginx/2008-January/003084.html

for details on why it was introduced.

Though it looks like it also filters out any real 
connection-specific errors as well, including SSL_ERROR_WANT_READ 
or SSL_ERROR_WANT_WRITE.  I don't think this was an intention.

I suspect the original idea was to filter out errors when 
SSL_get_error() returns SSL_ERROR_SYSCALL in the case when "an EOF 
was observed that violates the protocol", as per SSL_get_error() 
documentation:

       SSL_ERROR_SYSCALL
           Some I/O error occurred.  The OpenSSL error queue may contain more
           information on the error.  If the error queue is empty (i.e.
           ERR_get_error() returns 0), ret can be used to find out more about
           the error: If ret == 0, an EOF was observed that violates the
           protocol.  If ret == -1, the underlying BIO reported an I/O error
           (for socket I/O on Unix systems, consult errno for details).

Probably this needs to be rewritten.

> I have also tried to change the condition to just n < 0, and came to
> antoher issue. If client closes connection prematurely, there is
> usually SSL_write, that has failed with error WANT_WRITE. If then the
> SSL_shutdown is called repeatedly, it causes OpenSSL error (SSL:
> error:1409F07F:SSL routines:ssl3_write_pending:bad write retry),
> because pending SSL_write should have been called first.

In many places we try to avoid doing actual SSL shutdown if we 
know there was an error and/or we know the connection was already 
closed, by using c->ssl->no_send_shutdown flag.  Existing cases 
might not be enough though.

-- 
Maxim Dounin
http://mdounin.ru/

From arut at nginx.com  Tue Dec 11 14:23:00 2018
From: arut at nginx.com (Roman Arutyunyan)
Date: Tue, 11 Dec 2018 17:23:00 +0300
Subject: Copy regex unnamed captures to cloned subrequests
Message-ID: <hg.81d49f85afedac2ec723.1544538180@dev.nginx.com>


details:   https://hg.nginx.org/nginx/rev/81d49f85afed
branches:  
changeset: 7427:81d49f85afed
user:      Roman Arutyunyan <arut at nginx.com>
date:      Tue Dec 11 13:09:00 2018 +0300
description:
Copy regex unnamed captures to cloned subrequests.

Previously, unnamed regex captures matched in the parent request, were not
available in a cloned subrequest.  Now 3 fields related to unnamed captures
are copied to a cloned subrequest: r->ncaptures, r->captures and
r->captures_data.  Since r->captures cannot be changed by either request after
creating a clone, a new flag r->realloc_captures is introduced to force
reallocation of r->captures.

The issue was reported as a proxy_cache_background_update misbehavior in
http://mailman.nginx.org/pipermail/nginx/2018-December/057251.html.

diffstat:

 src/http/ngx_http_core_module.c |  8 ++++++++
 src/http/ngx_http_request.h     |  4 ++++
 src/http/ngx_http_variables.c   |  4 +++-
 3 files changed, 15 insertions(+), 1 deletions(-)

diffs (47 lines):

diff -r 4722b4b8aa93 -r 81d49f85afed src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c	Tue Dec 11 13:12:35 2018 +0300
+++ b/src/http/ngx_http_core_module.c	Tue Dec 11 13:09:00 2018 +0300
@@ -2386,6 +2386,14 @@ ngx_http_subrequest(ngx_http_request_t *
         sr->phase_handler = r->phase_handler;
         sr->write_event_handler = ngx_http_core_run_phases;
 
+#if (NGX_PCRE)
+        sr->ncaptures = r->ncaptures;
+        sr->captures = r->captures;
+        sr->captures_data = r->captures_data;
+        sr->realloc_captures = 1;
+        r->realloc_captures = 1;
+#endif
+
         ngx_http_update_location_config(sr);
     }
 
diff -r 4722b4b8aa93 -r 81d49f85afed src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h	Tue Dec 11 13:12:35 2018 +0300
+++ b/src/http/ngx_http_request.h	Tue Dec 11 13:09:00 2018 +0300
@@ -499,6 +499,10 @@ struct ngx_http_request_s {
     unsigned                          gzip_vary:1;
 #endif
 
+#if (NGX_PCRE)
+    unsigned                          realloc_captures:1;
+#endif
+
     unsigned                          proxy:1;
     unsigned                          bypass_cache:1;
     unsigned                          no_cache:1;
diff -r 4722b4b8aa93 -r 81d49f85afed src/http/ngx_http_variables.c
--- a/src/http/ngx_http_variables.c	Tue Dec 11 13:12:35 2018 +0300
+++ b/src/http/ngx_http_variables.c	Tue Dec 11 13:09:00 2018 +0300
@@ -2504,7 +2504,9 @@ ngx_http_regex_exec(ngx_http_request_t *
     if (re->ncaptures) {
         len = cmcf->ncaptures;
 
-        if (r->captures == NULL) {
+        if (r->captures == NULL || r->realloc_captures) {
+            r->realloc_captures = 0;
+
             r->captures = ngx_palloc(r->pool, len * sizeof(int));
             if (r->captures == NULL) {
                 return NGX_ERROR;



From arut at nginx.com  Wed Dec 12 14:47:08 2018
From: arut at nginx.com (Roman Arutyunyan)
Date: Wed, 12 Dec 2018 14:47:08 +0000
Subject: [nginx] Resolver: report SRV resolve failure if all A resolves failed.
Message-ID: <hg.cbc5dee8d5d2.1544626028.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/cbc5dee8d5d2
branches:  
changeset: 7428:cbc5dee8d5d2
user:      Roman Arutyunyan <arut at nginx.com>
date:      Tue Dec 11 19:41:22 2018 +0300
description:
Resolver: report SRV resolve failure if all A resolves failed.

Previously, if an SRV record was successfully resolved, but all of its A
records failed to resolve, NXDOMAIN was returned to the caller, which is
considered a successful resolve rather than an error.  This could result in
losing the result of a previous successful resolve by the caller.

Now NXDOMAIN is only returned if at least one A resolve completed with this
code.  Otherwise the error state of the first A resolve is returned.

diffstat:

 src/core/ngx_resolver.c |  10 +++++++++-
 1 files changed, 9 insertions(+), 1 deletions(-)

diffs (20 lines):

diff -r 81d49f85afed -r cbc5dee8d5d2 src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c	Tue Dec 11 13:09:00 2018 +0300
+++ b/src/core/ngx_resolver.c	Tue Dec 11 19:41:22 2018 +0300
@@ -4266,7 +4266,15 @@ ngx_resolver_report_srv(ngx_resolver_t *
     }
 
     if (naddrs == 0) {
-        ctx->state = NGX_RESOLVE_NXDOMAIN;
+        ctx->state = srvs[0].state;
+
+        for (i = 0; i < nsrvs; i++) {
+            if (srvs[i].state == NGX_RESOLVE_NXDOMAIN) {
+                ctx->state = NGX_RESOLVE_NXDOMAIN;
+                break;
+            }
+        }
+
         ctx->valid = ngx_time() + (r->valid ? r->valid : 10);
 
         ctx->handler(ctx);

From ru at nginx.com  Thu Dec 13 14:24:43 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 13 Dec 2018 14:24:43 +0000
Subject: [nginx] Upstream: implemented $upstream_bytes_sent.
Message-ID: <hg.e573d74299a0.1544711083.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/e573d74299a0
branches:  
changeset: 7429:e573d74299a0
user:      Ruslan Ermilov <ru at nginx.com>
date:      Thu Dec 13 17:23:07 2018 +0300
description:
Upstream: implemented $upstream_bytes_sent.

diffstat:

 src/http/ngx_http_upstream.c |  15 +++++++++++++++
 src/http/ngx_http_upstream.h |   1 +
 2 files changed, 16 insertions(+), 0 deletions(-)

diffs (57 lines):

diff -r cbc5dee8d5d2 -r e573d74299a0 src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Tue Dec 11 19:41:22 2018 +0300
+++ b/src/http/ngx_http_upstream.c	Thu Dec 13 17:23:07 2018 +0300
@@ -409,6 +409,10 @@ static ngx_http_variable_t  ngx_http_ups
       ngx_http_upstream_response_length_variable, 1,
       NGX_HTTP_VAR_NOCACHEABLE, 0 },
 
+    { ngx_string("upstream_bytes_sent"), NULL,
+      ngx_http_upstream_response_length_variable, 2,
+      NGX_HTTP_VAR_NOCACHEABLE, 0 },
+
 #if (NGX_HTTP_CACHE)
 
     { ngx_string("upstream_cache_status"), NULL,
@@ -4136,6 +4140,10 @@ ngx_http_upstream_next(ngx_http_request_
 
     if (u->peer.sockaddr) {
 
+        if (u->peer.connection) {
+            u->state->bytes_sent = u->peer.connection->sent;
+        }
+
         if (ft_type == NGX_HTTP_UPSTREAM_FT_HTTP_403
             || ft_type == NGX_HTTP_UPSTREAM_FT_HTTP_404)
         {
@@ -4319,6 +4327,10 @@ ngx_http_upstream_finalize_request(ngx_h
                                         - u->pipe->preread_size;
             u->state->response_length = u->pipe->read_length;
         }
+
+        if (u->peer.connection) {
+            u->state->bytes_sent = u->peer.connection->sent;
+        }
     }
 
     u->finalize_request(r, rc);
@@ -5502,6 +5514,9 @@ ngx_http_upstream_response_length_variab
         if (data == 1) {
             p = ngx_sprintf(p, "%O", state[i].bytes_received);
 
+        } else if (data == 2) {
+            p = ngx_sprintf(p, "%O", state[i].bytes_sent);
+
         } else {
             p = ngx_sprintf(p, "%O", state[i].response_length);
         }
diff -r cbc5dee8d5d2 -r e573d74299a0 src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h	Tue Dec 11 19:41:22 2018 +0300
+++ b/src/http/ngx_http_upstream.h	Thu Dec 13 17:23:07 2018 +0300
@@ -64,6 +64,7 @@ typedef struct {
     ngx_msec_t                       queue_time;
     off_t                            response_length;
     off_t                            bytes_received;
+    off_t                            bytes_sent;
 
     ngx_str_t                       *peer;
 } ngx_http_upstream_state_t;

From ru at nginx.com  Thu Dec 13 14:27:22 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 13 Dec 2018 17:27:22 +0300
Subject: [PATCH] Upstream: added $upstream_bytes_sent variable
In-Reply-To: <20181130153932.GB50234@lo0.su>
References: <fafbb3ee41e5bb03bcfb.1543314850@piotrsikora.c.googlers.com>
 <20181129150051.GK99070@mdounin.ru> <20181130153932.GB50234@lo0.su>
Message-ID: <20181213142722.GC69334@lo0.su>

https://hg.nginx.org/nginx/rev/e573d74299a0

From pluknet at nginx.com  Thu Dec 13 23:42:10 2018
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Fri, 14 Dec 2018 02:42:10 +0300
Subject: OpenSSL and Early data
In-Reply-To: <72f6b5db15d0c785276bf6e6f95063e970d1d5f1.camel@gmail.com>
References: <CACqAjk3tSwjBpS3bQn2P6NhPE7E8jAVf=NeFYX9VW6-P4VLQ+Q@mail.gmail.com>
 <005D72EE-CAD8-4EE3-B358-157F573AFFFB@nginx.com>
 <72f6b5db15d0c785276bf6e6f95063e970d1d5f1.camel@gmail.com>
Message-ID: <EFABCC25-2D62-4BDA-A4FE-AEE86A5AA44F@nginx.com>


> On 6 Dec 2018, at 19:20, Jan Pracha? <jan.prachar at gmail.com> wrote:
> 
> On Thu, 2018-12-06 at 18:13 +0300, Sergey Kandaurov wrote:
>>> On 6 Dec 2018, at 02:39, Honza Pracha? <jan.prachar at gmail.com>
>>> wrote:
>>> 
>>> Hello! FYI there is an issue with TLS 1.3 Early data in OpenSSL ? 
>>> https://github.com/openssl/openssl/issues/7757
>>> 
>>> So maybe you would want to consider ignoring Early data with HTTP/2
>>> and OpenSSL. Or try to fix the problem on the nginx side, i.e. do
>>> not call SSL_read_early_data() until all pending data is written
>>> with SSL_write_early_data().
>> 
>> Hello.
>> 
>> This is not strictly related to HTTP/2.
>> I could reproduce it with s_client -early_data over HTTP/1.1,
>> where 1st request is sent in 0-RTT, and 2nd - after handshake.
>> 
>> This quick workaround helped me.  The idea is that we block reading
>> if SSL_write_early_data returned SSL_ERROR_WANT_WRITE, until one of
>> the next SSL_write_early_data will succeed.  In practice, we won't
>> read until there's also no more data to send.  For static content,
>> that means that we will continue to read only after the whole file
>> was sent.  This doesn't look perfect but seems to work.
> 
> This patch works for me too. SSL_read_early_data waits until all
> requested files are sent. Then the handshake is finished.

Thanks.
It would be nice if you could also try this patch instead.
Unlike previous, this one is closer to what would be committed.

# HG changeset patch
# User Sergey Kandaurov <pluknet at nginx.com>
# Date 1544744260 -10800
#      Fri Dec 14 02:37:40 2018 +0300
# Node ID bdd5189649f5d788a0e21d0dcd6bc6f4593af401
# Parent  81d49f85afedac2ec7237ee3ce4e6ae927c94706
SSL: avoid reading on pending SSL_write_early_data().

If SSL_write_early_data() returned SSL_ERROR_WANT_WRITE, stop further reading
using a newly introduced c->ssl->write_blocked flag, as otherwise this would
result in SSL error "ssl3_write_bytes:bad length".  Eventually, normal reading
will be restored by read event posted from successful SSL_write_early_data().

While here, place "SSL_write_early_data: want write" debug on the path.

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -1839,6 +1839,10 @@ ngx_ssl_recv_early(ngx_connection_t *c, 
         buf += 1;
     }
 
+    if (c->ssl->write_blocked) {
+        return NGX_AGAIN;
+    }
+
     /*
      * SSL_read_early_data() may return data in parts, so try to read
      * until SSL_read_early_data() would return no data
@@ -2339,6 +2343,12 @@ ngx_ssl_write_early(ngx_connection_t *c,
             ngx_post_event(c->read, &ngx_posted_events);
         }
 
+        if (c->ssl->write_blocked) {
+            c->ssl->write_blocked = 0;
+
+            ngx_post_event(c->read, &ngx_posted_events);
+        }
+
         c->sent += written;
 
         return written;
@@ -2352,6 +2362,9 @@ ngx_ssl_write_early(ngx_connection_t *c,
 
     if (sslerr == SSL_ERROR_WANT_WRITE) {
 
+        ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0,
+                       "SSL_write_early_data: want write");
+
         if (c->ssl->saved_read_handler) {
 
             c->read->handler = c->ssl->saved_read_handler;
@@ -2366,6 +2379,15 @@ ngx_ssl_write_early(ngx_connection_t *c,
         }
 
         c->write->ready = 0;
+
+        /*
+         * OpenSSL 1.1.1a fails to handle SSL_read_early_data()
+         * if an SSL_write_early_data() call blocked on writing,
+         * see https://github.com/openssl/openssl/issues/7757
+         */
+
+        c->ssl->write_blocked = 1;
+
         return NGX_AGAIN;
     }
 
diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h
--- a/src/event/ngx_event_openssl.h
+++ b/src/event/ngx_event_openssl.h
@@ -98,6 +98,7 @@ struct ngx_ssl_connection_s {
     unsigned                    try_early_data:1;
     unsigned                    in_early:1;
     unsigned                    early_preread:1;
+    unsigned                    write_blocked:1;
 };
 
 

-- 
Sergey Kandaurov


From Elliot.Thomas at bbc.co.uk  Fri Dec 14 10:55:37 2018
From: Elliot.Thomas at bbc.co.uk (Elliot Thomas)
Date: Fri, 14 Dec 2018 10:55:37 +0000
Subject: [PATCH] Apply cache locking behaviour to stale cache entries.
In-Reply-To: <20181210190717.GV99070@mdounin.ru>
References: <D82ACDE2.2781E%Elliot.Thomas@bbc.co.uk>
 <20181210190717.GV99070@mdounin.ru>
Message-ID: <D8359B16.2A044%Elliot.Thomas@bbc.co.uk>


Hello!

On 10/12/2018, 19:07, "nginx-devel on behalf of Maxim Dounin"
<nginx-devel-bounces at nginx.org on behalf of mdounin at mdounin.ru> wrote:

...

> Note that posting patches to a public mailing list with such a
> disclaimer might not be a good idea. If you cannot remove it,
> please make sure to add an explicit comment that you understand
> that you are posting to a public mailing list, and you've read the
> http://nginx.org/en/docs/contributing_changes.html article. In
> particular, that you agree with the "License? part.

Unfortunately, I have no control over the disclaimer (which does look a
bit silly on a public mailing list).

I will add a disclaimer to the disclaimer where appropriate.

...

> This changes makes the "done" label useless. It can be removed,
> and corresponding gotos in the function replaced with explicit
> returns (as it was before the introduction of cache locks in
> revision 70ba81827472, http://hg.nginx.org/nginx/rev/70ba81827472)

Noted and changed.

...

> Note that c->lock is not really needed now, as
> u->conf->cache_lock can be used directly by the caller. It might
> be a good idea to get rid of it (not sure though).

I guess this could cause a change in behaviour. If my understanding of
Nginx is correct, u->conf->cache_lock can be different on a per-location
basis, but c->lock would be set by the first request.

But I can certainly remove this check/field if desired.

> I don't like the idea of pass-through of the return code. Rather,
> return code of the ngx_http_file_cache_lock() should be checked by
> the caller, and appropriate actions takens depending on it.

...

> Overall, this part of the ngx_http_upstream_cache() function seems
> to be overcomplicated, and it is really not clear how this is
> expected to interact with various possible rc values and various
> possible ngx_http_file_cache_lock() outcomes.
>
> It may be a good idea to move ngx_http_file_cache_lock() further
> down instead of trying to slip it in between these switches().
> This will require separate handling of ngx_http_file_cache_lock()
> return codes, but this actually looks like a good change, see
> comments above.

Sounds reasonable, I have reworked this part.

Now, for any "u->cacheable" upstream request that reaches the end of the
ngx_http_upstream() function, the locking function is called.

As the ngx_http_file_cache_lock() return code is no longer handled in
the same way as a ngx_http_file_cache_open() return code,
I took the liberty of changing them so they related to the action of
"acquiring a lock" rather than what ngx_http_file_cache_open() would do.

* NGX_OK - returned if acquiring the lock is successful or unnecessary.
    This was NGX_DECLINED.
* NGX_AGAIN - returned if lock is held (no change from previous usage).
    This has not changed.
* NGX_BUSY - returned if lock is held and lock timeout reached.
    This was NGX_HTTP_CACHE_SCARCE.

Attached is a revised patch.


Regards, Elliot.

---
Please ignore the following disclaimer, it?s a bit silly.
I have read the contribution guide, and am fine with it.





-----------------------------
http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and
may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in
error, please delete it from your system.
Do not use, copy or disclose the
information in any way nor act in reliance on it and notify the sender
immediately.
Please note that the BBC monitors e-mails
sent or received.
Further communication will signify your consent to
this.
-----------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: proxy-cache-lock-on-stale-v2.patch
Type: application/octet-stream
Size: 3263 bytes
Desc: proxy-cache-lock-on-stale-v2.patch
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20181214/3cc6233b/attachment.obj>

From jan.prachar at gmail.com  Fri Dec 14 11:17:23 2018
From: jan.prachar at gmail.com (Jan =?UTF-8?Q?Pracha=C5=99?=)
Date: Fri, 14 Dec 2018 12:17:23 +0100
Subject: OpenSSL and Early data
In-Reply-To: <EFABCC25-2D62-4BDA-A4FE-AEE86A5AA44F@nginx.com>
References: <CACqAjk3tSwjBpS3bQn2P6NhPE7E8jAVf=NeFYX9VW6-P4VLQ+Q@mail.gmail.com>
 <005D72EE-CAD8-4EE3-B358-157F573AFFFB@nginx.com>
 <72f6b5db15d0c785276bf6e6f95063e970d1d5f1.camel@gmail.com>
 <EFABCC25-2D62-4BDA-A4FE-AEE86A5AA44F@nginx.com>
Message-ID: <2c00386ffccf89719b29ab2ce9f648c6442f959f.camel@gmail.com>

On Fri, 2018-12-14 at 02:42 +0300, Sergey Kandaurov wrote:
> > On 6 Dec 2018, at 19:20, Jan Pracha? <jan.prachar at gmail.com> wrote:
> > 
> > On Thu, 2018-12-06 at 18:13 +0300, Sergey Kandaurov wrote:
> > > > On 6 Dec 2018, at 02:39, Honza Pracha? <jan.prachar at gmail.com>
> > > > wrote:
> > > > 
> > > > Hello! FYI there is an issue with TLS 1.3 Early data in OpenSSL
> > > > ? 
> > > > https://github.com/openssl/openssl/issues/7757
> > > > 
> > > > So maybe you would want to consider ignoring Early data with
> > > > HTTP/2
> > > > and OpenSSL. Or try to fix the problem on the nginx side, i.e.
> > > > do
> > > > not call SSL_read_early_data() until all pending data is
> > > > written
> > > > with SSL_write_early_data().
> > > 
> > > Hello.
> > > 
> > > This is not strictly related to HTTP/2.
> > > I could reproduce it with s_client -early_data over HTTP/1.1,
> > > where 1st request is sent in 0-RTT, and 2nd - after handshake.
> > > 
> > > This quick workaround helped me.  The idea is that we block
> > > reading
> > > if SSL_write_early_data returned SSL_ERROR_WANT_WRITE, until one
> > > of
> > > the next SSL_write_early_data will succeed.  In practice, we
> > > won't
> > > read until there's also no more data to send.  For static
> > > content,
> > > that means that we will continue to read only after the whole
> > > file
> > > was sent.  This doesn't look perfect but seems to work.
> > 
> > This patch works for me too. SSL_read_early_data waits until all
> > requested files are sent. Then the handshake is finished.
> 
> Thanks.
> It would be nice if you could also try this patch instead.
> Unlike previous, this one is closer to what would be committed.

I can confirm that provided patch works for me.


From mdounin at mdounin.ru  Fri Dec 14 16:53:29 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 14 Dec 2018 16:53:29 +0000
Subject: [nginx] Geo: fixed handling of AF_UNIX client addresses (ticket
 #1684).
Message-ID: <hg.286ae954009d.1544806409.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/286ae954009d
branches:  
changeset: 7430:286ae954009d
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Fri Dec 14 18:11:06 2018 +0300
description:
Geo: fixed handling of AF_UNIX client addresses (ticket #1684).

Previously, AF_UNIX client addresses were handled as AF_INET, leading
to unexpected results.

diffstat:

 src/http/modules/ngx_http_geo_module.c |  13 +++++++++++++
 src/stream/ngx_stream_geo_module.c     |  13 +++++++++++++
 2 files changed, 26 insertions(+), 0 deletions(-)

diffs (60 lines):

diff --git a/src/http/modules/ngx_http_geo_module.c b/src/http/modules/ngx_http_geo_module.c
--- a/src/http/modules/ngx_http_geo_module.c
+++ b/src/http/modules/ngx_http_geo_module.c
@@ -215,6 +215,13 @@ ngx_http_geo_cidr_variable(ngx_http_requ
         break;
 #endif
 
+#if (NGX_HAVE_UNIX_DOMAIN)
+    case AF_UNIX:
+        vv = (ngx_http_variable_value_t *)
+                  ngx_radix32tree_find(ctx->u.trees.tree, INADDR_NONE);
+        break;
+#endif
+
     default: /* AF_INET */
         sin = (struct sockaddr_in *) addr.sockaddr;
         inaddr = ntohl(sin->sin_addr.s_addr);
@@ -277,6 +284,12 @@ ngx_http_geo_range_variable(ngx_http_req
             break;
 #endif
 
+#if (NGX_HAVE_UNIX_DOMAIN)
+        case AF_UNIX:
+            inaddr = INADDR_NONE;
+            break;
+#endif
+
         default: /* AF_INET */
             sin = (struct sockaddr_in *) addr.sockaddr;
             inaddr = ntohl(sin->sin_addr.s_addr);
diff --git a/src/stream/ngx_stream_geo_module.c b/src/stream/ngx_stream_geo_module.c
--- a/src/stream/ngx_stream_geo_module.c
+++ b/src/stream/ngx_stream_geo_module.c
@@ -206,6 +206,13 @@ ngx_stream_geo_cidr_variable(ngx_stream_
         break;
 #endif
 
+#if (NGX_HAVE_UNIX_DOMAIN)
+    case AF_UNIX:
+        vv = (ngx_stream_variable_value_t *)
+                  ngx_radix32tree_find(ctx->u.trees.tree, INADDR_NONE);
+        break;
+#endif
+
     default: /* AF_INET */
         sin = (struct sockaddr_in *) addr.sockaddr;
         inaddr = ntohl(sin->sin_addr.s_addr);
@@ -268,6 +275,12 @@ ngx_stream_geo_range_variable(ngx_stream
             break;
 #endif
 
+#if (NGX_HAVE_UNIX_DOMAIN)
+        case AF_UNIX:
+            inaddr = INADDR_NONE;
+            break;
+#endif
+
         default: /* AF_INET */
             sin = (struct sockaddr_in *) addr.sockaddr;
             inaddr = ntohl(sin->sin_addr.s_addr);

From terence at honles.com  Mon Dec 17 03:18:19 2018
From: terence at honles.com (Terence Honles)
Date: Sun, 16 Dec 2018 19:18:19 -0800
Subject: [PATCH] better constrain IP-literal validation in
 ngx_http_validate_host()
Message-ID: <CAHtY7_pxP9JXLgJgpwzJRNs=9Vtd-1s6efzrktCu-7EYqpjUvA@mail.gmail.com>

# HG changeset patch
# User Terence Honles <terence at honles.com>
# Date 1542840079 28800
#      Wed Nov 21 14:41:19 2018 -0800
# Node ID 0763519f3dcce2c68ccd8894dcc02a4d6114b4c2
# Parent  be5cb9c67c05ccaf22dab7abba78aa4c1545a8ee
better constrain IP-literal validation in ngx_http_validate_host()

The existing validation in ngx_http_validate_host() would allow a IP-literal
such as "[127.0.0.1]" which is invalid according to RFC 3986 (See Appendix A.
for the Collected ABNF). This format is intended for IPv6 and IPv-future not
IPv4.

The following changeset does the following:

- Terminates the validation loop if the state is `sw_rest` (If checking this
  every character outweighs the benefit of early termination it can be
  omitted).
- If a "." is found in an IPv6 literal or immediately after the initial "["
  the hostname is determined to be invalid.
- If a "[" is found at any position other than the first character the
  hostname is determined to be invalid.
- Adds a "sw_literal_start" state which branches to "sw_literal_ip_v6" or
  "sw_literal_ip_future"  depending if the character "v" is found at the start
  of the IP-literal.
- If a non hex character (and not ":" because of the explicit case statement)
  is found in an IPv6 literal the hostname is determined to be invalid.
- If the validation loop has not ended in either the "sw_usual" or "sw_rest"
  state the hostname is determined to be invalid (This will occur in an
  unterminated IP-literal)

This changeset *does not* aim to validate IPv-future as it is currently very
permissive and there is no pressing need for this validation.

diff -r be5cb9c67c05 -r 0763519f3dcc src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c Wed Nov 21 20:23:16 2018 +0300
+++ b/src/http/ngx_http_request.c Wed Nov 21 14:41:19 2018 -0800
@@ -1958,12 +1958,14 @@
 static ngx_int_t
 ngx_http_validate_host(ngx_str_t *host, ngx_pool_t *pool, ngx_uint_t alloc)
 {
-    u_char  *h, ch;
+    u_char  *h, c, ch;
     size_t   i, dot_pos, host_len;

     enum {
         sw_usual = 0,
-        sw_literal,
+        sw_literal_start,
+        sw_literal_ip_v6,
+        sw_literal_ip_future,
         sw_rest
     } state;

@@ -1974,13 +1976,16 @@

     state = sw_usual;

-    for (i = 0; i < host->len; i++) {
+    for (i = 0; i < host->len && state != sw_rest; i++) {
         ch = h[i];

         switch (ch) {

         case '.':
-            if (dot_pos == i - 1) {
+            if (state == sw_literal_start
+                || state == sw_literal_ip_v6
+                || dot_pos == i - 1)
+            {
                 return NGX_DECLINED;
             }
             dot_pos = i;
@@ -1995,12 +2000,14 @@

         case '[':
             if (i == 0) {
-                state = sw_literal;
+                state = sw_literal_start;
+            } else {
+                return NGX_DECLINED;
             }
             break;

         case ']':
-            if (state == sw_literal) {
+            if (state == sw_literal_ip_v6 || state == sw_literal_ip_future) {
                 host_len = i + 1;
                 state = sw_rest;
             }
@@ -2015,6 +2022,17 @@
                 return NGX_DECLINED;
             }

+            if (state == sw_literal_start) {
+                state = ch == 'v' ? sw_literal_ip_future : sw_literal_ip_v6;
+            }
+
+            if (state == sw_literal_ip_v6) {
+                c = (u_char) (ch | 0x20);
+                if (!((ch >= '0' && ch <= '9') || (c >= 'a' || c <= 'f'))) {
+                    return NGX_DECLINED;
+                }
+            }
+
             if (ch >= 'A' && ch <= 'Z') {
                 alloc = 1;
             }
@@ -2023,6 +2041,10 @@
         }
     }

+    if (state != sw_usual && state != sw_rest) {
+        return NGX_DECLINED;
+    }
+
     if (dot_pos == host_len - 1) {
         host_len--;
     }

From mdounin at mdounin.ru  Mon Dec 17 16:17:40 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 17 Dec 2018 19:17:40 +0300
Subject: [PATCH] better constrain IP-literal validation in
 ngx_http_validate_host()
In-Reply-To: <CAHtY7_pxP9JXLgJgpwzJRNs=9Vtd-1s6efzrktCu-7EYqpjUvA@mail.gmail.com>
References: <CAHtY7_pxP9JXLgJgpwzJRNs=9Vtd-1s6efzrktCu-7EYqpjUvA@mail.gmail.com>
Message-ID: <20181217161740.GT99070@mdounin.ru>

Hello!

On Sun, Dec 16, 2018 at 07:18:19PM -0800, Terence Honles wrote:

> # HG changeset patch
> # User Terence Honles <terence at honles.com>
> # Date 1542840079 28800
> #      Wed Nov 21 14:41:19 2018 -0800
> # Node ID 0763519f3dcce2c68ccd8894dcc02a4d6114b4c2
> # Parent  be5cb9c67c05ccaf22dab7abba78aa4c1545a8ee
> better constrain IP-literal validation in ngx_http_validate_host()
> 
> The existing validation in ngx_http_validate_host() would allow a IP-literal
> such as "[127.0.0.1]" which is invalid according to RFC 3986 (See Appendix A.
> for the Collected ABNF). This format is intended for IPv6 and IPv-future not
> IPv4.

We've considered doing more strict checks when introducing IPv6 
literals in e7db97bfac25 (http://hg.nginx.org/nginx/rev/e7db97bfac25), 
yet decided that:

- it doesn't add anything to security,
- and may actually harm some future workloads, such as using 
  things like [unix:/path/to/unix.socket].

In particular, it doesn't looks like permitting [127.0.0.1] can be 
a problem.

Do you think that introducing more strict checks can be 
beneficial?  Could you please outline reasons?

[...]

-- 
Maxim Dounin
http://mdounin.ru/

From jackdev at mailbox.org  Tue Dec 18 09:21:31 2018
From: jackdev at mailbox.org (Jack Henschel)
Date: Tue, 18 Dec 2018 10:21:31 +0100
Subject: [PATCH] Clarify proxy modules $proxy_host variable content
In-Reply-To: <20181210173833.GU99070@mdounin.ru>
References: <2f1f0736-df17-172f-a215-fffa527734ff@mailbox.org>
 <20181210173833.GU99070@mdounin.ru>
Message-ID: <daee690c-95c9-c36a-8aab-62ecd38971b7@mailbox.org>

Good Morning,

On 12/10/18 6:38 PM, Maxim Dounin wrote:
> Hello!
> 
> On Tue, Dec 04, 2018 at 05:00:10PM +0100, Jack Henschel wrote:
> 
>> From the current documentation it was not immediately obvious (at least
>> to me and my colleague) that nginx does not set the $proxy_host variable
>> to the hosts specified in an upstream block, but literally uses the name
>> used in the proxy_pass directive.
>>
>> Maxim Dounin has explained to me the reason for this [1], nevertheless I
>> think it is worthwhile explicitly adding this behavior to the documentation.
> 
> Thank you for your patch.  It looks too explicit for me though.
> 

That's the point though: to be _explicit_ about this behavior.
I have talked to a few of my colleagues and none of them regarded it as
obvious.

Greetings
Jack

From pluknet at nginx.com  Tue Dec 18 12:26:10 2018
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Tue, 18 Dec 2018 12:26:10 +0000
Subject: [nginx] SSL: avoid reading on pending SSL_write_early_data().
Message-ID: <hg.294162223c7c.1545135970.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/294162223c7c
branches:  
changeset: 7431:294162223c7c
user:      Sergey Kandaurov <pluknet at nginx.com>
date:      Tue Dec 18 15:15:15 2018 +0300
description:
SSL: avoid reading on pending SSL_write_early_data().

If SSL_write_early_data() returned SSL_ERROR_WANT_WRITE, stop further reading
using a newly introduced c->ssl->write_blocked flag, as otherwise this would
result in SSL error "ssl3_write_bytes:bad length".  Eventually, normal reading
will be restored by read event posted from successful SSL_write_early_data().

While here, place "SSL_write_early_data: want write" debug on the path.

diffstat:

 src/event/ngx_event_openssl.c |  20 ++++++++++++++++++++
 src/event/ngx_event_openssl.h |   1 +
 2 files changed, 21 insertions(+), 0 deletions(-)

diffs (62 lines):

diff -r 286ae954009d -r 294162223c7c src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c	Fri Dec 14 18:11:06 2018 +0300
+++ b/src/event/ngx_event_openssl.c	Tue Dec 18 15:15:15 2018 +0300
@@ -1839,6 +1839,10 @@ ngx_ssl_recv_early(ngx_connection_t *c, 
         buf += 1;
     }
 
+    if (c->ssl->write_blocked) {
+        return NGX_AGAIN;
+    }
+
     /*
      * SSL_read_early_data() may return data in parts, so try to read
      * until SSL_read_early_data() would return no data
@@ -2339,6 +2343,11 @@ ngx_ssl_write_early(ngx_connection_t *c,
             ngx_post_event(c->read, &ngx_posted_events);
         }
 
+        if (c->ssl->write_blocked) {
+            c->ssl->write_blocked = 0;
+            ngx_post_event(c->read, &ngx_posted_events);
+        }
+
         c->sent += written;
 
         return written;
@@ -2352,6 +2361,9 @@ ngx_ssl_write_early(ngx_connection_t *c,
 
     if (sslerr == SSL_ERROR_WANT_WRITE) {
 
+        ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0,
+                       "SSL_write_early_data: want write");
+
         if (c->ssl->saved_read_handler) {
 
             c->read->handler = c->ssl->saved_read_handler;
@@ -2365,6 +2377,14 @@ ngx_ssl_write_early(ngx_connection_t *c,
             ngx_post_event(c->read, &ngx_posted_events);
         }
 
+        /*
+         * OpenSSL 1.1.1a fails to handle SSL_read_early_data()
+         * if an SSL_write_early_data() call blocked on writing,
+         * see https://github.com/openssl/openssl/issues/7757
+         */
+
+        c->ssl->write_blocked = 1;
+
         c->write->ready = 0;
         return NGX_AGAIN;
     }
diff -r 286ae954009d -r 294162223c7c src/event/ngx_event_openssl.h
--- a/src/event/ngx_event_openssl.h	Fri Dec 14 18:11:06 2018 +0300
+++ b/src/event/ngx_event_openssl.h	Tue Dec 18 15:15:15 2018 +0300
@@ -98,6 +98,7 @@ struct ngx_ssl_connection_s {
     unsigned                    try_early_data:1;
     unsigned                    in_early:1;
     unsigned                    early_preread:1;
+    unsigned                    write_blocked:1;
 };
 
 

From mdounin at mdounin.ru  Tue Dec 18 16:14:27 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 18 Dec 2018 19:14:27 +0300
Subject: [PATCH] Clarify proxy modules $proxy_host variable content
In-Reply-To: <daee690c-95c9-c36a-8aab-62ecd38971b7@mailbox.org>
References: <2f1f0736-df17-172f-a215-fffa527734ff@mailbox.org>
 <20181210173833.GU99070@mdounin.ru>
 <daee690c-95c9-c36a-8aab-62ecd38971b7@mailbox.org>
Message-ID: <20181218161427.GU99070@mdounin.ru>

Hello!

On Tue, Dec 18, 2018 at 10:21:31AM +0100, Jack Henschel wrote:

> Good Morning,
> 
> On 12/10/18 6:38 PM, Maxim Dounin wrote:
> > Hello!
> > 
> > On Tue, Dec 04, 2018 at 05:00:10PM +0100, Jack Henschel wrote:
> > 
> >> From the current documentation it was not immediately obvious (at least
> >> to me and my colleague) that nginx does not set the $proxy_host variable
> >> to the hosts specified in an upstream block, but literally uses the name
> >> used in the proxy_pass directive.
> >>
> >> Maxim Dounin has explained to me the reason for this [1], nevertheless I
> >> think it is worthwhile explicitly adding this behavior to the documentation.
> > 
> > Thank you for your patch.  It looks too explicit for me though.
> > 
> 
> That's the point though: to be _explicit_ about this behavior.
> I have talked to a few of my colleagues and none of them regarded it as
> obvious.

While this may not be obvious, this is a logical result of how 
things work.  Further, this is something already explicitly 
written.  And the challenge is to make it obvious and logical, 
rather simply stating something again.

(Just in case, Yaroslav Zhuravlev already tried to introduce the 
DNS analogy I've used in my explanation to you, but failed to 
produce something committable as well.)

-- 
Maxim Dounin
http://mdounin.ru/

From pablo at pablo.com.mx  Wed Dec 19 19:47:04 2018
From: pablo at pablo.com.mx (Pablo Fischer)
Date: Wed, 19 Dec 2018 11:47:04 -0800
Subject: Start nginx with all servers weighted=0
Message-ID: <CAJRPuowaq2rPUeYTR52zO96+RUmF=LXVfQ8=b9Gr4A+K3xy2Kg@mail.gmail.com>

Hi all,

Sorry for posting to nginx-devel but looking to see if I'm missing any
details. I'm building a service discovery integration on top of our
Nginx, things work fine but there are two caveats:

- Nginx does not allow you to start with an empty upstream (aka no hosts).
- Nginx errors out with invalid configuration if you set a weight to "0".

I'm wondering if there is any technical reason of why I _should not_
make a patch that would allow the weight to be 0 during startup
configuration (or reload), looking specifically at this:

https://github.com/nginx/nginx/blob/cb4dd56771c1af082bf3e810436712b4f48f2cf2/src/http/ngx_http_upstream.c#L5839

or... if there is a known patch that does this already then even better.

We have a module that allows us set the weights dynamically via a URL
we have without reloading nginx (we can't execute continuous reloads
since our buffers are maxed out so this could potentially throw the
host into a OOM case).

Thanks!
-- 
Pablo

From vl at nginx.com  Wed Dec 19 20:18:25 2018
From: vl at nginx.com (Vladimir Homutov)
Date: Wed, 19 Dec 2018 23:18:25 +0300
Subject: Start nginx with all servers weighted=0
In-Reply-To: <CAJRPuowaq2rPUeYTR52zO96+RUmF=LXVfQ8=b9Gr4A+K3xy2Kg@mail.gmail.com>
References: <CAJRPuowaq2rPUeYTR52zO96+RUmF=LXVfQ8=b9Gr4A+K3xy2Kg@mail.gmail.com>
Message-ID: <805a712c-7427-70b0-ea60-d931f67e4f5c@nginx.com>

On 19.12.2018 22:47, Pablo Fischer wrote:
> Hi all,
>
> Sorry for posting to nginx-devel but looking to see if I'm missing any
> details. I'm building a service discovery integration on top of our
> Nginx, things work fine but there are two caveats:
>
> - Nginx does not allow you to start with an empty upstream (aka no hosts).

the reason is that such configuration is meaningless,
at least with unmodified nginx version

> - Nginx errors out with invalid configuration if you set a weight to "0".

to disable a particular server, use 'down' flag.

> I'm wondering if there is any technical reason of why I _should not_
> make a patch that would allow the weight to be 0 during startup
> configuration (or reload), looking specifically at this:
>
>
https://github.com/nginx/nginx/blob/cb4dd56771c1af082bf3e810436712b4f48f2cf2/src/http/ngx_http_upstream.c#L5839
>
> or... if there is a known patch that does this already then even better
>
> We have a module that allows us set the weights dynamically via a URL
> we have without reloading nginx (we can't execute continuous reloads
> since our buffers are maxed out so this could potentially throw the
> host into a OOM case).

you may be interested in functionality, available in commercial version,
see http://nginx.org/en/docs/http/ngx_http_upstream_module.html#zone


From terence at honles.com  Fri Dec 21 19:59:27 2018
From: terence at honles.com (Terence Honles)
Date: Fri, 21 Dec 2018 11:59:27 -0800
Subject: [PATCH] better constrain IP-literal validation in
 ngx_http_validate_host()
In-Reply-To: <20181217161740.GT99070@mdounin.ru>
References: <CAHtY7_pxP9JXLgJgpwzJRNs=9Vtd-1s6efzrktCu-7EYqpjUvA@mail.gmail.com>
 <20181217161740.GT99070@mdounin.ru>
Message-ID: <CAHtY7_rG0zYJn+HUCgfzr7uDP5fYtzrmQPw85zGzyb05wH0dZA@mail.gmail.com>

Yes, thanks for the prompt response and pardon my delay in responding.

The reason I came across this code was I have NGINX handling HTTPS, and
proxying to Django via uWSGI. Django has the following RegEx [1]_. Which is is
compliant with the IPv6 literal notation, but causes Django to report an
error.

While I agree, there may not be an issue of security; Any down stream systems
may be confused and unable to handle a malformed hostname. Another alternative
would be to rewrite the hostname to not include the "[" and "]" if it is not a
valid IPv6 literal, but at that point you are validating the IPv6 literal and
to me it makes more sense to reject the connection and force the sending
system to fix the bug rather than have NGINX act in a non compliant manner.

I understand there may be a future effort to support something like:
``[unix:/path/to/unix.socket]``, but I would *NOT* want that to come over the
network via something akin to
``curl -H 'Host: [unix:/path/to/docker-etc.]' my-hostname``. To support other
workloads properly, I believe the validation may need to understand
differences between potentially trusted and untrusted input, and at that point
it may need to be rewritten. With that in mind, this change should not be seen
as a blocker to that potential future.

-Terence

.. [1] https://github.com/django/django/blob/194a4b526ca3f1e1b13cf27f6ed9aeec3a2e9f89/django/http/request.py#L22

On Mon, Dec 17, 2018 at 8:17 AM Maxim Dounin <mdounin at mdounin.ru> wrote:
>
> Hello!
>
> On Sun, Dec 16, 2018 at 07:18:19PM -0800, Terence Honles wrote:
>
> > # HG changeset patch
> > # User Terence Honles <terence at honles.com>
> > # Date 1542840079 28800
> > #      Wed Nov 21 14:41:19 2018 -0800
> > # Node ID 0763519f3dcce2c68ccd8894dcc02a4d6114b4c2
> > # Parent  be5cb9c67c05ccaf22dab7abba78aa4c1545a8ee
> > better constrain IP-literal validation in ngx_http_validate_host()
> >
> > The existing validation in ngx_http_validate_host() would allow a IP-literal
> > such as "[127.0.0.1]" which is invalid according to RFC 3986 (See Appendix A.
> > for the Collected ABNF). This format is intended for IPv6 and IPv-future not
> > IPv4.
>
> We've considered doing more strict checks when introducing IPv6
> literals in e7db97bfac25 (http://hg.nginx.org/nginx/rev/e7db97bfac25),
> yet decided that:
>
> - it doesn't add anything to security,
> - and may actually harm some future workloads, such as using
>   things like [unix:/path/to/unix.socket].
>
> In particular, it doesn't looks like permitting [127.0.0.1] can be
> a problem.
>
> Do you think that introducing more strict checks can be
> beneficial?  Could you please outline reasons?
>
> [...]
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>

From xeioex at nginx.com  Sun Dec 23 17:40:37 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Sun, 23 Dec 2018 17:40:37 +0000
Subject: [njs] Added the rest parameters support.
Message-ID: <hg.daebcd564289.1545586837.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/daebcd564289
branches:  
changeset: 691:daebcd564289
user:      Alexander Pyshchev <yftuls at gmail.com>
date:      Mon Dec 17 21:13:02 2018 +0200
description:
Added the rest parameters support.

This closes #21 issue on Github.

diffstat:

 njs/njs_function.c       |  60 ++++++++++++++++++++++++++++++++++++++++++++---
 njs/njs_function.h       |   3 ++
 njs/njs_lexer.c          |   9 +++++++
 njs/njs_parser.c         |  13 ++++++++++
 njs/njs_parser.h         |   1 +
 njs/test/njs_unit_test.c |  40 ++++++++++++++++++++++++++++++++
 6 files changed, 122 insertions(+), 4 deletions(-)

diffs (226 lines):

diff -r 0709c3d38212 -r daebcd564289 njs/njs_function.c
--- a/njs/njs_function.c	Fri Dec 07 18:58:27 2018 +0300
+++ b/njs/njs_function.c	Mon Dec 17 21:13:02 2018 +0200
@@ -168,6 +168,42 @@ njs_function_arguments_object_init(njs_v
 
 
 njs_ret_t
+njs_function_rest_parameters_init(njs_vm_t *vm, njs_native_frame_t *frame)
+{
+    uint32_t     length;
+    nxt_uint_t   nargs, n, i;
+    njs_array_t  *array;
+    njs_value_t  *rest_arguments;
+
+    nargs = frame->nargs;
+    n = frame->function->u.lambda->nargs;
+    length = (nargs >= n) ? (nargs - n + 1) : 0;
+
+    array = njs_array_alloc(vm, length, 0);
+    if (nxt_slow_path(array == NULL)) {
+        return NXT_ERROR;
+    }
+
+    if (n <= nargs) {
+        i = 0;
+        do {
+            /* GC: retain. */
+            array->start[i++] = frame->arguments[n++];
+        } while (n <= nargs);
+    }
+
+    rest_arguments = &frame->arguments[frame->function->u.lambda->nargs];
+
+    /* GC: retain. */
+    rest_arguments->type = NJS_ARRAY;
+    rest_arguments->data.u.array = array;
+    rest_arguments->data.truth = 1;
+
+    return NXT_OK;
+}
+
+
+njs_ret_t
 njs_function_arguments_thrower(njs_vm_t *vm, njs_value_t *value,
     njs_value_t *setval, njs_value_t *retval)
 {
@@ -482,6 +518,13 @@ njs_function_call(njs_vm_t *vm, njs_inde
         }
     }
 
+    if (lambda->rest_parameters) {
+        ret = njs_function_rest_parameters_init(vm, &frame->native);
+        if (nxt_slow_path(ret != NXT_OK)) {
+            return NXT_ERROR;
+        }
+    }
+
     vm->active_frame = frame;
 
     return NJS_APPLIED;
@@ -600,8 +643,9 @@ static njs_ret_t
 njs_function_prototype_length(njs_vm_t *vm, njs_value_t *value,
     njs_value_t *setval, njs_value_t *retval)
 {
-    nxt_uint_t      n;
-    njs_function_t  *function;
+    nxt_uint_t             n;
+    njs_function_t         *function;
+    njs_function_lambda_t  *lambda;
 
     function = value->data.u.function;
 
@@ -613,10 +657,18 @@ njs_function_prototype_length(njs_vm_t *
         }
 
     } else {
-        n = function->u.lambda->nargs + 1;
+        lambda = function->u.lambda;
+        n = lambda->nargs + 1 - lambda->rest_parameters;
     }
 
-    njs_value_number_set(retval, n - function->args_offset);
+    if (n >= function->args_offset) {
+        n -= function->args_offset;
+
+    } else {
+        n = 0;
+    }
+
+    njs_value_number_set(retval, n);
 
     return NXT_OK;
 }
diff -r 0709c3d38212 -r daebcd564289 njs/njs_function.h
--- a/njs/njs_function.h	Fri Dec 07 18:58:27 2018 +0300
+++ b/njs/njs_function.h	Mon Dec 17 21:13:02 2018 +0200
@@ -31,6 +31,7 @@ struct njs_function_lambda_s {
     uint8_t                        block_closures;    /* 4 bits */
 
     uint8_t                        arguments_object;  /* 1 bit */
+    uint8_t                        rest_parameters;   /* 1 bit */
 
     /* Initial values of local scope. */
     njs_value_t                    *local_scope;
@@ -151,6 +152,8 @@ njs_function_t *njs_function_value_copy(
 njs_native_frame_t *njs_function_frame_alloc(njs_vm_t *vm, size_t size);
 njs_ret_t njs_function_arguments_object_init(njs_vm_t *vm,
     njs_native_frame_t *frame);
+njs_ret_t njs_function_rest_parameters_init(njs_vm_t *vm,
+    njs_native_frame_t *frame);
 njs_ret_t njs_function_arguments_thrower(njs_vm_t *vm, njs_value_t *value,
     njs_value_t *setval, njs_value_t *retval);
 njs_ret_t njs_function_prototype_create(njs_vm_t *vm, njs_value_t *value,
diff -r 0709c3d38212 -r daebcd564289 njs/njs_lexer.c
--- a/njs/njs_lexer.c	Fri Dec 07 18:58:27 2018 +0300
+++ b/njs/njs_lexer.c	Mon Dec 17 21:13:02 2018 +0200
@@ -326,6 +326,15 @@ njs_lexer_next_token(njs_lexer_t *lexer)
         case NJS_TOKEN_DOT:
             p = lexer->start;
 
+            if (p + 1 < lexer->end
+                && njs_tokens[p[0]] == NJS_TOKEN_DOT
+                && njs_tokens[p[1]] == NJS_TOKEN_DOT)
+            {
+                lexer->text.length = (p - lexer->text.start) + 2;
+                lexer->start += 2;
+                return NJS_TOKEN_ELLIPSIS;
+            }
+
             if (p == lexer->end || njs_tokens[*p] != NJS_TOKEN_DIGIT) {
                 lexer->text.length = p - lexer->text.start;
                 return NJS_TOKEN_DOT;
diff -r 0709c3d38212 -r daebcd564289 njs/njs_parser.c
--- a/njs/njs_parser.c	Fri Dec 07 18:58:27 2018 +0300
+++ b/njs/njs_parser.c	Mon Dec 17 21:13:02 2018 +0200
@@ -629,6 +629,19 @@ njs_parser_function_lambda(njs_vm_t *vm,
 
     while (token != NJS_TOKEN_CLOSE_PARENTHESIS) {
 
+        if (nxt_slow_path(lambda->rest_parameters)) {
+            return NJS_TOKEN_ILLEGAL;
+        }
+
+        if (nxt_slow_path(token == NJS_TOKEN_ELLIPSIS)) {
+            lambda->rest_parameters = 1;
+
+            token = njs_parser_token(parser);
+            if (nxt_slow_path(token <= NJS_TOKEN_ILLEGAL)) {
+                return NJS_TOKEN_ILLEGAL;
+            }
+        }
+
         if (nxt_slow_path(token != NJS_TOKEN_NAME)) {
             return NJS_TOKEN_ILLEGAL;
         }
diff -r 0709c3d38212 -r daebcd564289 njs/njs_parser.h
--- a/njs/njs_parser.h	Fri Dec 07 18:58:27 2018 +0300
+++ b/njs/njs_parser.h	Mon Dec 17 21:13:02 2018 +0200
@@ -29,6 +29,7 @@ typedef enum {
 
     NJS_TOKEN_COMMA,
     NJS_TOKEN_DOT,
+    NJS_TOKEN_ELLIPSIS,
     NJS_TOKEN_SEMICOLON,
 
     NJS_TOKEN_COLON,
diff -r 0709c3d38212 -r daebcd564289 njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Fri Dec 07 18:58:27 2018 +0300
+++ b/njs/test/njs_unit_test.c	Mon Dec 17 21:13:02 2018 +0200
@@ -5310,9 +5310,19 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("function f() { }; f.length = 1"),
       nxt_string("TypeError: Cannot assign to read-only property 'length' of function") },
 
+    { nxt_string("function f(...rest) { }; f.length"),
+      nxt_string("0") },
+
+    { nxt_string("function f(...rest) { }; var binded = f.bind(this, [1,2]);"
+                 "binded.length"),
+      nxt_string("0") },
+
     { nxt_string("function f(a,b) { }; f.length"),
       nxt_string("2") },
 
+    { nxt_string("function f(a,...rest) { }; f.length"),
+      nxt_string("1") },
+
     { nxt_string("function f(a,b) { }; var ff = f.bind(f, 1); ff.length"),
       nxt_string("1") },
 
@@ -6063,6 +6073,36 @@ static njs_unit_test_t  njs_test[] =
                  "[concat('.',1,2,3), concat('+',1,2,3,4)]"),
       nxt_string("1.2.3,1+2+3+4") },
 
+    /* rest parameters. */
+
+    { nxt_string("function myFoo(a,b,...other) { return other };"
+                 "myFoo(1,2,3,4,5);" ),
+      nxt_string("3,4,5") },
+
+    { nxt_string("function myFoo(a,b,...other, c) { return other };"),
+      nxt_string("SyntaxError: Unexpected token \"c\" in 1") },
+
+    { nxt_string("function sum(a, b, c, ...other) { return a+b+c+other[2] };"
+                 "sum(\"one \",2,\" three \",\"four \",\"five \",\"the long enough sixth argument \");"),
+      nxt_string("one 2 three the long enough sixth argument ") },
+
+    { nxt_string("function myFoo1(a,...other) { return other };"
+                 "function myFoo2(a,b,...other) { return other };"
+                 "myFoo1(1,2,3,4,5,myFoo2(1,2,3,4));"),
+      nxt_string("2,3,4,5,3,4") },
+
+    { nxt_string("function myFoo(...other) { return (other instanceof Array) };"
+                 "myFoo(1);" ),
+      nxt_string("true") },
+
+    { nxt_string("function myFoo(a,...other) { return other.length };"
+                 "myFoo(1,2,3,4,5);" ),
+      nxt_string("4") },
+
+    { nxt_string("function myFoo(a,b,...other) { return other };"
+                 "myFoo(1,2);" ),
+      nxt_string("") },
+
     /* Scopes. */
 
     { nxt_string("function f(x) { a = x } var a; f(5); a"),

From hongzhidao at gmail.com  Sun Dec 23 19:38:00 2018
From: hongzhidao at gmail.com (=?UTF-8?B?5rSq5b+X6YGT?=)
Date: Mon, 24 Dec 2018 03:38:00 +0800
Subject: The reason of ignoring subrequest in access phase hander.
Message-ID: <CAJuLd0gLsxjgLbHOqSbEXwALgJOa5E2xfG-uqMXaT82LkBnzAQ@mail.gmail.com>

Hi.

I'm wondering the reason of ignoring subrequests in access phase hander.
Can anyone explain it to me?

1069 ngx_int_t
1070 ngx_http_core_access_phase(ngx_http_request_t *r,
ngx_http_phase_handler_t *ph)
1071 {
1072     ngx_int_t                  rc;
1073     ngx_http_core_loc_conf_t  *clcf;
1074
1075     if (r != r->main) {
1076         r->phase_handler = ph->next;
1077         return NGX_AGAIN;
1078     }
1079

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20181224/29f5591e/attachment.html>

From xeioex at nginx.com  Mon Dec 24 11:00:53 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Mon, 24 Dec 2018 11:00:53 +0000
Subject: [njs] Removed surplus for loop.
Message-ID: <hg.5c8802f61ad6.1545649253.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/5c8802f61ad6
branches:  
changeset: 692:5c8802f61ad6
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Dec 24 13:59:27 2018 +0300
description:
Removed surplus for loop.

Left over after simplifications in 4e62b7a295e0.

Found by Coverity (CID 1441823).

diffstat:

 njs/njs_parser_expression.c |  52 +++++++++++++++++++++-----------------------
 1 files changed, 25 insertions(+), 27 deletions(-)

diffs (66 lines):

diff -r daebcd564289 -r 5c8802f61ad6 njs/njs_parser_expression.c
--- a/njs/njs_parser_expression.c	Mon Dec 17 21:13:02 2018 +0200
+++ b/njs/njs_parser_expression.c	Mon Dec 24 13:59:27 2018 +0300
@@ -536,37 +536,35 @@ njs_parser_exponential_expression(njs_vm
         return token;
     }
 
-    for ( ;; ) {
-        if (token == NJS_TOKEN_EXPONENTIATION) {
-
-            node = njs_parser_node_alloc(vm);
-            if (nxt_slow_path(node == NULL)) {
-                return NJS_TOKEN_ERROR;
-            }
-
-            node->token = token;
-            node->u.operation = njs_vmcode_exponentiation;
-            node->scope = parser->scope;
-            node->left = parser->node;
-            node->left->dest = node;
+    if (token == NJS_TOKEN_EXPONENTIATION) {
 
-            token = njs_parser_token(parser);
-            if (nxt_slow_path(token <= NJS_TOKEN_ILLEGAL)) {
-                return token;
-            }
-
-            token = njs_parser_exponential_expression(vm, parser, NULL, token);
-            if (nxt_slow_path(token <= NJS_TOKEN_ILLEGAL)) {
-                return token;
-            }
-
-            node->right = parser->node;
-            node->right->dest = node;
-            parser->node = node;
+        node = njs_parser_node_alloc(vm);
+        if (nxt_slow_path(node == NULL)) {
+            return NJS_TOKEN_ERROR;
         }
 
-        return token;
+        node->token = token;
+        node->u.operation = njs_vmcode_exponentiation;
+        node->scope = parser->scope;
+        node->left = parser->node;
+        node->left->dest = node;
+
+        token = njs_parser_token(parser);
+        if (nxt_slow_path(token <= NJS_TOKEN_ILLEGAL)) {
+            return token;
+        }
+
+        token = njs_parser_exponential_expression(vm, parser, NULL, token);
+        if (nxt_slow_path(token <= NJS_TOKEN_ILLEGAL)) {
+            return token;
+        }
+
+        node->right = parser->node;
+        node->right->dest = node;
+        parser->node = node;
     }
+
+    return token;
 }
 
 

From mdounin at mdounin.ru  Mon Dec 24 12:58:59 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 24 Dec 2018 15:58:59 +0300
Subject: [PATCH] better constrain IP-literal validation in
 ngx_http_validate_host()
In-Reply-To: <CAHtY7_rG0zYJn+HUCgfzr7uDP5fYtzrmQPw85zGzyb05wH0dZA@mail.gmail.com>
References: <CAHtY7_pxP9JXLgJgpwzJRNs=9Vtd-1s6efzrktCu-7EYqpjUvA@mail.gmail.com>
 <20181217161740.GT99070@mdounin.ru>
 <CAHtY7_rG0zYJn+HUCgfzr7uDP5fYtzrmQPw85zGzyb05wH0dZA@mail.gmail.com>
Message-ID: <20181224125858.GH99070@mdounin.ru>

Hello!

On Fri, Dec 21, 2018 at 11:59:27AM -0800, Terence Honles wrote:

> The reason I came across this code was I have NGINX handling HTTPS, and
> proxying to Django via uWSGI. Django has the following RegEx [1]_. Which is is
> compliant with the IPv6 literal notation, but causes Django to report an
> error.

Well, as far as I see the regex in question will also report an 
error for perfectly valid IPvFuture literals.

> While I agree, there may not be an issue of security; Any down stream systems
> may be confused and unable to handle a malformed hostname.

If you want to prevent such names from hitting your backend 
servers, a better solution might be to configure nginx to only 
accept explicitly configured host names (including IP literals).  
This is more or less trivial to configure and something you 
probably should do anyway unless you are still supporting 
pre-Host-header clients.  E.g.:

server {
    listen 80 default;
    return 404;
}

server {
    listen 80;
    server_name example.com;
    ...
}

> Another alternative would be to rewrite the hostname to not 
> include the "[" and "]" if it is not a valid IPv6 literal,

Well, this _will_ be a security issue, since it can easily result 
in different processing of names in nginx and backends.

-- 
Maxim Dounin
http://mdounin.ru/

From mdounin at mdounin.ru  Mon Dec 24 14:06:55 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 24 Dec 2018 17:06:55 +0300
Subject: The reason of ignoring subrequest in access phase hander.
In-Reply-To: <CAJuLd0gLsxjgLbHOqSbEXwALgJOa5E2xfG-uqMXaT82LkBnzAQ@mail.gmail.com>
References: <CAJuLd0gLsxjgLbHOqSbEXwALgJOa5E2xfG-uqMXaT82LkBnzAQ@mail.gmail.com>
Message-ID: <20181224140654.GK99070@mdounin.ru>

Hello!

On Mon, Dec 24, 2018 at 03:38:00AM +0800, ??? wrote:

> Hi.
> 
> I'm wondering the reason of ignoring subrequests in access phase hander.
> Can anyone explain it to me?
> 
> 1069 ngx_int_t
> 1070 ngx_http_core_access_phase(ngx_http_request_t *r,
> ngx_http_phase_handler_t *ph)
> 1071 {
> 1072     ngx_int_t                  rc;
> 1073     ngx_http_core_loc_conf_t  *clcf;
> 1074
> 1075     if (r != r->main) {
> 1076         r->phase_handler = ph->next;
> 1077         return NGX_AGAIN;
> 1078     }
> 1079

Access checks are expected to apply to the main request, but make 
little to no sense for subrequests.  E.g., if you are using 
IP-based restrictions or Basic HTTP authentication, both are 
exactly identical for the main request and subrequests.  As such, 
it is enough to do the checks for the main request only, and this 
is what nginx does.

Note that it implies that access restrictions configured for the 
main requests are expected to be enough for all subrequests it can 
initiate.

-- 
Maxim Dounin
http://mdounin.ru/

From gmm at csdoc.com  Mon Dec 24 14:35:22 2018
From: gmm at csdoc.com (Gena Makhomed)
Date: Mon, 24 Dec 2018 16:35:22 +0200
Subject: [PATCH] Contrib: vim syntax,
 update core and 3rd party module directives.
Message-ID: <0b8fa025-fda8-0c7a-7a9e-addfab4e576f@csdoc.com>

# HG changeset patch
# User Gena Makhomed <gmm at csdoc.com>
# Date 1545661810 -7200
#      Mon Dec 24 16:30:10 2018 +0200
# Node ID 19f1cbe0be527cbbfea8d8e7e8693352e72e745b
# Parent  294162223c7c51a147c07eba36d0506d54edbf20
Contrib: vim syntax, update core and 3rd party module directives.

diff -r 294162223c7c -r 19f1cbe0be52 contrib/vim/syntax/nginx.vim
--- a/contrib/vim/syntax/nginx.vim	Tue Dec 18 15:15:15 2018 +0300
+++ b/contrib/vim/syntax/nginx.vim	Mon Dec 24 16:30:10 2018 +0200
@@ -108,6 +108,7 @@
  syn keyword ngxDirectiveError contained error_page
  syn keyword ngxDirectiveError contained post_action

+syn keyword ngxDirectiveDeprecated contained limit_zone
  syn keyword ngxDirectiveDeprecated contained proxy_downstream_buffer
  syn keyword ngxDirectiveDeprecated contained proxy_upstream_buffer
  syn keyword ngxDirectiveDeprecated contained spdy_chunk_size
@@ -118,6 +119,7 @@
  syn keyword ngxDirectiveDeprecated contained spdy_recv_buffer_size
  syn keyword ngxDirectiveDeprecated contained spdy_recv_timeout
  syn keyword ngxDirectiveDeprecated contained spdy_streams_index_size
+syn keyword ngxDirectiveDeprecated contained ssl
  syn keyword ngxDirectiveDeprecated contained upstream_conf

  syn keyword ngxDirective contained absolute_redirect
@@ -136,6 +138,7 @@
  syn keyword ngxDirective contained allow
  syn keyword ngxDirective contained ancient_browser
  syn keyword ngxDirective contained ancient_browser_value
+syn keyword ngxDirective contained api
  syn keyword ngxDirective contained auth_basic
  syn keyword ngxDirective contained auth_basic_user_file
  syn keyword ngxDirective contained auth_http
@@ -143,7 +146,11 @@
  syn keyword ngxDirective contained auth_http_pass_client_cert
  syn keyword ngxDirective contained auth_http_timeout
  syn keyword ngxDirective contained auth_jwt
+syn keyword ngxDirective contained auth_jwt_claim_set
+syn keyword ngxDirective contained auth_jwt_header_set
  syn keyword ngxDirective contained auth_jwt_key_file
+syn keyword ngxDirective contained auth_jwt_key_request
+syn keyword ngxDirective contained auth_jwt_leeway
  syn keyword ngxDirective contained auth_request
  syn keyword ngxDirective contained auth_request_set
  syn keyword ngxDirective contained autoindex
@@ -229,6 +236,7 @@
  syn keyword ngxDirective contained fastcgi_request_buffering
  syn keyword ngxDirective contained fastcgi_send_lowat
  syn keyword ngxDirective contained fastcgi_send_timeout
+syn keyword ngxDirective contained fastcgi_socket_keepalive
  syn keyword ngxDirective contained fastcgi_split_path_info
  syn keyword ngxDirective contained fastcgi_store
  syn keyword ngxDirective contained fastcgi_store_access
@@ -255,6 +263,7 @@
  syn keyword ngxDirective contained grpc_read_timeout
  syn keyword ngxDirective contained grpc_send_timeout
  syn keyword ngxDirective contained grpc_set_header
+syn keyword ngxDirective contained grpc_socket_keepalive
  syn keyword ngxDirective contained grpc_ssl_certificate
  syn keyword ngxDirective contained grpc_ssl_certificate_key
  syn keyword ngxDirective contained grpc_ssl_ciphers
@@ -330,6 +339,8 @@
  syn keyword ngxDirective contained keepalive_disable
  syn keyword ngxDirective contained keepalive_requests
  syn keyword ngxDirective contained keepalive_timeout
+syn keyword ngxDirective contained keyval
+syn keyword ngxDirective contained keyval_zone
  syn keyword ngxDirective contained kqueue_changes
  syn keyword ngxDirective contained kqueue_events
  syn keyword ngxDirective contained large_client_header_buffers
@@ -367,6 +378,7 @@
  syn keyword ngxDirective contained memcached_next_upstream_tries
  syn keyword ngxDirective contained memcached_read_timeout
  syn keyword ngxDirective contained memcached_send_timeout
+syn keyword ngxDirective contained memcached_socket_keepalive
  syn keyword ngxDirective contained merge_slashes
  syn keyword ngxDirective contained min_delete_depth
  syn keyword ngxDirective contained mirror
@@ -375,9 +387,9 @@
  syn keyword ngxDirective contained modern_browser_value
  syn keyword ngxDirective contained mp4
  syn keyword ngxDirective contained mp4_buffer_size
-syn keyword ngxDirective contained mp4_max_buffer_size
  syn keyword ngxDirective contained mp4_limit_rate
  syn keyword ngxDirective contained mp4_limit_rate_after
+syn keyword ngxDirective contained mp4_max_buffer_size
  syn keyword ngxDirective contained msie_padding
  syn keyword ngxDirective contained msie_refresh
  syn keyword ngxDirective contained multi_accept
@@ -456,11 +468,13 @@
  syn keyword ngxDirective contained proxy_read_timeout
  syn keyword ngxDirective contained proxy_redirect
  syn keyword ngxDirective contained proxy_request_buffering
+syn keyword ngxDirective contained proxy_requests
  syn keyword ngxDirective contained proxy_responses
  syn keyword ngxDirective contained proxy_send_lowat
  syn keyword ngxDirective contained proxy_send_timeout
  syn keyword ngxDirective contained proxy_set_body
  syn keyword ngxDirective contained proxy_set_header
+syn keyword ngxDirective contained proxy_socket_keepalive
  syn keyword ngxDirective contained proxy_ssl
  syn keyword ngxDirective contained proxy_ssl_certificate
  syn keyword ngxDirective contained proxy_ssl_certificate_key
@@ -481,6 +495,7 @@
  syn keyword ngxDirective contained proxy_timeout
  syn keyword ngxDirective contained proxy_upload_rate
  syn keyword ngxDirective contained queue
+syn keyword ngxDirective contained random
  syn keyword ngxDirective contained random_index
  syn keyword ngxDirective contained read_ahead
  syn keyword ngxDirective contained real_ip_header
@@ -533,6 +548,7 @@
  syn keyword ngxDirective contained scgi_read_timeout
  syn keyword ngxDirective contained scgi_request_buffering
  syn keyword ngxDirective contained scgi_send_timeout
+syn keyword ngxDirective contained scgi_socket_keepalive
  syn keyword ngxDirective contained scgi_store
  syn keyword ngxDirective contained scgi_store_access
  syn keyword ngxDirective contained scgi_temp_file_write_size
@@ -565,7 +581,6 @@
  syn keyword ngxDirective contained ssi_silent_errors
  syn keyword ngxDirective contained ssi_types
  syn keyword ngxDirective contained ssi_value_length
-syn keyword ngxDirective contained ssl
  syn keyword ngxDirective contained ssl_buffer_size
  syn keyword ngxDirective contained ssl_certificate
  syn keyword ngxDirective contained ssl_certificate_key
@@ -573,6 +588,7 @@
  syn keyword ngxDirective contained ssl_client_certificate
  syn keyword ngxDirective contained ssl_crl
  syn keyword ngxDirective contained ssl_dhparam
+syn keyword ngxDirective contained ssl_early_data
  syn keyword ngxDirective contained ssl_ecdh_curve
  syn keyword ngxDirective contained ssl_engine
  syn keyword ngxDirective contained ssl_handshake_timeout
@@ -664,6 +680,7 @@
  syn keyword ngxDirective contained uwsgi_read_timeout
  syn keyword ngxDirective contained uwsgi_request_buffering
  syn keyword ngxDirective contained uwsgi_send_timeout
+syn keyword ngxDirective contained uwsgi_socket_keepalive
  syn keyword ngxDirective contained uwsgi_ssl_certificate
  syn keyword ngxDirective contained uwsgi_ssl_certificate_key
  syn keyword ngxDirective contained uwsgi_ssl_ciphers
@@ -701,6 +718,26 @@
  syn keyword ngxDirective contained xslt_stylesheet
  syn keyword ngxDirective contained xslt_types
  syn keyword ngxDirective contained zone
+syn keyword ngxDirective contained zone_sync
+syn keyword ngxDirective contained zone_sync_buffers
+syn keyword ngxDirective contained zone_sync_connect_retry_interval
+syn keyword ngxDirective contained zone_sync_connect_timeout
+syn keyword ngxDirective contained zone_sync_interval
+syn keyword ngxDirective contained zone_sync_recv_buffer_size
+syn keyword ngxDirective contained zone_sync_server
+syn keyword ngxDirective contained zone_sync_ssl
+syn keyword ngxDirective contained zone_sync_ssl_certificate
+syn keyword ngxDirective contained zone_sync_ssl_certificate_key
+syn keyword ngxDirective contained zone_sync_ssl_ciphers
+syn keyword ngxDirective contained zone_sync_ssl_crl
+syn keyword ngxDirective contained zone_sync_ssl_name
+syn keyword ngxDirective contained zone_sync_ssl_password_file
+syn keyword ngxDirective contained zone_sync_ssl_protocols
+syn keyword ngxDirective contained zone_sync_ssl_server_name
+syn keyword ngxDirective contained zone_sync_ssl_trusted_certificate
+syn keyword ngxDirective contained zone_sync_ssl_verify
+syn keyword ngxDirective contained zone_sync_ssl_verify_depth
+syn keyword ngxDirective contained zone_sync_timeout

  " 3rd party modules list taken from
  " 
https://github.com/freebsd/freebsd-ports/blob/master/www/nginx-devel/Makefile
@@ -876,6 +913,8 @@

  " NGINX WebDAV missing commands support (PROPFIND & OPTIONS)
  " https://github.com/arut/nginx-dav-ext-module
+syn keyword ngxDirectiveThirdParty contained dav_ext_lock
+syn keyword ngxDirectiveThirdParty contained dav_ext_lock_zone
  syn keyword ngxDirectiveThirdParty contained dav_ext_methods

  " ngx_eval
@@ -895,6 +934,7 @@
  syn keyword ngxDirectiveThirdParty contained fancyindex_exact_size
  syn keyword ngxDirectiveThirdParty contained fancyindex_footer
  syn keyword ngxDirectiveThirdParty contained fancyindex_header
+syn keyword ngxDirectiveThirdParty contained fancyindex_hide_parent_dir
  syn keyword ngxDirectiveThirdParty contained fancyindex_hide_symlinks
  syn keyword ngxDirectiveThirdParty contained fancyindex_ignore
  syn keyword ngxDirectiveThirdParty contained fancyindex_localtime
@@ -937,8 +977,17 @@

  " nchan
  " https://github.com/slact/nchan
+syn keyword ngxDirectiveThirdParty contained 
nchan_access_control_allow_credentials
  syn keyword ngxDirectiveThirdParty contained 
nchan_access_control_allow_origin
  syn keyword ngxDirectiveThirdParty contained nchan_authorize_request
+syn keyword ngxDirectiveThirdParty contained nchan_benchmark
+syn keyword ngxDirectiveThirdParty contained nchan_benchmark_channels
+syn keyword ngxDirectiveThirdParty contained 
nchan_benchmark_message_padding_bytes
+syn keyword ngxDirectiveThirdParty contained 
nchan_benchmark_messages_per_channel_per_minute
+syn keyword ngxDirectiveThirdParty contained 
nchan_benchmark_publisher_distribution
+syn keyword ngxDirectiveThirdParty contained 
nchan_benchmark_subscriber_distribution
+syn keyword ngxDirectiveThirdParty contained 
nchan_benchmark_subscribers_per_channel
+syn keyword ngxDirectiveThirdParty contained nchan_benchmark_time
  syn keyword ngxDirectiveThirdParty contained nchan_channel_event_string
  syn keyword ngxDirectiveThirdParty contained 
nchan_channel_events_channel_id
  syn keyword ngxDirectiveThirdParty contained nchan_channel_group
@@ -974,15 +1023,19 @@
  syn keyword ngxDirectiveThirdParty contained nchan_pubsub
  syn keyword ngxDirectiveThirdParty contained nchan_pubsub_channel_id
  syn keyword ngxDirectiveThirdParty contained nchan_pubsub_location
+syn keyword ngxDirectiveThirdParty contained nchan_redis_connect_timeout
  syn keyword ngxDirectiveThirdParty contained 
nchan_redis_fakesub_timer_interval
  syn keyword ngxDirectiveThirdParty contained 
nchan_redis_idle_channel_cache_timeout
  syn keyword ngxDirectiveThirdParty contained nchan_redis_namespace
+syn keyword ngxDirectiveThirdParty contained 
nchan_redis_nostore_fastpublish
+syn keyword ngxDirectiveThirdParty contained nchan_redis_optimize_target
  syn keyword ngxDirectiveThirdParty contained nchan_redis_pass
  syn keyword ngxDirectiveThirdParty contained nchan_redis_pass_inheritable
  syn keyword ngxDirectiveThirdParty contained nchan_redis_ping_interval
  syn keyword ngxDirectiveThirdParty contained 
nchan_redis_publish_msgpacked_max_size
  syn keyword ngxDirectiveThirdParty contained nchan_redis_server
  syn keyword ngxDirectiveThirdParty contained nchan_redis_storage_mode
+syn keyword ngxDirectiveThirdParty contained nchan_redis_subscribe_weights
  syn keyword ngxDirectiveThirdParty contained nchan_redis_url
  syn keyword ngxDirectiveThirdParty contained 
nchan_redis_wait_after_connecting
  syn keyword ngxDirectiveThirdParty contained nchan_shared_memory_size
@@ -1280,6 +1333,7 @@
  syn keyword ngxDirectiveThirdParty contained lua_package_path
  syn keyword ngxDirectiveThirdParty contained lua_regex_cache_max_entries
  syn keyword ngxDirectiveThirdParty contained lua_regex_match_limit
+syn keyword ngxDirectiveThirdParty contained lua_sa_restart
  syn keyword ngxDirectiveThirdParty contained lua_shared_dict
  syn keyword ngxDirectiveThirdParty contained lua_socket_buffer_size
  syn keyword ngxDirectiveThirdParty contained lua_socket_connect_timeout
@@ -1355,9 +1409,15 @@
  " https://www.phusionpassenger.com/library/config/nginx/reference/
  syn keyword ngxDirectiveThirdParty contained 
passenger_abort_on_startup_error
  syn keyword ngxDirectiveThirdParty contained 
passenger_abort_websockets_on_process_shutdown
+syn keyword ngxDirectiveThirdParty contained 
passenger_admin_panel_auth_type
+syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_password
+syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_url
+syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_username
+syn keyword ngxDirectiveThirdParty contained 
passenger_anonymous_telemetry_proxy
  syn keyword ngxDirectiveThirdParty contained passenger_app_env
  syn keyword ngxDirectiveThirdParty contained 
passenger_app_file_descriptor_ulimit
  syn keyword ngxDirectiveThirdParty contained passenger_app_group_name
+syn keyword ngxDirectiveThirdParty contained passenger_app_log_file
  syn keyword ngxDirectiveThirdParty contained passenger_app_rights
  syn keyword ngxDirectiveThirdParty contained passenger_app_root
  syn keyword ngxDirectiveThirdParty contained passenger_app_type
@@ -1373,8 +1433,10 @@
  syn keyword ngxDirectiveThirdParty contained passenger_debugger
  syn keyword ngxDirectiveThirdParty contained passenger_default_group
  syn keyword ngxDirectiveThirdParty contained passenger_default_user
+syn keyword ngxDirectiveThirdParty contained 
passenger_disable_anonymous_telemetry
  syn keyword ngxDirectiveThirdParty contained 
passenger_disable_security_update_check
  syn keyword ngxDirectiveThirdParty contained passenger_document_root
+syn keyword ngxDirectiveThirdParty contained passenger_dump_config_manifest
  syn keyword ngxDirectiveThirdParty contained passenger_enabled
  syn keyword ngxDirectiveThirdParty contained passenger_env_var
  syn keyword ngxDirectiveThirdParty contained 
passenger_file_descriptor_log_file
@@ -1402,6 +1464,7 @@
  syn keyword ngxDirectiveThirdParty contained passenger_memory_limit
  syn keyword ngxDirectiveThirdParty contained passenger_meteor_app_settings
  syn keyword ngxDirectiveThirdParty contained passenger_min_instances
+syn keyword ngxDirectiveThirdParty contained passenger_monitor_log_file
  syn keyword ngxDirectiveThirdParty contained passenger_nodejs
  syn keyword ngxDirectiveThirdParty contained passenger_pass_header
  syn keyword ngxDirectiveThirdParty contained passenger_pool_idle_time
@@ -1778,6 +1841,8 @@
  syn keyword ngxDirectiveThirdParty contained 
vhost_traffic_status_filter_by_host
  syn keyword ngxDirectiveThirdParty contained 
vhost_traffic_status_filter_by_set_key
  syn keyword ngxDirectiveThirdParty contained 
vhost_traffic_status_filter_check_duplicate
+syn keyword ngxDirectiveThirdParty contained 
vhost_traffic_status_filter_max_node
+syn keyword ngxDirectiveThirdParty contained 
vhost_traffic_status_histogram_buckets
  syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_limit
  syn keyword ngxDirectiveThirdParty contained 
vhost_traffic_status_limit_check_duplicate
  syn keyword ngxDirectiveThirdParty contained 
vhost_traffic_status_limit_traffic
@@ -1899,11 +1964,11 @@

  " ngx_http_accounting_module
  " https://github.com/Lax/ngx_http_accounting_module
-syn keyword ngxDirectiveThirdParty contained http_accounting
-syn keyword ngxDirectiveThirdParty contained http_accounting_id
-syn keyword ngxDirectiveThirdParty contained http_accounting_interval
-syn keyword ngxDirectiveThirdParty contained http_accounting_log
-syn keyword ngxDirectiveThirdParty contained http_accounting_perturb
+syn keyword ngxDirectiveThirdParty contained accounting
+syn keyword ngxDirectiveThirdParty contained accounting_id
+syn keyword ngxDirectiveThirdParty contained accounting_interval
+syn keyword ngxDirectiveThirdParty contained accounting_log
+syn keyword ngxDirectiveThirdParty contained accounting_perturb

  " concatenating files in a given context: CSS and JS files usually
  " https://github.com/alibaba/nginx-http-concat


From hongzhidao at gmail.com  Mon Dec 24 15:48:31 2018
From: hongzhidao at gmail.com (=?UTF-8?B?5rSq5b+X6YGT?=)
Date: Mon, 24 Dec 2018 23:48:31 +0800
Subject: The reason of ignoring subrequest in access phase hander.
In-Reply-To: <20181224140654.GK99070@mdounin.ru>
References: <CAJuLd0gLsxjgLbHOqSbEXwALgJOa5E2xfG-uqMXaT82LkBnzAQ@mail.gmail.com>
 <20181224140654.GK99070@mdounin.ru>
Message-ID: <CAJuLd0htcS7=UmcJieyLVxvCcL5LHu1G3WNzgF0tkHkjb32MWg@mail.gmail.com>

Get it, thanks for your explanation.

I faced it by the following usage of lua, and I think it's resonable for
developers.

location /test {
   content_by_lua_block {
       local res = ngx.location.capture("/other"); -- post subrequest
   }
}

location /other {
   access_by_lua_block {
       ...
   }
}

It's ok for us to replace it with rewrite.
Anyway, do you still think it's better to restrict subrequests in access
phase handler?



On Mon, Dec 24, 2018 at 10:06 PM Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Mon, Dec 24, 2018 at 03:38:00AM +0800, ??? wrote:
>
> > Hi.
> >
> > I'm wondering the reason of ignoring subrequests in access phase hander.
> > Can anyone explain it to me?
> >
> > 1069 ngx_int_t
> > 1070 ngx_http_core_access_phase(ngx_http_request_t *r,
> > ngx_http_phase_handler_t *ph)
> > 1071 {
> > 1072     ngx_int_t                  rc;
> > 1073     ngx_http_core_loc_conf_t  *clcf;
> > 1074
> > 1075     if (r != r->main) {
> > 1076         r->phase_handler = ph->next;
> > 1077         return NGX_AGAIN;
> > 1078     }
> > 1079
>
> Access checks are expected to apply to the main request, but make
> little to no sense for subrequests.  E.g., if you are using
> IP-based restrictions or Basic HTTP authentication, both are
> exactly identical for the main request and subrequests.  As such,
> it is enough to do the checks for the main request only, and this
> is what nginx does.
>
> Note that it implies that access restrictions configured for the
> main requests are expected to be enough for all subrequests it can
> initiate.
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20181224/63bfedf6/attachment-0001.html>

From pluknet at nginx.com  Mon Dec 24 17:13:59 2018
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 24 Dec 2018 17:13:59 +0000
Subject: [nginx] Userid: using stub for AF_UNIX addresses.
Message-ID: <hg.ecc97cb0eda4.1545671639.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/ecc97cb0eda4
branches:  
changeset: 7432:ecc97cb0eda4
user:      Sergey Kandaurov <pluknet at nginx.com>
date:      Mon Dec 24 19:55:00 2018 +0300
description:
Userid: using stub for AF_UNIX addresses.

Previously, AF_UNIX addresses misbehaved as AF_INET, which typically resulted
in $uid_set composed from the middle of sun_path.

diffstat:

 src/http/modules/ngx_http_userid_filter_module.c |  7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diffs (17 lines):

diff -r 294162223c7c -r ecc97cb0eda4 src/http/modules/ngx_http_userid_filter_module.c
--- a/src/http/modules/ngx_http_userid_filter_module.c	Tue Dec 18 15:15:15 2018 +0300
+++ b/src/http/modules/ngx_http_userid_filter_module.c	Mon Dec 24 19:55:00 2018 +0300
@@ -545,6 +545,13 @@ ngx_http_userid_create_uid(ngx_http_requ
 
                 break;
 #endif
+
+#if (NGX_HAVE_UNIX_DOMAIN)
+            case AF_UNIX:
+                ctx->uid_set[0] = 0;
+                break;
+#endif
+
             default: /* AF_INET */
                 sin = (struct sockaddr_in *) c->local_sockaddr;
                 ctx->uid_set[0] = sin->sin_addr.s_addr;

From mdounin at mdounin.ru  Mon Dec 24 18:07:36 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 24 Dec 2018 18:07:36 +0000
Subject: [nginx] Win32: removed NGX_DIR_MASK concept.
Message-ID: <hg.061ec464813f.1545674856.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/061ec464813f
branches:  
changeset: 7433:061ec464813f
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Dec 24 21:07:05 2018 +0300
description:
Win32: removed NGX_DIR_MASK concept.

Previous interface of ngx_open_dir() assumed that passed directory name
has a room for NGX_DIR_MASK at the end (NGX_DIR_MASK_LEN bytes).  While all
direct users of ngx_dir_open() followed this interface, this also implied
similar requirements for indirect uses - in particular, via ngx_walk_tree().

Currently none of ngx_walk_tree() uses provides appropriate space, and
fixing this does not look like a right way to go.  Instead, ngx_dir_open()
interface was changed to not require any additional space and use
appropriate allocations instead.

diffstat:

 src/core/ngx_file.c                             |   4 ++--
 src/http/modules/ngx_http_autoindex_module.c    |   2 --
 src/http/modules/ngx_http_random_index_module.c |   2 +-
 src/os/unix/ngx_files.h                         |   3 ---
 src/os/win32/ngx_files.c                        |  21 ++++++++++++++++++---
 src/os/win32/ngx_files.h                        |   3 ---
 6 files changed, 21 insertions(+), 14 deletions(-)

diffs (107 lines):

diff --git a/src/core/ngx_file.c b/src/core/ngx_file.c
--- a/src/core/ngx_file.c
+++ b/src/core/ngx_file.c
@@ -1017,13 +1017,13 @@ ngx_walk_tree(ngx_tree_ctx_t *ctx, ngx_s
 
         file.len = tree->len + 1 + len;
 
-        if (file.len + NGX_DIR_MASK_LEN > buf.len) {
+        if (file.len > buf.len) {
 
             if (buf.len) {
                 ngx_free(buf.data);
             }
 
-            buf.len = tree->len + 1 + len + NGX_DIR_MASK_LEN;
+            buf.len = tree->len + 1 + len;
 
             buf.data = ngx_alloc(buf.len + 1, ctx->log);
             if (buf.data == NULL) {
diff --git a/src/http/modules/ngx_http_autoindex_module.c b/src/http/modules/ngx_http_autoindex_module.c
--- a/src/http/modules/ngx_http_autoindex_module.c
+++ b/src/http/modules/ngx_http_autoindex_module.c
@@ -186,8 +186,6 @@ ngx_http_autoindex_handler(ngx_http_requ
         return rc;
     }
 
-    /* NGX_DIR_MASK_LEN is lesser than NGX_HTTP_AUTOINDEX_PREALLOCATE */
-
     last = ngx_http_map_uri_to_path(r, &path, &root,
                                     NGX_HTTP_AUTOINDEX_PREALLOCATE);
     if (last == NULL) {
diff --git a/src/http/modules/ngx_http_random_index_module.c b/src/http/modules/ngx_http_random_index_module.c
--- a/src/http/modules/ngx_http_random_index_module.c
+++ b/src/http/modules/ngx_http_random_index_module.c
@@ -98,7 +98,7 @@ ngx_http_random_index_handler(ngx_http_r
     }
 
 #if (NGX_HAVE_D_TYPE)
-    len = NGX_DIR_MASK_LEN;
+    len = 0;
 #else
     len = NGX_HTTP_RANDOM_INDEX_PREALLOCATE;
 #endif
diff --git a/src/os/unix/ngx_files.h b/src/os/unix/ngx_files.h
--- a/src/os/unix/ngx_files.h
+++ b/src/os/unix/ngx_files.h
@@ -213,9 +213,6 @@ void ngx_close_file_mapping(ngx_file_map
 #endif
 
 
-#define NGX_DIR_MASK_LEN         0
-
-
 ngx_int_t ngx_open_dir(ngx_str_t *name, ngx_dir_t *dir);
 #define ngx_open_dir_n           "opendir()"
 
diff --git a/src/os/win32/ngx_files.c b/src/os/win32/ngx_files.c
--- a/src/os/win32/ngx_files.c
+++ b/src/os/win32/ngx_files.c
@@ -427,16 +427,31 @@ ngx_realpath(u_char *path, u_char *resol
 ngx_int_t
 ngx_open_dir(ngx_str_t *name, ngx_dir_t *dir)
 {
-    ngx_cpystrn(name->data + name->len, NGX_DIR_MASK, NGX_DIR_MASK_LEN + 1);
+    u_char     *pattern, *p;
+    ngx_err_t   err;
+
+    pattern = malloc(name->len + 3);
+    if (pattern == NULL) {
+        return NGX_ERROR;
+    }
 
-    dir->dir = FindFirstFile((const char *) name->data, &dir->finddata);
+    p = ngx_cpymem(pattern, name->data, name->len);
 
-    name->data[name->len] = '\0';
+    *p++ = '/';
+    *p++ = '*';
+    *p = '\0';
+
+    dir->dir = FindFirstFile((const char *) pattern, &dir->finddata);
 
     if (dir->dir == INVALID_HANDLE_VALUE) {
+        err = ngx_errno;
+        ngx_free(pattern);
+        ngx_set_errno(err);
         return NGX_ERROR;
     }
 
+    ngx_free(pattern);
+
     dir->valid_info = 1;
     dir->ready = 1;
 
diff --git a/src/os/win32/ngx_files.h b/src/os/win32/ngx_files.h
--- a/src/os/win32/ngx_files.h
+++ b/src/os/win32/ngx_files.h
@@ -181,9 +181,6 @@ u_char *ngx_realpath(u_char *path, u_cha
 #define NGX_HAVE_MAX_PATH           1
 #define NGX_MAX_PATH                MAX_PATH
 
-#define NGX_DIR_MASK                (u_char *) "/*"
-#define NGX_DIR_MASK_LEN            2
-
 
 ngx_int_t ngx_open_dir(ngx_str_t *name, ngx_dir_t *dir);
 #define ngx_open_dir_n              "FindFirstFile()"

From mdounin at mdounin.ru  Mon Dec 24 18:09:11 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 24 Dec 2018 18:09:11 +0000
Subject: [nginx] Contrib: vim syntax,
 update core and 3rd party module directives.
Message-ID: <hg.e3b262e7fc88.1545674951.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/e3b262e7fc88
branches:  
changeset: 7434:e3b262e7fc88
user:      Gena Makhomed <gmm at csdoc.com>
date:      Mon Dec 24 16:30:10 2018 +0200
description:
Contrib: vim syntax, update core and 3rd party module directives.

diffstat:

 contrib/vim/syntax/nginx.vim |  79 ++++++++++++++++++++++++++++++++++++++++---
 1 files changed, 72 insertions(+), 7 deletions(-)

diffs (288 lines):

diff --git a/contrib/vim/syntax/nginx.vim b/contrib/vim/syntax/nginx.vim
--- a/contrib/vim/syntax/nginx.vim
+++ b/contrib/vim/syntax/nginx.vim
@@ -108,6 +108,7 @@ syn keyword ngxDirectiveControl containe
 syn keyword ngxDirectiveError contained error_page
 syn keyword ngxDirectiveError contained post_action
 
+syn keyword ngxDirectiveDeprecated contained limit_zone
 syn keyword ngxDirectiveDeprecated contained proxy_downstream_buffer
 syn keyword ngxDirectiveDeprecated contained proxy_upstream_buffer
 syn keyword ngxDirectiveDeprecated contained spdy_chunk_size
@@ -118,6 +119,7 @@ syn keyword ngxDirectiveDeprecated conta
 syn keyword ngxDirectiveDeprecated contained spdy_recv_buffer_size
 syn keyword ngxDirectiveDeprecated contained spdy_recv_timeout
 syn keyword ngxDirectiveDeprecated contained spdy_streams_index_size
+syn keyword ngxDirectiveDeprecated contained ssl
 syn keyword ngxDirectiveDeprecated contained upstream_conf
 
 syn keyword ngxDirective contained absolute_redirect
@@ -136,6 +138,7 @@ syn keyword ngxDirective contained alias
 syn keyword ngxDirective contained allow
 syn keyword ngxDirective contained ancient_browser
 syn keyword ngxDirective contained ancient_browser_value
+syn keyword ngxDirective contained api
 syn keyword ngxDirective contained auth_basic
 syn keyword ngxDirective contained auth_basic_user_file
 syn keyword ngxDirective contained auth_http
@@ -143,7 +146,11 @@ syn keyword ngxDirective contained auth_
 syn keyword ngxDirective contained auth_http_pass_client_cert
 syn keyword ngxDirective contained auth_http_timeout
 syn keyword ngxDirective contained auth_jwt
+syn keyword ngxDirective contained auth_jwt_claim_set
+syn keyword ngxDirective contained auth_jwt_header_set
 syn keyword ngxDirective contained auth_jwt_key_file
+syn keyword ngxDirective contained auth_jwt_key_request
+syn keyword ngxDirective contained auth_jwt_leeway
 syn keyword ngxDirective contained auth_request
 syn keyword ngxDirective contained auth_request_set
 syn keyword ngxDirective contained autoindex
@@ -229,6 +236,7 @@ syn keyword ngxDirective contained fastc
 syn keyword ngxDirective contained fastcgi_request_buffering
 syn keyword ngxDirective contained fastcgi_send_lowat
 syn keyword ngxDirective contained fastcgi_send_timeout
+syn keyword ngxDirective contained fastcgi_socket_keepalive
 syn keyword ngxDirective contained fastcgi_split_path_info
 syn keyword ngxDirective contained fastcgi_store
 syn keyword ngxDirective contained fastcgi_store_access
@@ -255,6 +263,7 @@ syn keyword ngxDirective contained grpc_
 syn keyword ngxDirective contained grpc_read_timeout
 syn keyword ngxDirective contained grpc_send_timeout
 syn keyword ngxDirective contained grpc_set_header
+syn keyword ngxDirective contained grpc_socket_keepalive
 syn keyword ngxDirective contained grpc_ssl_certificate
 syn keyword ngxDirective contained grpc_ssl_certificate_key
 syn keyword ngxDirective contained grpc_ssl_ciphers
@@ -330,6 +339,8 @@ syn keyword ngxDirective contained keepa
 syn keyword ngxDirective contained keepalive_disable
 syn keyword ngxDirective contained keepalive_requests
 syn keyword ngxDirective contained keepalive_timeout
+syn keyword ngxDirective contained keyval
+syn keyword ngxDirective contained keyval_zone
 syn keyword ngxDirective contained kqueue_changes
 syn keyword ngxDirective contained kqueue_events
 syn keyword ngxDirective contained large_client_header_buffers
@@ -367,6 +378,7 @@ syn keyword ngxDirective contained memca
 syn keyword ngxDirective contained memcached_next_upstream_tries
 syn keyword ngxDirective contained memcached_read_timeout
 syn keyword ngxDirective contained memcached_send_timeout
+syn keyword ngxDirective contained memcached_socket_keepalive
 syn keyword ngxDirective contained merge_slashes
 syn keyword ngxDirective contained min_delete_depth
 syn keyword ngxDirective contained mirror
@@ -375,9 +387,9 @@ syn keyword ngxDirective contained moder
 syn keyword ngxDirective contained modern_browser_value
 syn keyword ngxDirective contained mp4
 syn keyword ngxDirective contained mp4_buffer_size
-syn keyword ngxDirective contained mp4_max_buffer_size
 syn keyword ngxDirective contained mp4_limit_rate
 syn keyword ngxDirective contained mp4_limit_rate_after
+syn keyword ngxDirective contained mp4_max_buffer_size
 syn keyword ngxDirective contained msie_padding
 syn keyword ngxDirective contained msie_refresh
 syn keyword ngxDirective contained multi_accept
@@ -456,11 +468,13 @@ syn keyword ngxDirective contained proxy
 syn keyword ngxDirective contained proxy_read_timeout
 syn keyword ngxDirective contained proxy_redirect
 syn keyword ngxDirective contained proxy_request_buffering
+syn keyword ngxDirective contained proxy_requests
 syn keyword ngxDirective contained proxy_responses
 syn keyword ngxDirective contained proxy_send_lowat
 syn keyword ngxDirective contained proxy_send_timeout
 syn keyword ngxDirective contained proxy_set_body
 syn keyword ngxDirective contained proxy_set_header
+syn keyword ngxDirective contained proxy_socket_keepalive
 syn keyword ngxDirective contained proxy_ssl
 syn keyword ngxDirective contained proxy_ssl_certificate
 syn keyword ngxDirective contained proxy_ssl_certificate_key
@@ -481,6 +495,7 @@ syn keyword ngxDirective contained proxy
 syn keyword ngxDirective contained proxy_timeout
 syn keyword ngxDirective contained proxy_upload_rate
 syn keyword ngxDirective contained queue
+syn keyword ngxDirective contained random
 syn keyword ngxDirective contained random_index
 syn keyword ngxDirective contained read_ahead
 syn keyword ngxDirective contained real_ip_header
@@ -533,6 +548,7 @@ syn keyword ngxDirective contained scgi_
 syn keyword ngxDirective contained scgi_read_timeout
 syn keyword ngxDirective contained scgi_request_buffering
 syn keyword ngxDirective contained scgi_send_timeout
+syn keyword ngxDirective contained scgi_socket_keepalive
 syn keyword ngxDirective contained scgi_store
 syn keyword ngxDirective contained scgi_store_access
 syn keyword ngxDirective contained scgi_temp_file_write_size
@@ -565,7 +581,6 @@ syn keyword ngxDirective contained ssi_m
 syn keyword ngxDirective contained ssi_silent_errors
 syn keyword ngxDirective contained ssi_types
 syn keyword ngxDirective contained ssi_value_length
-syn keyword ngxDirective contained ssl
 syn keyword ngxDirective contained ssl_buffer_size
 syn keyword ngxDirective contained ssl_certificate
 syn keyword ngxDirective contained ssl_certificate_key
@@ -573,6 +588,7 @@ syn keyword ngxDirective contained ssl_c
 syn keyword ngxDirective contained ssl_client_certificate
 syn keyword ngxDirective contained ssl_crl
 syn keyword ngxDirective contained ssl_dhparam
+syn keyword ngxDirective contained ssl_early_data
 syn keyword ngxDirective contained ssl_ecdh_curve
 syn keyword ngxDirective contained ssl_engine
 syn keyword ngxDirective contained ssl_handshake_timeout
@@ -664,6 +680,7 @@ syn keyword ngxDirective contained uwsgi
 syn keyword ngxDirective contained uwsgi_read_timeout
 syn keyword ngxDirective contained uwsgi_request_buffering
 syn keyword ngxDirective contained uwsgi_send_timeout
+syn keyword ngxDirective contained uwsgi_socket_keepalive
 syn keyword ngxDirective contained uwsgi_ssl_certificate
 syn keyword ngxDirective contained uwsgi_ssl_certificate_key
 syn keyword ngxDirective contained uwsgi_ssl_ciphers
@@ -701,6 +718,26 @@ syn keyword ngxDirective contained xslt_
 syn keyword ngxDirective contained xslt_stylesheet
 syn keyword ngxDirective contained xslt_types
 syn keyword ngxDirective contained zone
+syn keyword ngxDirective contained zone_sync
+syn keyword ngxDirective contained zone_sync_buffers
+syn keyword ngxDirective contained zone_sync_connect_retry_interval
+syn keyword ngxDirective contained zone_sync_connect_timeout
+syn keyword ngxDirective contained zone_sync_interval
+syn keyword ngxDirective contained zone_sync_recv_buffer_size
+syn keyword ngxDirective contained zone_sync_server
+syn keyword ngxDirective contained zone_sync_ssl
+syn keyword ngxDirective contained zone_sync_ssl_certificate
+syn keyword ngxDirective contained zone_sync_ssl_certificate_key
+syn keyword ngxDirective contained zone_sync_ssl_ciphers
+syn keyword ngxDirective contained zone_sync_ssl_crl
+syn keyword ngxDirective contained zone_sync_ssl_name
+syn keyword ngxDirective contained zone_sync_ssl_password_file
+syn keyword ngxDirective contained zone_sync_ssl_protocols
+syn keyword ngxDirective contained zone_sync_ssl_server_name
+syn keyword ngxDirective contained zone_sync_ssl_trusted_certificate
+syn keyword ngxDirective contained zone_sync_ssl_verify
+syn keyword ngxDirective contained zone_sync_ssl_verify_depth
+syn keyword ngxDirective contained zone_sync_timeout
 
 " 3rd party modules list taken from
 " https://github.com/freebsd/freebsd-ports/blob/master/www/nginx-devel/Makefile
@@ -876,6 +913,8 @@ syn keyword ngxDirectiveThirdParty conta
 
 " NGINX WebDAV missing commands support (PROPFIND & OPTIONS)
 " https://github.com/arut/nginx-dav-ext-module
+syn keyword ngxDirectiveThirdParty contained dav_ext_lock
+syn keyword ngxDirectiveThirdParty contained dav_ext_lock_zone
 syn keyword ngxDirectiveThirdParty contained dav_ext_methods
 
 " ngx_eval
@@ -895,6 +934,7 @@ syn keyword ngxDirectiveThirdParty conta
 syn keyword ngxDirectiveThirdParty contained fancyindex_exact_size
 syn keyword ngxDirectiveThirdParty contained fancyindex_footer
 syn keyword ngxDirectiveThirdParty contained fancyindex_header
+syn keyword ngxDirectiveThirdParty contained fancyindex_hide_parent_dir
 syn keyword ngxDirectiveThirdParty contained fancyindex_hide_symlinks
 syn keyword ngxDirectiveThirdParty contained fancyindex_ignore
 syn keyword ngxDirectiveThirdParty contained fancyindex_localtime
@@ -937,8 +977,17 @@ syn keyword ngxDirectiveThirdParty conta
 
 " nchan
 " https://github.com/slact/nchan
+syn keyword ngxDirectiveThirdParty contained nchan_access_control_allow_credentials
 syn keyword ngxDirectiveThirdParty contained nchan_access_control_allow_origin
 syn keyword ngxDirectiveThirdParty contained nchan_authorize_request
+syn keyword ngxDirectiveThirdParty contained nchan_benchmark
+syn keyword ngxDirectiveThirdParty contained nchan_benchmark_channels
+syn keyword ngxDirectiveThirdParty contained nchan_benchmark_message_padding_bytes
+syn keyword ngxDirectiveThirdParty contained nchan_benchmark_messages_per_channel_per_minute
+syn keyword ngxDirectiveThirdParty contained nchan_benchmark_publisher_distribution
+syn keyword ngxDirectiveThirdParty contained nchan_benchmark_subscriber_distribution
+syn keyword ngxDirectiveThirdParty contained nchan_benchmark_subscribers_per_channel
+syn keyword ngxDirectiveThirdParty contained nchan_benchmark_time
 syn keyword ngxDirectiveThirdParty contained nchan_channel_event_string
 syn keyword ngxDirectiveThirdParty contained nchan_channel_events_channel_id
 syn keyword ngxDirectiveThirdParty contained nchan_channel_group
@@ -974,15 +1023,19 @@ syn keyword ngxDirectiveThirdParty conta
 syn keyword ngxDirectiveThirdParty contained nchan_pubsub
 syn keyword ngxDirectiveThirdParty contained nchan_pubsub_channel_id
 syn keyword ngxDirectiveThirdParty contained nchan_pubsub_location
+syn keyword ngxDirectiveThirdParty contained nchan_redis_connect_timeout
 syn keyword ngxDirectiveThirdParty contained nchan_redis_fakesub_timer_interval
 syn keyword ngxDirectiveThirdParty contained nchan_redis_idle_channel_cache_timeout
 syn keyword ngxDirectiveThirdParty contained nchan_redis_namespace
+syn keyword ngxDirectiveThirdParty contained nchan_redis_nostore_fastpublish
+syn keyword ngxDirectiveThirdParty contained nchan_redis_optimize_target
 syn keyword ngxDirectiveThirdParty contained nchan_redis_pass
 syn keyword ngxDirectiveThirdParty contained nchan_redis_pass_inheritable
 syn keyword ngxDirectiveThirdParty contained nchan_redis_ping_interval
 syn keyword ngxDirectiveThirdParty contained nchan_redis_publish_msgpacked_max_size
 syn keyword ngxDirectiveThirdParty contained nchan_redis_server
 syn keyword ngxDirectiveThirdParty contained nchan_redis_storage_mode
+syn keyword ngxDirectiveThirdParty contained nchan_redis_subscribe_weights
 syn keyword ngxDirectiveThirdParty contained nchan_redis_url
 syn keyword ngxDirectiveThirdParty contained nchan_redis_wait_after_connecting
 syn keyword ngxDirectiveThirdParty contained nchan_shared_memory_size
@@ -1280,6 +1333,7 @@ syn keyword ngxDirectiveThirdParty conta
 syn keyword ngxDirectiveThirdParty contained lua_package_path
 syn keyword ngxDirectiveThirdParty contained lua_regex_cache_max_entries
 syn keyword ngxDirectiveThirdParty contained lua_regex_match_limit
+syn keyword ngxDirectiveThirdParty contained lua_sa_restart
 syn keyword ngxDirectiveThirdParty contained lua_shared_dict
 syn keyword ngxDirectiveThirdParty contained lua_socket_buffer_size
 syn keyword ngxDirectiveThirdParty contained lua_socket_connect_timeout
@@ -1355,9 +1409,15 @@ syn keyword ngxDirectiveThirdParty conta
 " https://www.phusionpassenger.com/library/config/nginx/reference/
 syn keyword ngxDirectiveThirdParty contained passenger_abort_on_startup_error
 syn keyword ngxDirectiveThirdParty contained passenger_abort_websockets_on_process_shutdown
+syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_auth_type
+syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_password
+syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_url
+syn keyword ngxDirectiveThirdParty contained passenger_admin_panel_username
+syn keyword ngxDirectiveThirdParty contained passenger_anonymous_telemetry_proxy
 syn keyword ngxDirectiveThirdParty contained passenger_app_env
 syn keyword ngxDirectiveThirdParty contained passenger_app_file_descriptor_ulimit
 syn keyword ngxDirectiveThirdParty contained passenger_app_group_name
+syn keyword ngxDirectiveThirdParty contained passenger_app_log_file
 syn keyword ngxDirectiveThirdParty contained passenger_app_rights
 syn keyword ngxDirectiveThirdParty contained passenger_app_root
 syn keyword ngxDirectiveThirdParty contained passenger_app_type
@@ -1373,8 +1433,10 @@ syn keyword ngxDirectiveThirdParty conta
 syn keyword ngxDirectiveThirdParty contained passenger_debugger
 syn keyword ngxDirectiveThirdParty contained passenger_default_group
 syn keyword ngxDirectiveThirdParty contained passenger_default_user
+syn keyword ngxDirectiveThirdParty contained passenger_disable_anonymous_telemetry
 syn keyword ngxDirectiveThirdParty contained passenger_disable_security_update_check
 syn keyword ngxDirectiveThirdParty contained passenger_document_root
+syn keyword ngxDirectiveThirdParty contained passenger_dump_config_manifest
 syn keyword ngxDirectiveThirdParty contained passenger_enabled
 syn keyword ngxDirectiveThirdParty contained passenger_env_var
 syn keyword ngxDirectiveThirdParty contained passenger_file_descriptor_log_file
@@ -1402,6 +1464,7 @@ syn keyword ngxDirectiveThirdParty conta
 syn keyword ngxDirectiveThirdParty contained passenger_memory_limit
 syn keyword ngxDirectiveThirdParty contained passenger_meteor_app_settings
 syn keyword ngxDirectiveThirdParty contained passenger_min_instances
+syn keyword ngxDirectiveThirdParty contained passenger_monitor_log_file
 syn keyword ngxDirectiveThirdParty contained passenger_nodejs
 syn keyword ngxDirectiveThirdParty contained passenger_pass_header
 syn keyword ngxDirectiveThirdParty contained passenger_pool_idle_time
@@ -1778,6 +1841,8 @@ syn keyword ngxDirectiveThirdParty conta
 syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_filter_by_host
 syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_filter_by_set_key
 syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_filter_check_duplicate
+syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_filter_max_node
+syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_histogram_buckets
 syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_limit
 syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_limit_check_duplicate
 syn keyword ngxDirectiveThirdParty contained vhost_traffic_status_limit_traffic
@@ -1899,11 +1964,11 @@ syn keyword ngxDirectiveThirdParty conta
 
 " ngx_http_accounting_module
 " https://github.com/Lax/ngx_http_accounting_module
-syn keyword ngxDirectiveThirdParty contained http_accounting
-syn keyword ngxDirectiveThirdParty contained http_accounting_id
-syn keyword ngxDirectiveThirdParty contained http_accounting_interval
-syn keyword ngxDirectiveThirdParty contained http_accounting_log
-syn keyword ngxDirectiveThirdParty contained http_accounting_perturb
+syn keyword ngxDirectiveThirdParty contained accounting
+syn keyword ngxDirectiveThirdParty contained accounting_id
+syn keyword ngxDirectiveThirdParty contained accounting_interval
+syn keyword ngxDirectiveThirdParty contained accounting_log
+syn keyword ngxDirectiveThirdParty contained accounting_perturb
 
 " concatenating files in a given context: CSS and JS files usually
 " https://github.com/alibaba/nginx-http-concat

From mdounin at mdounin.ru  Mon Dec 24 18:10:58 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 24 Dec 2018 21:10:58 +0300
Subject: [PATCH] Contrib: vim syntax,
 update core and 3rd party module directives.
In-Reply-To: <0b8fa025-fda8-0c7a-7a9e-addfab4e576f@csdoc.com>
References: <0b8fa025-fda8-0c7a-7a9e-addfab4e576f@csdoc.com>
Message-ID: <20181224181058.GO99070@mdounin.ru>

Hello!

On Mon, Dec 24, 2018 at 04:35:22PM +0200, Gena Makhomed wrote:

> # HG changeset patch
> # User Gena Makhomed <gmm at csdoc.com>
> # Date 1545661810 -7200
> #      Mon Dec 24 16:30:10 2018 +0200
> # Node ID 19f1cbe0be527cbbfea8d8e7e8693352e72e745b
> # Parent  294162223c7c51a147c07eba36d0506d54edbf20
> Contrib: vim syntax, update core and 3rd party module directives.

Committed, thanks.

-- 
Maxim Dounin
http://mdounin.ru/

From terence at honles.com  Mon Dec 24 21:47:36 2018
From: terence at honles.com (Terence Honles)
Date: Mon, 24 Dec 2018 13:47:36 -0800
Subject: [PATCH] better constrain IP-literal validation in
 ngx_http_validate_host()
In-Reply-To: <20181224125858.GH99070@mdounin.ru>
References: <CAHtY7_pxP9JXLgJgpwzJRNs=9Vtd-1s6efzrktCu-7EYqpjUvA@mail.gmail.com>
 <20181217161740.GT99070@mdounin.ru>
 <CAHtY7_rG0zYJn+HUCgfzr7uDP5fYtzrmQPw85zGzyb05wH0dZA@mail.gmail.com>
 <20181224125858.GH99070@mdounin.ru>
Message-ID: <CAHtY7_pKzzoP_YLBX=RE=Z=xBuPDjMaNZvOU3D8ZWpL1vEqFpA@mail.gmail.com>

Yes, the regex will fail for IPv future literals, but I don't believe they are
being used in practice. When they are, I'm sure the Django project will
welcome the change to the RegEx.

As for the configuration you proposed, we are already using that (with a 444
instead of 404), but the IP literal will still pass through because it is a
valid match (but an invalid hostname according to RFC 3986).

-Terence

On Mon, Dec 24, 2018 at 4:58 AM Maxim Dounin <mdounin at mdounin.ru> wrote:
>
> Hello!
>
> On Fri, Dec 21, 2018 at 11:59:27AM -0800, Terence Honles wrote:
>
> > The reason I came across this code was I have NGINX handling HTTPS, and
> > proxying to Django via uWSGI. Django has the following RegEx [1]_. Which is is
> > compliant with the IPv6 literal notation, but causes Django to report an
> > error.
>
> Well, as far as I see the regex in question will also report an
> error for perfectly valid IPvFuture literals.
>
> > While I agree, there may not be an issue of security; Any down stream systems
> > may be confused and unable to handle a malformed hostname.
>
> If you want to prevent such names from hitting your backend
> servers, a better solution might be to configure nginx to only
> accept explicitly configured host names (including IP literals).
> This is more or less trivial to configure and something you
> probably should do anyway unless you are still supporting
> pre-Host-header clients.  E.g.:
>
> server {
>     listen 80 default;
>     return 404;
> }
>
> server {
>     listen 80;
>     server_name example.com;
>     ...
> }
>
> > Another alternative would be to rewrite the hostname to not
> > include the "[" and "]" if it is not a valid IPv6 literal,
>
> Well, this _will_ be a security issue, since it can easily result
> in different processing of names in nginx and backends.
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>

From terence at honles.com  Mon Dec 24 22:11:20 2018
From: terence at honles.com (Terence Honles)
Date: Mon, 24 Dec 2018 14:11:20 -0800
Subject: [PATCH] better constrain IP-literal validation in
 ngx_http_validate_host()
In-Reply-To: <CAHtY7_pKzzoP_YLBX=RE=Z=xBuPDjMaNZvOU3D8ZWpL1vEqFpA@mail.gmail.com>
References: <CAHtY7_pxP9JXLgJgpwzJRNs=9Vtd-1s6efzrktCu-7EYqpjUvA@mail.gmail.com>
 <20181217161740.GT99070@mdounin.ru>
 <CAHtY7_rG0zYJn+HUCgfzr7uDP5fYtzrmQPw85zGzyb05wH0dZA@mail.gmail.com>
 <20181224125858.GH99070@mdounin.ru>
 <CAHtY7_pKzzoP_YLBX=RE=Z=xBuPDjMaNZvOU3D8ZWpL1vEqFpA@mail.gmail.com>
Message-ID: <CAHtY7_q=FD-n=b4cw7TQOCBwiKDxxfm1gu=iwZ2D_v8bP1meQg@mail.gmail.com>

I'll double check the configuration we have when I get back
into work next year, but until then have a Merry Christmas /
Happy holidays!

-Terence

On Mon, Dec 24, 2018, 13:47 Terence Honles <terence at honles.com wrote:

> Yes, the regex will fail for IPv future literals, but I don't believe they
> are
> being used in practice. When they are, I'm sure the Django project will
> welcome the change to the RegEx.
>
> As for the configuration you proposed, we are already using that (with a
> 444
> instead of 404), but the IP literal will still pass through because it is a
> valid match (but an invalid hostname according to RFC 3986).
>
> -Terence
>
> On Mon, Dec 24, 2018 at 4:58 AM Maxim Dounin <mdounin at mdounin.ru> wrote:
> >
> > Hello!
> >
> > On Fri, Dec 21, 2018 at 11:59:27AM -0800, Terence Honles wrote:
> >
> > > The reason I came across this code was I have NGINX handling HTTPS, and
> > > proxying to Django via uWSGI. Django has the following RegEx [1]_.
> Which is is
> > > compliant with the IPv6 literal notation, but causes Django to report
> an
> > > error.
> >
> > Well, as far as I see the regex in question will also report an
> > error for perfectly valid IPvFuture literals.
> >
> > > While I agree, there may not be an issue of security; Any down stream
> systems
> > > may be confused and unable to handle a malformed hostname.
> >
> > If you want to prevent such names from hitting your backend
> > servers, a better solution might be to configure nginx to only
> > accept explicitly configured host names (including IP literals).
> > This is more or less trivial to configure and something you
> > probably should do anyway unless you are still supporting
> > pre-Host-header clients.  E.g.:
> >
> > server {
> >     listen 80 default;
> >     return 404;
> > }
> >
> > server {
> >     listen 80;
> >     server_name example.com;
> >     ...
> > }
> >
> > > Another alternative would be to rewrite the hostname to not
> > > include the "[" and "]" if it is not a valid IPv6 literal,
> >
> > Well, this _will_ be a security issue, since it can easily result
> > in different processing of names in nginx and backends.
> >
> > --
> > Maxim Dounin
> > http://mdounin.ru/
> > _______________________________________________
> > nginx-devel mailing list
> > nginx-devel at nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx-devel
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20181224/88f1cd3c/attachment.html>

From xeioex at nginx.com  Tue Dec 25 06:52:14 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 25 Dec 2018 06:52:14 +0000
Subject: [njs] Version 0.2.7.
Message-ID: <hg.4624ba4f6497.1545720734.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/4624ba4f6497
branches:  
changeset: 693:4624ba4f6497
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Dec 25 09:49:52 2018 +0300
description:
Version 0.2.7.

diffstat:

 CHANGES |  26 ++++++++++++++++++++++++++
 1 files changed, 26 insertions(+), 0 deletions(-)

diffs (33 lines):

diff -r 5c8802f61ad6 -r 4624ba4f6497 CHANGES
--- a/CHANGES	Mon Dec 24 13:59:27 2018 +0300
+++ b/CHANGES	Tue Dec 25 09:49:52 2018 +0300
@@ -1,3 +1,29 @@
+
+Changes with njs 0.2.7                                           25 Dec 2018
+
+    Core:
+
+    *) Feature: rest parameters syntax (destructuring is not supported).
+       Thanks to Alexander Pyshchev.
+
+    *) Feature: added Object.entries() method.
+
+    *) Feature: added Object.values() method.
+
+    *) Improvement: code generator refactored and simplified.
+
+    *) Bugfix: fixed automatic semicolon insertion.
+
+    *) Bugfix: fixed assignment expression from compound assignment.
+
+    *) Bugfix: fixed comparison of Byte and UTF8 strings.
+
+    *) Bugfix: fixed type of iteration variable in for-in with array
+       values.
+
+    *) Bugfix: fixed building on paltforms without librt.
+
+    *) Bugfix: miscellaneous additional bugs have been fixed.
 
 Changes with njs 0.2.6                                           27 Nov 2018
 

From xeioex at nginx.com  Tue Dec 25 06:52:14 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 25 Dec 2018 06:52:14 +0000
Subject: [njs] Added tag 0.2.7 for changeset 4624ba4f6497
Message-ID: <hg.039a88aa833c.1545720734.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/039a88aa833c
branches:  
changeset: 694:039a88aa833c
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Dec 25 09:51:02 2018 +0300
description:
Added tag 0.2.7 for changeset 4624ba4f6497

diffstat:

 .hgtags |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (8 lines):

diff -r 4624ba4f6497 -r 039a88aa833c .hgtags
--- a/.hgtags	Tue Dec 25 09:49:52 2018 +0300
+++ b/.hgtags	Tue Dec 25 09:51:02 2018 +0300
@@ -21,3 +21,4 @@ e83f41520613987542472275d49633a9aa5955d1
 3e6c38f64bdbc53e783813541559034ed6890aee 0.2.4
 3315f6aa6000ce6d2dbc74c73660becf4178a549 0.2.5
 0709b8b4f11ebec95dd4f72d5bb38044682f77e6 0.2.6
+4624ba4f6497a3d10fe1c0a6f45fb453579502f5 0.2.7

From xeioex at nginx.com  Tue Dec 25 06:54:36 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 25 Dec 2018 06:54:36 +0000
Subject: [njs] Version bump.
Message-ID: <hg.7b1b3eb864e0.1545720876.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/7b1b3eb864e0
branches:  
changeset: 695:7b1b3eb864e0
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Dec 25 09:53:27 2018 +0300
description:
Version bump.

diffstat:

 njs/njs.h |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff -r 039a88aa833c -r 7b1b3eb864e0 njs/njs.h
--- a/njs/njs.h	Tue Dec 25 09:51:02 2018 +0300
+++ b/njs/njs.h	Tue Dec 25 09:53:27 2018 +0300
@@ -11,7 +11,7 @@
 
 #include <nxt_auto_config.h>
 
-#define NJS_VERSION                 "0.2.7"
+#define NJS_VERSION                 "0.2.8"
 
 
 #include <nxt_types.h>

From vl at nginx.com  Tue Dec 25 13:11:49 2018
From: vl at nginx.com (Vladimir Homutov)
Date: Tue, 25 Dec 2018 13:11:49 +0000
Subject: [nginx] Autoindex: fixed possible integer overflow on 32-bit systems.
Message-ID: <hg.a91b93f3f3e7.1545743509.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/a91b93f3f3e7
branches:  
changeset: 7435:a91b93f3f3e7
user:      Vladimir Homutov <vl at nginx.com>
date:      Tue Dec 25 12:59:24 2018 +0300
description:
Autoindex: fixed possible integer overflow on 32-bit systems.

diffstat:

 src/http/modules/ngx_http_autoindex_module.c |  66 +++++++++++++++++----------
 1 files changed, 42 insertions(+), 24 deletions(-)

diffs (117 lines):

diff -r e3b262e7fc88 -r a91b93f3f3e7 src/http/modules/ngx_http_autoindex_module.c
--- a/src/http/modules/ngx_http_autoindex_module.c	Mon Dec 24 16:30:10 2018 +0200
+++ b/src/http/modules/ngx_http_autoindex_module.c	Tue Dec 25 12:59:24 2018 +0300
@@ -434,7 +434,7 @@ ngx_http_autoindex_html(ngx_http_request
 {
     u_char                         *last, scale;
     off_t                           length;
-    size_t                          len, char_len, escape_html;
+    size_t                          len, entry_len, char_len, escape_html;
     ngx_tm_t                        tm;
     ngx_buf_t                      *b;
     ngx_int_t                       size;
@@ -499,17 +499,23 @@ ngx_http_autoindex_html(ngx_http_request
             entry[i].utf_len = entry[i].name.len;
         }
 
-        len += sizeof("<a href=\"") - 1
-            + entry[i].name.len + entry[i].escape
-            + 1                                          /* 1 is for "/" */
-            + sizeof("\">") - 1
-            + entry[i].name.len - entry[i].utf_len
-            + entry[i].escape_html
-            + NGX_HTTP_AUTOINDEX_NAME_LEN + sizeof("&gt;") - 2
-            + sizeof("</a>") - 1
-            + sizeof(" 28-Sep-1970 12:00 ") - 1
-            + 20                                         /* the file size */
-            + 2;
+        entry_len = sizeof("<a href=\"") - 1
+                  + entry[i].name.len + entry[i].escape
+                  + 1                                    /* 1 is for "/" */
+                  + sizeof("\">") - 1
+                  + entry[i].name.len - entry[i].utf_len
+                  + entry[i].escape_html
+                  + NGX_HTTP_AUTOINDEX_NAME_LEN + sizeof("&gt;") - 2
+                  + sizeof("</a>") - 1
+                  + sizeof(" 28-Sep-1970 12:00 ") - 1
+                  + 20                                   /* the file size */
+                  + 2;
+
+        if (len > NGX_MAX_SIZE_T_VALUE - entry_len) {
+            return NULL;
+        }
+
+        len += entry_len;
     }
 
     b = ngx_create_temp_buf(r->pool, len);
@@ -697,7 +703,7 @@ static ngx_buf_t *
 ngx_http_autoindex_json(ngx_http_request_t *r, ngx_array_t *entries,
     ngx_str_t *callback)
 {
-    size_t                       len;
+    size_t                       len, entry_len;
     ngx_buf_t                   *b;
     ngx_uint_t                   i;
     ngx_http_autoindex_entry_t  *entry;
@@ -714,15 +720,21 @@ ngx_http_autoindex_json(ngx_http_request
         entry[i].escape = ngx_escape_json(NULL, entry[i].name.data,
                                           entry[i].name.len);
 
-        len += sizeof("{  }," CRLF) - 1
-            + sizeof("\"name\":\"\"") - 1
-            + entry[i].name.len + entry[i].escape
-            + sizeof(", \"type\":\"directory\"") - 1
-            + sizeof(", \"mtime\":\"Wed, 31 Dec 1986 10:00:00 GMT\"") - 1;
+        entry_len = sizeof("{  }," CRLF) - 1
+                  + sizeof("\"name\":\"\"") - 1
+                  + entry[i].name.len + entry[i].escape
+                  + sizeof(", \"type\":\"directory\"") - 1
+                  + sizeof(", \"mtime\":\"Wed, 31 Dec 1986 10:00:00 GMT\"") - 1;
 
         if (entry[i].file) {
-            len += sizeof(", \"size\":") - 1 + NGX_OFF_T_LEN;
+            entry_len += sizeof(", \"size\":") - 1 + NGX_OFF_T_LEN;
         }
+
+        if (len > NGX_MAX_SIZE_T_VALUE - entry_len) {
+            return NULL;
+        }
+
+        len += entry_len;
     }
 
     b = ngx_create_temp_buf(r->pool, len);
@@ -841,7 +853,7 @@ ngx_http_autoindex_jsonp_callback(ngx_ht
 static ngx_buf_t *
 ngx_http_autoindex_xml(ngx_http_request_t *r, ngx_array_t *entries)
 {
-    size_t                          len;
+    size_t                          len, entry_len;
     ngx_tm_t                        tm;
     ngx_buf_t                      *b;
     ngx_str_t                       type;
@@ -859,13 +871,19 @@ ngx_http_autoindex_xml(ngx_http_request_
         entry[i].escape = ngx_escape_html(NULL, entry[i].name.data,
                                           entry[i].name.len);
 
-        len += sizeof("<directory></directory>" CRLF) - 1
-            + entry[i].name.len + entry[i].escape
-            + sizeof(" mtime=\"1986-12-31T10:00:00Z\"") - 1;
+        entry_len = sizeof("<directory></directory>" CRLF) - 1
+                  + entry[i].name.len + entry[i].escape
+                  + sizeof(" mtime=\"1986-12-31T10:00:00Z\"") - 1;
 
         if (entry[i].file) {
-            len += sizeof(" size=\"\"") - 1 + NGX_OFF_T_LEN;
+            entry_len += sizeof(" size=\"\"") - 1 + NGX_OFF_T_LEN;
         }
+
+        if (len > NGX_MAX_SIZE_T_VALUE - entry_len) {
+            return NULL;
+        }
+
+        len += entry_len;
     }
 
     b = ngx_create_temp_buf(r->pool, len);

From mdounin at mdounin.ru  Tue Dec 25 14:56:42 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 25 Dec 2018 14:56:42 +0000
Subject: [nginx] nginx-1.15.8-RELEASE
Message-ID: <hg.ee551e3f6dba.1545749802.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/ee551e3f6dba
branches:  
changeset: 7436:ee551e3f6dba
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Dec 25 17:53:03 2018 +0300
description:
nginx-1.15.8-RELEASE

diffstat:

 docs/xml/nginx/changes.xml |  76 ++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 76 insertions(+), 0 deletions(-)

diffs (86 lines):

diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml
--- a/docs/xml/nginx/changes.xml
+++ b/docs/xml/nginx/changes.xml
@@ -5,6 +5,82 @@
 <change_log title="nginx">
 
 
+<changes ver="1.15.8" date="2018-12-25">
+
+<change type="feature">
+<para lang="ru">
+?????????? $upstream_bytes_sent.<br/>
+??????? Piotr Sikora.
+</para>
+<para lang="en">
+the $upstream_bytes_sent variable.<br/>
+Thanks to Piotr Sikora.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+????? ????????? ? ???????? ????????? ?????????? ??? vim.<br/>
+??????? ???????? ????????.
+</para>
+<para lang="en">
+new directives in vim syntax highlighting scripts.<br/>
+Thanks to Gena Makhomed.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ????????? proxy_cache_background_update.
+</para>
+<para lang="en">
+in the "proxy_cache_background_update" directive.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ????????? geo ??? ????????????? unix domain listen-???????.
+</para>
+<para lang="en">
+in the "geo" directive when using unix domain listen sockets.
+</para>
+</change>
+
+<change type="workaround">
+<para lang="ru">
+??? ????????????? ????????? ssl_early_data ? OpenSSL
+? ????? ????? ?????????? ?????????
+"ignoring stale global SSL error ... bad length".
+</para>
+<para lang="en">
+the "ignoring stale global SSL error ... bad length"
+alerts might appear in logs
+when using the "ssl_early_data" directive with OpenSSL.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? nginx/Windows.
+</para>
+<para lang="en">
+in nginx/Windows.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ?????? ngx_http_autoindex_module ?? 32-?????? ??????????.
+</para>
+<para lang="en">
+in the ngx_http_autoindex_module on 32-bit platforms.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.15.7" date="2018-11-27">
 
 <change type="feature">

From mdounin at mdounin.ru  Tue Dec 25 14:56:43 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 25 Dec 2018 14:56:43 +0000
Subject: [nginx] release-1.15.8 tag
Message-ID: <hg.6d15e452fa2e.1545749803.6026610855610030274@dev.nginx.com>

details:   https://hg.nginx.org/nginx/rev/6d15e452fa2e
branches:  
changeset: 7437:6d15e452fa2e
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Dec 25 17:53:03 2018 +0300
description:
release-1.15.8 tag

diffstat:

 .hgtags |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (8 lines):

diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -433,3 +433,4 @@ 49d49835653857daa418e68d6cbfed4958c78fca
 f062e43d74fc2578bb100a9e82a953efa1eb9e4e release-1.15.5
 2351853ce6867b6166823bdf94333c0a76633c0a release-1.15.6
 051a039ce1c7e09144de4a4846669ec7116cecea release-1.15.7
+ee551e3f6dba336c0d875e266d7d55385f379b42 release-1.15.8

From mdounin at mdounin.ru  Tue Dec 25 15:42:34 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 25 Dec 2018 18:42:34 +0300
Subject: [PATCH] better constrain IP-literal validation in
 ngx_http_validate_host()
In-Reply-To: <CAHtY7_pKzzoP_YLBX=RE=Z=xBuPDjMaNZvOU3D8ZWpL1vEqFpA@mail.gmail.com>
References: <CAHtY7_pxP9JXLgJgpwzJRNs=9Vtd-1s6efzrktCu-7EYqpjUvA@mail.gmail.com>
 <20181217161740.GT99070@mdounin.ru>
 <CAHtY7_rG0zYJn+HUCgfzr7uDP5fYtzrmQPw85zGzyb05wH0dZA@mail.gmail.com>
 <20181224125858.GH99070@mdounin.ru>
 <CAHtY7_pKzzoP_YLBX=RE=Z=xBuPDjMaNZvOU3D8ZWpL1vEqFpA@mail.gmail.com>
Message-ID: <20181225154234.GD99070@mdounin.ru>

Hello!

On Mon, Dec 24, 2018 at 01:47:36PM -0800, Terence Honles wrote:

> Yes, the regex will fail for IPv future literals, but I don't believe they are
> being used in practice. When they are, I'm sure the Django project will
> welcome the change to the RegEx.

Sure.  The point is that there is no difference between perfectly 
valid and invalid literals.  Django will complain if it sees 
anything it doesn't understand (and that's perfectly fine, 
actually).

> As for the configuration you proposed, we are already using that (with a 444
> instead of 404), but the IP literal will still pass through because it is a
> valid match (but an invalid hostname according to RFC 3986).

With the configuration I proposed, names you haven't explicitly 
configured with the "server_name" directive will not be sent to 
backends.  And if you've explicitly configured an invalid name, I 
don't see why nginx should refuse doing what it was explicitly 
told to do.

Most likely, you've instead configured nginx to pass everything to 
Django, and this is what causes errors in your setup.  Consider 
switching to a more restricted configuration.

Happy holidays.

-- 
Maxim Dounin
http://mdounin.ru/

From xeioex at nginx.com  Wed Dec 26 10:23:40 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 26 Dec 2018 10:23:40 +0000
Subject: [njs] Fixed exit instructions inside try blocks.
Message-ID: <hg.e1e54922e085.1545819820.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/e1e54922e085
branches:  
changeset: 696:e1e54922e085
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Dec 26 13:23:16 2018 +0300
description:
Fixed exit instructions inside try blocks.

Fixed instructions: continue, break, return.

diffstat:

 njs/njs_disassembler.c       |   61 +++++-
 njs/njs_generator.c          |  468 ++++++++++++++++++++++++++++++++++++------
 njs/njs_vm.c                 |  113 +++++++++-
 njs/njs_vm.h                 |   27 ++-
 njs/test/njs_expect_test.exp |    4 +
 njs/test/njs_unit_test.c     |  280 +++++++++++++++++++++++++
 6 files changed, 858 insertions(+), 95 deletions(-)

diffs (truncated from 1325 to 1000 lines):

diff -r 7b1b3eb864e0 -r e1e54922e085 njs/njs_disassembler.c
--- a/njs/njs_disassembler.c	Tue Dec 25 09:53:27 2018 +0300
+++ b/njs/njs_disassembler.c	Wed Dec 26 13:23:16 2018 +0300
@@ -125,8 +125,6 @@ static njs_code_name_t  code_names[] = {
 
     { njs_vmcode_throw, sizeof(njs_vmcode_throw_t),
           nxt_string("THROW           ") },
-    { njs_vmcode_finally, sizeof(njs_vmcode_finally_t),
-          nxt_string("FINALLY         ") },
 
 };
 
@@ -162,15 +160,18 @@ njs_disassemble(u_char *start, u_char *e
     njs_vmcode_3addr_t           *code3;
     njs_vmcode_array_t           *array;
     njs_vmcode_catch_t           *catch;
+    njs_vmcode_finally_t         *finally;
     njs_vmcode_try_end_t         *try_end;
     njs_vmcode_try_start_t       *try_start;
     njs_vmcode_operation_t       operation;
     njs_vmcode_cond_jump_t       *cond_jump;
     njs_vmcode_test_jump_t       *test_jump;
     njs_vmcode_prop_next_t       *prop_next;
+    njs_vmcode_try_return_t      *try_return;
     njs_vmcode_equal_jump_t      *equal;
     njs_vmcode_prop_foreach_t    *prop_foreach;
     njs_vmcode_method_frame_t    *method;
+    njs_vmcode_try_trampoline_t  *try_tramp;
     njs_vmcode_function_frame_t  *function;
 
     p = start;
@@ -323,8 +324,9 @@ njs_disassemble(u_char *start, u_char *e
         if (operation == njs_vmcode_try_start) {
             try_start = (njs_vmcode_try_start_t *) p;
 
-            printf("%05zd TRY START         %04zX +%zd\n",
-                   p - start, (size_t) try_start->value,
+            printf("%05zd TRY START         %04zX %04zX +%zd\n",
+                   p - start, (size_t) try_start->exception_value,
+                   (size_t) try_start->exit_value,
                    (size_t) try_start->offset);
 
             p += sizeof(njs_vmcode_try_start_t);
@@ -332,6 +334,43 @@ njs_disassemble(u_char *start, u_char *e
             continue;
         }
 
+        if (operation == njs_vmcode_try_break) {
+            try_tramp = (njs_vmcode_try_trampoline_t *) p;
+
+            printf("%05zd TRY BREAK         %04zX %zd\n",
+                   p - start, (size_t) try_tramp->exit_value,
+                   (size_t) try_tramp->offset);
+
+            p += sizeof(njs_vmcode_try_trampoline_t);
+
+            continue;
+        }
+
+        if (operation == njs_vmcode_try_continue) {
+            try_tramp = (njs_vmcode_try_trampoline_t *) p;
+
+            printf("%05zd TRY CONTINUE      %04zX %zd\n",
+                   p - start, (size_t) try_tramp->exit_value,
+                   (size_t) try_tramp->offset);
+
+            p += sizeof(njs_vmcode_try_trampoline_t);
+
+            continue;
+        }
+
+        if (operation == njs_vmcode_try_return) {
+            try_return = (njs_vmcode_try_return_t *) p;
+
+            printf("%05zd TRY RETURN        %04zX %04zX +%zd\n",
+                   p - start, (size_t) try_return->save,
+                   (size_t) try_return->retval,
+                   (size_t) try_return->offset);
+
+            p += sizeof(njs_vmcode_try_return_t);
+
+            continue;
+        }
+
         if (operation == njs_vmcode_catch) {
             catch = (njs_vmcode_catch_t *) p;
 
@@ -355,6 +394,20 @@ njs_disassemble(u_char *start, u_char *e
             continue;
         }
 
+        if (operation == njs_vmcode_finally) {
+            finally = (njs_vmcode_finally_t *) p;
+
+            printf("%05zd TRY FINALLY       %04zX %04zX +%zd +%zd\n",
+                   p - start, (size_t) finally->retval,
+                   (size_t) finally->exit_value,
+                   (size_t) finally->continue_offset,
+                   (size_t) finally->break_offset);
+
+            p += sizeof(njs_vmcode_finally_t);
+
+            continue;
+        }
+
         code_name = code_names;
         n = nxt_nitems(code_names);
 
diff -r 7b1b3eb864e0 -r e1e54922e085 njs/njs_generator.c
--- a/njs/njs_generator.c	Tue Dec 25 09:53:27 2018 +0300
+++ b/njs/njs_generator.c	Wed Dec 26 13:23:16 2018 +0300
@@ -20,13 +20,24 @@ struct njs_generator_patch_s {
      */
     njs_ret_t                       jump_offset;
     njs_generator_patch_t           *next;
+    /*
+     * index_offset is used for patching vmcode_try_return instruction
+     * inside try blocks.
+     */
+    njs_index_t                     index_offset;
 };
 
 
 typedef enum {
-    NJS_GENERATOR_BLOCK = 0,
-    NJS_GENERATOR_LOOP,
-    NJS_GENERATOR_SWITCH,
+    NJS_GENERATOR_BLOCK = 1,
+    NJS_GENERATOR_LOOP = 2,
+    NJS_GENERATOR_SWITCH = 4,
+    NJS_GENERATOR_TRY = 8,
+
+#define NJS_GENERATOR_ALL          (NJS_GENERATOR_BLOCK                      \
+                                    | NJS_GENERATOR_LOOP                     \
+                                    | NJS_GENERATOR_SWITCH                   \
+                                    | NJS_GENERATOR_TRY)
 } njs_generator_block_type_t;
 
 
@@ -70,8 +81,16 @@ static nxt_int_t njs_generate_for_in_sta
 static nxt_noinline nxt_int_t njs_generate_start_block(njs_vm_t *vm,
     njs_generator_t *generator, njs_generator_block_type_t type,
     const nxt_str_t *label);
-static nxt_noinline void njs_generate_patch_loop_continuation(njs_vm_t *vm,
-    njs_generator_t *generator);
+static njs_generator_block_t *njs_generate_find_block(
+    njs_generator_block_t *block, uint32_t mask);
+static nxt_int_t njs_generate_make_continuation_patch(njs_vm_t *vm,
+    njs_generator_t *generator, njs_generator_block_t *block, njs_ret_t offset);
+static nxt_noinline void njs_generate_patch_block(njs_vm_t *vm,
+    njs_generator_t *generator, njs_generator_patch_t *list);
+static nxt_noinline void njs_generate_patch_try_exit_block(njs_vm_t *vm,
+    njs_generator_t *generator, njs_generator_patch_t *list, njs_index_t dest);
+static nxt_int_t njs_generate_make_exit_patch(njs_vm_t *vm,
+    njs_generator_t *generator, njs_generator_block_t *block, njs_ret_t offset);
 static nxt_noinline void njs_generate_patch_block_exit(njs_vm_t *vm,
     njs_generator_t *generator);
 static nxt_int_t njs_generate_continue_statement(njs_vm_t *vm,
@@ -866,6 +885,7 @@ njs_generate_switch_statement(njs_vm_t *
                 return NXT_ERROR;
             }
 
+            patch->index_offset = 0;
             patch->jump_offset = njs_code_offset(generator, equal)
                                  + offsetof(njs_vmcode_equal_jump_t, offset);
 
@@ -966,7 +986,7 @@ njs_generate_while_statement(njs_vm_t *v
 
     /* The loop condition. */
 
-    njs_generate_patch_loop_continuation(vm, generator);
+    njs_generate_patch_block(vm, generator, generator->block->continuation);
 
     njs_code_set_jump_offset(generator, njs_vmcode_jump_t, jump_offset);
 
@@ -1016,7 +1036,7 @@ njs_generate_do_while_statement(njs_vm_t
 
     /* The loop condition. */
 
-    njs_generate_patch_loop_continuation(vm, generator);
+    njs_generate_patch_block(vm, generator, generator->block->continuation);
 
     condition = node->right;
 
@@ -1100,7 +1120,7 @@ njs_generate_for_statement(njs_vm_t *vm,
 
     /* The loop update. */
 
-    njs_generate_patch_loop_continuation(vm, generator);
+    njs_generate_patch_block(vm, generator, generator->block->continuation);
 
     update = node->right;
 
@@ -1199,7 +1219,7 @@ njs_generate_for_in_statement(njs_vm_t *
 
     /* The loop iterator. */
 
-    njs_generate_patch_loop_continuation(vm, generator);
+    njs_generate_patch_block(vm, generator, generator->block->continuation);
 
     njs_code_set_jump_offset(generator, njs_vmcode_prop_foreach_t, prop_offset);
 
@@ -1258,15 +1278,51 @@ njs_generate_start_block(njs_vm_t *vm, n
 }
 
 
+static njs_generator_block_t *
+njs_generate_find_block(njs_generator_block_t *block, uint32_t mask)
+{
+    while (block != NULL) {
+        if (block->type & mask) {
+            return block;
+        }
+
+        block = block->next;
+    }
+
+    return NULL;
+}
+
+
+static nxt_int_t
+njs_generate_make_continuation_patch(njs_vm_t *vm, njs_generator_t *generator,
+    njs_generator_block_t *block, njs_ret_t offset)
+{
+    njs_generator_patch_t  *patch;
+
+    patch = nxt_mem_cache_alloc(vm->mem_cache_pool,
+                                sizeof(njs_generator_patch_t));
+    if (nxt_slow_path(patch == NULL)) {
+        njs_memory_error(vm);
+        return NXT_ERROR;
+    }
+
+    patch->next = block->continuation;
+    block->continuation = patch;
+
+    patch->index_offset = 0;
+    patch->jump_offset = offset;
+
+    return NXT_OK;
+}
+
+
 static nxt_noinline void
-njs_generate_patch_loop_continuation(njs_vm_t *vm, njs_generator_t *generator)
+njs_generate_patch_block(njs_vm_t *vm, njs_generator_t *generator,
+    njs_generator_patch_t *list)
 {
-    njs_generator_block_t  *block;
     njs_generator_patch_t  *patch, *next;
 
-    block = generator->block;
-
-    for (patch = block->continuation; patch != NULL; patch = next) {
+    for (patch = list; patch != NULL; patch = next) {
         njs_code_update_offset(generator, patch);
         next = patch->next;
 
@@ -1276,20 +1332,56 @@ njs_generate_patch_loop_continuation(njs
 
 
 static nxt_noinline void
+njs_generate_patch_try_exit_block(njs_vm_t *vm, njs_generator_t *generator,
+    njs_generator_patch_t *list, njs_index_t dest)
+{
+    njs_generator_patch_t  *patch, *next;
+
+    for (patch = list; patch != NULL; patch = next) {
+        njs_code_update_offset(generator, patch);
+        next = patch->next;
+
+        if (patch->index_offset != 0) {
+            *(njs_code_ptr(generator, njs_index_t, patch->index_offset)) = dest;
+        }
+
+        nxt_mem_cache_free(vm->mem_cache_pool, patch);
+    }
+}
+
+
+static nxt_int_t
+njs_generate_make_exit_patch(njs_vm_t *vm, njs_generator_t *generator,
+    njs_generator_block_t *block, njs_ret_t offset)
+{
+    njs_generator_patch_t  *patch;
+
+    patch = nxt_mem_cache_alloc(vm->mem_cache_pool,
+                                sizeof(njs_generator_patch_t));
+    if (nxt_slow_path(patch == NULL)) {
+        njs_memory_error(vm);
+        return NXT_ERROR;
+    }
+
+    patch->next = block->exit;
+    block->exit = patch;
+
+    patch->index_offset = 0;
+    patch->jump_offset = offset;
+
+    return NXT_OK;
+}
+
+
+static nxt_noinline void
 njs_generate_patch_block_exit(njs_vm_t *vm, njs_generator_t *generator)
 {
     njs_generator_block_t  *block;
-    njs_generator_patch_t  *patch, *next;
 
     block = generator->block;
     generator->block = block->next;
 
-    for (patch = block->exit; patch != NULL; patch = next) {
-        njs_code_update_offset(generator, patch);
-        next = patch->next;
-
-        nxt_mem_cache_free(vm->mem_cache_pool, patch);
-    }
+    njs_generate_patch_block(vm, generator, block->exit);
 
     nxt_mem_cache_free(vm->mem_cache_pool, block);
 }
@@ -1299,22 +1391,22 @@ static nxt_int_t
 njs_generate_continue_statement(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node)
 {
-    njs_vmcode_jump_t   *jump;
+    njs_vmcode_jump_t      *jump;
     njs_generator_patch_t  *patch;
     njs_generator_block_t  *block;
 
-    for (block = generator->block; block != NULL; block = block->next) {
-        if (block->type == NJS_GENERATOR_LOOP) {
-            goto found;
-        }
+    block = njs_generate_find_block(generator->block,
+                                    NJS_GENERATOR_LOOP | NJS_GENERATOR_TRY);
+
+    if (nxt_slow_path(block == NULL)) {
+        goto syntax_error;
     }
 
-    njs_generate_syntax_error(vm, node->token_line,
-                              "Illegal continue statement");
-
-    return NXT_ERROR;
-
-found:
+    if (block->type == NJS_GENERATOR_TRY
+        && njs_generate_find_block(block->next, NJS_GENERATOR_LOOP) == NULL)
+    {
+        goto syntax_error;
+    }
 
     /* TODO: LABEL */
 
@@ -1331,11 +1423,19 @@ found:
         jump->code.retval = NJS_VMCODE_NO_RETVAL;
         jump->offset = offsetof(njs_vmcode_jump_t, offset);
 
+        patch->index_offset = 0;
         patch->jump_offset = njs_code_offset(generator, jump)
                              + offsetof(njs_vmcode_jump_t, offset);
     }
 
     return NXT_OK;
+
+syntax_error:
+
+    njs_generate_syntax_error(vm, node->token_line,
+                              "Illegal continue statement");
+
+    return NXT_ERROR;
 }
 
 
@@ -1343,24 +1443,22 @@ static nxt_int_t
 njs_generate_break_statement(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node)
 {
-    njs_vmcode_jump_t   *jump;
+    njs_vmcode_jump_t      *jump;
     njs_generator_patch_t  *patch;
     njs_generator_block_t  *block;
 
-    for (block = generator->block; block != NULL; block = block->next) {
-        if (block->type == NJS_GENERATOR_LOOP
-            || block->type == NJS_GENERATOR_SWITCH)
-        {
-            goto found;
-        }
+    block = njs_generate_find_block(generator->block, NJS_GENERATOR_ALL);
+
+    if (nxt_slow_path(block == NULL)) {
+        goto syntax_error;
+     }
+
+    if (block->type == NJS_GENERATOR_TRY
+        && njs_generate_find_block(block->next, NJS_GENERATOR_ALL) == NULL)
+    {
+        goto syntax_error;
     }
 
-    njs_generate_syntax_error(vm, node->token_line, "Illegal break statement");
-
-    return NXT_ERROR;
-
-found:
-
     /* TODO: LABEL: loop and switch may have label, block must have label. */
 
     patch = nxt_mem_cache_alloc(vm->mem_cache_pool,
@@ -1376,11 +1474,18 @@ found:
         jump->code.retval = NJS_VMCODE_NO_RETVAL;
         jump->offset = offsetof(njs_vmcode_jump_t, offset);
 
+        patch->index_offset = 0;
         patch->jump_offset = njs_code_offset(generator, jump)
                              + offsetof(njs_vmcode_jump_t, offset);
     }
 
     return NXT_OK;
+
+syntax_error:
+
+    njs_generate_syntax_error(vm, node->token_line, "Illegal break statement");
+
+    return NXT_ERROR;
 }
 
 
@@ -2356,30 +2461,65 @@ static nxt_int_t
 njs_generate_return_statement(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node)
 {
-    nxt_int_t            ret;
-    njs_index_t          index;
-    njs_vmcode_return_t  *code;
+    nxt_int_t                ret;
+    njs_index_t              index;
+    njs_vmcode_return_t      *code;
+    njs_generator_patch_t    *patch;
+    njs_generator_block_t    *block;
+    njs_vmcode_try_return_t  *try_return;
 
     ret = njs_generator(vm, generator, node->right);
 
-    if (nxt_fast_path(ret == NXT_OK)) {
+    if (nxt_slow_path(ret != NXT_OK)) {
+        return ret;
+    }
+
+    if (node->right != NULL) {
+        index = node->right->index;
+
+    } else {
+        index = njs_value_index(vm, &njs_value_void,
+                                generator->runtime);
+    }
+
+    block = njs_generate_find_block(generator->block, NJS_GENERATOR_TRY);
+
+    if (nxt_fast_path(block == NULL)) {
         njs_generate_code(generator, njs_vmcode_return_t, code);
         code->code.operation = njs_vmcode_return;
         code->code.operands = NJS_VMCODE_1OPERAND;
         code->code.retval = NJS_VMCODE_NO_RETVAL;
 
-        if (node->right != NULL) {
-            index = node->right->index;
-
-        } else {
-            index = njs_value_index(vm, &njs_value_void, generator->runtime);
-        }
-
         code->retval = index;
         node->index = index;
+
+        return NXT_OK;
     }
 
-    return ret;
+    patch = nxt_mem_cache_alloc(vm->mem_cache_pool,
+                                sizeof(njs_generator_patch_t));
+    if (nxt_slow_path(patch == NULL)) {
+        return NXT_ERROR;
+    }
+
+    patch->next = block->exit;
+    block->exit = patch;
+
+    njs_generate_code(generator, njs_vmcode_try_return_t, try_return);
+    try_return->code.operation = njs_vmcode_try_return;
+    try_return->code.operands = NJS_VMCODE_2OPERANDS;
+    try_return->code.retval = NJS_VMCODE_RETVAL;
+    try_return->retval = index;
+
+    try_return->save = index;
+    patch->index_offset = njs_code_offset(generator, try_return)
+                          + offsetof(njs_vmcode_try_return_t, save);
+
+    try_return->offset = offsetof(njs_vmcode_try_return_t, offset);
+    patch->jump_offset = njs_code_offset(generator, try_return)
+                         + offsetof(njs_vmcode_try_return_t, offset);
+
+    return NXT_OK;
 }
 
 
@@ -2534,13 +2674,16 @@ static nxt_int_t
 njs_generate_try_statement(njs_vm_t *vm, njs_generator_t *generator,
     njs_parser_node_t *node)
 {
-    njs_ret_t               try_offset, catch_offset;
-    nxt_int_t               ret;
-    njs_index_t             index, catch_index;
-    njs_vmcode_catch_t      *catch;
-    njs_vmcode_finally_t    *finally;
-    njs_vmcode_try_end_t    *try_end, *catch_end;
-    njs_vmcode_try_start_t  *try_start;
+    njs_ret_t                    try_offset, try_end_offset, catch_offset,
+                                 catch_end_offset;
+    nxt_int_t                    ret;
+    njs_index_t                  exception_index, exit_index, catch_index;
+    njs_vmcode_catch_t           *catch;
+    njs_vmcode_finally_t         *finally;
+    njs_vmcode_try_end_t         *try_end, *catch_end;
+    njs_generator_block_t        *block, *try_block, *catch_block;
+    njs_vmcode_try_start_t       *try_start;
+    njs_vmcode_try_trampoline_t  *try_break, *try_continue;
 
     njs_generate_code(generator, njs_vmcode_try_start_t, try_start);
     try_offset = njs_code_offset(generator, try_start);
@@ -2548,25 +2691,82 @@ njs_generate_try_statement(njs_vm_t *vm,
     try_start->code.operands = NJS_VMCODE_2OPERANDS;
     try_start->code.retval = NJS_VMCODE_NO_RETVAL;
 
-    index = njs_generate_temp_index_get(vm, generator, node);
-    if (nxt_slow_path(index == NJS_INDEX_ERROR)) {
+    exception_index = njs_generate_temp_index_get(vm, generator, node);
+    if (nxt_slow_path(exception_index == NJS_INDEX_ERROR)) {
         return NXT_ERROR;
     }
 
-    try_start->value = index;
+    try_start->exception_value = exception_index;
+
+    /*
+     * exit_value is used in njs_vmcode_finally to make a decision
+     * which way to go after "break", "continue" and "return" instruction
+     * inside "try" or "catch" blocks.
+     */
+
+    exit_index = njs_generate_temp_index_get(vm, generator, node);
+    if (nxt_slow_path(exit_index == NJS_INDEX_ERROR)) {
+        return NXT_ERROR;
+    }
+
+    try_start->exit_value = exit_index;
+
+    ret = njs_generate_start_block(vm, generator, NJS_GENERATOR_TRY, &no_label);
+    if (nxt_slow_path(ret != NXT_OK)) {
+        return ret;
+    }
 
     ret = njs_generator(vm, generator, node->left);
     if (nxt_slow_path(ret != NXT_OK)) {
         return ret;
     }
 
+    try_block = generator->block;
+
     njs_generate_code(generator, njs_vmcode_try_end_t, try_end);
+    try_end_offset = njs_code_offset(generator, try_end);
     try_end->code.operation = njs_vmcode_try_end;
     try_end->code.operands = NJS_VMCODE_NO_OPERAND;
     try_end->code.retval = NJS_VMCODE_NO_RETVAL;
 
+    if (try_block->exit != NULL) {
+        njs_generate_patch_try_exit_block(vm, generator, try_block->exit,
+                                          exit_index);
+
+        njs_generate_code(generator, njs_vmcode_try_trampoline_t, try_break);
+        try_break->code.operation = njs_vmcode_try_break;
+        try_break->code.operands = NJS_VMCODE_2OPERANDS;
+        try_break->code.retval = NJS_VMCODE_NO_RETVAL;
+
+        try_break->exit_value = exit_index;
+
+        try_break->offset = -sizeof(njs_vmcode_try_end_t);
+
+    } else {
+        try_break = NULL;
+    }
+
+    if (try_block->continuation != NULL) {
+        njs_generate_patch_block(vm, generator, try_block->continuation);
+
+        njs_generate_code(generator, njs_vmcode_try_trampoline_t, try_continue);
+        try_continue->code.operation = njs_vmcode_try_continue;
+        try_continue->code.operands = NJS_VMCODE_2OPERANDS;
+        try_continue->code.retval = NJS_VMCODE_NO_RETVAL;
+
+        try_continue->exit_value = exit_index;
+
+        try_continue->offset = -sizeof(njs_vmcode_try_end_t);
+
+        if (try_break != NULL) {
+            try_continue->offset -= sizeof(njs_vmcode_try_trampoline_t);
+        }
+    }
+
+    generator->block = try_block->next;
+
     njs_code_set_jump_offset(generator, njs_vmcode_try_start_t, try_offset);
-    try_offset = njs_code_offset(generator, try_end);
+    try_offset = try_end_offset;
 
     node = node->right;
 
@@ -2592,6 +2792,43 @@ njs_generate_try_statement(njs_vm_t *vm,
 
         njs_code_set_jump_offset(generator, njs_vmcode_try_end_t, try_offset);
 
+        if (try_block->continuation != NULL || try_block->exit != NULL) {
+            njs_generate_code(generator, njs_vmcode_finally_t, finally);
+            finally->code.operation = njs_vmcode_finally;
+            finally->code.operands = NJS_VMCODE_2OPERANDS;
+            finally->code.retval = NJS_VMCODE_NO_RETVAL;
+            finally->retval = exception_index;
+            finally->exit_value = exit_index;
+            finally->continue_offset = offsetof(njs_vmcode_finally_t,
+                                                continue_offset);
+            finally->break_offset = offsetof(njs_vmcode_finally_t,
+                                             break_offset);
+
+            if (try_block->continuation != NULL) {
+                /*
+                 * block != NULL is checked
+                 * by njs_generate_continue_statement()
+                 */
+                block = njs_generate_find_block(generator->block,
+                                                NJS_GENERATOR_LOOP);
+
+                njs_generate_make_continuation_patch(vm, generator, block,
+                            njs_code_offset(generator, finally)
+                             + offsetof(njs_vmcode_finally_t, continue_offset));
+            }
+
+            if (try_block->exit != NULL) {
+                block = njs_generate_find_block(generator->block,
+                                                NJS_GENERATOR_ALL);
+
+                if (block != NULL) {
+                    njs_generate_make_exit_patch(vm, generator, block,
+                                njs_code_offset(generator, finally)
+                                + offsetof(njs_vmcode_finally_t, break_offset));
+                }
+            }
+        }
+
         /* TODO: release exception variable index. */
 
     } else {
@@ -2610,19 +2847,67 @@ njs_generate_try_statement(njs_vm_t *vm,
             catch->code.retval = NJS_VMCODE_NO_RETVAL;
             catch->exception = catch_index;
 
+            ret = njs_generate_start_block(vm, generator, NJS_GENERATOR_TRY,
+                                           &no_label);
+            if (nxt_slow_path(ret != NXT_OK)) {
+                return ret;
+            }
+
             ret = njs_generator(vm, generator, node->left->right);
             if (nxt_slow_path(ret != NXT_OK)) {
                 return ret;
             }
 
+            catch_block = generator->block;
+
             njs_generate_code(generator, njs_vmcode_try_end_t, catch_end);
+            catch_end_offset = njs_code_offset(generator, catch_end);
             catch_end->code.operation = njs_vmcode_try_end;
             catch_end->code.operands = NJS_VMCODE_NO_OPERAND;
             catch_end->code.retval = NJS_VMCODE_NO_RETVAL;
 
+            if (catch_block->exit != NULL) {
+                njs_generate_patch_try_exit_block(vm, generator,
+                                                  catch_block->exit,
+                                                  exit_index);
+
+                njs_generate_code(generator, njs_vmcode_try_trampoline_t,
+                                  try_break);
+                try_break->code.operation = njs_vmcode_try_break;
+                try_break->code.operands = NJS_VMCODE_2OPERANDS;
+                try_break->code.retval = NJS_VMCODE_NO_RETVAL;
+
+                try_break->exit_value = exit_index;
+
+                try_break->offset = -sizeof(njs_vmcode_try_end_t);
+
+            } else {
+                try_break = NULL;
+            }
+
+            if (catch_block->continuation != NULL) {
+                njs_generate_patch_block(vm, generator,
+                                         catch_block->continuation);
+
+                njs_generate_code(generator, njs_vmcode_try_trampoline_t,
+                                  try_continue);
+                try_continue->code.operation = njs_vmcode_try_continue;
+                try_continue->code.operands = NJS_VMCODE_2OPERANDS;
+                try_continue->code.retval = NJS_VMCODE_NO_RETVAL;
+
+                try_continue->exit_value = exit_index;
+
+                try_continue->offset = -sizeof(njs_vmcode_try_end_t);
+
+                if (try_break != NULL) {
+                    try_continue->offset -= sizeof(njs_vmcode_try_trampoline_t);
+                }
+            }
+
+            generator->block = catch_block->next;
+
             njs_code_set_jump_offset(generator, njs_vmcode_catch_t,
                                      catch_offset);
-            catch_offset = njs_code_offset(generator, catch_end);
 
             /* TODO: release exception variable index. */
 
@@ -2631,10 +2916,10 @@ njs_generate_try_statement(njs_vm_t *vm,
             catch->code.operands = NJS_VMCODE_2OPERANDS;
             catch->code.retval = NJS_VMCODE_NO_RETVAL;
             catch->offset = sizeof(njs_vmcode_catch_t);
-            catch->exception = index;
+            catch->exception = exception_index;
 
             njs_code_set_jump_offset(generator, njs_vmcode_try_end_t,
-                                     catch_offset);
+                                     catch_end_offset);
 
         } else {
             /* A try/finally case. */
@@ -2644,7 +2929,9 @@ njs_generate_try_statement(njs_vm_t *vm,
             catch->code.operands = NJS_VMCODE_2OPERANDS;
             catch->code.retval = NJS_VMCODE_NO_RETVAL;
             catch->offset = sizeof(njs_vmcode_catch_t);
-            catch->exception = index;
+            catch->exception = exception_index;
+
+            catch_block = NULL;
         }
 
         njs_code_set_jump_offset(generator, njs_vmcode_try_end_t, try_offset);
@@ -2656,12 +2943,43 @@ njs_generate_try_statement(njs_vm_t *vm,
 
         njs_generate_code(generator, njs_vmcode_finally_t, finally);
         finally->code.operation = njs_vmcode_finally;
-        finally->code.operands = NJS_VMCODE_1OPERAND;
+        finally->code.operands = NJS_VMCODE_2OPERANDS;
         finally->code.retval = NJS_VMCODE_NO_RETVAL;
-        finally->retval = index;
+        finally->retval = exception_index;
+        finally->exit_value = exit_index;
+        finally->continue_offset = offsetof(njs_vmcode_finally_t,
+                                            continue_offset);
+        finally->break_offset = offsetof(njs_vmcode_finally_t, break_offset);
+
+        if (try_block->continuation != NULL
+            || (catch_block && catch_block->continuation != NULL))
+        {
+            /*
+             * block != NULL is checked
+             * by njs_generate_continue_statement()
+             */
+            block = njs_generate_find_block(generator->block,
+                                            NJS_GENERATOR_LOOP);
+
+            njs_generate_make_continuation_patch(vm, generator, block,
+                         njs_code_offset(generator, finally)
+                         + offsetof(njs_vmcode_finally_t, continue_offset));
+        }
+
+        if (try_block->exit != NULL
+            || (catch_block != NULL && catch_block->exit != NULL))
+        {
+            block = njs_generate_find_block(generator->block,
+                                            NJS_GENERATOR_ALL);
+            if (block != NULL) {
+                njs_generate_make_exit_patch(vm, generator, block,
+                                njs_code_offset(generator, finally)
+                                + offsetof(njs_vmcode_finally_t, break_offset));
+            }
+        }
     }
 
-    return njs_generate_index_release(vm, generator, index);
+    return njs_generate_index_release(vm, generator, exception_index);
 }
 
 
diff -r 7b1b3eb864e0 -r e1e54922e085 njs/njs_vm.c
--- a/njs/njs_vm.c	Tue Dec 25 09:53:27 2018 +0300
+++ b/njs/njs_vm.c	Wed Dec 26 13:23:16 2018 +0300
@@ -132,7 +132,8 @@ start:
          *   njs_vmcode_function_call(),
          *   njs_vmcode_return(),
          *   njs_vmcode_try_start(),
-         *   njs_vmcode_try_next(),
+         *   njs_vmcode_try_continue(),
+         *   njs_vmcode_try_break(),
          *   njs_vmcode_try_end(),
          *   njs_vmcode_catch().
          *   njs_vmcode_throw().
@@ -2580,9 +2581,12 @@ njs_vmcode_stop(njs_vm_t *vm, njs_value_
  */
 
 njs_ret_t
-njs_vmcode_try_start(njs_vm_t *vm, njs_value_t *value, njs_value_t *offset)
+njs_vmcode_try_start(njs_vm_t *vm, njs_value_t *exception_value,
+    njs_value_t *offset)
 {
-    njs_exception_t  *e;
+    njs_value_t             *exit_value;
+    njs_exception_t         *e;
+    njs_vmcode_try_start_t  *try_start;
 
     if (vm->top_frame->exception.catch != NULL) {
         e = nxt_mem_cache_alloc(vm->mem_cache_pool, sizeof(njs_exception_t));
@@ -2597,13 +2601,68 @@ njs_vmcode_try_start(njs_vm_t *vm, njs_v
 
     vm->top_frame->exception.catch = vm->current + (njs_ret_t) offset;
 
-    njs_set_invalid(value);
+    njs_set_invalid(exception_value);
+
+    try_start = (njs_vmcode_try_start_t *) vm->current;
+    exit_value = njs_vmcode_operand(vm, try_start->exit_value);
+
+    njs_set_invalid(exit_value);
+    exit_value->data.u.number = 0;
 
     return sizeof(njs_vmcode_try_start_t);
 }
 
 
 /*
+ * njs_vmcode_try_break() sets exit_value to INVALID 1, and jumps to
+ * the nearest try_end block. The exit_value is checked by njs_vmcode_finally().
+ */
+
+nxt_noinline njs_ret_t
+njs_vmcode_try_break(njs_vm_t *vm, njs_value_t *exit_value,
+    njs_value_t *offset)
+{
+    exit_value->data.u.number = 1;
+
+    return (njs_ret_t) offset;
+}
+
+
+/*
+ * njs_vmcode_try_break() sets exit_value to INVALID -1, and jumps to
+ * the nearest try_end block. The exit_value is checked by njs_vmcode_finally().
+ */
+
+nxt_noinline njs_ret_t
+njs_vmcode_try_continue(njs_vm_t *vm, njs_value_t *exit_value,
+    njs_value_t *offset)
+{
+    exit_value->data.u.number = -1;
+
+    return (njs_ret_t) offset;
+}
+
+/*
+ * njs_vmcode_try_return() saves a return value to use it later by
+ * njs_vmcode_finally(), and jumps to the nearest try_end block.
+ */
+
+nxt_noinline njs_ret_t
+njs_vmcode_try_return(njs_vm_t *vm, njs_value_t *value, njs_value_t *invld)
+{
+    njs_vmcode_try_return_t  *try_return;
+
+    vm->retval = *value;
+
+    njs_retain(value);
+
+    try_return = (njs_vmcode_try_return_t *) vm->current;
+
+    return try_return->offset;
+}
+
+
+/*
  * njs_vmcode_try_end() is set on the end of a "try" block to remove the block.
  * It is also set on the end of a "catch" block followed by a "finally" block.
  */
@@ -2664,24 +2723,48 @@ njs_vmcode_catch(njs_vm_t *vm, njs_value
 
 
 /*
- * njs_vmcode_finally() is set on the end of a "finally" block to throw
- * uncaught exception.
+ * njs_vmcode_finally() is set on the end of a "finally" or a "catch" block.
+ *   1) to throw uncaught exception.
+ *   2) to make a jump to an enslosing loop exit if "continue" or "break"
+ *      statement was used inside try block.
+ *   3) to finalize "return" instruction from "try" block.
  */
 
 njs_ret_t
 njs_vmcode_finally(njs_vm_t *vm, njs_value_t *invld, njs_value_t *retval)
 {
-    njs_value_t  *value;
-
-    value = njs_vmcode_operand(vm, retval);
-
-    if (!njs_is_valid(value)) {
-        return sizeof(njs_vmcode_finally_t);
+    njs_value_t           *exception_value, *exit_value;
+    njs_vmcode_finally_t  *finally;
+
+    exception_value = njs_vmcode_operand(vm, retval);
+
+    if (njs_is_valid(exception_value)) {
+        vm->retval = *exception_value;
+
+        return NXT_ERROR;
     }
 
-    vm->retval = *value;
-
-    return NXT_ERROR;
+    finally = (njs_vmcode_finally_t *) vm->current;
+    exit_value = njs_vmcode_operand(vm, finally->exit_value);
+
+    /*
+     * exit_value is set by:
+     *   vmcode_try_start to INVALID 0
+     *   vmcode_try_break to INVALID 1
+     *   vmcode_try_continue to INVALID -1
+     *   vmcode_try_return to a valid return value
+     */
+
+    if (njs_is_valid(exit_value)) {
+        return njs_vmcode_return(vm, NULL, exit_value);
+
+    } else if (exit_value->data.u.number != 0) {
+        return (njs_ret_t) (exit_value->data.u.number > 0)
+                                ? finally->break_offset
+                                : finally->continue_offset;
+    }
+
+    return sizeof(njs_vmcode_finally_t);
 }
 
 
diff -r 7b1b3eb864e0 -r e1e54922e085 njs/njs_vm.h
--- a/njs/njs_vm.h	Tue Dec 25 09:53:27 2018 +0300
+++ b/njs/njs_vm.h	Wed Dec 26 13:23:16 2018 +0300
@@ -774,13 +774,21 @@ typedef struct {
 typedef struct {
     njs_vmcode_t               code;
     njs_ret_t                  offset;
-    njs_index_t                value;
+    njs_index_t                exception_value;
+    njs_index_t                exit_value;
 } njs_vmcode_try_start_t;
 
 
 typedef struct {
     njs_vmcode_t               code;
     njs_ret_t                  offset;
+    njs_index_t                exit_value;
+} njs_vmcode_try_trampoline_t;
+
+
+typedef struct {
+    njs_vmcode_t               code;
+    njs_ret_t                  offset;
     njs_index_t                exception;
 } njs_vmcode_catch_t;
 
@@ -799,7 +807,18 @@ typedef struct {
 
 typedef struct {
     njs_vmcode_t               code;
+    njs_index_t                save;
     njs_index_t                retval;
+    njs_ret_t                  offset;
+} njs_vmcode_try_return_t;
+
+
+typedef struct {

From xeioex at nginx.com  Wed Dec 26 13:06:48 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 26 Dec 2018 13:06:48 +0000
Subject: [njs] Added support of native function calls in njs_vm_call().
Message-ID: <hg.36989f329e16.1545829608.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/36989f329e16
branches:  
changeset: 697:36989f329e16
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Dec 26 16:05:23 2018 +0300
description:
Added support of native function calls in njs_vm_call().

diffstat:

 njs/njs.c          |  44 ++++++++++++++++++++++++++++++++------------
 njs/njs_function.c |   2 +-
 njs/njs_function.h |   2 +-
 3 files changed, 34 insertions(+), 14 deletions(-)

diffs (87 lines):

diff -r e1e54922e085 -r 36989f329e16 njs/njs.c
--- a/njs/njs.c	Wed Dec 26 13:23:16 2018 +0300
+++ b/njs/njs.c	Wed Dec 26 16:05:23 2018 +0300
@@ -461,9 +461,10 @@ nxt_int_t
 njs_vm_call(njs_vm_t *vm, njs_function_t *function, const njs_value_t *args,
     nxt_uint_t nargs)
 {
-    u_char       *current;
-    njs_ret_t    ret;
-    njs_value_t  *this;
+    u_char              *current;
+    njs_ret_t           ret;
+    njs_value_t         *this;
+    njs_continuation_t  *cont;
 
     static const njs_vmcode_stop_t  stop[] = {
         { .code = { .operation = njs_vmcode_stop,
@@ -474,17 +475,36 @@ njs_vm_call(njs_vm_t *vm, njs_function_t
 
     this = (njs_value_t *) &njs_value_void;
 
-    ret = njs_function_frame(vm, function, this, args, nargs, 0);
-    if (nxt_slow_path(ret != NXT_OK)) {
-        return ret;
-    }
+    current = vm->current;
+
+    if (function->native) {
+        ret = njs_function_native_frame(vm, function, this, &args[0],
+                                        nargs, NJS_CONTINUATION_SIZE, 0);
+        if (ret != NJS_OK) {
+            return NJS_ERROR;
+        }
+
+        cont = njs_vm_continuation(vm);
+
+        cont->function = function->u.native;
+        cont->args_types = function->args_types;
+        cont->retval = NJS_INDEX_GLOBAL_RETVAL;
 
-    current = vm->current;
-    vm->current = (u_char *) stop;
+        cont->return_address = (u_char *) stop;
+        vm->current = (u_char *) njs_continuation_nexus;
 
-    ret = njs_function_call(vm, NJS_INDEX_GLOBAL_RETVAL, 0);
-    if (nxt_slow_path(ret == NXT_ERROR)) {
-        return ret;
+    } else {
+        ret = njs_function_frame(vm, function, this, args, nargs, 0);
+        if (nxt_slow_path(ret != NXT_OK)) {
+            return ret;
+        }
+
+        vm->current = (u_char *) stop;
+
+        ret = njs_function_call(vm, NJS_INDEX_GLOBAL_RETVAL, 0);
+        if (nxt_slow_path(ret == NXT_ERROR)) {
+            return ret;
+        }
     }
 
     ret = njs_vmcode_interpreter(vm);
diff -r e1e54922e085 -r 36989f329e16 njs/njs_function.c
--- a/njs/njs_function.c	Wed Dec 26 13:23:16 2018 +0300
+++ b/njs/njs_function.c	Wed Dec 26 16:05:23 2018 +0300
@@ -214,7 +214,7 @@ njs_function_arguments_thrower(njs_vm_t 
 
 njs_ret_t
 njs_function_native_frame(njs_vm_t *vm, njs_function_t *function,
-    const njs_value_t *this, njs_value_t *args, nxt_uint_t nargs,
+    const njs_value_t *this, const njs_value_t *args, nxt_uint_t nargs,
     size_t reserve, nxt_bool_t ctor)
 {
     size_t              size;
diff -r e1e54922e085 -r 36989f329e16 njs/njs_function.h
--- a/njs/njs_function.h	Wed Dec 26 13:23:16 2018 +0300
+++ b/njs/njs_function.h	Wed Dec 26 16:05:23 2018 +0300
@@ -165,7 +165,7 @@ njs_ret_t njs_function_constructor(njs_v
 njs_ret_t njs_function_apply(njs_vm_t *vm, njs_function_t *function,
     njs_value_t *args, nxt_uint_t nargs, njs_index_t retval);
 njs_ret_t njs_function_native_frame(njs_vm_t *vm, njs_function_t *function,
-    const njs_value_t *this, njs_value_t *args, nxt_uint_t nargs,
+    const njs_value_t *this, const njs_value_t *args, nxt_uint_t nargs,
     size_t reserve, nxt_bool_t ctor);
 njs_ret_t njs_function_frame(njs_vm_t *vm, njs_function_t *function,
     const njs_value_t *this, const njs_value_t *args, nxt_uint_t nargs,

From xeioex at nginx.com  Wed Dec 26 17:18:34 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 26 Dec 2018 17:18:34 +0000
Subject: [njs] Refactored functions related to variables.
Message-ID: <hg.1cac2060faae.1545844714.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/1cac2060faae
branches:  
changeset: 698:1cac2060faae
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Dec 26 19:57:41 2018 +0300
description:
Refactored functions related to variables.

    1) njs_builtin_add() is refactored out as a minor variant of
       njs_variable_add().
    2) parser is refactored out from arguments of the functions.

diffstat:

 njs/njs_parser.c   |  87 ++++++++++++++++++++++++++++++++++------------
 njs/njs_parser.h   |   7 +--
 njs/njs_variable.c |  99 +++++++++++++++--------------------------------------
 njs/njs_variable.h |   8 ++-
 njs/njs_vm.h       |   1 +
 5 files changed, 101 insertions(+), 101 deletions(-)

diffs (442 lines):

diff -r 36989f329e16 -r 1cac2060faae njs/njs_parser.c
--- a/njs/njs_parser.c	Wed Dec 26 16:05:23 2018 +0300
+++ b/njs/njs_parser.c	Wed Dec 26 19:57:41 2018 +0300
@@ -430,6 +430,23 @@ njs_parser_match(njs_vm_t *vm, njs_parse
 }
 
 
+nxt_inline njs_variable_t *
+njs_parser_variable_add(njs_vm_t *vm, njs_parser_t *parser,
+    njs_variable_type_t type)
+{
+    return njs_variable_add(vm, parser->scope, &parser->lexer->text,
+                            parser->lexer->key_hash, type);
+}
+
+nxt_inline njs_ret_t
+njs_parser_variable_reference(njs_vm_t *vm, njs_parser_t *parser,
+    njs_parser_node_t *node, njs_variable_reference_t type)
+{
+    return njs_variable_reference(vm, parser->scope, node, &parser->lexer->text,
+                                  parser->lexer->key_hash, type);
+}
+
+
 static njs_token_t
 njs_parser_function_declaration(njs_vm_t *vm, njs_parser_t *parser)
 {
@@ -463,12 +480,12 @@ njs_parser_function_declaration(njs_vm_t
         return NJS_TOKEN_ILLEGAL;
     }
 
-    var = njs_variable_add(vm, parser, NJS_VARIABLE_FUNCTION);
+    var = njs_parser_variable_add(vm, parser, NJS_VARIABLE_FUNCTION);
     if (nxt_slow_path(var == NULL)) {
         return NJS_TOKEN_ERROR;
     }
 
-    ret = njs_variable_reference(vm, parser, node, NJS_DECLARATION);
+    ret = njs_parser_variable_reference(vm, parser, node, NJS_DECLARATION);
     if (nxt_slow_path(ret != NXT_OK)) {
         return NJS_TOKEN_ERROR;
     }
@@ -542,7 +559,7 @@ njs_parser_function_expression(njs_vm_t 
     }
 
     if (token == NJS_TOKEN_NAME) {
-        var = njs_variable_add(vm, parser, NJS_VARIABLE_SHIM);
+        var = njs_parser_variable_add(vm, parser, NJS_VARIABLE_SHIM);
         if (nxt_slow_path(var == NULL)) {
             return NJS_TOKEN_ERROR;
         }
@@ -646,7 +663,7 @@ njs_parser_function_lambda(njs_vm_t *vm,
             return NJS_TOKEN_ILLEGAL;
         }
 
-        arg = njs_variable_add(vm, parser, NJS_VARIABLE_VAR);
+        arg = njs_parser_variable_add(vm, parser, NJS_VARIABLE_VAR);
         if (nxt_slow_path(arg == NULL)) {
             return NJS_TOKEN_ERROR;
         }
@@ -836,7 +853,7 @@ njs_parser_var_statement(njs_vm_t *vm, n
             return NJS_TOKEN_ILLEGAL;
         }
 
-        var = njs_variable_add(vm, parser, NJS_VARIABLE_VAR);
+        var = njs_parser_variable_add(vm, parser, NJS_VARIABLE_VAR);
         if (nxt_slow_path(var == NULL)) {
             return NJS_TOKEN_ERROR;
         }
@@ -848,7 +865,7 @@ njs_parser_var_statement(njs_vm_t *vm, n
 
         name->token = NJS_TOKEN_NAME;
 
-        ret = njs_variable_reference(vm, parser, name, NJS_DECLARATION);
+        ret = njs_parser_variable_reference(vm, parser, name, NJS_DECLARATION);
         if (nxt_slow_path(ret != NXT_OK)) {
             return NJS_TOKEN_ERROR;
         }
@@ -1299,7 +1316,7 @@ njs_parser_for_var_statement(njs_vm_t *v
             return NJS_TOKEN_ILLEGAL;
         }
 
-        var = njs_variable_add(vm, parser, NJS_VARIABLE_VAR);
+        var = njs_parser_variable_add(vm, parser, NJS_VARIABLE_VAR);
         if (nxt_slow_path(var == NULL)) {
             return NJS_TOKEN_ERROR;
         }
@@ -1311,7 +1328,7 @@ njs_parser_for_var_statement(njs_vm_t *v
 
         name->token = NJS_TOKEN_NAME;
 
-        ret = njs_variable_reference(vm, parser, name, NJS_DECLARATION);
+        ret = njs_parser_variable_reference(vm, parser, name, NJS_DECLARATION);
         if (nxt_slow_path(ret != NXT_OK)) {
             return NJS_TOKEN_ERROR;
         }
@@ -1584,7 +1601,7 @@ njs_parser_try_statement(njs_vm_t *vm, n
             return NJS_TOKEN_ERROR;
         }
 
-        var = njs_variable_add(vm, parser, NJS_VARIABLE_CATCH);
+        var = njs_parser_variable_add(vm, parser, NJS_VARIABLE_CATCH);
         if (nxt_slow_path(var == NULL)) {
             return NJS_TOKEN_ERROR;
         }
@@ -1596,7 +1613,7 @@ njs_parser_try_statement(njs_vm_t *vm, n
 
         node->token = NJS_TOKEN_NAME;
 
-        ret = njs_variable_reference(vm, parser, node, NJS_DECLARATION);
+        ret = njs_parser_variable_reference(vm, parser, node, NJS_DECLARATION);
         if (nxt_slow_path(ret != NXT_OK)) {
             return NJS_TOKEN_ERROR;
         }
@@ -1816,7 +1833,7 @@ njs_parser_terminal(njs_vm_t *vm, njs_pa
             break;
         }
 
-        ret = njs_variable_reference(vm, parser, node, NJS_REFERENCE);
+        ret = njs_parser_variable_reference(vm, parser, node, NJS_REFERENCE);
         if (nxt_slow_path(ret != NXT_OK)) {
             return NJS_TOKEN_ERROR;
         }
@@ -2075,11 +2092,23 @@ static njs_token_t
 njs_parser_builtin_object(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node)
 {
-    njs_ret_t       ret;
-    nxt_uint_t      index;
-    njs_variable_t  *var;
-
-    var = njs_builtin_add(vm, parser);
+    uint32_t            hash;
+    nxt_str_t           *name;
+    njs_ret_t           ret;
+    nxt_uint_t          index;
+    njs_variable_t      *var;
+    njs_parser_scope_t  *scope;
+
+    scope = parser->scope;
+
+    while (scope->type != NJS_SCOPE_GLOBAL) {
+        scope = scope->parent;
+    }
+
+    hash = parser->lexer->key_hash;
+    name = &parser->lexer->text;
+
+    var = njs_variable_add(vm, scope, name, hash, NJS_VARIABLE_VAR);
     if (nxt_slow_path(var == NULL)) {
         return NJS_TOKEN_ERROR;
     }
@@ -2090,7 +2119,7 @@ njs_parser_builtin_object(njs_vm_t *vm, 
     var->value.type = NJS_OBJECT;
     var->value.data.truth = 1;
 
-    ret = njs_variable_reference(vm, parser, node, NJS_REFERENCE);
+    ret = njs_variable_reference(vm, scope, node, name, hash, NJS_REFERENCE);
     if (nxt_slow_path(ret != NXT_OK)) {
         return NJS_TOKEN_ERROR;
     }
@@ -2107,11 +2136,23 @@ static njs_token_t
 njs_parser_builtin_function(njs_vm_t *vm, njs_parser_t *parser,
     njs_parser_node_t *node)
 {
-    njs_ret_t       ret;
-    nxt_uint_t      index;
-    njs_variable_t  *var;
-
-    var = njs_builtin_add(vm, parser);
+    uint32_t            hash;
+    nxt_str_t           *name;
+    njs_ret_t           ret;
+    nxt_uint_t          index;
+    njs_variable_t      *var;
+    njs_parser_scope_t  *scope;
+
+    scope = parser->scope;
+
+    while (scope->type != NJS_SCOPE_GLOBAL) {
+        scope = scope->parent;
+    }
+
+    hash = parser->lexer->key_hash;
+    name = &parser->lexer->text;
+
+    var = njs_variable_add(vm, scope, name, hash, NJS_VARIABLE_VAR);
     if (nxt_slow_path(var == NULL)) {
         return NJS_TOKEN_ERROR;
     }
@@ -2122,7 +2163,7 @@ njs_parser_builtin_function(njs_vm_t *vm
     var->value.type = NJS_FUNCTION;
     var->value.data.truth = 1;
 
-    ret = njs_variable_reference(vm, parser, node, NJS_REFERENCE);
+    ret = njs_variable_reference(vm, scope, node, name, hash, NJS_REFERENCE);
     if (nxt_slow_path(ret != NXT_OK)) {
         return NJS_TOKEN_ERROR;
     }
diff -r 36989f329e16 -r 1cac2060faae njs/njs_parser.h
--- a/njs/njs_parser.h	Wed Dec 26 16:05:23 2018 +0300
+++ b/njs/njs_parser.h	Wed Dec 26 19:57:41 2018 +0300
@@ -250,8 +250,6 @@ struct njs_parser_scope_s {
 };
 
 
-typedef struct njs_parser_node_s    njs_parser_node_t;
-
 struct njs_parser_node_s {
     njs_token_t                     token:16;
     uint8_t                         ctor:1;
@@ -328,8 +326,9 @@ njs_token_t njs_parser_property_name(njs
 njs_token_t njs_parser_property_token(njs_parser_t *parser);
 njs_token_t njs_parser_token(njs_parser_t *parser);
 nxt_int_t njs_parser_string_create(njs_vm_t *vm, njs_value_t *value);
-njs_ret_t njs_variable_reference(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node, njs_variable_reference_t reference);
+njs_ret_t njs_variable_reference(njs_vm_t *vm, njs_parser_scope_t *scope,
+    njs_parser_node_t *node, nxt_str_t *name, uint32_t hash,
+    njs_variable_reference_t reference);
 njs_variable_t *njs_variable_get(njs_vm_t *vm, njs_parser_node_t *node);
 njs_index_t njs_variable_typeof(njs_vm_t *vm, njs_parser_node_t *node);
 njs_index_t njs_variable_index(njs_vm_t *vm, njs_parser_node_t *node);
diff -r 36989f329e16 -r 1cac2060faae njs/njs_variable.c
--- a/njs/njs_variable.c	Wed Dec 26 16:05:23 2018 +0300
+++ b/njs/njs_variable.c	Wed Dec 26 19:57:41 2018 +0300
@@ -1,6 +1,7 @@
 
 /*
  * Copyright (C) Igor Sysoev
+ * Copyright (C) Dmitry Volyntsev
  * Copyright (C) NGINX, Inc.
  */
 
@@ -15,8 +16,8 @@ typedef struct {
 } njs_variable_scope_t;
 
 
-static njs_ret_t njs_variable_find(njs_vm_t *vm, njs_parser_node_t *node,
-    njs_variable_scope_t *vs);
+static njs_ret_t  njs_variable_find(njs_vm_t *vm, njs_parser_scope_t *scope,
+    njs_variable_scope_t *vs, nxt_str_t *name, uint32_t hash);
 static njs_variable_t *njs_variable_alloc(njs_vm_t *vm, nxt_str_t *name,
     njs_variable_type_t type);
 
@@ -48,67 +49,17 @@ const nxt_lvlhsh_proto_t  njs_variables_
 
 
 njs_variable_t *
-njs_builtin_add(njs_vm_t *vm, njs_parser_t *parser)
+njs_variable_add(njs_vm_t *vm, njs_parser_scope_t *scope, nxt_str_t *name,
+    uint32_t hash, njs_variable_type_t type)
 {
     nxt_int_t           ret;
     njs_variable_t      *var;
-    njs_parser_scope_t  *scope;
     nxt_lvlhsh_query_t  lhq;
 
-    lhq.key_hash = parser->lexer->key_hash;
-    lhq.key = parser->lexer->text;
+    lhq.key_hash = hash;
+    lhq.key = *name;
     lhq.proto = &njs_variables_hash_proto;
 
-    scope = parser->scope;
-
-    while (scope->type != NJS_SCOPE_GLOBAL) {
-        scope = scope->parent;
-    }
-
-    if (nxt_lvlhsh_find(&scope->variables, &lhq) == NXT_OK) {
-        var = lhq.value;
-
-        return var;
-    }
-
-    var = njs_variable_alloc(vm, &lhq.key, NJS_VARIABLE_VAR);
-    if (nxt_slow_path(var == NULL)) {
-        return var;
-    }
-
-    lhq.replace = 0;
-    lhq.value = var;
-    lhq.pool = vm->mem_cache_pool;
-
-    ret = nxt_lvlhsh_insert(&scope->variables, &lhq);
-
-    if (nxt_fast_path(ret == NXT_OK)) {
-        return var;
-    }
-
-    njs_internal_error(vm, "lvlhsh insert failed");
-
-    nxt_mem_cache_free(vm->mem_cache_pool, var->name.start);
-    nxt_mem_cache_free(vm->mem_cache_pool, var);
-
-    return NULL;
-}
-
-
-njs_variable_t *
-njs_variable_add(njs_vm_t *vm, njs_parser_t *parser, njs_variable_type_t type)
-{
-    nxt_int_t           ret;
-    njs_variable_t      *var;
-    njs_parser_scope_t  *scope;
-    nxt_lvlhsh_query_t  lhq;
-
-    lhq.key_hash = parser->lexer->key_hash;
-    lhq.key = parser->lexer->text;
-    lhq.proto = &njs_variables_hash_proto;
-
-    scope = parser->scope;
-
     if (type >= NJS_VARIABLE_VAR) {
         /*
          * A "var" and "function" declarations are
@@ -175,17 +126,18 @@ const nxt_lvlhsh_proto_t  njs_reference_
 
 
 njs_ret_t
-njs_variable_reference(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node, njs_variable_reference_t reference)
+njs_variable_reference(njs_vm_t *vm, njs_parser_scope_t *scope,
+    njs_parser_node_t *node, nxt_str_t *name, uint32_t hash,
+    njs_variable_reference_t reference)
 {
     njs_ret_t           ret;
     nxt_lvlhsh_query_t  lhq;
 
-    ret = njs_name_copy(vm, &node->u.variable_name, &parser->lexer->text);
+    ret = njs_name_copy(vm, &node->u.variable_name, name);
 
     if (nxt_fast_path(ret == NXT_OK)) {
-        node->variable_name_hash = parser->lexer->key_hash;
-        node->scope = parser->scope;
+        node->variable_name_hash = hash;
+        node->scope = scope;
         node->reference = reference;
 
         lhq.key_hash = node->variable_name_hash;
@@ -195,7 +147,7 @@ njs_variable_reference(njs_vm_t *vm, njs
         lhq.value = node;
         lhq.pool = vm->mem_cache_pool;
 
-        ret = nxt_lvlhsh_insert(&parser->scope->references, &lhq);
+        ret = nxt_lvlhsh_insert(&scope->references, &lhq);
 
         if (nxt_slow_path(ret != NXT_ERROR)) {
             ret = NXT_OK;
@@ -241,7 +193,9 @@ njs_variables_scope_resolve(njs_vm_t *vm
             }
 
             if (!local_scope) {
-                ret = njs_variable_find(vm, node, &vs);
+                ret = njs_variable_find(vm, node->scope, &vs,
+                                        &node->u.variable_name,
+                                        node->variable_name_hash);
                 if (nxt_slow_path(ret != NXT_OK)) {
                     continue;
                 }
@@ -308,7 +262,8 @@ njs_variable_typeof(njs_vm_t *vm, njs_pa
         return node->index;
     }
 
-    ret = njs_variable_find(vm, node, &vs);
+    ret = njs_variable_find(vm, node->scope, &vs, &node->u.variable_name,
+                            node->variable_name_hash);
 
     if (nxt_fast_path(ret == NXT_OK)) {
         return vs.variable->index;
@@ -348,7 +303,8 @@ njs_variable_get(njs_vm_t *vm, njs_parse
     njs_variable_t        *var;
     njs_variable_scope_t  vs;
 
-    ret = njs_variable_find(vm, node, &vs);
+    ret = njs_variable_find(vm, node->scope, &vs, &node->u.variable_name,
+                            node->variable_name_hash);
 
     if (nxt_slow_path(ret != NXT_OK)) {
         goto not_found;
@@ -446,17 +402,16 @@ not_found:
 
 
 static njs_ret_t
-njs_variable_find(njs_vm_t *vm, njs_parser_node_t *node,
-    njs_variable_scope_t *vs)
+njs_variable_find(njs_vm_t *vm, njs_parser_scope_t *scope,
+    njs_variable_scope_t *vs, nxt_str_t *name, uint32_t hash)
 {
-    njs_parser_scope_t  *scope, *parent, *previous;
+    njs_parser_scope_t  *parent, *previous;
 
-    vs->lhq.key_hash = node->variable_name_hash;
-    vs->lhq.key = node->u.variable_name;
+    vs->lhq.key_hash = hash;
+    vs->lhq.key = *name;
     vs->lhq.proto = &njs_variables_hash_proto;
 
     previous = NULL;
-    scope = node->scope;
 
     for ( ;; ) {
         if (nxt_lvlhsh_find(&scope->variables, &vs->lhq) == NXT_OK) {
@@ -517,6 +472,8 @@ njs_variable_alloc(njs_vm_t *vm, nxt_str
 
     nxt_mem_cache_free(vm->mem_cache_pool, var);
 
+    njs_memory_error(vm);
+
     return NULL;
 }
 
diff -r 36989f329e16 -r 1cac2060faae njs/njs_variable.h
--- a/njs/njs_variable.h	Wed Dec 26 16:05:23 2018 +0300
+++ b/njs/njs_variable.h	Wed Dec 26 19:57:41 2018 +0300
@@ -42,9 +42,11 @@ typedef struct {
          + njs_scope_offset((var)->index) - NJS_INDEX_GLOBAL_OFFSET)
 
 
-njs_variable_t *njs_builtin_add(njs_vm_t *vm, njs_parser_t *parser);
-njs_variable_t *njs_variable_add(njs_vm_t *vm, njs_parser_t *parser,
-    njs_variable_type_t type);
+njs_variable_t *njs_variable_add(njs_vm_t *vm, njs_parser_scope_t *scope,
+    nxt_str_t *name, uint32_t hash, njs_variable_type_t type);
+njs_ret_t njs_variable_reference(njs_vm_t *vm, njs_parser_scope_t *scope,
+    njs_parser_node_t *node, nxt_str_t *name, uint32_t hash,
+    njs_variable_reference_t reference);
 njs_ret_t njs_variables_scope_reference(njs_vm_t *vm,
     njs_parser_scope_t *scope);
 njs_ret_t njs_name_copy(njs_vm_t *vm, nxt_str_t *dst, nxt_str_t *src);
diff -r 36989f329e16 -r 1cac2060faae njs/njs_vm.h
--- a/njs/njs_vm.h	Wed Dec 26 16:05:23 2018 +0300
+++ b/njs/njs_vm.h	Wed Dec 26 19:57:41 2018 +0300
@@ -165,6 +165,7 @@ typedef struct njs_frame_s            nj
 typedef struct njs_native_frame_s     njs_native_frame_t;
 typedef struct njs_property_next_s    njs_property_next_t;
 typedef struct njs_parser_scope_s     njs_parser_scope_t;
+typedef struct njs_parser_node_s      njs_parser_node_t;
 
 
 union njs_value_s {

From xeioex at nginx.com  Thu Dec 27 07:29:01 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Thu, 27 Dec 2018 07:29:01 +0000
Subject: [njs] Style.
Message-ID: <hg.a8dff774ab62.1545895741.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/a8dff774ab62
branches:  
changeset: 699:a8dff774ab62
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Thu Dec 27 10:27:46 2018 +0300
description:
Style.

diffstat:

 njs/njs_variable.c |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff -r 1cac2060faae -r a8dff774ab62 njs/njs_variable.c
--- a/njs/njs_variable.c	Wed Dec 26 19:57:41 2018 +0300
+++ b/njs/njs_variable.c	Thu Dec 27 10:27:46 2018 +0300
@@ -16,7 +16,7 @@ typedef struct {
 } njs_variable_scope_t;
 
 
-static njs_ret_t  njs_variable_find(njs_vm_t *vm, njs_parser_scope_t *scope,
+static njs_ret_t njs_variable_find(njs_vm_t *vm, njs_parser_scope_t *scope,
     njs_variable_scope_t *vs, nxt_str_t *name, uint32_t hash);
 static njs_variable_t *njs_variable_alloc(njs_vm_t *vm, nxt_str_t *name,
     njs_variable_type_t type);

From xeioex at nginx.com  Thu Dec 27 08:49:37 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Thu, 27 Dec 2018 08:49:37 +0000
Subject: [njs] Improved naming in njs_variables_scope_resolve().
Message-ID: <hg.9ab8d11c151d.1545900577.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/9ab8d11c151d
branches:  
changeset: 700:9ab8d11c151d
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Thu Dec 27 10:35:07 2018 +0300
description:
Improved naming in njs_variables_scope_resolve().

Thanks to ??? (Hong Zhi Dao).

diffstat:

 njs/njs_variable.c |  10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

diffs (45 lines):

diff -r a8dff774ab62 -r 9ab8d11c151d njs/njs_variable.c
--- a/njs/njs_variable.c	Thu Dec 27 10:27:46 2018 +0300
+++ b/njs/njs_variable.c	Thu Dec 27 10:35:07 2018 +0300
@@ -160,7 +160,7 @@ njs_variable_reference(njs_vm_t *vm, njs
 
 static njs_ret_t
 njs_variables_scope_resolve(njs_vm_t *vm, njs_parser_scope_t *scope,
-    nxt_bool_t local_scope)
+    nxt_bool_t closure)
 {
     njs_ret_t             ret;
     nxt_queue_t           *nested;
@@ -178,7 +178,7 @@ njs_variables_scope_resolve(njs_vm_t *vm
     {
         scope = nxt_queue_link_data(lnk, njs_parser_scope_t, link);
 
-        ret = njs_variables_scope_resolve(vm, scope, local_scope);
+        ret = njs_variables_scope_resolve(vm, scope, closure);
         if (nxt_slow_path(ret != NXT_OK)) {
             return NXT_ERROR;
         }
@@ -192,7 +192,7 @@ njs_variables_scope_resolve(njs_vm_t *vm
                 break;
             }
 
-            if (!local_scope) {
+            if (closure) {
                 ret = njs_variable_find(vm, node->scope, &vs,
                                         &node->u.variable_name,
                                         node->variable_name_hash);
@@ -238,12 +238,12 @@ njs_variables_scope_reference(njs_vm_t *
      * only in the local scope (reference and definition nestings are the same).
      */
 
-    ret = njs_variables_scope_resolve(vm, scope, 0);
+    ret = njs_variables_scope_resolve(vm, scope, 1);
     if (nxt_slow_path(ret != NXT_OK)) {
         return NXT_ERROR;
     }
 
-    ret = njs_variables_scope_resolve(vm, scope, 1);
+    ret = njs_variables_scope_resolve(vm, scope, 0);
     if (nxt_slow_path(ret != NXT_OK)) {
         return NXT_ERROR;
     }

From xeioex at nginx.com  Fri Dec 28 09:46:26 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 28 Dec 2018 09:46:26 +0000
Subject: [njs] njs_vm_pending() is split in njs_vm_posted() and
 njs_vm_waiting().
Message-ID: <hg.bdd4c516a078.1545990386.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/bdd4c516a078
branches:  
changeset: 702:bdd4c516a078
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Dec 28 12:43:22 2018 +0300
description:
njs_vm_pending() is split in njs_vm_posted() and njs_vm_waiting().

There are two types of events in njs:
    Posted - are ready to be executed by njs_vm_run().
    Waiting - await external async events.

diffstat:

 njs/njs.c       |  15 +++++++++++----
 njs/njs.h       |  14 +++++++++++++-
 njs/njs_event.h |   4 +++-
 3 files changed, 27 insertions(+), 6 deletions(-)

diffs (82 lines):

diff -r fa1c446b6823 -r bdd4c516a078 njs/njs.c
--- a/njs/njs.c	Fri Dec 28 12:42:29 2018 +0300
+++ b/njs/njs.c	Fri Dec 28 12:43:22 2018 +0300
@@ -196,7 +196,7 @@ njs_vm_destroy(njs_vm_t *vm)
     njs_event_t        *event;
     nxt_lvlhsh_each_t  lhe;
 
-    if (njs_is_pending_events(vm)) {
+    if (njs_waiting_events(vm)) {
         nxt_lvlhsh_each_init(&lhe, &njs_event_hash_proto);
 
         for ( ;; ) {
@@ -558,9 +558,16 @@ njs_vm_del_event(njs_vm_t *vm, njs_vm_ev
 
 
 nxt_int_t
-njs_vm_pending(njs_vm_t *vm)
+njs_vm_waiting(njs_vm_t *vm)
 {
-    return njs_is_pending_events(vm);
+    return njs_waiting_events(vm);
+}
+
+
+nxt_int_t
+njs_vm_posted(njs_vm_t *vm)
+{
+    return njs_posted_events(vm);
 }
 
 
@@ -652,7 +659,7 @@ njs_vm_handle_events(njs_vm_t *vm)
         }
     }
 
-    return njs_is_pending_events(vm) ? NJS_AGAIN : NJS_OK;
+    return njs_posted_events(vm) ? NJS_AGAIN : NJS_OK;
 }
 
 
diff -r fa1c446b6823 -r bdd4c516a078 njs/njs.h
--- a/njs/njs.h	Fri Dec 28 12:42:29 2018 +0300
+++ b/njs/njs.h	Fri Dec 28 12:43:22 2018 +0300
@@ -167,11 +167,23 @@ NXT_EXPORT njs_vm_event_t njs_vm_add_eve
     njs_function_t *function, nxt_uint_t once, njs_host_event_t host_ev,
     njs_event_destructor destructor);
 NXT_EXPORT void njs_vm_del_event(njs_vm_t *vm, njs_vm_event_t vm_event);
-NXT_EXPORT nxt_int_t njs_vm_pending(njs_vm_t *vm);
 NXT_EXPORT nxt_int_t njs_vm_post_event(njs_vm_t *vm, njs_vm_event_t vm_event,
     const njs_value_t *args, nxt_uint_t nargs);
 
 /*
+ * Returns 1 if async events are present.
+ */
+NXT_EXPORT nxt_int_t njs_vm_waiting(njs_vm_t *vm);
+
+/*
+ * Returns 1 if posted events are ready to be executed.
+ */
+NXT_EXPORT nxt_int_t njs_vm_posted(njs_vm_t *vm);
+
+#define njs_vm_pending(vm)  (njs_vm_waiting(vm) || njs_vm_posted(vm))
+
+
+/*
  * Runs the specified function with provided arguments.
  *  NJS_OK successful run.
  *  NJS_ERROR some exception or internal error happens.
diff -r fa1c446b6823 -r bdd4c516a078 njs/njs_event.h
--- a/njs/njs_event.h	Fri Dec 28 12:42:29 2018 +0300
+++ b/njs/njs_event.h	Fri Dec 28 12:43:22 2018 +0300
@@ -12,7 +12,9 @@
 #define NJS_EVENT_DELETE       2
 
 
-#define njs_is_pending_events(vm) (!nxt_lvlhsh_is_empty(&(vm)->events_hash))
+#define njs_waiting_events(vm) (!nxt_lvlhsh_is_empty(&(vm)->events_hash))
+
+#define njs_posted_events(vm) (!nxt_queue_is_empty(&(vm)->posted_events))
 
 
 typedef struct {

From xeioex at nginx.com  Fri Dec 28 09:46:26 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 28 Dec 2018 09:46:26 +0000
Subject: [njs] njs_vm_run() is rectified.
Message-ID: <hg.fa1c446b6823.1545990386.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/fa1c446b6823
branches:  
changeset: 701:fa1c446b6823
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Dec 28 12:42:29 2018 +0300
description:
njs_vm_run() is rectified.

Previously, both njs_vm_call() and njs_vm_run() can be used to run njs
code. njs_vm_call() was used to invoke a single function, while
njs_vm_run() was used to run global code as well to process the events.

At first invocation njs_vm_run() executed global code, all the next
invocations it processed pending events.

The solution is splitting njs_vm_run() into two functions. One for
events processing and another for running the global code.

diffstat:

 nginx/ngx_http_js_module.c      |   2 +-
 nginx/ngx_stream_js_module.c    |   2 +-
 njs/njs.c                       |  25 ++++++++++++-------------
 njs/njs.h                       |  29 +++++++++++++++++++++++++++--
 njs/njs_shell.c                 |   2 +-
 njs/test/njs_interactive_test.c |   2 +-
 njs/test/njs_unit_test.c        |   2 +-
 7 files changed, 44 insertions(+), 20 deletions(-)

diffs (159 lines):

diff -r 9ab8d11c151d -r fa1c446b6823 nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c	Thu Dec 27 10:35:07 2018 +0300
+++ b/nginx/ngx_http_js_module.c	Fri Dec 28 12:42:29 2018 +0300
@@ -913,7 +913,7 @@ ngx_http_js_init_vm(ngx_http_request_t *
     cln->handler = ngx_http_js_cleanup_ctx;
     cln->data = ctx;
 
-    if (njs_vm_run(ctx->vm) == NJS_ERROR) {
+    if (njs_vm_start(ctx->vm) == NJS_ERROR) {
         njs_vm_retval_to_ext_string(ctx->vm, &exception);
 
         ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
diff -r 9ab8d11c151d -r fa1c446b6823 nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c	Thu Dec 27 10:35:07 2018 +0300
+++ b/nginx/ngx_stream_js_module.c	Fri Dec 28 12:42:29 2018 +0300
@@ -727,7 +727,7 @@ ngx_stream_js_init_vm(ngx_stream_session
     cln->handler = ngx_stream_js_cleanup_ctx;
     cln->data = ctx;
 
-    if (njs_vm_run(ctx->vm) == NJS_ERROR) {
+    if (njs_vm_start(ctx->vm) == NJS_ERROR) {
         njs_vm_retval_to_ext_string(ctx->vm, &exception);
 
         ngx_log_error(NGX_LOG_ERR, s->connection->log, 0,
diff -r 9ab8d11c151d -r fa1c446b6823 njs/njs.c
--- a/njs/njs.c	Thu Dec 27 10:35:07 2018 +0300
+++ b/njs/njs.c	Fri Dec 28 12:42:29 2018 +0300
@@ -595,27 +595,26 @@ njs_vm_post_event(njs_vm_t *vm, njs_vm_e
 nxt_int_t
 njs_vm_run(njs_vm_t *vm)
 {
-    nxt_int_t  ret;
-
     if (nxt_slow_path(vm->backtrace != NULL)) {
         nxt_array_reset(vm->backtrace);
     }
 
+    return njs_vm_handle_events(vm);
+}
+
+
+nxt_int_t
+njs_vm_start(njs_vm_t *vm)
+{
+    njs_ret_t  ret;
+
     ret = njs_vmcode_interpreter(vm);
 
     if (ret == NJS_STOP) {
-        ret = njs_vm_handle_events(vm);
+        ret = NJS_OK;
     }
 
-    switch (ret) {
-    case NJS_STOP:
-        return NJS_OK;
-
-    case NXT_AGAIN:
-    case NXT_ERROR:
-    default:
-        return ret;
-    }
+    return ret;
 }
 
 
@@ -653,7 +652,7 @@ njs_vm_handle_events(njs_vm_t *vm)
         }
     }
 
-    return njs_is_pending_events(vm) ? NJS_AGAIN : NJS_STOP;
+    return njs_is_pending_events(vm) ? NJS_AGAIN : NJS_OK;
 }
 
 
diff -r 9ab8d11c151d -r fa1c446b6823 njs/njs.h
--- a/njs/njs.h	Thu Dec 27 10:35:07 2018 +0300
+++ b/njs/njs.h	Fri Dec 28 12:42:29 2018 +0300
@@ -162,8 +162,6 @@ NXT_EXPORT void njs_vm_destroy(njs_vm_t 
 
 NXT_EXPORT nxt_int_t njs_vm_compile(njs_vm_t *vm, u_char **start, u_char *end);
 NXT_EXPORT njs_vm_t *njs_vm_clone(njs_vm_t *vm, njs_external_ptr_t external);
-NXT_EXPORT nxt_int_t njs_vm_call(njs_vm_t *vm, njs_function_t *function,
-    const njs_value_t *args, nxt_uint_t nargs);
 
 NXT_EXPORT njs_vm_event_t njs_vm_add_event(njs_vm_t *vm,
     njs_function_t *function, nxt_uint_t once, njs_host_event_t host_ev,
@@ -173,8 +171,35 @@ NXT_EXPORT nxt_int_t njs_vm_pending(njs_
 NXT_EXPORT nxt_int_t njs_vm_post_event(njs_vm_t *vm, njs_vm_event_t vm_event,
     const njs_value_t *args, nxt_uint_t nargs);
 
+/*
+ * Runs the specified function with provided arguments.
+ *  NJS_OK successful run.
+ *  NJS_ERROR some exception or internal error happens.
+ *
+ *  njs_vm_retval(vm) can be used to get the retval or exception value.
+ */
+NXT_EXPORT nxt_int_t njs_vm_call(njs_vm_t *vm, njs_function_t *function,
+    const njs_value_t *args, nxt_uint_t nargs);
+
+/*
+ * Runs posted events.
+ *  NJS_OK successfully processed all posted events, no more events.
+ *  NJS_AGAIN successfully processed all events, some posted events are
+ *    still pending.
+ *  NJS_ERROR some exception or internal error happens.
+ *    njs_vm_retval(vm) can be used to get the retval or exception value.
+ */
 NXT_EXPORT nxt_int_t njs_vm_run(njs_vm_t *vm);
 
+/*
+ * Runs the global code.
+ *   NJS_OK successful run.
+ *   NJS_ERROR some exception or internal error happens.
+ *
+ *   njs_vm_retval(vm) can be used to get the retval or exception value.
+ */
+NXT_EXPORT nxt_int_t njs_vm_start(njs_vm_t *vm);
+
 NXT_EXPORT const njs_extern_t *njs_vm_external_prototype(njs_vm_t *vm,
     njs_external_t *external);
 NXT_EXPORT nxt_int_t njs_vm_external_create(njs_vm_t *vm,
diff -r 9ab8d11c151d -r fa1c446b6823 njs/njs_shell.c
--- a/njs/njs_shell.c	Thu Dec 27 10:35:07 2018 +0300
+++ b/njs/njs_shell.c	Fri Dec 28 12:42:29 2018 +0300
@@ -490,7 +490,7 @@ njs_process_script(njs_vm_t *vm, njs_opt
             printf("\n");
         }
 
-        ret = njs_vm_run(vm);
+        ret = njs_vm_start(vm);
     }
 
     if (njs_vm_retval_dump(vm, out, 1) != NXT_OK) {
diff -r 9ab8d11c151d -r fa1c446b6823 njs/test/njs_interactive_test.c
--- a/njs/test/njs_interactive_test.c	Thu Dec 27 10:35:07 2018 +0300
+++ b/njs/test/njs_interactive_test.c	Fri Dec 28 12:42:29 2018 +0300
@@ -285,7 +285,7 @@ njs_interactive_test(nxt_bool_t verbose)
 
             ret = njs_vm_compile(vm, &start, end);
             if (ret == NXT_OK) {
-                ret = njs_vm_run(vm);
+                ret = njs_vm_start(vm);
             }
         }
 
diff -r 9ab8d11c151d -r fa1c446b6823 njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Thu Dec 27 10:35:07 2018 +0300
+++ b/njs/test/njs_unit_test.c	Fri Dec 28 12:42:29 2018 +0300
@@ -11801,7 +11801,7 @@ njs_unit_test(njs_unit_test_t tests[], s
                 goto done;
             }
 
-            ret = njs_vm_run(nvm);
+            ret = njs_vm_start(nvm);
 
             if (njs_vm_retval_to_ext_string(nvm, &s) != NXT_OK) {
                 printf("njs_vm_retval_to_ext_string() failed\n");

From xeioex at nginx.com  Fri Dec 28 09:46:26 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 28 Dec 2018 09:46:26 +0000
Subject: [njs] Interactive shell: initial support of posted events.
Message-ID: <hg.f08c5fc1dea8.1545990386.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/f08c5fc1dea8
branches:  
changeset: 703:f08c5fc1dea8
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Dec 28 12:43:36 2018 +0300
description:
Interactive shell: initial support of posted events.

diffstat:

 njs/njs_shell.c |  49 ++++++++++++++++++++++++++++++++++++-------------
 1 files changed, 36 insertions(+), 13 deletions(-)

diffs (103 lines):

diff -r bdd4c516a078 -r f08c5fc1dea8 njs/njs_shell.c
--- a/njs/njs_shell.c	Fri Dec 28 12:43:22 2018 +0300
+++ b/njs/njs_shell.c	Fri Dec 28 12:43:36 2018 +0300
@@ -54,7 +54,7 @@ static nxt_int_t njs_interactive_shell(n
     njs_vm_opt_t *vm_options);
 static nxt_int_t njs_process_file(njs_opts_t *opts, njs_vm_opt_t *vm_options);
 static nxt_int_t njs_process_script(njs_vm_t *vm, njs_opts_t *opts,
-    const nxt_str_t *script, nxt_str_t *out);
+    const nxt_str_t *script);
 static nxt_int_t njs_editline_init(njs_vm_t *vm);
 static char **njs_completion_handler(const char *text, int start, int end);
 static char *njs_completion_generator(const char *text, int state);
@@ -297,7 +297,7 @@ static nxt_int_t
 njs_interactive_shell(njs_opts_t *opts, njs_vm_opt_t *vm_options)
 {
     njs_vm_t   *vm;
-    nxt_str_t  line, out;
+    nxt_str_t  line;
 
     vm = njs_vm_create(vm_options);
     if (vm == NULL) {
@@ -335,9 +335,7 @@ njs_interactive_shell(njs_opts_t *opts, 
 
         add_history((char *) line.start);
 
-        njs_process_script(vm, opts, &line, &out);
-
-        printf("%.*s\n", (int) out.length, out.start);
+        njs_process_script(vm, opts, &line);
 
         /* editline allocs a new buffer every time. */
         free(line.start);
@@ -357,7 +355,7 @@ njs_process_file(njs_opts_t *opts, njs_v
     ssize_t      n;
     njs_vm_t     *vm;
     nxt_int_t    ret;
-    nxt_str_t    out, script;
+    nxt_str_t    script;
     struct stat  sb;
 
     file = opts->file;
@@ -448,9 +446,8 @@ njs_process_file(njs_opts_t *opts, njs_v
         goto done;
     }
 
-    ret = njs_process_script(vm, opts, &script, &out);
+    ret = njs_process_script(vm, opts, &script);
     if (ret != NXT_OK) {
-        fprintf(stderr, "%.*s\n", (int) out.length, out.start);
         ret = NXT_ERROR;
         goto done;
     }
@@ -473,9 +470,27 @@ close_fd:
 }
 
 
+static void
+njs_output(njs_vm_t *vm, njs_opts_t *opts, njs_ret_t ret)
+{
+    nxt_str_t  out;
+
+    if (njs_vm_retval_dump(vm, &out, 1) != NXT_OK) {
+        out = nxt_string_value("failed to get retval from VM");
+        ret = NJS_ERROR;
+    }
+
+    if (ret != NJS_OK) {
+        fprintf(stderr, "%.*s\n", (int) out.length, out.start);
+
+    } else if (opts->interactive) {
+        printf("%.*s\n", (int) out.length, out.start);
+    }
+}
+
+
 static nxt_int_t
-njs_process_script(njs_vm_t *vm, njs_opts_t *opts, const nxt_str_t *script,
-    nxt_str_t *out)
+njs_process_script(njs_vm_t *vm, njs_opts_t *opts, const nxt_str_t *script)
 {
     u_char     *start;
     nxt_int_t  ret;
@@ -493,9 +508,17 @@ njs_process_script(njs_vm_t *vm, njs_opt
         ret = njs_vm_start(vm);
     }
 
-    if (njs_vm_retval_dump(vm, out, 1) != NXT_OK) {
-        *out = nxt_string_value("failed to get retval from VM");
-        return NXT_ERROR;
+    njs_output(vm, opts, ret);
+
+    if (ret == NJS_OK) {
+        while (njs_vm_posted(vm)) {
+            ret = njs_vm_run(vm);
+
+            if (ret == NJS_ERROR) {
+                njs_output(vm, opts, ret);
+                return ret;
+            }
+        }
     }
 
     return ret;

From xeioex at nginx.com  Fri Dec 28 09:46:26 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 28 Dec 2018 09:46:26 +0000
Subject: [njs] Added setImmediate().
Message-ID: <hg.269fe12bbabe.1545990386.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/269fe12bbabe
branches:  
changeset: 704:269fe12bbabe
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Dec 28 12:44:56 2018 +0300
description:
Added setImmediate().

diffstat:

 njs/njs.c               |   2 +-
 njs/njs.h               |  10 +++++-----
 njs/njs_builtin.c       |   3 +++
 njs/njs_event.h         |  18 +++++++++---------
 njs/njs_generator.c     |   1 +
 njs/njs_lexer_keyword.c |   1 +
 njs/njs_parser.c        |   1 +
 njs/njs_parser.h        |   1 +
 njs/njs_time.c          |  41 +++++++++++++++++++++++++++++++++--------
 njs/njs_time.h          |   3 +++
 njs/njs_vm.h            |   1 +
 11 files changed, 59 insertions(+), 23 deletions(-)

diffs (258 lines):

diff -r f08c5fc1dea8 -r 269fe12bbabe njs/njs.c
--- a/njs/njs.c	Fri Dec 28 12:43:36 2018 +0300
+++ b/njs/njs.c	Fri Dec 28 12:44:56 2018 +0300
@@ -521,7 +521,7 @@ njs_vm_call(njs_vm_t *vm, njs_function_t
 
 njs_vm_event_t
 njs_vm_add_event(njs_vm_t *vm, njs_function_t *function, nxt_uint_t once,
-    njs_host_event_t host_ev, njs_event_destructor destructor)
+    njs_host_event_t host_ev, njs_event_destructor_t destructor)
 {
     njs_event_t  *event;
 
diff -r f08c5fc1dea8 -r 269fe12bbabe njs/njs.h
--- a/njs/njs.h	Fri Dec 28 12:43:36 2018 +0300
+++ b/njs/njs.h	Fri Dec 28 12:44:56 2018 +0300
@@ -125,15 +125,15 @@ typedef void *                      njs_
 typedef void *                      njs_host_event_t;
 typedef void *                      njs_external_ptr_t;
 
-typedef njs_host_event_t (*njs_set_timer)(njs_external_ptr_t external,
+typedef njs_host_event_t (*njs_set_timer_t)(njs_external_ptr_t external,
     uint64_t delay, njs_vm_event_t vm_event);
-typedef void (*njs_event_destructor)(njs_external_ptr_t external,
+typedef void (*njs_event_destructor_t)(njs_external_ptr_t external,
     njs_host_event_t event);
 
 
 typedef struct {
-    njs_set_timer                   set_timer;
-    njs_event_destructor            clear_timer;
+    njs_set_timer_t                 set_timer;
+    njs_event_destructor_t          clear_timer;
 } njs_vm_ops_t;
 
 
@@ -165,7 +165,7 @@ NXT_EXPORT njs_vm_t *njs_vm_clone(njs_vm
 
 NXT_EXPORT njs_vm_event_t njs_vm_add_event(njs_vm_t *vm,
     njs_function_t *function, nxt_uint_t once, njs_host_event_t host_ev,
-    njs_event_destructor destructor);
+    njs_event_destructor_t destructor);
 NXT_EXPORT void njs_vm_del_event(njs_vm_t *vm, njs_vm_event_t vm_event);
 NXT_EXPORT nxt_int_t njs_vm_post_event(njs_vm_t *vm, njs_vm_event_t vm_event,
     const njs_value_t *args, nxt_uint_t nargs);
diff -r f08c5fc1dea8 -r 269fe12bbabe njs/njs_builtin.c
--- a/njs/njs_builtin.c	Fri Dec 28 12:43:36 2018 +0300
+++ b/njs/njs_builtin.c	Fri Dec 28 12:44:56 2018 +0300
@@ -111,6 +111,7 @@ const njs_object_init_t    *njs_function
     &njs_decode_uri_component_function_init,
     &njs_require_function_init,
     &njs_set_timeout_function_init,
+    &njs_set_immediate_function_init,
     &njs_clear_timeout_function_init,
     NULL
 };
@@ -132,6 +133,8 @@ const njs_function_init_t  njs_native_fu
     { njs_module_require,              { NJS_SKIP_ARG, NJS_STRING_ARG } },
     { njs_set_timeout,
       { NJS_SKIP_ARG, NJS_FUNCTION_ARG, NJS_NUMBER_ARG } },
+    { njs_set_immediate,
+      { NJS_SKIP_ARG, NJS_FUNCTION_ARG } },
     { njs_clear_timeout,               { NJS_SKIP_ARG, NJS_NUMBER_ARG } },
 };
 
diff -r f08c5fc1dea8 -r 269fe12bbabe njs/njs_event.h
--- a/njs/njs_event.h	Fri Dec 28 12:43:36 2018 +0300
+++ b/njs/njs_event.h	Fri Dec 28 12:44:56 2018 +0300
@@ -18,17 +18,17 @@
 
 
 typedef struct {
-    njs_function_t        *function;
-    njs_value_t           *args;
-    nxt_uint_t            nargs;
-    njs_host_event_t      host_event;
-    njs_event_destructor  destructor;
+    njs_function_t          *function;
+    njs_value_t             *args;
+    nxt_uint_t              nargs;
+    njs_host_event_t        host_event;
+    njs_event_destructor_t  destructor;
 
-    njs_value_t           id;
-    nxt_queue_link_t      link;
+    njs_value_t             id;
+    nxt_queue_link_t        link;
 
-    unsigned              posted:1;
-    unsigned              once:1;
+    unsigned                posted:1;
+    unsigned                once:1;
 } njs_event_t;
 
 
diff -r f08c5fc1dea8 -r 269fe12bbabe njs/njs_generator.c
--- a/njs/njs_generator.c	Fri Dec 28 12:43:36 2018 +0300
+++ b/njs/njs_generator.c	Fri Dec 28 12:44:56 2018 +0300
@@ -406,6 +406,7 @@ njs_generator(njs_vm_t *vm, njs_generato
     case NJS_TOKEN_DECODE_URI_COMPONENT:
     case NJS_TOKEN_REQUIRE:
     case NJS_TOKEN_SET_TIMEOUT:
+    case NJS_TOKEN_SET_IMMEDIATE:
     case NJS_TOKEN_CLEAR_TIMEOUT:
         return njs_generate_builtin_object(vm, generator, node);
 
diff -r f08c5fc1dea8 -r 269fe12bbabe njs/njs_lexer_keyword.c
--- a/njs/njs_lexer_keyword.c	Fri Dec 28 12:43:36 2018 +0300
+++ b/njs/njs_lexer_keyword.c	Fri Dec 28 12:44:56 2018 +0300
@@ -90,6 +90,7 @@ static const njs_keyword_t  njs_keywords
     { nxt_string("decodeURIComponent"),  NJS_TOKEN_DECODE_URI_COMPONENT, 0 },
     { nxt_string("require"),      NJS_TOKEN_REQUIRE, 0 },
     { nxt_string("setTimeout"),   NJS_TOKEN_SET_TIMEOUT, 0 },
+    { nxt_string("setImmediate"), NJS_TOKEN_SET_IMMEDIATE, 0 },
     { nxt_string("clearTimeout"), NJS_TOKEN_CLEAR_TIMEOUT, 0 },
 
     /* Reserved words. */
diff -r f08c5fc1dea8 -r 269fe12bbabe njs/njs_parser.c
--- a/njs/njs_parser.c	Fri Dec 28 12:43:36 2018 +0300
+++ b/njs/njs_parser.c	Fri Dec 28 12:44:56 2018 +0300
@@ -2075,6 +2075,7 @@ njs_parser_terminal(njs_vm_t *vm, njs_pa
     case NJS_TOKEN_DECODE_URI_COMPONENT:
     case NJS_TOKEN_REQUIRE:
     case NJS_TOKEN_SET_TIMEOUT:
+    case NJS_TOKEN_SET_IMMEDIATE:
     case NJS_TOKEN_CLEAR_TIMEOUT:
         return njs_parser_builtin_function(vm, parser, node);
 
diff -r f08c5fc1dea8 -r 269fe12bbabe njs/njs_parser.h
--- a/njs/njs_parser.h	Fri Dec 28 12:43:36 2018 +0300
+++ b/njs/njs_parser.h	Fri Dec 28 12:44:56 2018 +0300
@@ -199,6 +199,7 @@ typedef enum {
     NJS_TOKEN_DECODE_URI_COMPONENT,
     NJS_TOKEN_REQUIRE,
     NJS_TOKEN_SET_TIMEOUT,
+    NJS_TOKEN_SET_IMMEDIATE,
     NJS_TOKEN_CLEAR_TIMEOUT,
 
     NJS_TOKEN_RESERVED,
diff -r f08c5fc1dea8 -r 269fe12bbabe njs/njs_time.c
--- a/njs/njs_time.c	Fri Dec 28 12:43:36 2018 +0300
+++ b/njs/njs_time.c	Fri Dec 28 12:44:56 2018 +0300
@@ -10,10 +10,11 @@
 #include <stdio.h>
 
 
-njs_ret_t
-njs_set_timeout(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
-    njs_index_t unused)
+static njs_ret_t
+njs_set_timer(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
+    njs_index_t unused, nxt_bool_t immediate)
 {
+    nxt_uint_t     n;
     uint64_t       delay;
     njs_event_t   *event;
     njs_vm_ops_t  *ops;
@@ -36,7 +37,7 @@ njs_set_timeout(njs_vm_t *vm, njs_value_
 
     delay = 0;
 
-    if (nargs >= 3 && njs_is_number(&args[2])) {
+    if (!immediate && nargs >= 3 && njs_is_number(&args[2])) {
         delay = args[2].data.u.number;
     }
 
@@ -45,9 +46,11 @@ njs_set_timeout(njs_vm_t *vm, njs_value_
         goto memory_error;
     }
 
-    event->destructor = ops->clear_timer;
+    n = immediate ? 2 : 3;
+
+    event->destructor = (ops != NULL) ? ops->clear_timer : NULL;
     event->function = args[1].data.u.function;
-    event->nargs = (nargs >= 3) ? nargs - 3 : 0;
+    event->nargs = (nargs >= n) ? nargs - n : 0;
     event->once = 1;
     event->posted = 0;
 
@@ -58,11 +61,11 @@ njs_set_timeout(njs_vm_t *vm, njs_value_
             goto memory_error;
         }
 
-        memcpy(event->args, &args[3], sizeof(njs_value_t) * event->nargs);
+        memcpy(event->args, &args[n], sizeof(njs_value_t) * event->nargs);
     }
 
     event->host_event = ops->set_timer(vm->external, delay, event);
-    if (event->host_event == NULL) {
+    if (nxt_slow_path(event->host_event == NULL)) {
         njs_internal_error(vm, "set_timer() failed");
         return NJS_ERROR;
     }
@@ -78,6 +81,22 @@ memory_error:
 
 
 njs_ret_t
+njs_set_timeout(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
+    njs_index_t unused)
+{
+    return njs_set_timer(vm, args, nargs, unused, 0);
+}
+
+
+njs_ret_t
+njs_set_immediate(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
+    njs_index_t unused)
+{
+    return njs_set_timer(vm, args, nargs, unused, 1);
+}
+
+
+njs_ret_t
 njs_clear_timeout(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
     njs_index_t unused)
 {
@@ -116,6 +135,12 @@ const njs_object_init_t  njs_set_timeout
     0,
 };
 
+const njs_object_init_t  njs_set_immediate_function_init = {
+    nxt_string("setImmediate"),
+    NULL,
+    0,
+};
+
 
 const njs_object_init_t  njs_clear_timeout_function_init = {
     nxt_string("clearTimeout"),
diff -r f08c5fc1dea8 -r 269fe12bbabe njs/njs_time.h
--- a/njs/njs_time.h	Fri Dec 28 12:43:36 2018 +0300
+++ b/njs/njs_time.h	Fri Dec 28 12:44:56 2018 +0300
@@ -10,11 +10,14 @@
 
 njs_ret_t njs_set_timeout(njs_vm_t *vm, njs_value_t *args,
     nxt_uint_t nargs, njs_index_t unused);
+njs_ret_t njs_set_immediate(njs_vm_t *vm, njs_value_t *args,
+    nxt_uint_t nargs, njs_index_t unused);
 njs_ret_t njs_clear_timeout(njs_vm_t *vm, njs_value_t *args,
     nxt_uint_t nargs, njs_index_t unused);
 
 
 extern const njs_object_init_t  njs_set_timeout_function_init;
+extern const njs_object_init_t  njs_set_immediate_function_init;
 extern const njs_object_init_t  njs_clear_timeout_function_init;
 
 #endif /* _NJS_TIMEOUT_H_INCLUDED_ */
diff -r f08c5fc1dea8 -r 269fe12bbabe njs/njs_vm.h
--- a/njs/njs_vm.h	Fri Dec 28 12:43:36 2018 +0300
+++ b/njs/njs_vm.h	Fri Dec 28 12:44:56 2018 +0300
@@ -950,6 +950,7 @@ enum njs_function_e {
     NJS_FUNCTION_STRING_DECODE_URI_COMPONENT,
     NJS_FUNCTION_REQUIRE,
     NJS_FUNCTION_SET_TIMEOUT,
+    NJS_FUNCTION_SET_IMMEDIATE,
     NJS_FUNCTION_CLEAR_TIMEOUT,
 #define NJS_FUNCTION_MAX       (NJS_FUNCTION_CLEAR_TIMEOUT + 1)
 };

From xeioex at nginx.com  Fri Dec 28 12:40:23 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 28 Dec 2018 12:40:23 +0000
Subject: [njs] Fixed grammar in njs_vm.h.
Message-ID: <hg.2866557dc7a0.1546000823.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/2866557dc7a0
branches:  
changeset: 705:2866557dc7a0
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Dec 28 15:39:59 2018 +0300
description:
Fixed grammar in njs_vm.h.

diffstat:

 njs/njs_vm.h |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (21 lines):

diff -r 269fe12bbabe -r 2866557dc7a0 njs/njs_vm.h
--- a/njs/njs_vm.h	Fri Dec 28 12:44:56 2018 +0300
+++ b/njs/njs_vm.h	Fri Dec 28 15:39:59 2018 +0300
@@ -828,7 +828,7 @@ typedef enum {
     NJS_SCOPE_GLOBAL = 1,
     NJS_SCOPE_CALLEE_ARGUMENTS = 2,
     /*
-     * The argument and local VM scopes should separate because a
+     * The argument and local VM scopes should be separated because a
      * function may be called with any number of arguments.
      */
     NJS_SCOPE_ARGUMENTS = 3,
@@ -838,7 +838,7 @@ typedef enum {
     NJS_SCOPE_CLOSURE = 5,
     /*
      * The block and shim scopes are not really VM scopes.
-     * They used only on parsing phase.
+     * They are used only on parsing phase.
      */
     NJS_SCOPE_BLOCK = 16,
     NJS_SCOPE_SHIM = 17,

From xeioex at nginx.com  Fri Dec 28 17:34:27 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 28 Dec 2018 17:34:27 +0000
Subject: [njs] Interactive shell: immediate events support.
Message-ID: <hg.c4ec36309dda.1546018467.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/c4ec36309dda
branches:  
changeset: 706:c4ec36309dda
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Dec 28 19:36:24 2018 +0300
description:
Interactive shell: immediate events support.

This closes #66 issue on Github.

diffstat:

 njs/njs.c                    |    2 +-
 njs/njs_shell.c              |  331 ++++++++++++++++++++++++++++++++++++------
 njs/test/njs_expect_test.exp |   61 +++++++
 3 files changed, 341 insertions(+), 53 deletions(-)

diffs (622 lines):

diff -r 2866557dc7a0 -r c4ec36309dda njs/njs.c
--- a/njs/njs.c	Fri Dec 28 15:39:59 2018 +0300
+++ b/njs/njs.c	Fri Dec 28 19:36:24 2018 +0300
@@ -645,7 +645,7 @@ njs_vm_handle_events(njs_vm_t *vm)
         ev = nxt_queue_link_data(link, njs_event_t, link);
 
         if (ev->once) {
-            njs_del_event(vm, ev, NJS_EVENT_DELETE);
+            njs_del_event(vm, ev, NJS_EVENT_RELEASE | NJS_EVENT_DELETE);
 
         } else {
             ev->posted = 0;
diff -r 2866557dc7a0 -r c4ec36309dda njs/njs_shell.c
--- a/njs/njs_shell.c	Fri Dec 28 15:39:59 2018 +0300
+++ b/njs/njs_shell.c	Fri Dec 28 19:36:24 2018 +0300
@@ -20,13 +20,6 @@
 #include <readline.h>
 
 
-typedef enum {
-    NJS_COMPLETION_VAR = 0,
-    NJS_COMPLETION_SUFFIX,
-    NJS_COMPLETION_GLOBAL
-} njs_completion_phase_t;
-
-
 typedef struct {
     char                    *file;
     nxt_int_t               version;
@@ -40,22 +33,45 @@ typedef struct {
 typedef struct {
     size_t                  index;
     size_t                  length;
-    njs_vm_t                *vm;
     nxt_array_t             *completions;
     nxt_array_t             *suffix_completions;
     nxt_lvlhsh_each_t       lhe;
-    njs_completion_phase_t  phase;
+
+    enum {
+       NJS_COMPLETION_VAR = 0,
+       NJS_COMPLETION_SUFFIX,
+       NJS_COMPLETION_GLOBAL
+    }                       phase;
 } njs_completion_t;
 
 
+typedef struct {
+    njs_vm_event_t          vm_event;
+    nxt_queue_link_t        link;
+} njs_ev_t;
+
+
+typedef struct {
+    njs_vm_t                *vm;
+
+    nxt_lvlhsh_t            events;  /* njs_ev_t * */
+    nxt_queue_t             posted_events;
+
+    uint64_t                time;
+
+    njs_completion_t        completion;
+} njs_console_t;
+
+
 static nxt_int_t njs_get_options(njs_opts_t *opts, int argc, char **argv);
-static nxt_int_t njs_externals_init(njs_vm_t *vm);
+static nxt_int_t njs_console_init(njs_vm_t *vm, njs_console_t *console);
+static nxt_int_t njs_externals_init(njs_vm_t *vm, njs_console_t *console);
 static nxt_int_t njs_interactive_shell(njs_opts_t *opts,
     njs_vm_opt_t *vm_options);
 static nxt_int_t njs_process_file(njs_opts_t *opts, njs_vm_opt_t *vm_options);
-static nxt_int_t njs_process_script(njs_vm_t *vm, njs_opts_t *opts,
+static nxt_int_t njs_process_script(njs_console_t *console, njs_opts_t *opts,
     const nxt_str_t *script);
-static nxt_int_t njs_editline_init(njs_vm_t *vm);
+static nxt_int_t njs_editline_init(void);
 static char **njs_completion_handler(const char *text, int start, int end);
 static char *njs_completion_generator(const char *text, int state);
 
@@ -70,6 +86,15 @@ static njs_ret_t njs_ext_console_time(nj
 static njs_ret_t njs_ext_console_time_end(njs_vm_t *vm, njs_value_t *args,
     nxt_uint_t nargs, njs_index_t unused);
 
+static njs_host_event_t njs_console_set_timer(njs_external_ptr_t external,
+    uint64_t delay, njs_vm_event_t vm_event);
+static void njs_console_clear_timer(njs_external_ptr_t external,
+    njs_host_event_t event);
+
+static nxt_int_t lvlhsh_key_test(nxt_lvlhsh_query_t *lhq, void *data);
+static void *lvlhsh_pool_alloc(void *pool, size_t size, nxt_uint_t nalloc);
+static void lvlhsh_pool_free(void *pool, void *p, size_t size);
+
 
 static njs_external_t  njs_ext_console[] = {
 
@@ -150,10 +175,22 @@ static njs_external_t  njs_externals[] =
 };
 
 
-static njs_completion_t  njs_completion;
+static const nxt_lvlhsh_proto_t  lvlhsh_proto  nxt_aligned(64) = {
+    NXT_LVLHSH_LARGE_SLAB,
+    0,
+    lvlhsh_key_test,
+    lvlhsh_pool_alloc,
+    lvlhsh_pool_free,
+};
 
 
-static uint64_t njs_console_time = UINT64_MAX;
+static njs_vm_ops_t njs_console_ops = {
+    njs_console_set_timer,
+    njs_console_clear_timer
+};
+
+
+static njs_console_t  njs_console;
 
 
 int
@@ -182,6 +219,8 @@ main(int argc, char **argv)
     vm_options.accumulative = opts.interactive;
     vm_options.backtrace = 1;
     vm_options.sandbox = opts.sandbox;
+    vm_options.ops = &njs_console_ops;
+    vm_options.external = &njs_console;
 
     if (opts.interactive) {
         ret = njs_interactive_shell(&opts, &vm_options);
@@ -259,7 +298,26 @@ njs_get_options(njs_opts_t *opts, int ar
 
 
 static nxt_int_t
-njs_externals_init(njs_vm_t *vm)
+njs_console_init(njs_vm_t *vm, njs_console_t *console)
+{
+    console->vm = vm;
+
+    nxt_lvlhsh_init(&console->events);
+    nxt_queue_init(&console->posted_events);
+
+    console->time = UINT64_MAX;
+
+    console->completion.completions = njs_vm_completions(vm, NULL);
+    if (console->completion.completions == NULL) {
+        return NXT_ERROR;
+    }
+
+    return NXT_OK;
+}
+
+
+static nxt_int_t
+njs_externals_init(njs_vm_t *vm, njs_console_t *console)
 {
     nxt_uint_t          ret;
     njs_value_t         *value;
@@ -279,7 +337,7 @@ njs_externals_init(njs_vm_t *vm)
         return NXT_ERROR;
     }
 
-    ret = njs_vm_external_create(vm, value, proto, NULL);
+    ret = njs_vm_external_create(vm, value, proto, console);
     if (ret != NXT_OK) {
         return NXT_ERROR;
     }
@@ -289,6 +347,11 @@ njs_externals_init(njs_vm_t *vm)
         return NXT_ERROR;
     }
 
+    ret = njs_console_init(vm, console);
+    if (ret != NXT_OK) {
+        return NXT_ERROR;
+    }
+
     return NXT_OK;
 }
 
@@ -299,22 +362,22 @@ njs_interactive_shell(njs_opts_t *opts, 
     njs_vm_t   *vm;
     nxt_str_t  line;
 
+    if (njs_editline_init() != NXT_OK) {
+        fprintf(stderr, "failed to init completions\n");
+        return NXT_ERROR;
+    }
+
     vm = njs_vm_create(vm_options);
     if (vm == NULL) {
         fprintf(stderr, "failed to create vm\n");
         return NXT_ERROR;
     }
 
-    if (njs_externals_init(vm) != NXT_OK) {
+    if (njs_externals_init(vm, vm_options->external) != NXT_OK) {
         fprintf(stderr, "failed to add external protos\n");
         return NXT_ERROR;
     }
 
-    if (njs_editline_init(vm) != NXT_OK) {
-        fprintf(stderr, "failed to init completions\n");
-        return NXT_ERROR;
-    }
-
     if (!opts->quiet) {
         printf("interactive njs %s\n\n", NJS_VERSION);
 
@@ -335,7 +398,7 @@ njs_interactive_shell(njs_opts_t *opts, 
 
         add_history((char *) line.start);
 
-        njs_process_script(vm, opts, &line);
+        njs_process_script(vm_options->external, opts, &line);
 
         /* editline allocs a new buffer every time. */
         free(line.start);
@@ -439,14 +502,14 @@ njs_process_file(njs_opts_t *opts, njs_v
         goto done;
     }
 
-    ret = njs_externals_init(vm);
+    ret = njs_externals_init(vm, vm_options->external);
     if (ret != NXT_OK) {
         fprintf(stderr, "failed to add external protos\n");
         ret = NXT_ERROR;
         goto done;
     }
 
-    ret = njs_process_script(vm, opts, &script);
+    ret = njs_process_script(vm_options->external, opts, &script);
     if (ret != NXT_OK) {
         ret = NXT_ERROR;
         goto done;
@@ -490,11 +553,43 @@ njs_output(njs_vm_t *vm, njs_opts_t *opt
 
 
 static nxt_int_t
-njs_process_script(njs_vm_t *vm, njs_opts_t *opts, const nxt_str_t *script)
+njs_process_events(njs_console_t *console, njs_opts_t *opts)
+{
+    njs_ev_t          *ev;
+    nxt_queue_t       *events;
+    nxt_queue_link_t  *link;
+
+    events = &console->posted_events;
+
+    for ( ;; ) {
+        link = nxt_queue_first(events);
+
+        if (link == nxt_queue_tail(events)) {
+            break;
+        }
+
+        ev = nxt_queue_link_data(link, njs_ev_t, link);
+
+        nxt_queue_remove(&ev->link);
+        ev->link.prev = NULL;
+        ev->link.next = NULL;
+
+        njs_vm_post_event(console->vm, ev->vm_event, NULL, 0);
+    }
+
+    return NXT_OK;
+}
+
+
+static nxt_int_t
+njs_process_script(njs_console_t *console, njs_opts_t *opts,
+    const nxt_str_t *script)
 {
     u_char     *start;
+    njs_vm_t   *vm;
     nxt_int_t  ret;
 
+    vm = console->vm;
     start = script->start;
 
     ret = njs_vm_compile(vm, &start, start + script->length);
@@ -510,14 +605,31 @@ njs_process_script(njs_vm_t *vm, njs_opt
 
     njs_output(vm, opts, ret);
 
-    if (ret == NJS_OK) {
-        while (njs_vm_posted(vm)) {
-            ret = njs_vm_run(vm);
+    for ( ;; ) {
+        if (!njs_vm_pending(vm)) {
+            break;
+        }
+
+        ret = njs_process_events(console, opts);
+        if (nxt_slow_path(ret != NXT_OK)) {
+            fprintf(stderr, "njs_process_events() failed\n");
+            ret = NJS_ERROR;
+            break;
+        }
 
-            if (ret == NJS_ERROR) {
-                njs_output(vm, opts, ret);
-                return ret;
-            }
+        if (njs_vm_waiting(vm) && !njs_vm_posted(vm)) {
+            /*TODO: async events. */
+
+            fprintf(stderr, "njs_process_script(): "
+                    "async events unsupported\n");
+            ret = NJS_ERROR;
+            break;
+        }
+
+        ret = njs_vm_run(vm);
+
+        if (ret == NJS_ERROR) {
+            njs_output(vm, opts, ret);
         }
     }
 
@@ -526,7 +638,7 @@ njs_process_script(njs_vm_t *vm, njs_opt
 
 
 static nxt_int_t
-njs_editline_init(njs_vm_t *vm)
+njs_editline_init(void)
 {
     rl_completion_append_character = '\0';
     rl_attempted_completion_function = njs_completion_handler;
@@ -534,13 +646,6 @@ njs_editline_init(njs_vm_t *vm)
 
     setlocale(LC_ALL, "");
 
-    njs_completion.completions = njs_vm_completions(vm, NULL);
-    if (njs_completion.completions == NULL) {
-        return NXT_ERROR;
-    }
-
-    njs_completion.vm = vm;
-
     return NXT_OK;
 }
 
@@ -571,10 +676,12 @@ njs_completion_generator(const char *tex
     size_t            len;
     nxt_str_t         expression, *suffix;
     const char        *p;
+    njs_vm_t          *vm;
     njs_variable_t    *var;
     njs_completion_t  *cmpl;
 
-    cmpl = &njs_completion;
+    vm = njs_console.vm;
+    cmpl = &njs_console.completion;
 
     if (state == 0) {
         cmpl->phase = 0;
@@ -589,12 +696,12 @@ next:
 
     switch (cmpl->phase) {
     case NJS_COMPLETION_VAR:
-        if (cmpl->vm->parser == NULL) {
+        if (vm->parser == NULL) {
             njs_next_phase(cmpl);
         }
 
         for ( ;; ) {
-            var = nxt_lvlhsh_each(&cmpl->vm->parser->scope->variables,
+            var = nxt_lvlhsh_each(&vm->parser->scope->variables,
                                   &cmpl->lhe);
 
             if (var == NULL) {
@@ -630,8 +737,7 @@ next:
             expression.start = (u_char *) text;
             expression.length = p - text;
 
-            cmpl->suffix_completions = njs_vm_completions(cmpl->vm,
-                                                          &expression);
+            cmpl->suffix_completions = njs_vm_completions(vm, &expression);
             if (cmpl->suffix_completions == NULL) {
                 njs_next_phase(cmpl);
             }
@@ -797,14 +903,21 @@ static njs_ret_t
 njs_ext_console_time(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
     njs_index_t unused)
 {
+    njs_console_t  *console;
+
     if (!njs_value_is_void(njs_arg(args, nargs, 1))) {
         njs_vm_error(vm, "labels not implemented");
         return NJS_ERROR;
     }
 
-    vm->retval = njs_value_void;
+    console = njs_vm_external(vm, njs_arg(args, nargs, 0));
+    if (nxt_slow_path(console == NULL)) {
+        return NJS_ERROR;
+    }
 
-    njs_console_time = nxt_time();
+    console->time = nxt_time();
+
+    vm->retval = njs_value_void;
 
     return NJS_OK;
 }
@@ -814,7 +927,8 @@ static njs_ret_t
 njs_ext_console_time_end(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
     njs_index_t unused)
 {
-    uint64_t  ns, ms;
+    uint64_t       ns, ms;
+    njs_console_t  *console;
 
     ns = nxt_time();
 
@@ -823,16 +937,21 @@ njs_ext_console_time_end(njs_vm_t *vm, n
         return NJS_ERROR;
     }
 
-    if (nxt_fast_path(njs_console_time != UINT64_MAX)) {
+    console = njs_vm_external(vm, njs_arg(args, nargs, 0));
+    if (nxt_slow_path(console == NULL)) {
+        return NJS_ERROR;
+    }
 
-        ns = ns - njs_console_time;
+    if (nxt_fast_path(console->time != UINT64_MAX)) {
+
+        ns = ns - console->time;
 
         ms = ns / 1000000;
         ns = ns % 1000000;
 
         printf("default: %" PRIu64 ".%06" PRIu64 "ms\n", ms, ns);
 
-        njs_console_time = UINT64_MAX;
+        console->time = UINT64_MAX;
 
     } else {
         printf("Timer \"default\" doesn?t exist.\n");
@@ -842,3 +961,111 @@ njs_ext_console_time_end(njs_vm_t *vm, n
 
     return NJS_OK;
 }
+
+
+static njs_host_event_t
+njs_console_set_timer(njs_external_ptr_t external, uint64_t delay,
+    njs_vm_event_t vm_event)
+{
+    njs_ev_t            *ev;
+    njs_vm_t            *vm;
+    nxt_int_t           ret;
+    njs_console_t       *console;
+    nxt_lvlhsh_query_t  lhq;
+
+    if (delay != 0) {
+        fprintf(stderr, "njs_console_set_timer(): async timers unsupported\n");
+        return NULL;
+    }
+
+    console = external;
+    vm = console->vm;
+
+    ev = nxt_mem_cache_alloc(vm->mem_cache_pool, sizeof(njs_ev_t));
+    if (nxt_slow_path(ev == NULL)) {
+        return NULL;
+    }
+
+    ev->vm_event = vm_event;
+
+    lhq.key.start = (u_char *) &ev->vm_event;
+    lhq.key.length = sizeof(njs_vm_event_t);
+    lhq.key_hash = nxt_djb_hash(lhq.key.start, lhq.key.length);
+
+    lhq.replace = 0;
+    lhq.value = ev;
+    lhq.proto = &lvlhsh_proto;
+    lhq.pool = vm->mem_cache_pool;
+
+    ret = nxt_lvlhsh_insert(&console->events, &lhq);
+    if (nxt_slow_path(ret != NXT_OK)) {
+        return NULL;
+    }
+
+    nxt_queue_insert_tail(&console->posted_events, &ev->link);
+
+    return (njs_host_event_t) ev;
+}
+
+
+static void
+njs_console_clear_timer(njs_external_ptr_t external, njs_host_event_t event)
+{
+    njs_vm_t            *vm;
+    njs_ev_t            *ev;
+    nxt_int_t           ret;
+    njs_console_t       *console;
+    nxt_lvlhsh_query_t  lhq;
+
+    ev = event;
+    console = external;
+    vm = console->vm;
+
+    lhq.key.start = (u_char *) &ev->vm_event;
+    lhq.key.length = sizeof(njs_vm_event_t);
+    lhq.key_hash = nxt_djb_hash(lhq.key.start, lhq.key.length);
+
+    lhq.proto = &lvlhsh_proto;
+    lhq.pool = vm->mem_cache_pool;
+
+    if (ev->link.prev != NULL) {
+        nxt_queue_remove(&ev->link);
+    }
+
+    ret = nxt_lvlhsh_delete(&console->events, &lhq);
+    if (ret != NXT_OK) {
+        fprintf(stderr, "nxt_lvlhsh_delete() failed\n");
+    }
+
+    nxt_mem_cache_free(vm->mem_cache_pool, ev);
+}
+
+
+static nxt_int_t
+lvlhsh_key_test(nxt_lvlhsh_query_t *lhq, void *data)
+{
+    njs_ev_t  *ev;
+
+    ev = data;
+
+    if (memcmp(&ev->vm_event, lhq->key.start, sizeof(njs_vm_event_t)) == 0) {
+        return NXT_OK;
+    }
+
+    return NXT_DECLINED;
+}
+
+
+static void *
+lvlhsh_pool_alloc(void *pool, size_t size, nxt_uint_t nalloc)
+{
+    return nxt_mem_cache_align(pool, size, size);
+}
+
+
+static void
+lvlhsh_pool_free(void *pool, void *p, size_t size)
+{
+    nxt_mem_cache_free(pool, p);
+}
+
diff -r 2866557dc7a0 -r c4ec36309dda njs/test/njs_expect_test.exp
--- a/njs/test/njs_expect_test.exp	Fri Dec 28 15:39:59 2018 +0300
+++ b/njs/test/njs_expect_test.exp	Fri Dec 28 19:36:24 2018 +0300
@@ -292,6 +292,67 @@ njs_test {
      "'?'"}
 }
 
+# Immediate events
+
+njs_test {
+    {"var t = setImmediate(console.log, 'a', 'aa')\r\n"
+     "undefined\r\n'a' 'aa'"}
+}
+
+njs_test {
+    {"var a = 1 + 1; setTimeout(function (x) {a = x}, 0, 'a'); a\r\n"
+     "2"}
+    {"a\r\n"
+     "a\r\n'a'"}
+}
+
+njs_test {
+    {"setTimeout(function () {}, 1, 'a')\r\n"
+     "njs_console_set_timer(): async timers unsupported"}
+}
+
+njs_test {
+    {"var a = 1 + 1; setTimeout(function (x) { setTimeout(function (y) {a = y}, 0, x)}, 0, 'a'); a\r\n"
+     "2"}
+    {"a\r\n"
+     "a\r\n'a'"}
+}
+
+njs_test {
+    {"var a = 1 + 1; setImmediate(function (x) { setImmediate(function (y) {a = y}, x)}, 'a'); a\r\n"
+     "2"}
+    {"a\r\n"
+     "a\r\n'a'"}
+}
+
+njs_test {
+    {"var i = 0; (function x() { if (i < 10) setImmediate(x); i++; })()\r\n"
+     "undefined"}
+    {"i\r\n"
+     "i\r\n11"}
+}
+
+njs_test {
+    {"var a = 0, t = setImmediate(function() {a = 1}); clearTimeout(t)\r\n"
+     "undefined"}
+    {"a\r\n"
+     "a\r\n0"}
+}
+
+njs_test {
+    {"var i = 0; (function x() { if (i < 3) setImmediate(x); i++; throw 'Oops';})()\r\n"
+     "Oops"}
+    {"i\r\n"
+     "i\r\n4"}
+}
+
+njs_test {
+    {"var i = 0, queue = []; (function x() { if (i < 5) setImmediate(x); queue.push(i++); })()\r\n"
+     "undefined"}
+    {"queue.toString()\r\n"
+     "queue.toString()\r\n'0,1,2,3,4,5'"}
+}
+
 # require('fs')
 
 set file [open njs_test_file w]

From xeioex at nginx.com  Sat Dec 29 07:33:10 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Sat, 29 Dec 2018 07:33:10 +0000
Subject: [njs] Improved njs_generate_typeof_operation().
Message-ID: <hg.792dbba9b902.1546068790.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/792dbba9b902
branches:  
changeset: 707:792dbba9b902
user:      hongzhidao <hongzhidao at gmail.com>
date:      Sat Dec 29 01:43:44 2018 +0800
description:
Improved njs_generate_typeof_operation().

njs_variable_typeof() cannot fail.

diffstat:

 njs/njs_generator.c |  8 +-------
 1 files changed, 1 insertions(+), 7 deletions(-)

diffs (24 lines):

diff -r c4ec36309dda -r 792dbba9b902 njs/njs_generator.c
--- a/njs/njs_generator.c	Fri Dec 28 19:36:24 2018 +0300
+++ b/njs/njs_generator.c	Sat Dec 29 01:43:44 2018 +0800
@@ -2103,19 +2103,13 @@ njs_generate_typeof_operation(njs_vm_t *
     njs_parser_node_t *node)
 {
     nxt_int_t           ret;
-    njs_index_t         index;
     njs_parser_node_t   *expr;
     njs_vmcode_2addr_t  *code;
 
     expr = node->left;
 
     if (expr->token == NJS_TOKEN_NAME) {
-        index = njs_variable_typeof(vm, expr);
-        if (nxt_slow_path(index == NJS_INDEX_ERROR)) {
-            return NXT_ERROR;
-        }
-
-        expr->index = index;
+        expr->index = njs_variable_typeof(vm, expr);
 
     } else {
         ret = njs_generator(vm, generator, node->left);

From xeioex at nginx.com  Sat Dec 29 07:33:11 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Sat, 29 Dec 2018 07:33:11 +0000
Subject: [njs] Variables handling during code generation is improved.
Message-ID: <hg.9fb7bcaf07ba.1546068791.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/9fb7bcaf07ba
branches:  
changeset: 708:9fb7bcaf07ba
user:      hongzhidao <hongzhidao at gmail.com>
date:      Sat Dec 29 01:43:44 2018 +0800
description:
Variables handling during code generation is improved.

All the necessary fields related to a variable are moved into
njs_variable_reference_t.

diffstat:

 njs/njs_parser.c            |    2 +-
 njs/njs_parser.h            |    4 +-
 njs/njs_parser_expression.c |    2 +-
 njs/njs_variable.c          |  127 +++++++++++++++++++++----------------------
 njs/njs_variable.h          |   25 +++++--
 5 files changed, 83 insertions(+), 77 deletions(-)

diffs (382 lines):

diff -r 792dbba9b902 -r 9fb7bcaf07ba njs/njs_parser.c
--- a/njs/njs_parser.c	Sat Dec 29 01:43:44 2018 +0800
+++ b/njs/njs_parser.c	Sat Dec 29 01:43:44 2018 +0800
@@ -440,7 +440,7 @@ njs_parser_variable_add(njs_vm_t *vm, nj
 
 nxt_inline njs_ret_t
 njs_parser_variable_reference(njs_vm_t *vm, njs_parser_t *parser,
-    njs_parser_node_t *node, njs_variable_reference_t type)
+    njs_parser_node_t *node, njs_reference_type_t type)
 {
     return njs_variable_reference(vm, parser->scope, node, &parser->lexer->text,
                                   parser->lexer->key_hash, type);
diff -r 792dbba9b902 -r 9fb7bcaf07ba njs/njs_parser.h
--- a/njs/njs_parser.h	Sat Dec 29 01:43:44 2018 +0800
+++ b/njs/njs_parser.h	Sat Dec 29 01:43:44 2018 +0800
@@ -254,14 +254,12 @@ struct njs_parser_scope_s {
 struct njs_parser_node_s {
     njs_token_t                     token:16;
     uint8_t                         ctor:1;
-    njs_variable_reference_t        reference:2;
     uint8_t                         temporary;    /* 1 bit  */
     uint32_t                        token_line;
-    uint32_t                        variable_name_hash;
 
     union {
         uint32_t                    length;
-        nxt_str_t                   variable_name;
+        njs_variable_reference_t    reference;
         njs_value_t                 value;
         njs_vmcode_operation_t      operation;
         njs_parser_node_t           *object;
diff -r 792dbba9b902 -r 9fb7bcaf07ba njs/njs_parser_expression.c
--- a/njs/njs_parser_expression.c	Sat Dec 29 01:43:44 2018 +0800
+++ b/njs/njs_parser_expression.c	Sat Dec 29 01:43:44 2018 +0800
@@ -670,7 +670,7 @@ njs_parser_unary_expression(njs_vm_t *vm
     }
 
     if (token == NJS_TOKEN_TYPEOF && node->token == NJS_TOKEN_NAME) {
-        node->reference = NJS_TYPEOF;
+        node->u.reference.type = NJS_TYPEOF;
     }
 
     node = njs_parser_node_alloc(vm);
diff -r 792dbba9b902 -r 9fb7bcaf07ba njs/njs_variable.c
--- a/njs/njs_variable.c	Sat Dec 29 01:43:44 2018 +0800
+++ b/njs/njs_variable.c	Sat Dec 29 01:43:44 2018 +0800
@@ -9,15 +9,8 @@
 #include <string.h>
 
 
-typedef struct {
-    nxt_lvlhsh_query_t  lhq;
-    njs_variable_t      *variable;
-    njs_parser_scope_t  *scope;
-} njs_variable_scope_t;
-
-
 static njs_ret_t njs_variable_find(njs_vm_t *vm, njs_parser_scope_t *scope,
-    njs_variable_scope_t *vs, nxt_str_t *name, uint32_t hash);
+    njs_variable_reference_t *vr);
 static njs_variable_t *njs_variable_alloc(njs_vm_t *vm, nxt_str_t *name,
     njs_variable_type_t type);
 
@@ -106,7 +99,7 @@ njs_reference_hash_test(nxt_lvlhsh_query
 
     node = data;
 
-    if (nxt_strstr_eq(&lhq->key, &node->u.variable_name)) {
+    if (nxt_strstr_eq(&lhq->key, &node->u.reference.name)) {
         return NXT_OK;
     }
 
@@ -128,20 +121,23 @@ const nxt_lvlhsh_proto_t  njs_reference_
 njs_ret_t
 njs_variable_reference(njs_vm_t *vm, njs_parser_scope_t *scope,
     njs_parser_node_t *node, nxt_str_t *name, uint32_t hash,
-    njs_variable_reference_t reference)
+    njs_reference_type_t type)
 {
-    njs_ret_t           ret;
-    nxt_lvlhsh_query_t  lhq;
+    njs_ret_t                 ret;
+    nxt_lvlhsh_query_t        lhq;
+    njs_variable_reference_t  *vr;
 
-    ret = njs_name_copy(vm, &node->u.variable_name, name);
+    vr = &node->u.reference;
+
+    ret = njs_name_copy(vm, &vr->name, name);
 
     if (nxt_fast_path(ret == NXT_OK)) {
-        node->variable_name_hash = hash;
         node->scope = scope;
-        node->reference = reference;
+        vr->hash = hash;
+        vr->type = type;
 
-        lhq.key_hash = node->variable_name_hash;
-        lhq.key = node->u.variable_name;
+        lhq.key_hash = hash;
+        lhq.key = vr->name;
         lhq.proto = &njs_reference_hash_proto;
         lhq.replace = 0;
         lhq.value = node;
@@ -162,13 +158,13 @@ static njs_ret_t
 njs_variables_scope_resolve(njs_vm_t *vm, njs_parser_scope_t *scope,
     nxt_bool_t closure)
 {
-    njs_ret_t             ret;
-    nxt_queue_t           *nested;
-    njs_variable_t        *var;
-    nxt_queue_link_t      *lnk;
-    njs_parser_node_t     *node;
-    nxt_lvlhsh_each_t     lhe;
-    njs_variable_scope_t  vs;
+    njs_ret_t                 ret;
+    nxt_queue_t               *nested;
+    njs_variable_t            *var;
+    nxt_queue_link_t          *lnk;
+    njs_parser_node_t         *node;
+    nxt_lvlhsh_each_t         lhe;
+    njs_variable_reference_t  *vr;
 
     nested = &scope->nested;
 
@@ -192,19 +188,19 @@ njs_variables_scope_resolve(njs_vm_t *vm
                 break;
             }
 
+            vr = &node->u.reference;
+
             if (closure) {
-                ret = njs_variable_find(vm, node->scope, &vs,
-                                        &node->u.variable_name,
-                                        node->variable_name_hash);
+                ret = njs_variable_find(vm, node->scope, vr);
                 if (nxt_slow_path(ret != NXT_OK)) {
                     continue;
                 }
 
-                if (vs.scope->type == NJS_SCOPE_GLOBAL) {
+                if (vr->scope->type == NJS_SCOPE_GLOBAL) {
                     continue;
                 }
 
-                if (node->scope->nesting == vs.scope->nesting) {
+                if (node->scope->nesting == vr->scope->nesting) {
                     /*
                      * A variable is referenced locally here, but may be
                      * referenced non-locally in other places, skipping.
@@ -216,7 +212,7 @@ njs_variables_scope_resolve(njs_vm_t *vm
             var = njs_variable_get(vm, node);
 
             if (nxt_slow_path(var == NULL)) {
-                if (node->reference != NJS_TYPEOF) {
+                if (vr->type != NJS_TYPEOF) {
                     return NXT_ERROR;
                 }
             }
@@ -255,18 +251,19 @@ njs_variables_scope_reference(njs_vm_t *
 njs_index_t
 njs_variable_typeof(njs_vm_t *vm, njs_parser_node_t *node)
 {
-    nxt_int_t             ret;
-    njs_variable_scope_t  vs;
+    nxt_int_t                 ret;
+    njs_variable_reference_t  *vr;
 
     if (node->index != NJS_INDEX_NONE) {
         return node->index;
     }
 
-    ret = njs_variable_find(vm, node->scope, &vs, &node->u.variable_name,
-                            node->variable_name_hash);
+    vr = &node->u.reference;
+
+    ret = njs_variable_find(vm, node->scope, vr);
 
     if (nxt_fast_path(ret == NXT_OK)) {
-        return vs.variable->index;
+        return vr->variable->index;
     }
 
     return NJS_INDEX_NONE;
@@ -295,16 +292,17 @@ njs_variable_index(njs_vm_t *vm, njs_par
 njs_variable_t *
 njs_variable_get(njs_vm_t *vm, njs_parser_node_t *node)
 {
-    nxt_int_t             ret;
-    nxt_uint_t            scope_index;
-    nxt_array_t           *values;
-    njs_index_t           index;
-    njs_value_t           *value;
-    njs_variable_t        *var;
-    njs_variable_scope_t  vs;
+    nxt_int_t                 ret;
+    nxt_uint_t                scope_index;
+    nxt_array_t               *values;
+    njs_index_t               index;
+    njs_value_t               *value;
+    njs_variable_t            *var;
+    njs_variable_reference_t  *vr;
 
-    ret = njs_variable_find(vm, node->scope, &vs, &node->u.variable_name,
-                            node->variable_name_hash);
+    vr = &node->u.reference;
+
+    ret = njs_variable_find(vm, node->scope, vr);
 
     if (nxt_slow_path(ret != NXT_OK)) {
         goto not_found;
@@ -312,11 +310,11 @@ njs_variable_get(njs_vm_t *vm, njs_parse
 
     scope_index = 0;
 
-    if (vs.scope->type > NJS_SCOPE_GLOBAL) {
-        scope_index = (node->scope->nesting != vs.scope->nesting);
+    if (vr->scope->type > NJS_SCOPE_GLOBAL) {
+        scope_index = (node->scope->nesting != vr->scope->nesting);
     }
 
-    var = vs.variable;
+    var = vr->variable;
     index = var->index;
 
     if (index != NJS_INDEX_NONE) {
@@ -327,10 +325,10 @@ njs_variable_get(njs_vm_t *vm, njs_parse
             return var;
         }
 
-        vs.scope->argument_closures++;
+        vr->scope->argument_closures++;
         index = (index >> NJS_SCOPE_SHIFT) + 1;
 
-        if (index > 255 || vs.scope->argument_closures == 0) {
+        if (index > 255 || vr->scope->argument_closures == 0) {
             njs_internal_error(vm, "too many argument closures");
 
             return NULL;
@@ -339,11 +337,11 @@ njs_variable_get(njs_vm_t *vm, njs_parse
         var->argument = index;
     }
 
-    if (node->reference != NJS_DECLARATION && var->type <= NJS_VARIABLE_LET) {
+    if (vr->type != NJS_DECLARATION && var->type <= NJS_VARIABLE_LET) {
         goto not_found;
     }
 
-    if (vm->options.accumulative && vs.scope->type == NJS_SCOPE_GLOBAL) {
+    if (vm->options.accumulative && vr->scope->type == NJS_SCOPE_GLOBAL) {
         /*
          * When non-clonable VM runs in accumulative mode all
          * global variables should be allocated in absolute scope
@@ -359,7 +357,7 @@ njs_variable_get(njs_vm_t *vm, njs_parse
         index = (njs_index_t) value;
 
     } else {
-        values = vs.scope->values[scope_index];
+        values = vr->scope->values[scope_index];
 
         if (values == NULL) {
             values = nxt_array_create(4, sizeof(njs_value_t),
@@ -368,7 +366,7 @@ njs_variable_get(njs_vm_t *vm, njs_parse
                 return NULL;
             }
 
-            vs.scope->values[scope_index] = values;
+            vr->scope->values[scope_index] = values;
         }
 
         value = nxt_array_add(values, &njs_array_mem_proto, vm->mem_cache_pool);
@@ -376,8 +374,8 @@ njs_variable_get(njs_vm_t *vm, njs_parse
             return NULL;
         }
 
-        index = vs.scope->next_index[scope_index];
-        vs.scope->next_index[scope_index] += sizeof(njs_value_t);
+        index = vr->scope->next_index[scope_index];
+        vr->scope->next_index[scope_index] += sizeof(njs_value_t);
     }
 
     if (njs_is_object(&var->value)) {
@@ -395,7 +393,7 @@ njs_variable_get(njs_vm_t *vm, njs_parse
 not_found:
 
     njs_parser_ref_error(vm, vm->parser, "\"%.*s\" is not defined",
-                         (int) vs.lhq.key.length, vs.lhq.key.start);
+                         (int) vr->name.length, vr->name.start);
 
     return NULL;
 }
@@ -403,19 +401,20 @@ not_found:
 
 static njs_ret_t
 njs_variable_find(njs_vm_t *vm, njs_parser_scope_t *scope,
-    njs_variable_scope_t *vs, nxt_str_t *name, uint32_t hash)
+    njs_variable_reference_t *vr)
 {
+    nxt_lvlhsh_query_t  lhq;
     njs_parser_scope_t  *parent, *previous;
 
-    vs->lhq.key_hash = hash;
-    vs->lhq.key = *name;
-    vs->lhq.proto = &njs_variables_hash_proto;
+    lhq.key_hash = vr->hash;
+    lhq.key = vr->name;
+    lhq.proto = &njs_variables_hash_proto;
 
     previous = NULL;
 
     for ( ;; ) {
-        if (nxt_lvlhsh_find(&scope->variables, &vs->lhq) == NXT_OK) {
-            vs->variable = vs->lhq.value;
+        if (nxt_lvlhsh_find(&scope->variables, &lhq) == NXT_OK) {
+            vr->variable = lhq.value;
 
             if (scope->type == NJS_SCOPE_SHIM) {
                 scope = previous;
@@ -430,7 +429,7 @@ njs_variable_find(njs_vm_t *vm, njs_pars
                 }
             }
 
-            vs->scope = scope;
+            vr->scope = scope;
 
             return NXT_OK;
         }
@@ -439,7 +438,7 @@ njs_variable_find(njs_vm_t *vm, njs_pars
 
         if (parent == NULL) {
             /* A global scope. */
-            vs->scope = scope;
+            vr->scope = scope;
 
             return NXT_DECLINED;
         }
diff -r 792dbba9b902 -r 9fb7bcaf07ba njs/njs_variable.h
--- a/njs/njs_variable.h	Sat Dec 29 01:43:44 2018 +0800
+++ b/njs/njs_variable.h	Sat Dec 29 01:43:44 2018 +0800
@@ -18,13 +18,6 @@ typedef enum {
 } njs_variable_type_t;
 
 
-typedef enum {
-    NJS_DECLARATION = 0,
-    NJS_REFERENCE,
-    NJS_TYPEOF,
-} njs_variable_reference_t;
-
-
 typedef struct {
     nxt_str_t             name;
 
@@ -42,11 +35,27 @@ typedef struct {
          + njs_scope_offset((var)->index) - NJS_INDEX_GLOBAL_OFFSET)
 
 
+typedef enum {
+    NJS_DECLARATION = 0,
+    NJS_REFERENCE,
+    NJS_TYPEOF,
+} njs_reference_type_t;
+
+
+typedef struct {
+    njs_reference_type_t  type;
+    uint32_t              hash;
+    nxt_str_t             name;
+    njs_variable_t        *variable;
+    njs_parser_scope_t    *scope;
+} njs_variable_reference_t;
+
+
 njs_variable_t *njs_variable_add(njs_vm_t *vm, njs_parser_scope_t *scope,
     nxt_str_t *name, uint32_t hash, njs_variable_type_t type);
 njs_ret_t njs_variable_reference(njs_vm_t *vm, njs_parser_scope_t *scope,
     njs_parser_node_t *node, nxt_str_t *name, uint32_t hash,
-    njs_variable_reference_t reference);
+    njs_reference_type_t type);
 njs_ret_t njs_variables_scope_reference(njs_vm_t *vm,
     njs_parser_scope_t *scope);
 njs_ret_t njs_name_copy(njs_vm_t *vm, nxt_str_t *dst, nxt_str_t *src);

From xeioex at nginx.com  Sat Dec 29 07:33:11 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Sat, 29 Dec 2018 07:33:11 +0000
Subject: [njs] Removed duplicate njs_variable_reference() declaration.
Message-ID: <hg.9fc5ed136eb6.1546068791.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/9fc5ed136eb6
branches:  
changeset: 709:9fc5ed136eb6
user:      hongzhidao <hongzhidao at gmail.com>
date:      Sat Dec 29 01:43:44 2018 +0800
description:
Removed duplicate njs_variable_reference() declaration.

diffstat:

 njs/njs_parser.h |  3 ---
 1 files changed, 0 insertions(+), 3 deletions(-)

diffs (13 lines):

diff -r 9fb7bcaf07ba -r 9fc5ed136eb6 njs/njs_parser.h
--- a/njs/njs_parser.h	Sat Dec 29 01:43:44 2018 +0800
+++ b/njs/njs_parser.h	Sat Dec 29 01:43:44 2018 +0800
@@ -325,9 +325,6 @@ njs_token_t njs_parser_property_name(njs
 njs_token_t njs_parser_property_token(njs_parser_t *parser);
 njs_token_t njs_parser_token(njs_parser_t *parser);
 nxt_int_t njs_parser_string_create(njs_vm_t *vm, njs_value_t *value);
-njs_ret_t njs_variable_reference(njs_vm_t *vm, njs_parser_scope_t *scope,
-    njs_parser_node_t *node, nxt_str_t *name, uint32_t hash,
-    njs_variable_reference_t reference);
 njs_variable_t *njs_variable_get(njs_vm_t *vm, njs_parser_node_t *node);
 njs_index_t njs_variable_typeof(njs_vm_t *vm, njs_parser_node_t *node);
 njs_index_t njs_variable_index(njs_vm_t *vm, njs_parser_node_t *node);

From xeioex at nginx.com  Sat Dec 29 12:17:12 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Sat, 29 Dec 2018 12:17:12 +0000
Subject: [njs] Removed dead code in njs_set_timer().
Message-ID: <hg.157bbb5d50eb.1546085832.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/157bbb5d50eb
branches:  
changeset: 710:157bbb5d50eb
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Sat Dec 29 15:16:59 2018 +0300
description:
Removed dead code in njs_set_timer().

Found by Coverity (CID 1442096).

diffstat:

 njs/njs_time.c |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff -r 9fc5ed136eb6 -r 157bbb5d50eb njs/njs_time.c
--- a/njs/njs_time.c	Sat Dec 29 01:43:44 2018 +0800
+++ b/njs/njs_time.c	Sat Dec 29 15:16:59 2018 +0300
@@ -48,7 +48,7 @@ njs_set_timer(njs_vm_t *vm, njs_value_t 
 
     n = immediate ? 2 : 3;
 
-    event->destructor = (ops != NULL) ? ops->clear_timer : NULL;
+    event->destructor = ops->clear_timer;
     event->function = args[1].data.u.function;
     event->nargs = (nargs >= n) ? nargs - n : 0;
     event->once = 1;

From xeioex at nginx.com  Sun Dec 30 11:13:30 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Sun, 30 Dec 2018 11:13:30 +0000
Subject: [njs] Improved working with scope indexes of variables.
Message-ID: <hg.5cf68fb9152e.1546168410.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/5cf68fb9152e
branches:  
changeset: 711:5cf68fb9152e
user:      hongzhidao <hongzhidao at gmail.com>
date:      Sat Dec 29 16:03:57 2018 +0800
description:
Improved working with scope indexes of variables.

diffstat:

 njs/njs_parser.h   |   7 +++----
 njs/njs_variable.c |  35 +++++++++++++++++------------------
 njs/njs_variable.h |   1 +
 3 files changed, 21 insertions(+), 22 deletions(-)

diffs (115 lines):

diff -r 157bbb5d50eb -r 5cf68fb9152e njs/njs_parser.h
--- a/njs/njs_parser.h	Sat Dec 29 15:16:59 2018 +0300
+++ b/njs/njs_parser.h	Sat Dec 29 16:03:57 2018 +0800
@@ -238,10 +238,9 @@ struct njs_parser_scope_s {
     nxt_lvlhsh_t                    variables;
     nxt_lvlhsh_t                    references;
 
-    /*
-     * 0: local scope index;
-     * 1: closure scope index.
-     */
+#define NJS_SCOPE_INDEX_LOCAL       0
+#define NJS_SCOPE_INDEX_CLOSURE     1
+
     nxt_array_t                     *values[2];  /* Array of njs_value_t. */
     njs_index_t                     next_index[2];
 
diff -r 157bbb5d50eb -r 5cf68fb9152e njs/njs_variable.c
--- a/njs/njs_variable.c	Sat Dec 29 15:16:59 2018 +0300
+++ b/njs/njs_variable.c	Sat Dec 29 16:03:57 2018 +0800
@@ -9,7 +9,7 @@
 #include <string.h>
 
 
-static njs_ret_t njs_variable_find(njs_vm_t *vm, njs_parser_scope_t *scope,
+static njs_ret_t njs_variable_find(njs_vm_t *vm, njs_parser_scope_t *node_scope,
     njs_variable_reference_t *vr);
 static njs_variable_t *njs_variable_alloc(njs_vm_t *vm, nxt_str_t *name,
     njs_variable_type_t type);
@@ -196,15 +196,7 @@ njs_variables_scope_resolve(njs_vm_t *vm
                     continue;
                 }
 
-                if (vr->scope->type == NJS_SCOPE_GLOBAL) {
-                    continue;
-                }
-
-                if (node->scope->nesting == vr->scope->nesting) {
-                    /*
-                     * A variable is referenced locally here, but may be
-                     * referenced non-locally in other places, skipping.
-                     */
+                if (vr->scope_index == NJS_SCOPE_INDEX_LOCAL) {
                     continue;
                 }
             }
@@ -308,18 +300,16 @@ njs_variable_get(njs_vm_t *vm, njs_parse
         goto not_found;
     }
 
-    scope_index = 0;
-
-    if (vr->scope->type > NJS_SCOPE_GLOBAL) {
-        scope_index = (node->scope->nesting != vr->scope->nesting);
-    }
+    scope_index = vr->scope_index;
 
     var = vr->variable;
     index = var->index;
 
     if (index != NJS_INDEX_NONE) {
 
-        if (scope_index == 0 || njs_scope_type(index) != NJS_SCOPE_ARGUMENTS) {
+        if (scope_index == NJS_SCOPE_INDEX_LOCAL
+            || njs_scope_type(index) != NJS_SCOPE_ARGUMENTS)
+        {
             node->index = index;
 
             return var;
@@ -400,16 +390,17 @@ not_found:
 
 
 static njs_ret_t
-njs_variable_find(njs_vm_t *vm, njs_parser_scope_t *scope,
+njs_variable_find(njs_vm_t *vm, njs_parser_scope_t *node_scope,
     njs_variable_reference_t *vr)
 {
     nxt_lvlhsh_query_t  lhq;
-    njs_parser_scope_t  *parent, *previous;
+    njs_parser_scope_t  *scope, *parent, *previous;
 
     lhq.key_hash = vr->hash;
     lhq.key = vr->name;
     lhq.proto = &njs_variables_hash_proto;
 
+    scope = node_scope;
     previous = NULL;
 
     for ( ;; ) {
@@ -431,6 +422,14 @@ njs_variable_find(njs_vm_t *vm, njs_pars
 
             vr->scope = scope;
 
+            vr->scope_index = NJS_SCOPE_INDEX_LOCAL;
+
+            if (vr->scope->type > NJS_SCOPE_GLOBAL
+                && node_scope->nesting != vr->scope->nesting)
+            {
+                vr->scope_index = NJS_SCOPE_INDEX_CLOSURE;
+            }
+
             return NXT_OK;
         }
 
diff -r 157bbb5d50eb -r 5cf68fb9152e njs/njs_variable.h
--- a/njs/njs_variable.h	Sat Dec 29 15:16:59 2018 +0300
+++ b/njs/njs_variable.h	Sat Dec 29 16:03:57 2018 +0800
@@ -48,6 +48,7 @@ typedef struct {
     nxt_str_t             name;
     njs_variable_t        *variable;
     njs_parser_scope_t    *scope;
+    nxt_uint_t            scope_index;  /* NJS_SCOPE_INDEX_... */
 } njs_variable_reference_t;
 
 

From xeioex at nginx.com  Sun Dec 30 11:13:30 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Sun, 30 Dec 2018 11:13:30 +0000
Subject: [njs] Improved function naming in variable code.
Message-ID: <hg.74776ba6a1a2.1546168410.5965299922797593991@dev.nginx.com>

details:   https://hg.nginx.org/njs/rev/74776ba6a1a2
branches:  
changeset: 712:74776ba6a1a2
user:      hongzhidao <hongzhidao at gmail.com>
date:      Sat Dec 29 22:35:31 2018 +0800
description:
Improved function naming in variable code.

Renaming:
1) njs_variable_get() to njs_variable_resolve().
2) njs_variable_find() to njs_variable_reference_resolve().

To better reflect what they do.

diffstat:

 njs/njs_generator.c |   4 ++--
 njs/njs_parser.h    |   2 +-
 njs/njs_variable.c  |  20 ++++++++++----------
 3 files changed, 13 insertions(+), 13 deletions(-)

diffs (112 lines):

diff -r 5cf68fb9152e -r 74776ba6a1a2 njs/njs_generator.c
--- a/njs/njs_generator.c	Sat Dec 29 16:03:57 2018 +0800
+++ b/njs/njs_generator.c	Sat Dec 29 22:35:31 2018 +0800
@@ -488,7 +488,7 @@ njs_generate_name(njs_vm_t *vm, njs_gene
     njs_variable_t            *var;
     njs_vmcode_object_copy_t  *copy;
 
-    var = njs_variable_get(vm, node);
+    var = njs_variable_resolve(vm, node);
     if (nxt_slow_path(var == NULL)) {
         return NXT_ERROR;
     }
@@ -2257,7 +2257,7 @@ njs_generate_function_declaration(njs_vm
     njs_variable_t         *var;
     njs_function_lambda_t  *lambda;
 
-    var = njs_variable_get(vm, node);
+    var = njs_variable_resolve(vm, node);
     if (nxt_slow_path(var == NULL)) {
         return NXT_ERROR;
     }
diff -r 5cf68fb9152e -r 74776ba6a1a2 njs/njs_parser.h
--- a/njs/njs_parser.h	Sat Dec 29 16:03:57 2018 +0800
+++ b/njs/njs_parser.h	Sat Dec 29 22:35:31 2018 +0800
@@ -324,7 +324,7 @@ njs_token_t njs_parser_property_name(njs
 njs_token_t njs_parser_property_token(njs_parser_t *parser);
 njs_token_t njs_parser_token(njs_parser_t *parser);
 nxt_int_t njs_parser_string_create(njs_vm_t *vm, njs_value_t *value);
-njs_variable_t *njs_variable_get(njs_vm_t *vm, njs_parser_node_t *node);
+njs_variable_t *njs_variable_resolve(njs_vm_t *vm, njs_parser_node_t *node);
 njs_index_t njs_variable_typeof(njs_vm_t *vm, njs_parser_node_t *node);
 njs_index_t njs_variable_index(njs_vm_t *vm, njs_parser_node_t *node);
 nxt_bool_t njs_parser_has_side_effect(njs_parser_node_t *node);
diff -r 5cf68fb9152e -r 74776ba6a1a2 njs/njs_variable.c
--- a/njs/njs_variable.c	Sat Dec 29 16:03:57 2018 +0800
+++ b/njs/njs_variable.c	Sat Dec 29 22:35:31 2018 +0800
@@ -9,8 +9,8 @@
 #include <string.h>
 
 
-static njs_ret_t njs_variable_find(njs_vm_t *vm, njs_parser_scope_t *node_scope,
-    njs_variable_reference_t *vr);
+static njs_ret_t njs_variable_reference_resolve(njs_vm_t *vm,
+    njs_variable_reference_t *vr, njs_parser_scope_t *node_scope);
 static njs_variable_t *njs_variable_alloc(njs_vm_t *vm, nxt_str_t *name,
     njs_variable_type_t type);
 
@@ -191,7 +191,7 @@ njs_variables_scope_resolve(njs_vm_t *vm
             vr = &node->u.reference;
 
             if (closure) {
-                ret = njs_variable_find(vm, node->scope, vr);
+                ret = njs_variable_reference_resolve(vm, vr, node->scope);
                 if (nxt_slow_path(ret != NXT_OK)) {
                     continue;
                 }
@@ -201,7 +201,7 @@ njs_variables_scope_resolve(njs_vm_t *vm
                 }
             }
 
-            var = njs_variable_get(vm, node);
+            var = njs_variable_resolve(vm, node);
 
             if (nxt_slow_path(var == NULL)) {
                 if (vr->type != NJS_TYPEOF) {
@@ -252,7 +252,7 @@ njs_variable_typeof(njs_vm_t *vm, njs_pa
 
     vr = &node->u.reference;
 
-    ret = njs_variable_find(vm, node->scope, vr);
+    ret = njs_variable_reference_resolve(vm, vr, node->scope);
 
     if (nxt_fast_path(ret == NXT_OK)) {
         return vr->variable->index;
@@ -271,7 +271,7 @@ njs_variable_index(njs_vm_t *vm, njs_par
         return node->index;
     }
 
-    var = njs_variable_get(vm, node);
+    var = njs_variable_resolve(vm, node);
 
     if (nxt_fast_path(var != NULL)) {
         return var->index;
@@ -282,7 +282,7 @@ njs_variable_index(njs_vm_t *vm, njs_par
 
 
 njs_variable_t *
-njs_variable_get(njs_vm_t *vm, njs_parser_node_t *node)
+njs_variable_resolve(njs_vm_t *vm, njs_parser_node_t *node)
 {
     nxt_int_t                 ret;
     nxt_uint_t                scope_index;
@@ -294,7 +294,7 @@ njs_variable_get(njs_vm_t *vm, njs_parse
 
     vr = &node->u.reference;
 
-    ret = njs_variable_find(vm, node->scope, vr);
+    ret = njs_variable_reference_resolve(vm, vr, node->scope);
 
     if (nxt_slow_path(ret != NXT_OK)) {
         goto not_found;
@@ -390,8 +390,8 @@ not_found:
 
 
 static njs_ret_t
-njs_variable_find(njs_vm_t *vm, njs_parser_scope_t *node_scope,
-    njs_variable_reference_t *vr)
+njs_variable_reference_resolve(njs_vm_t *vm, njs_variable_reference_t *vr,
+    njs_parser_scope_t *node_scope)
 {
     nxt_lvlhsh_query_t  lhq;
     njs_parser_scope_t  *scope, *parent, *previous;