From gmm at csdoc.com  Thu Feb  1 09:19:01 2018
From: gmm at csdoc.com (Gena Makhomed)
Date: Thu, 1 Feb 2018 11:19:01 +0200
Subject: [PATCH 1 of 2] Contrib: vim syntax, update core module directives.
Message-ID: <d5143118-5013-c7dd-373c-b2440b7fa054@csdoc.com>

# HG changeset patch
# User Gena Makhomed <gmm at csdoc.com>
# Date 1517476175 -7200
#      Thu Feb 01 11:09:35 2018 +0200
# Node ID 6db472f8dc8ec04754a42ba22e9663cb7e91f2ab
# Parent  0b72d545f098b407351a1972fba5d29f2e904507
Contrib: vim syntax, update core module directives.

"match" is block directive, "upstream_conf" is deprecated
by patch http://hg.nginx.org/nginx.org/rev/27c53e1cb4b6

diff -r 0b72d545f098 -r 6db472f8dc8e contrib/vim/syntax/nginx.vim
--- a/contrib/vim/syntax/nginx.vim      Tue Jan 30 22:23:58 2018 +0300
+++ b/contrib/vim/syntax/nginx.vim      Thu Feb 01 11:09:35 2018 +0200
@@ -87,6 +87,7 @@
  syn keyword ngxDirectiveBlock contained geo
  syn keyword ngxDirectiveBlock contained map
  syn keyword ngxDirectiveBlock contained split_clients
+syn keyword ngxDirectiveBlock contained match

  syn keyword ngxDirectiveImportant contained include
  syn keyword ngxDirectiveImportant contained root
@@ -117,6 +118,7 @@
  syn keyword ngxDirectiveDeprecated contained spdy_recv_buffer_size
  syn keyword ngxDirectiveDeprecated contained spdy_recv_timeout
  syn keyword ngxDirectiveDeprecated contained spdy_streams_index_size
+syn keyword ngxDirectiveDeprecated contained upstream_conf

  syn keyword ngxDirective contained absolute_redirect
  syn keyword ngxDirective contained accept_mutex
@@ -324,7 +326,6 @@
  syn keyword ngxDirective contained log_subrequest
  syn keyword ngxDirective contained map_hash_bucket_size
  syn keyword ngxDirective contained map_hash_max_size
-syn keyword ngxDirective contained match
  syn keyword ngxDirective contained master_process
  syn keyword ngxDirective contained max_ranges
  syn keyword ngxDirective contained memcached_bind
@@ -582,7 +583,6 @@
  syn keyword ngxDirective contained types_hash_max_size
  syn keyword ngxDirective contained underscores_in_headers
  syn keyword ngxDirective contained uninitialized_variable_warn
-syn keyword ngxDirective contained upstream_conf
  syn keyword ngxDirective contained use
  syn keyword ngxDirective contained user
  syn keyword ngxDirective contained userid


From gmm at csdoc.com  Thu Feb  1 09:20:11 2018
From: gmm at csdoc.com (Gena Makhomed)
Date: Thu, 1 Feb 2018 11:20:11 +0200
Subject: [PATCH 2 of 2] Contrib: vim syntax,
 update 3rd party module directives.
Message-ID: <9d244deb-2903-5dd3-ae7e-0be674a18fb3@csdoc.com>

# HG changeset patch
# User Gena Makhomed <gmm at csdoc.com>
# Date 1517476514 -7200
#      Thu Feb 01 11:15:14 2018 +0200
# Node ID 2e0de79805eee669cca7e2455010689a3191f268
# Parent  6db472f8dc8ec04754a42ba22e9663cb7e91f2ab
Contrib: vim syntax, update 3rd party module directives.

Add new directives for 3rd party modules.

diff -r 6db472f8dc8e -r 2e0de79805ee contrib/vim/syntax/nginx.vim
--- a/contrib/vim/syntax/nginx.vim      Thu Feb 01 11:09:35 2018 +0200
+++ b/contrib/vim/syntax/nginx.vim      Thu Feb 01 11:15:14 2018 +0200
@@ -954,6 +954,7 @@
  syn keyword ngxDirectiveThirdParty contained nchan_redis_server
  syn keyword ngxDirectiveThirdParty contained nchan_redis_storage_mode
  syn keyword ngxDirectiveThirdParty contained nchan_redis_url
+syn keyword ngxDirectiveThirdParty contained 
nchan_redis_wait_after_connecting
  syn keyword ngxDirectiveThirdParty contained nchan_shared_memory_size
  syn keyword ngxDirectiveThirdParty contained nchan_storage_engine
  syn keyword ngxDirectiveThirdParty contained nchan_store_messages
@@ -1080,6 +1081,8 @@
  syn keyword ngxDirectiveThirdParty contained tnt_select_limit_max
  syn keyword ngxDirectiveThirdParty contained tnt_send_timeout
  syn keyword ngxDirectiveThirdParty contained tnt_set_header
+syn keyword ngxDirectiveThirdParty contained tnt_update
+syn keyword ngxDirectiveThirdParty contained tnt_upsert

  " A module for nginx web server for handling file uploads using 
multipart/form-data encoding (RFC 1867)
  " https://github.com/Austinb/nginx-upload-module
@@ -2106,8 +2109,10 @@

  " Embed the power of Lua into NGINX TCP/UDP servers
  " https://github.com/openresty/stream-lua-nginx-module
+syn keyword ngxDirectiveThirdParty contained lua_add_variable
  syn keyword ngxDirectiveThirdParty contained preread_by_lua_block
  syn keyword ngxDirectiveThirdParty contained preread_by_lua_file
+syn keyword ngxDirectiveThirdParty contained preread_by_lua_no_postpone

  " nginx-upsync-module
  " https://github.com/weibocom/nginx-upsync-module


From imhongxiaolong at gmail.com  Thu Feb  1 12:34:11 2018
From: imhongxiaolong at gmail.com (Xiaolong Hong)
Date: Thu, 1 Feb 2018 20:34:11 +0800
Subject: Fixed upstream->read timer when downstream->write not ready
In-Reply-To: <20180122163415.GE34136@mdounin.ru>
References: <CAJcM5sy0dR_f6BzBMvA_+bmiX-rf7VOe9a=OjHCo5Jxfqr59jg@mail.gmail.com>
 <20180122163415.GE34136@mdounin.ru>
Message-ID: <CAJcM5szspaW1NEw=P1SAdk=Ua8=XHhq2ENy4vc2nSz1aO0_7pw@mail.gmail.com>

> Hello!
> Thank you for the patch.
>
> The problem looks valid - indeed, if in non-buffered proxy mode
> writing to the client blocked, it is possible that we'll continue
> reading from upstream after the response was fully received, and
> will keep upstream read timer set.  This in turn can result in
> proxy_read_timeout being triggered if it is smaller than send_timeout.

Yes, we were plagued with this problem because we needed to make a
distinction between upstream timeout and client timeout in our production
system.

> I can't say I like the suggested change though.  The resulting
> conditions look overcomplicated.  Also, likely we have similar
> problem in ngx_http_upstream_process_upgraded(), and it probably
> needs fixing too.

> Also, suggested condition is incorrect, as it will keep timer if
> upstream->read->eof is set, but u->length isn't -1, or if
> upstream->read->error is set.

I'm sorry I missed another condition and the correct one may be

```
if (upstream->read->active && !upstream->read->ready
    && !(u->length == 0 || (upstream->read->eof && u->length == -1))
    && !upstream->read->error && upstream->read->eof)
{
```

It is assuredly overcomplicated, or it can be in another function to check
the correct condition.

> In case of ngx_http_upstream_process_non_buffered_request() a
> better approach might be to don't try to do any processing in the
> proxy module after response is received from upstream, but rather
> finalize the upstream connection and pass responsibility to
> ngx_writer() instead, as we do in the normal (aka buffered) case.
> For ngx_http_upstream_process_upgraded(), additional ->eof /
> ->error checks are probably needed.

In case of ngx_http_upstream_process_non_buffered_request(), NGINX receives
from upstream and immediately sends to downstream. I'm afraid the problem
still will not be solved accord with "after response is received from
upstream".
Furthermore, kqueue notifies about the end of file or a pending error while
epoll don't.

Do you have any other solutions or advices?

Thanks to inform me.


2018-01-23 0:34 GMT+08:00 Maxim Dounin <mdounin at mdounin.ru>:

> Hello!
>
> On Fri, Jan 19, 2018 at 08:43:06PM +0800, xiaolong hong wrote:
>
> > # HG changeset patch
> > # User Xiaolong Hong <imhongxiaolong at gmail.com>
> > # Date 1516354115 -28800
> > #      Fri Jan 19 17:28:35 2018 +0800
> > # Node ID f017b8c1a99433cc3321475968556aee50609145
> > # Parent  93abb5a855d6534f0356882f45be49f8c6a95a8b
> > Fixed upstream->read timer when downstream->write not ready.
> >
> > diff -r 93abb5a855d6 -r f017b8c1a994 src/http/ngx_http_upstream.c
> > --- a/src/http/ngx_http_upstream.c      Thu Jan 11 21:43:49 2018 +0300
> > +++ b/src/http/ngx_http_upstream.c      Fri Jan 19 17:28:35 2018 +0800
> > @@ -3625,7 +3625,9 @@ ngx_http_upstream_process_non_buffered_r
> >          return;
> >      }
> >
> > -    if (upstream->read->active && !upstream->read->ready) {
> > +    if (upstream->read->active && !upstream->read->ready
> > +        && !(u->length == 0 || (upstream->read->eof && u->length ==
> -1)))
> > +    {
> >          ngx_add_timer(upstream->read, u->conf->read_timeout);
> >
> >      } else if (upstream->read->timer_set) {
> >
> > --------
> >
> > When downstream hung and nginx received the last buffer from upstream,
> both
> > downstream->write timer and upstream->read timer would be added in the
> > meantime because downstream->write->ready and upstream->read->ready would
> > opportunely be 0.
> >
> > Actually if clcf->send_timeout less then u->conf->read_timeout,
> > upstream->read timer would be waked before downstream->write timer, it
> > caused mistakes to report the "upstream timed out" error deviating from
> the
> > fact that upstream worked normally but downstream hung.
> >
> > This problem could be fixed to check upstream eof when trying to add
> > upstream->read timer.
>
> Thank you for the patch.
>
> The problem looks valid - indeed, if in non-buffered proxy mode
> writing to the client blocked, it is possible that we'll continue
> reading from upstream after the response was fully received, and
> will keep upstream read timer set.  This in turn can result in
> proxy_read_timeout being triggered if it is smaller than send_timeout.
>
> I can't say I like the suggested change though.  The resulting
> conditions look overcomplicated.  Also, likely we have similar
> problem in ngx_http_upstream_process_upgraded(), and it probably
> needs fixing too.
>
> Also, suggested condition is incorrect, as it will keep timer if
> upstream->read->eof is set, but u->length isn't -1, or if
> upstream->read->error is set.
>
> In case of ngx_http_upstream_process_non_buffered_request() a
> better approach might be to don't try to do any processing in the
> proxy module after response is received from upstream, but rather
> finalize the upstream connection and pass responsibility to
> ngx_writer() instead, as we do in the normal (aka buffered) case.
> For ngx_http_upstream_process_upgraded(), additional ->eof /
> ->error checks are probably needed.
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180201/cd8af985/attachment.html>

From niklas.fiekas at backscattering.de  Fri Feb  2 15:03:29 2018
From: niklas.fiekas at backscattering.de (Niklas Fiekas)
Date: Fri, 2 Feb 2018 16:03:29 +0100
Subject: [PATCH] Add application/webassembly content type
Message-ID: <cfc86240-0c0a-116c-f969-cfba5957c958@backscattering.de>

Dear maintainers,

please consider adding the application/webassembly content type for wasm 
files.

With this header recent browsers allow streaming compilation.

Best regards,
Niklas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: webassembly-mime.patch
Type: text/x-patch
Size: 983 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180202/29274c3e/attachment.bin>

From niklas.fiekas at backscattering.de  Fri Feb  2 15:16:53 2018
From: niklas.fiekas at backscattering.de (Niklas Fiekas)
Date: Fri, 2 Feb 2018 16:16:53 +0100
Subject: [PATCH] Add application/webassembly content type
In-Reply-To: <cfc86240-0c0a-116c-f969-cfba5957c958@backscattering.de>
References: <cfc86240-0c0a-116c-f969-cfba5957c958@backscattering.de>
Message-ID: <49808609-e59e-4b81-a7cc-bcb6e48fb2f7@backscattering.de>

Well, this is embarrassing. The correct content type is actually 
application/wasm. (application/webassembly was overruled because 
Chromium went ahead with the other one.)

https://github.com/WebAssembly/spec/issues/573

Best,
Niklas

On 02.02.2018 16:03, Niklas Fiekas wrote:
> Dear maintainers,
> 
> please consider adding the application/webassembly content type for wasm 
> files.
> 
> With this header recent browsers allow streaming compilation.
> 
> Best regards,
> Niklas
> 
> 
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wasm-mime.patch
Type: text/x-patch
Size: 975 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180202/c52f66ab/attachment.bin>

From matthew.marangoni at gmail.com  Fri Feb  2 19:10:57 2018
From: matthew.marangoni at gmail.com (Matthew Marangoni)
Date: Fri, 02 Feb 2018 11:10:57 -0800
Subject: [PATCH] Optimized murmurhash performance
Message-ID: <e84b174c99ca39155e80.1517598657@localhost.localdomain>

# HG changeset patch
# User Matthew Marangoni <matthew.marangoni at gmail.com>
# Date 1516655725 28800
#      Mon Jan 22 13:15:25 2018 -0800
# Node ID e84b174c99ca39155e80dbb7458181e20190b70b
# Parent  cbf59d483c9cd94dc0fb05f1978601d02af69c20
Optimized murmurhash performance.

Improved performance for little endian machines.

diff -r cbf59d483c9c -r e84b174c99ca src/core/ngx_murmurhash.c
--- a/src/core/ngx_murmurhash.c	Tue Jan 16 13:52:03 2018 +0300
+++ b/src/core/ngx_murmurhash.c	Mon Jan 22 13:15:25 2018 -0800
@@ -16,10 +16,14 @@
     h = 0 ^ len;
 
     while (len >= 4) {
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+        k = *(u_int32_t*)data;
+#else
         k  = data[0];
         k |= data[1] << 8;
         k |= data[2] << 16;
         k |= data[3] << 24;
+#endif
 
         k *= 0x5bd1e995;
         k ^= k >> 24;

From ru at nginx.com  Sat Feb  3 13:29:45 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Sat, 3 Feb 2018 16:29:45 +0300
Subject: [PATCH] Optimized murmurhash performance
In-Reply-To: <e84b174c99ca39155e80.1517598657@localhost.localdomain>
References: <e84b174c99ca39155e80.1517598657@localhost.localdomain>
Message-ID: <20180203132945.GH2607@lo0.su>

On Fri, Feb 02, 2018 at 11:10:57AM -0800, Matthew Marangoni wrote:
> # HG changeset patch
> # User Matthew Marangoni <matthew.marangoni at gmail.com>
> # Date 1516655725 28800
> #      Mon Jan 22 13:15:25 2018 -0800
> # Node ID e84b174c99ca39155e80dbb7458181e20190b70b
> # Parent  cbf59d483c9cd94dc0fb05f1978601d02af69c20
> Optimized murmurhash performance.
> 
> Improved performance for little endian machines.
> 
> diff -r cbf59d483c9c -r e84b174c99ca src/core/ngx_murmurhash.c
> --- a/src/core/ngx_murmurhash.c	Tue Jan 16 13:52:03 2018 +0300
> +++ b/src/core/ngx_murmurhash.c	Mon Jan 22 13:15:25 2018 -0800
> @@ -16,10 +16,14 @@
>      h = 0 ^ len;
>  
>      while (len >= 4) {
> +#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__

Please use NGX_HAVE_LITTLE_ENDIAN instead.

> +        k = *(u_int32_t*)data;

Please use uint32_t instead.

> +#else
>          k  = data[0];
>          k |= data[1] << 8;
>          k |= data[2] << 16;
>          k |= data[3] << 24;
> +#endif
>  
>          k *= 0x5bd1e995;
>          k ^= k >> 24;
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
> 

-- 
Ruslan Ermilov
Assume stupidity not malice

From matthew.marangoni at gmail.com  Sat Feb  3 16:39:54 2018
From: matthew.marangoni at gmail.com (Matthew Marangoni)
Date: Sat, 03 Feb 2018 08:39:54 -0800
Subject: [PATCH 1 of 2] Optimized murmurhash performance
Message-ID: <e84b174c99ca39155e80.1517675994@localhost.localdomain>

# HG changeset patch
# User Matthew Marangoni <matthew.marangoni at gmail.com>
# Date 1516655725 28800
#      Mon Jan 22 13:15:25 2018 -0800
# Node ID e84b174c99ca39155e80dbb7458181e20190b70b
# Parent  cbf59d483c9cd94dc0fb05f1978601d02af69c20
Optimized murmurhash performance.

Improved performance for little endian machines.

diff -r cbf59d483c9c -r e84b174c99ca src/core/ngx_murmurhash.c
--- a/src/core/ngx_murmurhash.c	Tue Jan 16 13:52:03 2018 +0300
+++ b/src/core/ngx_murmurhash.c	Mon Jan 22 13:15:25 2018 -0800
@@ -16,10 +16,14 @@
     h = 0 ^ len;
 
     while (len >= 4) {
+#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+        k = *(u_int32_t*)data;
+#else
         k  = data[0];
         k |= data[1] << 8;
         k |= data[2] << 16;
         k |= data[3] << 24;
+#endif
 
         k *= 0x5bd1e995;
         k ^= k >> 24;

From matthew.marangoni at gmail.com  Sat Feb  3 16:39:55 2018
From: matthew.marangoni at gmail.com (Matthew Marangoni)
Date: Sat, 03 Feb 2018 08:39:55 -0800
Subject: [PATCH 2 of 2] Updated __ORDER_LITTLE_ENDIAN__ to match NGX standards
 and changed to uint32_t data type
In-Reply-To: <e84b174c99ca39155e80.1517675994@localhost.localdomain>
References: <e84b174c99ca39155e80.1517675994@localhost.localdomain>
Message-ID: <4fe15c57e1d5c7fa4e2d.1517675995@localhost.localdomain>

# HG changeset patch
# User Matthew Marangoni <matthew.marangoni at gmail.com>
# Date 1517675878 28800
#      Sat Feb 03 08:37:58 2018 -0800
# Node ID 4fe15c57e1d5c7fa4e2daa365f82fd00d0a1f6f4
# Parent  e84b174c99ca39155e80dbb7458181e20190b70b
Updated __ORDER_LITTLE_ENDIAN__ to match NGX standards and changed to uint32_t data type

diff -r e84b174c99ca -r 4fe15c57e1d5 src/core/ngx_murmurhash.c
--- a/src/core/ngx_murmurhash.c	Mon Jan 22 13:15:25 2018 -0800
+++ b/src/core/ngx_murmurhash.c	Sat Feb 03 08:37:58 2018 -0800
@@ -16,8 +16,8 @@
     h = 0 ^ len;
 
     while (len >= 4) {
-#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
-        k = *(u_int32_t*)data;
+#if (NGX_HAVE_LITTLE_ENDIAN)
+        k = *(uint32_t*)data;
 #else
         k  = data[0];
         k |= data[1] << 8;

From vbart at nginx.com  Sun Feb  4 01:21:13 2018
From: vbart at nginx.com (Valentin V. Bartenev)
Date: Sun, 04 Feb 2018 04:21:13 +0300
Subject: [PATCH 2 of 2] Updated __ORDER_LITTLE_ENDIAN__ to match NGX
 standards and changed to uint32_t data type
In-Reply-To: <4fe15c57e1d5c7fa4e2d.1517675995@localhost.localdomain>
References: <e84b174c99ca39155e80.1517675994@localhost.localdomain>
 <4fe15c57e1d5c7fa4e2d.1517675995@localhost.localdomain>
Message-ID: <1777243.zDY3EbAnj5@vbart-laptop>

On Saturday, 3 February 2018 19:39:55 MSK Matthew Marangoni wrote:
> # HG changeset patch
> # User Matthew Marangoni <matthew.marangoni at gmail.com>
> # Date 1517675878 28800
> #      Sat Feb 03 08:37:58 2018 -0800
> # Node ID 4fe15c57e1d5c7fa4e2daa365f82fd00d0a1f6f4
> # Parent  e84b174c99ca39155e80dbb7458181e20190b70b
> Updated __ORDER_LITTLE_ENDIAN__ to match NGX standards and changed to uint32_t data type
> 
> diff -r e84b174c99ca -r 4fe15c57e1d5 src/core/ngx_murmurhash.c
> --- a/src/core/ngx_murmurhash.c	Mon Jan 22 13:15:25 2018 -0800
> +++ b/src/core/ngx_murmurhash.c	Sat Feb 03 08:37:58 2018 -0800
> @@ -16,8 +16,8 @@
>      h = 0 ^ len;
>  
>      while (len >= 4) {
> -#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
> -        k = *(u_int32_t*)data;
> +#if (NGX_HAVE_LITTLE_ENDIAN)
> +        k = *(uint32_t*)data;
>  #else
>          k  = data[0];
>          k |= data[1] << 8;


Your patch will break nginx on platforms that have alignment requirements.

  wbr, Valentin V. Bartenev

From hongzhidao at gmail.com  Sun Feb  4 12:49:12 2018
From: hongzhidao at gmail.com (=?UTF-8?B?5rSq5b+X6YGT?=)
Date: Sun, 4 Feb 2018 20:49:12 +0800
Subject: [nginx] enhance options
Message-ID: <CAJuLd0i-EdJjFa_ATBTq7+wLGx3sYG8=r3NOGJ2Ecu3CCsYyfA@mail.gmail.com>

Hi!

Maybe it's better to keep the options more complete, and consider the
follow operation as a mistake.

/usr/local/nginx/sbin/nginx -

hg diff
diff -r cbf59d483c9c src/core/nginx.c
--- a/src/core/nginx.c Tue Jan 16 13:52:03 2018 +0300
+++ b/src/core/nginx.c Sun Feb 04 07:41:36 2018 -0500
@@ -753,7 +753,12 @@
             return NGX_ERROR;
         }

-        while (*p) {
+        do {
+
+            if (*p == '\0') {
+                ngx_log_stderr(0, "missing option: \"%s\"", argv[i]);
+                return NGX_ERROR;
+            }

             switch (*p++) {

@@ -855,7 +860,8 @@
                 ngx_log_stderr(0, "invalid option: \"%c\"", *(p - 1));
                 return NGX_ERROR;
             }
-        }
+
+        } while (*p);

     next:

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180204/02674b0d/attachment.html>

From mdounin at mdounin.ru  Sun Feb  4 18:27:11 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sun, 4 Feb 2018 21:27:11 +0300
Subject: [PATCH] Add application/webassembly content type
In-Reply-To: <49808609-e59e-4b81-a7cc-bcb6e48fb2f7@backscattering.de>
References: <cfc86240-0c0a-116c-f969-cfba5957c958@backscattering.de>
 <49808609-e59e-4b81-a7cc-bcb6e48fb2f7@backscattering.de>
Message-ID: <20180204182711.GN24410@mdounin.ru>

Hello!

On Fri, Feb 02, 2018 at 04:16:53PM +0100, Niklas Fiekas wrote:

> Well, this is embarrassing. The correct content type is actually 
> application/wasm. (application/webassembly was overruled because 
> Chromium went ahead with the other one.)
> 
> https://github.com/WebAssembly/spec/issues/573

As far as I see from the link provided and the IANA official list 
as available at [1], niether application/webassembly nor 
application/wasm are currently registered MIME types.  Once any of 
them is registered, we can consider adding it to nginx default 
mime.types.

[1] https://www.iana.org/assignments/media-types/media-types.xhtml

-- 
Maxim Dounin
http://mdounin.ru/

From b.rossgardt at googlemail.com  Sun Feb  4 18:53:39 2018
From: b.rossgardt at googlemail.com (=?utf-8?Q?Benedikt_Ro=C3=9Fgardt?=)
Date: Sun, 4 Feb 2018 19:53:39 +0100
Subject: Fwd: Proposal: Change folder for PHP CGI scripts
References: <2BD7BE3B-2D8A-466B-B333-A5E971AD0ECC@googlemail.com>
Message-ID: <4D09074E-B572-4154-AC70-637A376E6034@googlemail.com>

Hi,

When using the default configuration from the nginx.conf file, I was running into "File not found" issues. The problem for me was that the default configuration is pointing to the /script folder instead of my document root.

Maybe there is a reason for it, but I propose to use the root document folder instead.

Best,
Benedikt



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180204/8b615846/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch.diff
Type: application/octet-stream
Size: 498 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180204/8b615846/attachment.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180204/8b615846/attachment-0001.html>

From zhjwpku at gmail.com  Mon Feb  5 01:07:16 2018
From: zhjwpku at gmail.com (Junwang Zhao)
Date: Mon, 5 Feb 2018 09:07:16 +0800
Subject: [nginx] enhance options
In-Reply-To: <CAJuLd0i-EdJjFa_ATBTq7+wLGx3sYG8=r3NOGJ2Ecu3CCsYyfA@mail.gmail.com>
References: <CAJuLd0i-EdJjFa_ATBTq7+wLGx3sYG8=r3NOGJ2Ecu3CCsYyfA@mail.gmail.com>
Message-ID: <CAEG8a3Jy=hfrsRoZiri+eRwV_hx8EszF0naLiZx4jbhkAAB36A@mail.gmail.com>

LGTM

On Sun, Feb 4, 2018 at 8:49 PM, ??? <hongzhidao at gmail.com> wrote:

> Hi!
>
> Maybe it's better to keep the options more complete, and consider the
> follow operation as a mistake.
>
> /usr/local/nginx/sbin/nginx -
>
> hg diff
> diff -r cbf59d483c9c src/core/nginx.c
> --- a/src/core/nginx.c Tue Jan 16 13:52:03 2018 +0300
> +++ b/src/core/nginx.c Sun Feb 04 07:41:36 2018 -0500
> @@ -753,7 +753,12 @@
>              return NGX_ERROR;
>          }
>
> -        while (*p) {
> +        do {
> +
> +            if (*p == '\0') {
> +                ngx_log_stderr(0, "missing option: \"%s\"", argv[i]);
> +                return NGX_ERROR;
> +            }
>
>              switch (*p++) {
>
> @@ -855,7 +860,8 @@
>                  ngx_log_stderr(0, "invalid option: \"%c\"", *(p - 1));
>                  return NGX_ERROR;
>              }
> -        }
> +
> +        } while (*p);
>
>      next:
>
> Thanks.
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180205/f2ce15bf/attachment.html>

From mdounin at mdounin.ru  Mon Feb  5 15:27:08 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 5 Feb 2018 18:27:08 +0300
Subject: [nginx] enhance options
In-Reply-To: <CAJuLd0i-EdJjFa_ATBTq7+wLGx3sYG8=r3NOGJ2Ecu3CCsYyfA@mail.gmail.com>
References: <CAJuLd0i-EdJjFa_ATBTq7+wLGx3sYG8=r3NOGJ2Ecu3CCsYyfA@mail.gmail.com>
Message-ID: <20180205152708.GW24410@mdounin.ru>

Hello!

On Sun, Feb 04, 2018 at 08:49:12PM +0800, ??? wrote:

> Hi!
> 
> Maybe it's better to keep the options more complete, and consider the
> follow operation as a mistake.
> 
> /usr/local/nginx/sbin/nginx -
> 
> hg diff
> diff -r cbf59d483c9c src/core/nginx.c
> --- a/src/core/nginx.c Tue Jan 16 13:52:03 2018 +0300
> +++ b/src/core/nginx.c Sun Feb 04 07:41:36 2018 -0500
> @@ -753,7 +753,12 @@
>              return NGX_ERROR;
>          }
> 
> -        while (*p) {
> +        do {
> +
> +            if (*p == '\0') {
> +                ngx_log_stderr(0, "missing option: \"%s\"", argv[i]);
> +                return NGX_ERROR;
> +            }
> 
>              switch (*p++) {
> 
> @@ -855,7 +860,8 @@
>                  ngx_log_stderr(0, "invalid option: \"%c\"", *(p - 1));
>                  return NGX_ERROR;
>              }
> -        }
> +
> +        } while (*p);
> 
>      next:

Checking *p twice on each loop iteration certainly looks 
superfluous.  Rather, a more logical check would look like:

diff --git a/src/core/nginx.c b/src/core/nginx.c
--- a/src/core/nginx.c
+++ b/src/core/nginx.c
@@ -748,7 +748,7 @@ ngx_get_options(int argc, char *const *a
 
         p = (u_char *) argv[i];
 
-        if (*p++ != '-') {
+        if (*p++ != '-' || *p == '\0') {
             ngx_log_stderr(0, "invalid option: \"%s\"", argv[i]);
             return NGX_ERROR;
         }

Not sure it worth the change though.

-- 
Maxim Dounin
http://mdounin.ru/

From mdounin at mdounin.ru  Mon Feb  5 15:43:05 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 05 Feb 2018 15:43:05 +0000
Subject: [nginx] Contrib: vim syntax, update core module directives.
Message-ID: <hg.6db472f8dc8e.1517845385.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/6db472f8dc8e
branches:  
changeset: 7195:6db472f8dc8e
user:      Gena Makhomed <gmm at csdoc.com>
date:      Thu Feb 01 11:09:35 2018 +0200
description:
Contrib: vim syntax, update core module directives.

"match" is block directive, "upstream_conf" is deprecated
by patch http://hg.nginx.org/nginx.org/rev/27c53e1cb4b6

diffstat:

 contrib/vim/syntax/nginx.vim |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (35 lines):

diff --git a/contrib/vim/syntax/nginx.vim b/contrib/vim/syntax/nginx.vim
--- a/contrib/vim/syntax/nginx.vim
+++ b/contrib/vim/syntax/nginx.vim
@@ -87,6 +87,7 @@ syn keyword ngxDirectiveBlock contained 
 syn keyword ngxDirectiveBlock contained geo
 syn keyword ngxDirectiveBlock contained map
 syn keyword ngxDirectiveBlock contained split_clients
+syn keyword ngxDirectiveBlock contained match
 
 syn keyword ngxDirectiveImportant contained include
 syn keyword ngxDirectiveImportant contained root
@@ -117,6 +118,7 @@ syn keyword ngxDirectiveDeprecated conta
 syn keyword ngxDirectiveDeprecated contained spdy_recv_buffer_size
 syn keyword ngxDirectiveDeprecated contained spdy_recv_timeout
 syn keyword ngxDirectiveDeprecated contained spdy_streams_index_size
+syn keyword ngxDirectiveDeprecated contained upstream_conf
 
 syn keyword ngxDirective contained absolute_redirect
 syn keyword ngxDirective contained accept_mutex
@@ -324,7 +326,6 @@ syn keyword ngxDirective contained log_n
 syn keyword ngxDirective contained log_subrequest
 syn keyword ngxDirective contained map_hash_bucket_size
 syn keyword ngxDirective contained map_hash_max_size
-syn keyword ngxDirective contained match
 syn keyword ngxDirective contained master_process
 syn keyword ngxDirective contained max_ranges
 syn keyword ngxDirective contained memcached_bind
@@ -582,7 +583,6 @@ syn keyword ngxDirective contained types
 syn keyword ngxDirective contained types_hash_max_size
 syn keyword ngxDirective contained underscores_in_headers
 syn keyword ngxDirective contained uninitialized_variable_warn
-syn keyword ngxDirective contained upstream_conf
 syn keyword ngxDirective contained use
 syn keyword ngxDirective contained user
 syn keyword ngxDirective contained userid

From mdounin at mdounin.ru  Mon Feb  5 15:43:06 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 05 Feb 2018 15:43:06 +0000
Subject: [nginx] Contrib: vim syntax, update 3rd party module directives.
Message-ID: <hg.2e0de79805ee.1517845386.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/2e0de79805ee
branches:  
changeset: 7196:2e0de79805ee
user:      Gena Makhomed <gmm at csdoc.com>
date:      Thu Feb 01 11:15:14 2018 +0200
description:
Contrib: vim syntax, update 3rd party module directives.

Add new directives for 3rd party modules.

diffstat:

 contrib/vim/syntax/nginx.vim |  5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diffs (31 lines):

diff --git a/contrib/vim/syntax/nginx.vim b/contrib/vim/syntax/nginx.vim
--- a/contrib/vim/syntax/nginx.vim
+++ b/contrib/vim/syntax/nginx.vim
@@ -954,6 +954,7 @@ syn keyword ngxDirectiveThirdParty conta
 syn keyword ngxDirectiveThirdParty contained nchan_redis_server
 syn keyword ngxDirectiveThirdParty contained nchan_redis_storage_mode
 syn keyword ngxDirectiveThirdParty contained nchan_redis_url
+syn keyword ngxDirectiveThirdParty contained nchan_redis_wait_after_connecting
 syn keyword ngxDirectiveThirdParty contained nchan_shared_memory_size
 syn keyword ngxDirectiveThirdParty contained nchan_storage_engine
 syn keyword ngxDirectiveThirdParty contained nchan_store_messages
@@ -1080,6 +1081,8 @@ syn keyword ngxDirectiveThirdParty conta
 syn keyword ngxDirectiveThirdParty contained tnt_select_limit_max
 syn keyword ngxDirectiveThirdParty contained tnt_send_timeout
 syn keyword ngxDirectiveThirdParty contained tnt_set_header
+syn keyword ngxDirectiveThirdParty contained tnt_update
+syn keyword ngxDirectiveThirdParty contained tnt_upsert
 
 " A module for nginx web server for handling file uploads using multipart/form-data encoding (RFC 1867)
 " https://github.com/Austinb/nginx-upload-module
@@ -2106,8 +2109,10 @@ syn keyword ngxDirectiveThirdParty conta
 
 " Embed the power of Lua into NGINX TCP/UDP servers
 " https://github.com/openresty/stream-lua-nginx-module
+syn keyword ngxDirectiveThirdParty contained lua_add_variable
 syn keyword ngxDirectiveThirdParty contained preread_by_lua_block
 syn keyword ngxDirectiveThirdParty contained preread_by_lua_file
+syn keyword ngxDirectiveThirdParty contained preread_by_lua_no_postpone
 
 " nginx-upsync-module
 " https://github.com/weibocom/nginx-upsync-module

From mdounin at mdounin.ru  Mon Feb  5 15:43:15 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 5 Feb 2018 18:43:15 +0300
Subject: [PATCH 1 of 2] Contrib: vim syntax, update core module directives.
In-Reply-To: <d5143118-5013-c7dd-373c-b2440b7fa054@csdoc.com>
References: <d5143118-5013-c7dd-373c-b2440b7fa054@csdoc.com>
Message-ID: <20180205154315.GX24410@mdounin.ru>

Hello!

On Thu, Feb 01, 2018 at 11:19:01AM +0200, Gena Makhomed wrote:

> # HG changeset patch
> # User Gena Makhomed <gmm at csdoc.com>
> # Date 1517476175 -7200
> #      Thu Feb 01 11:09:35 2018 +0200
> # Node ID 6db472f8dc8ec04754a42ba22e9663cb7e91f2ab
> # Parent  0b72d545f098b407351a1972fba5d29f2e904507
> Contrib: vim syntax, update core module directives.
> 
> "match" is block directive, "upstream_conf" is deprecated
> by patch http://hg.nginx.org/nginx.org/rev/27c53e1cb4b6

[...]

Committed, thanks.

-- 
Maxim Dounin
http://mdounin.ru/

From mdounin at mdounin.ru  Mon Feb  5 15:43:28 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 5 Feb 2018 18:43:28 +0300
Subject: [PATCH 2 of 2] Contrib: vim syntax,
 update 3rd party module directives.
In-Reply-To: <9d244deb-2903-5dd3-ae7e-0be674a18fb3@csdoc.com>
References: <9d244deb-2903-5dd3-ae7e-0be674a18fb3@csdoc.com>
Message-ID: <20180205154328.GY24410@mdounin.ru>

Hello!

On Thu, Feb 01, 2018 at 11:20:11AM +0200, Gena Makhomed wrote:

> # HG changeset patch
> # User Gena Makhomed <gmm at csdoc.com>
> # Date 1517476514 -7200
> #      Thu Feb 01 11:15:14 2018 +0200
> # Node ID 2e0de79805eee669cca7e2455010689a3191f268
> # Parent  6db472f8dc8ec04754a42ba22e9663cb7e91f2ab
> Contrib: vim syntax, update 3rd party module directives.
> 
> Add new directives for 3rd party modules.

[...]

Committed, thanks.

-- 
Maxim Dounin
http://mdounin.ru/

From gmm at csdoc.com  Mon Feb  5 15:49:28 2018
From: gmm at csdoc.com (Gena Makhomed)
Date: Mon, 5 Feb 2018 17:49:28 +0200
Subject: nginx undocumented directives
In-Reply-To: <20171228131735.GJ34136@mdounin.ru>
References: <418fda51-227c-221f-804f-aa7e88322a53@csdoc.com>
 <20171228122533.GC15734@lo0.su> <20171228131735.GJ34136@mdounin.ru>
Message-ID: <f8be7289-9327-13c2-7cc1-759f0db72c0a@csdoc.com>

On 28.12.2017 15:17, Maxim Dounin wrote:

>>> smtp_client_buffer
>>> smtp_greeting_delay
>>
>> There are no notes of why these aren't documented.
> 
> I guess the reason is that mail documentation is mostly
> contributed, and was never seriously considered by the
> documentation project.
> 
> The smtp_client_buffer directive is identical to
> imap_client_buffer, but for the smtp module.
> 
> The smtp_greeting_delay directive allows introducing an artificial
> delay before sending an SMTP greeting, and rejecting clients who
> fail to wait for a greeting before sending commands.

One month later:

directives smtp_client_buffer and smtp_greeting_delay

still not documented.

-- 
Best regards,
  Gena

From mdounin at mdounin.ru  Mon Feb  5 16:08:06 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 5 Feb 2018 19:08:06 +0300
Subject: Fwd: Proposal: Change folder for PHP CGI scripts
In-Reply-To: <4D09074E-B572-4154-AC70-637A376E6034@googlemail.com>
References: <2BD7BE3B-2D8A-466B-B333-A5E971AD0ECC@googlemail.com>
 <4D09074E-B572-4154-AC70-637A376E6034@googlemail.com>
Message-ID: <20180205160806.GZ24410@mdounin.ru>

Hello!

On Sun, Feb 04, 2018 at 07:53:39PM +0100, Benedikt Ro?gardt via nginx-devel wrote:

> Hi,
> 
> When using the default configuration from the nginx.conf file, I 
> was running into "File not found" issues. The problem for me was 
> that the default configuration is pointing to the /script folder 
> instead of my document root.
> 
> Maybe there is a reason for it, but I propose to use the root 
> document folder instead.

The example configuration in question is just an example, and it 
assumes scripts are in the /scripts folder.  In particular, it 
demostrates that php scripts can be stored in a folder distrinct 
from the document root.

For a typical configuration where php scripts are located under 
the document root there is a fastcgi.conf include file, which 
combines fastcgi_params and "fastcgi_param SCRIPT_FILENAME 
$document_root$fastcgi_script_name".

-- 
Maxim Dounin
http://mdounin.ru/

From mdounin at mdounin.ru  Mon Feb  5 16:39:34 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 5 Feb 2018 19:39:34 +0300
Subject: nginx undocumented directives
In-Reply-To: <f8be7289-9327-13c2-7cc1-759f0db72c0a@csdoc.com>
References: <418fda51-227c-221f-804f-aa7e88322a53@csdoc.com>
 <20171228122533.GC15734@lo0.su> <20171228131735.GJ34136@mdounin.ru>
 <f8be7289-9327-13c2-7cc1-759f0db72c0a@csdoc.com>
Message-ID: <20180205163934.GC24410@mdounin.ru>

Hello!

On Mon, Feb 05, 2018 at 05:49:28PM +0200, Gena Makhomed wrote:

> On 28.12.2017 15:17, Maxim Dounin wrote:
> 
> >>> smtp_client_buffer
> >>> smtp_greeting_delay
> >>
> >> There are no notes of why these aren't documented.
> > 
> > I guess the reason is that mail documentation is mostly
> > contributed, and was never seriously considered by the
> > documentation project.
> > 
> > The smtp_client_buffer directive is identical to
> > imap_client_buffer, but for the smtp module.
> > 
> > The smtp_greeting_delay directive allows introducing an artificial
> > delay before sending an SMTP greeting, and rejecting clients who
> > fail to wait for a greeting before sending commands.
> 
> One month later:
> 
> directives smtp_client_buffer and smtp_greeting_delay
> 
> still not documented.

That's how opensource works.  If you want something to happen, you 
have two basic options:

1. Wait for someone to do it.

2. Do it yourself.

-- 
Maxim Dounin
http://mdounin.ru/

From pluknet at nginx.com  Tue Feb  6 21:06:15 2018
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Tue, 06 Feb 2018 21:06:15 +0000
Subject: [nginx] HTTP/2: removed unused field from ngx_http_v2_stream_t.
Message-ID: <hg.ca8f8a443c11.1517951175.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/ca8f8a443c11
branches:  
changeset: 7197:ca8f8a443c11
user:      Sergey Kandaurov <pluknet at nginx.com>
date:      Tue Feb 06 20:02:59 2018 +0300
description:
HTTP/2: removed unused field from ngx_http_v2_stream_t.

diffstat:

 src/http/v2/ngx_http_v2.h |  2 --
 1 files changed, 0 insertions(+), 2 deletions(-)

diffs (12 lines):

diff -r 2e0de79805ee -r ca8f8a443c11 src/http/v2/ngx_http_v2.h
--- a/src/http/v2/ngx_http_v2.h	Thu Feb 01 11:15:14 2018 +0200
+++ b/src/http/v2/ngx_http_v2.h	Tue Feb 06 20:02:59 2018 +0300
@@ -190,8 +190,6 @@ struct ngx_http_v2_stream_s {
 
     ngx_array_t                     *cookies;
 
-    size_t                           header_limit;
-
     ngx_pool_t                      *pool;
 
     unsigned                         waiting:1;

From arut at nginx.com  Wed Feb  7 13:49:02 2018
From: arut at nginx.com (Roman Arutyunyan)
Date: Wed, 07 Feb 2018 13:49:02 +0000
Subject: [nginx] Dav: added error logging.
Message-ID: <hg.573f20116163.1518011342.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/573f20116163
branches:  
changeset: 7198:573f20116163
user:      Roman Arutyunyan <arut at nginx.com>
date:      Wed Feb 07 16:44:29 2018 +0300
description:
Dav: added error logging.

Previously, when request body was not available or was previously read in
memory rather than a file, client received HTTP 500 error, but no explanation
was logged in error log.  This could happen, for example, if request body was
read or discarded prior to error_page redirect, or if mirroring was enabled
along with dav.

diffstat:

 src/http/modules/ngx_http_dav_module.c |  11 ++++++++++-
 1 files changed, 10 insertions(+), 1 deletions(-)

diffs (21 lines):

diff -r ca8f8a443c11 -r 573f20116163 src/http/modules/ngx_http_dav_module.c
--- a/src/http/modules/ngx_http_dav_module.c	Tue Feb 06 20:02:59 2018 +0300
+++ b/src/http/modules/ngx_http_dav_module.c	Wed Feb 07 16:44:29 2018 +0300
@@ -213,7 +213,16 @@ ngx_http_dav_put_handler(ngx_http_reques
     ngx_ext_rename_file_t     ext;
     ngx_http_dav_loc_conf_t  *dlcf;
 
-    if (r->request_body == NULL || r->request_body->temp_file == NULL) {
+    if (r->request_body == NULL) {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "PUT request body is unavailable");
+        ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
+        return;
+    }
+
+    if (r->request_body->temp_file == NULL) {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "PUT request body must be in a file");
         ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
         return;
     }

From ru at nginx.com  Thu Feb  8 06:58:08 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 08 Feb 2018 06:58:08 +0000
Subject: [nginx] Basic support of the Link response header.
Message-ID: <hg.6ba68ad8b24c.1518073088.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/6ba68ad8b24c
branches:  
changeset: 7199:6ba68ad8b24c
user:      Ruslan Ermilov <ru at nginx.com>
date:      Thu Feb 08 09:54:18 2018 +0300
description:
Basic support of the Link response header.

diffstat:

 src/http/modules/ngx_http_headers_filter_module.c |  42 ++++++++++++----------
 src/http/ngx_http_request.h                       |   1 +
 src/http/ngx_http_upstream.c                      |   5 ++
 src/http/ngx_http_variables.c                     |   3 +
 4 files changed, 32 insertions(+), 19 deletions(-)

diffs (124 lines):

diff -r 573f20116163 -r 6ba68ad8b24c src/http/modules/ngx_http_headers_filter_module.c
--- a/src/http/modules/ngx_http_headers_filter_module.c	Wed Feb 07 16:44:29 2018 +0300
+++ b/src/http/modules/ngx_http_headers_filter_module.c	Thu Feb 08 09:54:18 2018 +0300
@@ -56,7 +56,7 @@ static ngx_int_t ngx_http_set_expires(ng
     ngx_http_headers_conf_t *conf);
 static ngx_int_t ngx_http_parse_expires(ngx_str_t *value,
     ngx_http_expires_t *expires, time_t *expires_time, char **err);
-static ngx_int_t ngx_http_add_cache_control(ngx_http_request_t *r,
+static ngx_int_t ngx_http_add_multi_header_lines(ngx_http_request_t *r,
     ngx_http_header_val_t *hv, ngx_str_t *value);
 static ngx_int_t ngx_http_add_header(ngx_http_request_t *r,
     ngx_http_header_val_t *hv, ngx_str_t *value);
@@ -77,7 +77,13 @@ static char *ngx_http_headers_add(ngx_co
 
 static ngx_http_set_header_t  ngx_http_set_headers[] = {
 
-    { ngx_string("Cache-Control"), 0, ngx_http_add_cache_control },
+    { ngx_string("Cache-Control"),
+                 offsetof(ngx_http_headers_out_t, cache_control),
+                 ngx_http_add_multi_header_lines },
+
+    { ngx_string("Link"),
+                 offsetof(ngx_http_headers_out_t, link),
+                 ngx_http_add_multi_header_lines },
 
     { ngx_string("Last-Modified"),
                  offsetof(ngx_http_headers_out_t, last_modified),
@@ -555,42 +561,40 @@ ngx_http_add_header(ngx_http_request_t *
 
 
 static ngx_int_t
-ngx_http_add_cache_control(ngx_http_request_t *r, ngx_http_header_val_t *hv,
-    ngx_str_t *value)
+ngx_http_add_multi_header_lines(ngx_http_request_t *r,
+    ngx_http_header_val_t *hv, ngx_str_t *value)
 {
-    ngx_table_elt_t  *cc, **ccp;
+    ngx_array_t      *pa;
+    ngx_table_elt_t  *h, **ph;
 
     if (value->len == 0) {
         return NGX_OK;
     }
 
-    ccp = r->headers_out.cache_control.elts;
-
-    if (ccp == NULL) {
+    pa = (ngx_array_t *) ((char *) &r->headers_out + hv->offset);
 
-        if (ngx_array_init(&r->headers_out.cache_control, r->pool,
-                           1, sizeof(ngx_table_elt_t *))
-            != NGX_OK)
+    if (pa->elts == NULL) {
+        if (ngx_array_init(pa, r->pool, 1, sizeof(ngx_table_elt_t *)) != NGX_OK)
         {
             return NGX_ERROR;
         }
     }
 
-    cc = ngx_list_push(&r->headers_out.headers);
-    if (cc == NULL) {
+    h = ngx_list_push(&r->headers_out.headers);
+    if (h == NULL) {
         return NGX_ERROR;
     }
 
-    cc->hash = 1;
-    ngx_str_set(&cc->key, "Cache-Control");
-    cc->value = *value;
+    h->hash = 1;
+    h->key = hv->key;
+    h->value = *value;
 
-    ccp = ngx_array_push(&r->headers_out.cache_control);
-    if (ccp == NULL) {
+    ph = ngx_array_push(pa);
+    if (ph == NULL) {
         return NGX_ERROR;
     }
 
-    *ccp = cc;
+    *ph = h;
 
     return NGX_OK;
 }
diff -r 573f20116163 -r 6ba68ad8b24c src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h	Wed Feb 07 16:44:29 2018 +0300
+++ b/src/http/ngx_http_request.h	Thu Feb 08 09:54:18 2018 +0300
@@ -279,6 +279,7 @@ typedef struct {
     ngx_uint_t                        content_type_hash;
 
     ngx_array_t                       cache_control;
+    ngx_array_t                       link;
 
     off_t                             content_length_n;
     off_t                             content_offset;
diff -r 573f20116163 -r 6ba68ad8b24c src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Wed Feb 07 16:44:29 2018 +0300
+++ b/src/http/ngx_http_upstream.c	Thu Feb 08 09:54:18 2018 +0300
@@ -284,6 +284,11 @@ static ngx_http_upstream_header_t  ngx_h
                  ngx_http_upstream_process_vary, 0,
                  ngx_http_upstream_copy_header_line, 0, 0 },
 
+    { ngx_string("Link"),
+                 ngx_http_upstream_ignore_header_line, 0,
+                 ngx_http_upstream_copy_multi_header_lines,
+                 offsetof(ngx_http_headers_out_t, link), 0 },
+
     { ngx_string("X-Accel-Expires"),
                  ngx_http_upstream_process_accel_expires, 0,
                  ngx_http_upstream_copy_header_line, 0, 0 },
diff -r 573f20116163 -r 6ba68ad8b24c src/http/ngx_http_variables.c
--- a/src/http/ngx_http_variables.c	Wed Feb 07 16:44:29 2018 +0300
+++ b/src/http/ngx_http_variables.c	Thu Feb 08 09:54:18 2018 +0300
@@ -318,6 +318,9 @@ static ngx_http_variable_t  ngx_http_cor
     { ngx_string("sent_http_cache_control"), NULL, ngx_http_variable_headers,
       offsetof(ngx_http_request_t, headers_out.cache_control), 0, 0 },
 
+    { ngx_string("sent_http_link"), NULL, ngx_http_variable_headers,
+      offsetof(ngx_http_request_t, headers_out.link), 0, 0 },
+
     { ngx_string("limit_rate"), ngx_http_variable_request_set_size,
       ngx_http_variable_request_get_size,
       offsetof(ngx_http_request_t, limit_rate),

From ru at nginx.com  Thu Feb  8 06:58:09 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 08 Feb 2018 06:58:09 +0000
Subject: [nginx] HTTP/2: changed prototypes of request pseudo-headers parsers.
Message-ID: <hg.cadb43014c7c.1518073089.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/cadb43014c7c
branches:  
changeset: 7200:cadb43014c7c
user:      Ruslan Ermilov <ru at nginx.com>
date:      Thu Feb 08 09:54:49 2018 +0300
description:
HTTP/2: changed prototypes of request pseudo-headers parsers.

No functional changes.

diffstat:

 src/http/v2/ngx_http_v2.c |  49 +++++++++++++++++++++++-----------------------
 1 files changed, 24 insertions(+), 25 deletions(-)

diffs (164 lines):

diff -r 6ba68ad8b24c -r cadb43014c7c src/http/v2/ngx_http_v2.c
--- a/src/http/v2/ngx_http_v2.c	Thu Feb 08 09:54:18 2018 +0300
+++ b/src/http/v2/ngx_http_v2.c	Thu Feb 08 09:54:49 2018 +0300
@@ -150,13 +150,13 @@ static ngx_int_t ngx_http_v2_validate_he
 static ngx_int_t ngx_http_v2_pseudo_header(ngx_http_request_t *r,
     ngx_http_v2_header_t *header);
 static ngx_int_t ngx_http_v2_parse_path(ngx_http_request_t *r,
-    ngx_http_v2_header_t *header);
+    ngx_str_t *value);
 static ngx_int_t ngx_http_v2_parse_method(ngx_http_request_t *r,
-    ngx_http_v2_header_t *header);
+    ngx_str_t *value);
 static ngx_int_t ngx_http_v2_parse_scheme(ngx_http_request_t *r,
-    ngx_http_v2_header_t *header);
+    ngx_str_t *value);
 static ngx_int_t ngx_http_v2_parse_authority(ngx_http_request_t *r,
-    ngx_http_v2_header_t *header);
+    ngx_str_t *value);
 static ngx_int_t ngx_http_v2_construct_request_line(ngx_http_request_t *r);
 static ngx_int_t ngx_http_v2_cookie(ngx_http_request_t *r,
     ngx_http_v2_header_t *header);
@@ -3082,7 +3082,7 @@ ngx_http_v2_pseudo_header(ngx_http_reque
         if (ngx_memcmp(header->name.data, "path", sizeof("path") - 1)
             == 0)
         {
-            return ngx_http_v2_parse_path(r, header);
+            return ngx_http_v2_parse_path(r, &header->value);
         }
 
         break;
@@ -3091,13 +3091,13 @@ ngx_http_v2_pseudo_header(ngx_http_reque
         if (ngx_memcmp(header->name.data, "method", sizeof("method") - 1)
             == 0)
         {
-            return ngx_http_v2_parse_method(r, header);
+            return ngx_http_v2_parse_method(r, &header->value);
         }
 
         if (ngx_memcmp(header->name.data, "scheme", sizeof("scheme") - 1)
             == 0)
         {
-            return ngx_http_v2_parse_scheme(r, header);
+            return ngx_http_v2_parse_scheme(r, &header->value);
         }
 
         break;
@@ -3106,7 +3106,7 @@ ngx_http_v2_pseudo_header(ngx_http_reque
         if (ngx_memcmp(header->name.data, "authority", sizeof("authority") - 1)
             == 0)
         {
-            return ngx_http_v2_parse_authority(r, header);
+            return ngx_http_v2_parse_authority(r, &header->value);
         }
 
         break;
@@ -3121,7 +3121,7 @@ ngx_http_v2_pseudo_header(ngx_http_reque
 
 
 static ngx_int_t
-ngx_http_v2_parse_path(ngx_http_request_t *r, ngx_http_v2_header_t *header)
+ngx_http_v2_parse_path(ngx_http_request_t *r, ngx_str_t *value)
 {
     if (r->unparsed_uri.len) {
         ngx_log_error(NGX_LOG_INFO, r->connection->log, 0,
@@ -3130,20 +3130,19 @@ ngx_http_v2_parse_path(ngx_http_request_
         return NGX_DECLINED;
     }
 
-    if (header->value.len == 0) {
+    if (value->len == 0) {
         ngx_log_error(NGX_LOG_INFO, r->connection->log, 0,
                       "client sent empty :path header");
 
         return NGX_DECLINED;
     }
 
-    r->uri_start = header->value.data;
-    r->uri_end = header->value.data + header->value.len;
+    r->uri_start = value->data;
+    r->uri_end = value->data + value->len;
 
     if (ngx_http_parse_uri(r) != NGX_OK) {
         ngx_log_error(NGX_LOG_INFO, r->connection->log, 0,
-                      "client sent invalid :path header: \"%V\"",
-                      &header->value);
+                      "client sent invalid :path header: \"%V\"", value);
 
         return NGX_DECLINED;
     }
@@ -3161,7 +3160,7 @@ ngx_http_v2_parse_path(ngx_http_request_
 
 
 static ngx_int_t
-ngx_http_v2_parse_method(ngx_http_request_t *r, ngx_http_v2_header_t *header)
+ngx_http_v2_parse_method(ngx_http_request_t *r, ngx_str_t *value)
 {
     size_t         k, len;
     ngx_uint_t     n;
@@ -3201,15 +3200,15 @@ ngx_http_v2_parse_method(ngx_http_reques
         return NGX_DECLINED;
     }
 
-    if (header->value.len == 0) {
+    if (value->len == 0) {
         ngx_log_error(NGX_LOG_INFO, r->connection->log, 0,
                       "client sent empty :method header");
 
         return NGX_DECLINED;
     }
 
-    r->method_name.len = header->value.len;
-    r->method_name.data = header->value.data;
+    r->method_name.len = value->len;
+    r->method_name.data = value->data;
 
     len = r->method_name.len;
     n = sizeof(tests) / sizeof(tests[0]);
@@ -3256,7 +3255,7 @@ ngx_http_v2_parse_method(ngx_http_reques
 
 
 static ngx_int_t
-ngx_http_v2_parse_scheme(ngx_http_request_t *r, ngx_http_v2_header_t *header)
+ngx_http_v2_parse_scheme(ngx_http_request_t *r, ngx_str_t *value)
 {
     if (r->schema_start) {
         ngx_log_error(NGX_LOG_INFO, r->connection->log, 0,
@@ -3265,22 +3264,22 @@ ngx_http_v2_parse_scheme(ngx_http_reques
         return NGX_DECLINED;
     }
 
-    if (header->value.len == 0) {
+    if (value->len == 0) {
         ngx_log_error(NGX_LOG_INFO, r->connection->log, 0,
                       "client sent empty :scheme header");
 
         return NGX_DECLINED;
     }
 
-    r->schema_start = header->value.data;
-    r->schema_end = header->value.data + header->value.len;
+    r->schema_start = value->data;
+    r->schema_end = value->data + value->len;
 
     return NGX_OK;
 }
 
 
 static ngx_int_t
-ngx_http_v2_parse_authority(ngx_http_request_t *r, ngx_http_v2_header_t *header)
+ngx_http_v2_parse_authority(ngx_http_request_t *r, ngx_str_t *value)
 {
     ngx_table_elt_t            *h;
     ngx_http_header_t          *hh;
@@ -3298,8 +3297,8 @@ ngx_http_v2_parse_authority(ngx_http_req
     h->key.len = host.len;
     h->key.data = host.data;
 
-    h->value.len = header->value.len;
-    h->value.data = header->value.data;
+    h->value.len = value->len;
+    h->value.data = value->data;
 
     h->lowcase_key = host.data;
 

From ru at nginx.com  Thu Feb  8 06:58:11 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 08 Feb 2018 06:58:11 +0000
Subject: [nginx] HTTP/2: server push.
Message-ID: <hg.641306096f5b.1518073091.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/641306096f5b
branches:  
changeset: 7201:641306096f5b
user:      Ruslan Ermilov <ru at nginx.com>
date:      Thu Feb 08 09:55:03 2018 +0300
description:
HTTP/2: server push.

Resources to be pushed are configured with the "http2_push" directive.

Also, preload links from the Link response headers, as described in
https://www.w3.org/TR/preload/#server-push-http-2, can be pushed, if
enabled with the "http2_push_preload" directive.

Only relative URIs with absolute paths can be pushed.

The number of concurrent pushes is normally limited by a client, but
cannot exceed a hard limit set by the "http2_max_concurrent_pushes"
directive.

diffstat:

 src/http/v2/ngx_http_v2.c               |  216 +++++++++++-
 src/http/v2/ngx_http_v2.h               |   11 +
 src/http/v2/ngx_http_v2_filter_module.c |  559 +++++++++++++++++++++++++++++++-
 src/http/v2/ngx_http_v2_module.c        |  101 +++++
 src/http/v2/ngx_http_v2_module.h        |    6 +
 5 files changed, 874 insertions(+), 19 deletions(-)

diffs (truncated from 1234 to 1000 lines):

diff -r cadb43014c7c -r 641306096f5b src/http/v2/ngx_http_v2.c
--- a/src/http/v2/ngx_http_v2.c	Thu Feb 08 09:54:49 2018 +0300
+++ b/src/http/v2/ngx_http_v2.c	Thu Feb 08 09:55:03 2018 +0300
@@ -35,12 +35,11 @@
 #define NGX_HTTP_V2_GOAWAY_SIZE                  8
 #define NGX_HTTP_V2_WINDOW_UPDATE_SIZE           4
 
-#define NGX_HTTP_V2_STREAM_ID_SIZE               4
-
 #define NGX_HTTP_V2_SETTINGS_PARAM_SIZE          6
 
 /* settings fields */
 #define NGX_HTTP_V2_HEADER_TABLE_SIZE_SETTING    0x1
+#define NGX_HTTP_V2_ENABLE_PUSH_SETTING          0x2
 #define NGX_HTTP_V2_MAX_STREAMS_SETTING          0x3
 #define NGX_HTTP_V2_INIT_WINDOW_SIZE_SETTING     0x4
 #define NGX_HTTP_V2_MAX_FRAME_SIZE_SETTING       0x5
@@ -121,7 +120,7 @@ static ngx_int_t ngx_http_v2_parse_int(n
     u_char **pos, u_char *end, ngx_uint_t prefix);
 
 static ngx_http_v2_stream_t *ngx_http_v2_create_stream(
-    ngx_http_v2_connection_t *h2c);
+    ngx_http_v2_connection_t *h2c, ngx_uint_t push);
 static ngx_http_v2_node_t *ngx_http_v2_get_node_by_id(
     ngx_http_v2_connection_t *h2c, ngx_uint_t sid, ngx_uint_t alloc);
 static ngx_http_v2_node_t *ngx_http_v2_get_closed_node(
@@ -162,6 +161,7 @@ static ngx_int_t ngx_http_v2_cookie(ngx_
     ngx_http_v2_header_t *header);
 static ngx_int_t ngx_http_v2_construct_cookie_header(ngx_http_request_t *r);
 static void ngx_http_v2_run_request(ngx_http_request_t *r);
+static void ngx_http_v2_run_request_handler(ngx_event_t *ev);
 static ngx_int_t ngx_http_v2_process_request_body(ngx_http_request_t *r,
     u_char *pos, size_t size, ngx_uint_t last);
 static ngx_int_t ngx_http_v2_filter_request_body(ngx_http_request_t *r);
@@ -249,6 +249,8 @@ ngx_http_v2_init(ngx_event_t *rev)
 
     h2scf = ngx_http_get_module_srv_conf(hc->conf_ctx, ngx_http_v2_module);
 
+    h2c->concurrent_pushes = h2scf->concurrent_pushes;
+
     h2c->pool = ngx_create_pool(h2scf->pool_size, h2c->connection->log);
     if (h2c->pool == NULL) {
         ngx_http_close_connection(c);
@@ -366,7 +368,9 @@ ngx_http_v2_read_handler(ngx_event_t *re
             break;
         }
 
-        if (n == 0 && (h2c->state.incomplete || h2c->processing)) {
+        if (n == 0
+            && (h2c->state.incomplete || h2c->processing || h2c->pushing))
+        {
             ngx_log_error(NGX_LOG_INFO, c->log, 0,
                           "client prematurely closed connection");
         }
@@ -405,7 +409,7 @@ ngx_http_v2_read_handler(ngx_event_t *re
 
     h2c->blocked = 0;
 
-    if (h2c->processing) {
+    if (h2c->processing || h2c->pushing) {
         if (rev->timer_set) {
             ngx_del_timer(rev);
         }
@@ -589,7 +593,7 @@ ngx_http_v2_handle_connection(ngx_http_v
     ngx_connection_t        *c;
     ngx_http_v2_srv_conf_t  *h2scf;
 
-    if (h2c->last_out || h2c->processing) {
+    if (h2c->last_out || h2c->processing || h2c->pushing) {
         return;
     }
 
@@ -1123,7 +1127,7 @@ ngx_http_v2_state_headers(ngx_http_v2_co
         h2c->closed_nodes--;
     }
 
-    stream = ngx_http_v2_create_stream(h2c);
+    stream = ngx_http_v2_create_stream(h2c, 0);
     if (stream == NULL) {
         return ngx_http_v2_connection_error(h2c, NGX_HTTP_V2_INTERNAL_ERROR);
     }
@@ -1909,6 +1913,11 @@ ngx_http_v2_state_rst_stream(ngx_http_v2
                       "client canceled stream %ui", h2c->state.sid);
         break;
 
+    case NGX_HTTP_V2_REFUSED_STREAM:
+        ngx_log_error(NGX_LOG_INFO, fc->log, 0,
+                      "client refused stream %ui", h2c->state.sid);
+        break;
+
     case NGX_HTTP_V2_INTERNAL_ERROR:
         ngx_log_error(NGX_LOG_INFO, fc->log, 0,
                       "client terminated stream %ui due to internal error",
@@ -1966,6 +1975,7 @@ ngx_http_v2_state_settings_params(ngx_ht
 {
     ssize_t                   window_delta;
     ngx_uint_t                id, value;
+    ngx_http_v2_srv_conf_t   *h2scf;
     ngx_http_v2_out_frame_t  *frame;
 
     window_delta = 0;
@@ -2016,6 +2026,27 @@ ngx_http_v2_state_settings_params(ngx_ht
             h2c->frame_size = value;
             break;
 
+        case NGX_HTTP_V2_ENABLE_PUSH_SETTING:
+
+            if (value > 1) {
+                ngx_log_error(NGX_LOG_INFO, h2c->connection->log, 0,
+                              "client sent SETTINGS frame with incorrect "
+                              "ENABLE_PUSH value %ui", value);
+
+                return ngx_http_v2_connection_error(h2c,
+                                                    NGX_HTTP_V2_PROTOCOL_ERROR);
+            }
+
+            h2c->push_disabled = !value;
+            break;
+
+        case NGX_HTTP_V2_MAX_STREAMS_SETTING:
+            h2scf = ngx_http_get_module_srv_conf(h2c->http_connection->conf_ctx,
+                                                 ngx_http_v2_module);
+
+            h2c->concurrent_pushes = ngx_min(value, h2scf->concurrent_pushes);
+            break;
+
         default:
             break;
         }
@@ -2483,6 +2514,119 @@ ngx_http_v2_parse_int(ngx_http_v2_connec
 }
 
 
+ngx_int_t
+ngx_http_v2_push_stream(ngx_http_v2_connection_t *h2c, ngx_uint_t depend,
+    size_t request_length, ngx_str_t *path, ngx_str_t *authority)
+{
+    ngx_int_t              rc;
+    ngx_str_t              value;
+    ngx_connection_t      *fc;
+    ngx_http_request_t    *r;
+    ngx_http_v2_node_t    *node;
+    ngx_http_v2_stream_t  *stream;
+
+    node = ngx_http_v2_get_node_by_id(h2c, h2c->last_push, 1);
+
+    if (node == NULL) {
+        return NGX_ERROR;
+    }
+
+    if (node->parent) {
+        ngx_queue_remove(&node->reuse);
+        h2c->closed_nodes--;
+    }
+
+    stream = ngx_http_v2_create_stream(h2c, 1);
+    if (stream == NULL) {
+        return NGX_ERROR;
+    }
+
+    stream->pool = ngx_create_pool(1024, h2c->connection->log);
+    if (stream->pool == NULL) {
+        return NGX_ERROR;
+    }
+
+    r = stream->request;
+    fc = r->connection;
+
+    r->request_length = request_length;
+
+    stream->in_closed = 1;
+    stream->node = node;
+
+    node->stream = stream;
+
+    ngx_log_debug2(NGX_LOG_DEBUG_HTTP, h2c->connection->log, 0,
+                   "http2 push stream sid:%ui "
+                   "depends on %ui excl:0 weight:16",
+                   h2c->last_push, depend);
+
+    node->weight = NGX_HTTP_V2_DEFAULT_WEIGHT;
+    ngx_http_v2_set_dependency(h2c, node, depend, 0);
+
+    r->method_name = ngx_http_core_get_method;
+    r->method = NGX_HTTP_GET;
+
+    r->schema_start = (u_char *) "https";
+
+#if (NGX_HTTP_SSL)
+    if (fc->ssl) {
+        r->schema_end = r->schema_start + 5;
+
+    } else
+#endif
+    {
+        r->schema_end = r->schema_start + 4;
+    }
+
+    value.len = authority->len;
+
+    value.data = ngx_pstrdup(stream->pool, authority);
+    if (value.data == NULL) {
+        return NGX_ERROR;
+    }
+
+    rc = ngx_http_v2_parse_authority(r, &value);
+
+    if (rc != NGX_OK) {
+        goto error;
+    }
+
+    value.len = path->len;
+
+    value.data = ngx_pstrdup(stream->pool, path);
+    if (value.data == NULL) {
+        return NGX_ERROR;
+    }
+
+    rc = ngx_http_v2_parse_path(r, &value);
+
+    if (rc != NGX_OK) {
+        goto error;
+    }
+
+    fc->write->handler = ngx_http_v2_run_request_handler;
+    ngx_post_event(fc->write, &ngx_posted_events);
+
+    return NGX_OK;
+
+error:
+
+    if (rc == NGX_ABORT) {
+        return NGX_ERROR;
+    }
+
+    if (rc == NGX_DECLINED) {
+        ngx_http_finalize_request(r, NGX_HTTP_BAD_REQUEST);
+        return NGX_ERROR;
+    }
+
+    (void) ngx_http_v2_connection_error(h2c, NGX_HTTP_V2_INTERNAL_ERROR);
+
+    return NGX_ERROR;
+}
+
+
 static ngx_int_t
 ngx_http_v2_send_settings(ngx_http_v2_connection_t *h2c)
 {
@@ -2743,7 +2887,7 @@ ngx_http_v2_frame_handler(ngx_http_v2_co
 
 
 static ngx_http_v2_stream_t *
-ngx_http_v2_create_stream(ngx_http_v2_connection_t *h2c)
+ngx_http_v2_create_stream(ngx_http_v2_connection_t *h2c, ngx_uint_t push)
 {
     ngx_log_t                 *log;
     ngx_event_t               *rev, *wev;
@@ -2798,7 +2942,13 @@ ngx_http_v2_create_stream(ngx_http_v2_co
     ngx_memcpy(log, h2c->connection->log, sizeof(ngx_log_t));
 
     log->data = ctx;
-    log->action = "reading client request headers";
+
+    if (push) {
+        log->action = "processing pushed request headers";
+
+    } else {
+        log->action = "reading client request headers";
+    }
 
     ngx_memzero(rev, sizeof(ngx_event_t));
 
@@ -2870,7 +3020,12 @@ ngx_http_v2_create_stream(ngx_http_v2_co
     stream->send_window = h2c->init_window;
     stream->recv_window = h2scf->preread_size;
 
-    h2c->processing++;
+    if (push) {
+        h2c->pushing++;
+
+    } else {
+        h2c->processing++;
+    }
 
     return stream;
 }
@@ -3532,6 +3687,22 @@ ngx_http_v2_run_request(ngx_http_request
 }
 
 
+static void
+ngx_http_v2_run_request_handler(ngx_event_t *ev)
+{
+    ngx_connection_t    *fc;
+    ngx_http_request_t  *r;
+
+    fc = ev->data;
+    r = fc->data;
+
+    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                   "http2 run request handler");
+
+    ngx_http_v2_run_request(r);
+}
+
+
 ngx_int_t
 ngx_http_v2_read_request_body(ngx_http_request_t *r)
 {
@@ -4003,6 +4174,7 @@ void
 ngx_http_v2_close_stream(ngx_http_v2_stream_t *stream, ngx_int_t rc)
 {
     ngx_pool_t                *pool;
+    ngx_uint_t                 push;
     ngx_event_t               *ev;
     ngx_connection_t          *fc;
     ngx_http_v2_node_t        *node;
@@ -4011,9 +4183,10 @@ ngx_http_v2_close_stream(ngx_http_v2_str
     h2c = stream->connection;
     node = stream->node;
 
-    ngx_log_debug3(NGX_LOG_DEBUG_HTTP, h2c->connection->log, 0,
-                   "http2 close stream %ui, queued %ui, processing %ui",
-                   node->id, stream->queued, h2c->processing);
+    ngx_log_debug4(NGX_LOG_DEBUG_HTTP, h2c->connection->log, 0,
+                   "http2 close stream %ui, queued %ui, "
+                   "processing %ui, pushing %ui",
+                   node->id, stream->queued, h2c->processing, h2c->pushing);
 
     fc = stream->request->connection;
 
@@ -4069,6 +4242,8 @@ ngx_http_v2_close_stream(ngx_http_v2_str
         h2c->state.stream = NULL;
     }
 
+    push = stream->node->id % 2 == 0;
+
     node->stream = NULL;
 
     ngx_queue_insert_tail(&h2c->closed, &node->reuse);
@@ -4116,9 +4291,14 @@ ngx_http_v2_close_stream(ngx_http_v2_str
     fc->data = h2c->free_fake_connections;
     h2c->free_fake_connections = fc;
 
-    h2c->processing--;
-
-    if (h2c->processing || h2c->blocked) {
+    if (push) {
+        h2c->pushing--;
+
+    } else {
+        h2c->processing--;
+    }
+
+    if (h2c->processing || h2c->pushing || h2c->blocked) {
         return;
     }
 
@@ -4267,7 +4447,7 @@ ngx_http_v2_finalize_connection(ngx_http
 
     c->error = 1;
 
-    if (!h2c->processing) {
+    if (!h2c->processing && !h2c->pushing) {
         ngx_http_close_connection(c);
         return;
     }
@@ -4316,7 +4496,7 @@ ngx_http_v2_finalize_connection(ngx_http
 
     h2c->blocked = 0;
 
-    if (h2c->processing) {
+    if (h2c->processing || h2c->pushing) {
         return;
     }
 
diff -r cadb43014c7c -r 641306096f5b src/http/v2/ngx_http_v2.h
--- a/src/http/v2/ngx_http_v2.h	Thu Feb 08 09:54:49 2018 +0300
+++ b/src/http/v2/ngx_http_v2.h	Thu Feb 08 09:55:03 2018 +0300
@@ -24,6 +24,8 @@
 #define NGX_HTTP_V2_MAX_FIELD                                                 \
     (127 + (1 << (NGX_HTTP_V2_INT_OCTETS - 1) * 7) - 1)
 
+#define NGX_HTTP_V2_STREAM_ID_SIZE       4
+
 #define NGX_HTTP_V2_FRAME_HEADER_SIZE    9
 
 /* frame types */
@@ -118,6 +120,9 @@ struct ngx_http_v2_connection_s {
 
     ngx_uint_t                       processing;
 
+    ngx_uint_t                       pushing;
+    ngx_uint_t                       concurrent_pushes;
+
     size_t                           send_window;
     size_t                           recv_window;
     size_t                           init_window;
@@ -143,12 +148,14 @@ struct ngx_http_v2_connection_s {
     ngx_queue_t                      closed;
 
     ngx_uint_t                       last_sid;
+    ngx_uint_t                       last_push;
 
     unsigned                         closed_nodes:8;
     unsigned                         settings_ack:1;
     unsigned                         table_update:1;
     unsigned                         blocked:1;
     unsigned                         goaway:1;
+    unsigned                         push_disabled:1;
 };
 
 
@@ -276,6 +283,10 @@ void ngx_http_v2_init(ngx_event_t *rev);
 ngx_int_t ngx_http_v2_read_request_body(ngx_http_request_t *r);
 ngx_int_t ngx_http_v2_read_unbuffered_request_body(ngx_http_request_t *r);
 
+ngx_int_t ngx_http_v2_push_stream(ngx_http_v2_connection_t *h2c,
+    ngx_uint_t depend, size_t request_length, ngx_str_t *path,
+    ngx_str_t *authority);
+
 void ngx_http_v2_close_stream(ngx_http_v2_stream_t *stream, ngx_int_t rc);
 
 ngx_int_t ngx_http_v2_send_output_queue(ngx_http_v2_connection_t *h2c);
diff -r cadb43014c7c -r 641306096f5b src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c	Thu Feb 08 09:54:49 2018 +0300
+++ b/src/http/v2/ngx_http_v2_filter_module.c	Thu Feb 08 09:55:03 2018 +0300
@@ -2,6 +2,7 @@
 /*
  * Copyright (C) Nginx, Inc.
  * Copyright (C) Valentin V. Bartenev
+ * Copyright (C) Ruslan Ermilov
  */
 
 
@@ -33,6 +34,13 @@
 #define NGX_HTTP_V2_ENCODE_RAW            0
 #define NGX_HTTP_V2_ENCODE_HUFF           0x80
 
+#define NGX_HTTP_V2_AUTHORITY_INDEX       1
+#define NGX_HTTP_V2_METHOD_GET_INDEX      2
+#define NGX_HTTP_V2_PATH_INDEX            4
+
+#define NGX_HTTP_V2_SCHEME_HTTP_INDEX     6
+#define NGX_HTTP_V2_SCHEME_HTTPS_INDEX    7
+
 #define NGX_HTTP_V2_STATUS_INDEX          8
 #define NGX_HTTP_V2_STATUS_200_INDEX      8
 #define NGX_HTTP_V2_STATUS_204_INDEX      9
@@ -53,12 +61,18 @@
 #define NGX_HTTP_V2_NO_TRAILERS           (ngx_http_v2_out_frame_t *) -1
 
 
+static ngx_int_t ngx_http_v2_push_resources(ngx_http_request_t *r);
+static ngx_int_t ngx_http_v2_push_resource(ngx_http_request_t *r,
+    ngx_str_t *path, ngx_str_t *authority);
+
 static u_char *ngx_http_v2_string_encode(u_char *dst, u_char *src, size_t len,
     u_char *tmp, ngx_uint_t lower);
 static u_char *ngx_http_v2_write_int(u_char *pos, ngx_uint_t prefix,
     ngx_uint_t value);
 static ngx_http_v2_out_frame_t *ngx_http_v2_create_headers_frame(
     ngx_http_request_t *r, u_char *pos, u_char *end, ngx_uint_t fin);
+static ngx_http_v2_out_frame_t *ngx_http_v2_create_push_frame(
+    ngx_http_request_t *r, u_char *pos, u_char *end);
 static ngx_http_v2_out_frame_t *ngx_http_v2_create_trailers_frame(
     ngx_http_request_t *r);
 
@@ -81,6 +95,8 @@ static ngx_inline ngx_int_t ngx_http_v2_
 
 static ngx_int_t ngx_http_v2_headers_frame_handler(
     ngx_http_v2_connection_t *h2c, ngx_http_v2_out_frame_t *frame);
+static ngx_int_t ngx_http_v2_push_frame_handler(
+    ngx_http_v2_connection_t *h2c, ngx_http_v2_out_frame_t *frame);
 static ngx_int_t ngx_http_v2_data_frame_handler(
     ngx_http_v2_connection_t *h2c, ngx_http_v2_out_frame_t *frame);
 static ngx_inline void ngx_http_v2_handle_frame(
@@ -241,6 +257,15 @@ ngx_http_v2_header_filter(ngx_http_reque
 
     h2c = stream->connection;
 
+    if (!h2c->push_disabled && !h2c->goaway
+        && stream->node->id % 2 == 1
+        && r->method != NGX_HTTP_HEAD)
+    {
+        if (ngx_http_v2_push_resources(r) != NGX_OK) {
+            return NGX_ERROR;
+        }
+    }
+
     len = h2c->table_update ? 1 : 0;
 
     len += status ? 1 : 1 + ngx_http_v2_literal_size("418");
@@ -638,7 +663,7 @@ ngx_http_v2_header_filter(ngx_http_reque
 
     ngx_http_v2_queue_blocked_frame(h2c, frame);
 
-    stream->queued = 1;
+    stream->queued++;
 
     cln = ngx_http_cleanup_add(r, 0);
     if (cln == NULL) {
@@ -655,6 +680,365 @@ ngx_http_v2_header_filter(ngx_http_reque
 }
 
 
+static ngx_int_t
+ngx_http_v2_push_resources(ngx_http_request_t *r)
+{
+    u_char                     *start, *end, *last;
+    ngx_int_t                   rc;
+    ngx_str_t                   path, authority;
+    ngx_uint_t                  i, push;
+    ngx_table_elt_t           **h;
+    ngx_connection_t           *fc;
+    ngx_http_v2_stream_t       *stream;
+    ngx_http_v2_loc_conf_t     *h2lcf;
+    ngx_http_v2_connection_t   *h2c;
+    ngx_http_complex_value_t   *pushes;
+
+    fc = r->connection;
+
+    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, fc->log, 0, "http2 push resources");
+
+    stream = r->stream;
+    h2c = stream->connection;
+
+    ngx_str_null(&authority);
+
+    h2lcf = ngx_http_get_module_loc_conf(r, ngx_http_v2_module);
+
+    if (h2lcf->pushes) {
+        pushes = h2lcf->pushes->elts;
+
+        for (i = 0; i < h2lcf->pushes->nelts; i++) {
+
+            if (ngx_http_complex_value(r, &pushes[i], &path) != NGX_OK) {
+                return NGX_ERROR;
+            }
+
+            if (path.len == 0) {
+                continue;
+            }
+
+            if (path.len == 3 && ngx_strncmp(path.data, "off", 3) == 0) {
+                continue;
+            }
+
+            rc = ngx_http_v2_push_resource(r, &path, &authority);
+
+            if (rc == NGX_ERROR) {
+                return NGX_ERROR;
+            }
+
+            if (rc == NGX_ABORT) {
+                return NGX_OK;
+            }
+
+            /* NGX_OK, NGX_DECLINED */
+        }
+    }
+
+    if (!h2lcf->push_preload) {
+        return NGX_OK;
+    }
+
+    h = r->headers_out.link.elts;
+
+    for (i = 0; i < r->headers_out.link.nelts; i++) {
+
+        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 parse link: \"%V\"", &h[i]->value);
+
+        start = h[i]->value.data;
+        end = h[i]->value.data + h[i]->value.len;
+
+    next_link:
+
+        while (start < end && *start == ' ') { start++; }
+
+        if (start == end || *start++ != '<') {
+            continue;
+        }
+
+        while (start < end && *start == ' ') { start++; }
+
+        for (last = start; last < end && *last != '>'; last++) {
+            /* void */
+        }
+
+        if (last == start || last == end) {
+            continue;
+        }
+
+        path.len = last - start;
+        path.data = start;
+
+        start = last + 1;
+
+        while (start < end && *start == ' ') { start++; }
+
+        if (start == end) {
+            continue;
+        }
+
+        if (*start == ',') {
+            start++;
+            goto next_link;
+        }
+
+        if (*start++ != ';') {
+            continue;
+        }
+
+        last = ngx_strlchr(start, end, ',');
+
+        if (last == NULL) {
+            last = end;
+        }
+
+        push = 0;
+
+        for ( ;; ) {
+
+            while (start < last && *start == ' ') { start++; }
+
+            if (last - start >= 6
+                && ngx_strncasecmp(start, (u_char *) "nopush", 6) == 0)
+            {
+                start += 6;
+
+                if (start == last || *start == ' ' || *start == ';') {
+                    push = 0;
+                    break;
+                }
+
+                goto next_param;
+            }
+
+            if (last - start >= 11
+                && ngx_strncasecmp(start, (u_char *) "rel=preload", 11) == 0)
+            {
+                start += 11;
+
+                if (start == last || *start == ' ' || *start == ';') {
+                    push = 1;
+                }
+
+                goto next_param;
+            }
+
+            if (last - start >= 4
+                && ngx_strncasecmp(start, (u_char *) "rel=", 4) == 0)
+            {
+                start += 4;
+
+                while (start < last && *start == ' ') { start++; }
+
+                if (start == last || *start++ != '"') {
+                    goto next_param;
+                }
+
+                for ( ;; ) {
+
+                    while (start < last && *start == ' ') { start++; }
+
+                    if (last - start >= 7
+                        && ngx_strncasecmp(start, (u_char *) "preload", 7) == 0)
+                    {
+                        start += 7;
+
+                        if (start < last && (*start == ' ' || *start == '"')) {
+                            push = 1;
+                            break;
+                        }
+                    }
+
+                    while (start < last && *start != ' ' && *start != '"') {
+                        start++;
+                    }
+
+                    if (start == last) {
+                        break;
+                    }
+
+                    if (*start == '"') {
+                        break;
+                    }
+
+                    start++;
+                }
+            }
+
+        next_param:
+
+            start = ngx_strlchr(start, last, ';');
+
+            if (start == NULL) {
+                break;
+            }
+
+            start++;
+        }
+
+        if (push) {
+            while (path.len && path.data[path.len - 1] == ' ') {
+                path.len--;
+            }
+        }
+
+        if (push && path.len
+            && !(path.len > 1 && path.data[0] == '/' && path.data[1] == '/'))
+        {
+            rc = ngx_http_v2_push_resource(r, &path, &authority);
+
+            if (rc == NGX_ERROR) {
+                return NGX_ERROR;
+            }
+
+            if (rc == NGX_ABORT) {
+                return NGX_OK;
+            }
+
+            /* NGX_OK, NGX_DECLINED */
+        }
+
+        if (last < end) {
+            start = last + 1;
+            goto next_link;
+        }
+    }
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_http_v2_push_resource(ngx_http_request_t *r, ngx_str_t *path,
+    ngx_str_t *authority)
+{
+    u_char                    *start, *pos, *tmp;
+    size_t                     len;
+    ngx_table_elt_t           *host;
+    ngx_connection_t          *fc;
+    ngx_http_v2_stream_t      *stream;
+    ngx_http_v2_out_frame_t   *frame;
+    ngx_http_v2_connection_t  *h2c;
+
+    fc = r->connection;
+
+    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, fc->log, 0, "http2 push resource");
+
+    stream = r->stream;
+    h2c = stream->connection;
+
+    if (!ngx_path_separator(path->data[0])) {
+        ngx_log_error(NGX_LOG_WARN, fc->log, 0,
+                      "non-absolute path \"%V\" not pushed", path);
+        return NGX_DECLINED;
+    }
+
+    ngx_log_debug2(NGX_LOG_DEBUG_HTTP, h2c->connection->log, 0,
+                  "http2 pushing:%ui limit:%ui",
+                  h2c->pushing, h2c->concurrent_pushes);
+
+    if (h2c->pushing >= h2c->concurrent_pushes) {
+        return NGX_ABORT;
+    }
+
+    if (h2c->last_push == 0x7ffffffe) {
+        return NGX_ABORT;
+    }
+
+    if (path->len > NGX_HTTP_V2_MAX_FIELD) {
+        return NGX_DECLINED;
+    }
+
+    host = r->headers_in.host;
+
+    if (authority->len == 0 && host) {
+
+        len = 1 + NGX_HTTP_V2_INT_OCTETS + host->value.len;
+
+        tmp = ngx_palloc(r->pool, len);
+        pos = ngx_pnalloc(r->pool, len);
+
+        if (pos == NULL || tmp == NULL) {
+            return NGX_ERROR;
+        }
+
+        authority->data = pos;
+
+        *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_AUTHORITY_INDEX);
+        pos = ngx_http_v2_write_value(pos, host->value.data, host->value.len,
+                                      tmp);
+
+        authority->len = pos - authority->data;
+    }
+
+    len = (h2c->table_update ? 1 : 0)
+          + 1
+          + 1 + NGX_HTTP_V2_INT_OCTETS + path->len
+          + authority->len
+          + 1;
+
+    tmp = ngx_palloc(r->pool, len);
+    pos = ngx_pnalloc(r->pool, len);
+
+    if (pos == NULL || tmp == NULL) {
+        return NGX_ERROR;
+    }
+
+    start = pos;
+
+    if (h2c->table_update) {
+        ngx_log_debug0(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 table size update: 0");
+        *pos++ = (1 << 5) | 0;
+        h2c->table_update = 0;
+    }
+
+    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                   "http2 push header: \":method: GET\"");
+
+    *pos++ = ngx_http_v2_indexed(NGX_HTTP_V2_METHOD_GET_INDEX);
+
+    ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                   "http2 push header: \":path: %V\"", path);
+
+    *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_PATH_INDEX);
+    pos = ngx_http_v2_write_value(pos, path->data, path->len, tmp);
+
+    ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                   "http2 push header: \":authority: %V\"", &host->value);
+
+    pos = ngx_cpymem(pos, authority->data, authority->len);
+
+#if (NGX_HTTP_SSL)
+    if (fc->ssl) {
+        ngx_log_debug0(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 push header: \":scheme: https\"");
+        *pos++ = ngx_http_v2_indexed(NGX_HTTP_V2_SCHEME_HTTPS_INDEX);
+
+    } else
+#endif
+    {
+        ngx_log_debug0(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 push header: \":scheme: http\"");
+        *pos++ = ngx_http_v2_indexed(NGX_HTTP_V2_SCHEME_HTTP_INDEX);
+    }
+
+    frame = ngx_http_v2_create_push_frame(r, start, pos);
+    if (frame == NULL) {
+        return NGX_ERROR;
+    }
+
+    ngx_http_v2_queue_blocked_frame(h2c, frame);
+
+    stream->queued++;
+
+    return ngx_http_v2_push_stream(h2c, stream->node->id, pos - start,
+                                   path, &host->value);
+}
+
+
 static u_char *
 ngx_http_v2_string_encode(u_char *dst, u_char *src, size_t len, u_char *tmp,
     ngx_uint_t lower)
@@ -809,6 +1193,125 @@ ngx_http_v2_create_headers_frame(ngx_htt
 
 
 static ngx_http_v2_out_frame_t *
+ngx_http_v2_create_push_frame(ngx_http_request_t *r, u_char *pos, u_char *end)
+{
+    u_char                     type, flags;
+    size_t                     rest, frame_size, len;
+    ngx_buf_t                 *b;
+    ngx_chain_t               *cl, **ll;
+    ngx_http_v2_stream_t      *stream;
+    ngx_http_v2_out_frame_t   *frame;
+    ngx_http_v2_connection_t  *h2c;
+
+    stream = r->stream;
+    h2c = stream->connection;
+    rest = NGX_HTTP_V2_STREAM_ID_SIZE + (end - pos);
+
+    frame = ngx_palloc(r->pool, sizeof(ngx_http_v2_out_frame_t));
+    if (frame == NULL) {
+        return NULL;
+    }
+
+    frame->handler = ngx_http_v2_push_frame_handler;
+    frame->stream = stream;
+    frame->length = rest;
+    frame->blocked = 1;
+    frame->fin = 0;
+
+    ll = &frame->first;
+
+    type = NGX_HTTP_V2_PUSH_PROMISE_FRAME;
+    flags = NGX_HTTP_V2_NO_FLAG;
+    frame_size = h2c->frame_size;
+
+    for ( ;; ) {
+        if (rest <= frame_size) {
+            frame_size = rest;
+            flags |= NGX_HTTP_V2_END_HEADERS_FLAG;
+        }
+
+        b = ngx_create_temp_buf(r->pool,
+                                NGX_HTTP_V2_FRAME_HEADER_SIZE
+                                + ((type == NGX_HTTP_V2_PUSH_PROMISE_FRAME)
+                                   ? NGX_HTTP_V2_STREAM_ID_SIZE : 0));
+        if (b == NULL) {
+            return NULL;
+        }
+
+        b->last = ngx_http_v2_write_len_and_type(b->last, frame_size, type);
+        *b->last++ = flags;
+        b->last = ngx_http_v2_write_sid(b->last, stream->node->id);
+
+        b->tag = (ngx_buf_tag_t) &ngx_http_v2_module;
+
+        if (type == NGX_HTTP_V2_PUSH_PROMISE_FRAME) {
+            h2c->last_push += 2;
+
+            b->last = ngx_http_v2_write_sid(b->last, h2c->last_push);
+            len = frame_size - NGX_HTTP_V2_STREAM_ID_SIZE;
+
+        } else {
+            len = frame_size;
+        }
+
+        cl = ngx_alloc_chain_link(r->pool);
+        if (cl == NULL) {
+            return NULL;
+        }
+
+        cl->buf = b;
+
+        *ll = cl;
+        ll = &cl->next;
+
+        b = ngx_calloc_buf(r->pool);
+        if (b == NULL) {
+            return NULL;
+        }
+
+        b->pos = pos;
+
+        pos += len;
+
+        b->last = pos;
+        b->start = b->pos;
+        b->end = b->last;
+        b->temporary = 1;
+
+        cl = ngx_alloc_chain_link(r->pool);
+        if (cl == NULL) {
+            return NULL;
+        }
+
+        cl->buf = b;
+
+        *ll = cl;
+        ll = &cl->next;
+
+        rest -= frame_size;
+
+        if (rest) {
+            frame->length += NGX_HTTP_V2_FRAME_HEADER_SIZE;
+
+            type = NGX_HTTP_V2_CONTINUATION_FRAME;
+            continue;
+        }
+
+        cl->next = NULL;
+        frame->last = cl;
+
+        ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
+                       "http2:%ui create PUSH_PROMISE frame %p: "
+                       "sid:%ui len:%uz",
+                       stream->node->id, frame, h2c->last_push,
+                       frame->length);
+
+        return frame;
+    }
+}
+
+
+static ngx_http_v2_out_frame_t *
 ngx_http_v2_create_trailers_frame(ngx_http_request_t *r)
 {
     u_char            *pos, *start, *tmp;
@@ -1377,6 +1880,60 @@ ngx_http_v2_headers_frame_handler(ngx_ht
 
 
 static ngx_int_t
+ngx_http_v2_push_frame_handler(ngx_http_v2_connection_t *h2c,

From ru at nginx.com  Thu Feb  8 09:12:41 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 08 Feb 2018 09:12:41 +0000
Subject: [nginx] HTTP/2: fixed build with -Werror=unused-but-set-variable.
Message-ID: <hg.a49af443656f.1518081161.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/a49af443656f
branches:  
changeset: 7202:a49af443656f
user:      Ruslan Ermilov <ru at nginx.com>
date:      Thu Feb 08 12:11:30 2018 +0300
description:
HTTP/2: fixed build with -Werror=unused-but-set-variable.

diffstat:

 src/http/v2/ngx_http_v2_filter_module.c |  13 +++----------
 1 files changed, 3 insertions(+), 10 deletions(-)

diffs (33 lines):

diff -r 641306096f5b -r a49af443656f src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c	Thu Feb 08 09:55:03 2018 +0300
+++ b/src/http/v2/ngx_http_v2_filter_module.c	Thu Feb 08 12:11:30 2018 +0300
@@ -688,18 +688,11 @@ ngx_http_v2_push_resources(ngx_http_requ
     ngx_str_t                   path, authority;
     ngx_uint_t                  i, push;
     ngx_table_elt_t           **h;
-    ngx_connection_t           *fc;
-    ngx_http_v2_stream_t       *stream;
     ngx_http_v2_loc_conf_t     *h2lcf;
-    ngx_http_v2_connection_t   *h2c;
     ngx_http_complex_value_t   *pushes;
 
-    fc = r->connection;
-
-    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, fc->log, 0, "http2 push resources");
-
-    stream = r->stream;
-    h2c = stream->connection;
+    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
+                   "http2 push resources");
 
     ngx_str_null(&authority);
 
@@ -744,7 +737,7 @@ ngx_http_v2_push_resources(ngx_http_requ
 
     for (i = 0; i < r->headers_out.link.nelts; i++) {
 
-        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                        "http2 parse link: \"%V\"", &h[i]->value);
 
         start = h[i]->value.data;

From alessandro at ghedini.me  Thu Feb  8 16:52:59 2018
From: alessandro at ghedini.me (Alessandro Ghedini)
Date: Thu, 08 Feb 2018 16:52:59 +0000
Subject: [PATCH] HTTP/2: expose function to push single resource to modules
Message-ID: <1bb98b06d5536dfc80a4.1518108779@mandy.ghedini.home>

# HG changeset patch
# User Alessandro Ghedini <alessandro at ghedini.me>
# Date 1518108716 0
#      Thu Feb 08 16:51:56 2018 +0000
# Branch expose-push
# Node ID 1bb98b06d5536dfc80a407aabd8d06f9309f8df6
# Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
HTTP/2: expose function to push single resource to modules.

This makes it possible for 3rd party modules to implement alternative
methods for deciding which resources to push to clients on a per-request
basis (e.g. by parsing HTML from the response body, by using a custom
Link header parser, ...).

No functional changes.

diff -r a49af443656f -r 1bb98b06d553 src/http/v2/ngx_http_v2.h
--- a/src/http/v2/ngx_http_v2.h	Thu Feb 08 12:11:30 2018 +0300
+++ b/src/http/v2/ngx_http_v2.h	Thu Feb 08 16:51:56 2018 +0000
@@ -304,6 +304,9 @@
 size_t ngx_http_v2_huff_encode(u_char *src, size_t len, u_char *dst,
     ngx_uint_t lower);
 
+ngx_int_t ngx_http_v2_push_resource(ngx_http_request_t *r, ngx_str_t *path,
+    ngx_str_t *authority);
+
 
 #define ngx_http_v2_prefix(bits)  ((1 << (bits)) - 1)
 
diff -r a49af443656f -r 1bb98b06d553 src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c	Thu Feb 08 12:11:30 2018 +0300
+++ b/src/http/v2/ngx_http_v2_filter_module.c	Thu Feb 08 16:51:56 2018 +0000
@@ -62,8 +62,6 @@
 
 
 static ngx_int_t ngx_http_v2_push_resources(ngx_http_request_t *r);
-static ngx_int_t ngx_http_v2_push_resource(ngx_http_request_t *r,
-    ngx_str_t *path, ngx_str_t *authority);
 
 static u_char *ngx_http_v2_string_encode(u_char *dst, u_char *src, size_t len,
     u_char *tmp, ngx_uint_t lower);
@@ -903,7 +901,7 @@
 }
 
 
-static ngx_int_t
+ngx_int_t
 ngx_http_v2_push_resource(ngx_http_request_t *r, ngx_str_t *path,
     ngx_str_t *authority)
 {

From alessandro at ghedini.me  Thu Feb  8 17:04:58 2018
From: alessandro at ghedini.me (Alessandro Ghedini)
Date: Thu, 08 Feb 2018 17:04:58 +0000
Subject: [PATCH] HTTP/2: copy additional headers in the pushed requests
Message-ID: <4f7f42e6d13add2ab0c7.1518109498@mandy.ghedini.home>

# HG changeset patch
# User Alessandro Ghedini <alessandro at cloudflare.com>
# Date 1518109032 0
#      Thu Feb 08 16:57:12 2018 +0000
# Branch push-copy-headers
# Node ID 4f7f42e6d13add2ab0c7a9654472bb74085181d1
# Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
HTTP/2: copy additional headers in the pushed requests.

To ensure pushed requests are processed consistently with the original
client request, some headers need to be copied from the original request
into the pushed one.

The headers currently copied are User-Agent, Accept, Accept-Language and
Accept-Encoding.

diff -r a49af443656f -r 4f7f42e6d13a src/http/v2/ngx_http_v2.c
--- a/src/http/v2/ngx_http_v2.c	Thu Feb 08 12:11:30 2018 +0300
+++ b/src/http/v2/ngx_http_v2.c	Thu Feb 08 16:57:12 2018 +0000
@@ -156,6 +156,8 @@
     ngx_str_t *value);
 static ngx_int_t ngx_http_v2_parse_authority(ngx_http_request_t *r,
     ngx_str_t *value);
+static ngx_int_t ngx_http_v2_copy_header(ngx_http_request_t *r,
+    ngx_table_elt_t *hdr);
 static ngx_int_t ngx_http_v2_construct_request_line(ngx_http_request_t *r);
 static ngx_int_t ngx_http_v2_cookie(ngx_http_request_t *r,
     ngx_http_v2_header_t *header);
@@ -2516,7 +2518,8 @@
 
 ngx_int_t
 ngx_http_v2_push_stream(ngx_http_v2_connection_t *h2c, ngx_uint_t depend,
-    size_t request_length, ngx_str_t *path, ngx_str_t *authority)
+    size_t request_length, ngx_str_t *path, ngx_str_t *authority,
+    ngx_http_headers_in_t *headers_in)
 {
     ngx_int_t              rc;
     ngx_str_t              value;
@@ -2605,6 +2608,34 @@
         goto error;
     }
 
+    rc = ngx_http_v2_copy_header(r, headers_in->user_agent);
+
+    if (rc != NGX_OK) {
+        goto error;
+    }
+
+#if (NGX_HTTP_HEADERS)
+    rc = ngx_http_v2_copy_header(r, headers_in->accept);
+
+    if (rc != NGX_OK) {
+        goto error;
+    }
+
+    rc = ngx_http_v2_copy_header(r, headers_in->accept_language);
+
+    if (rc != NGX_OK) {
+        goto error;
+    }
+#endif
+
+#if (NGX_HTTP_GZIP)
+    rc = ngx_http_v2_copy_header(r, headers_in->accept_encoding);
+
+    if (rc != NGX_OK) {
+        goto error;
+    }
+#endif
+
     fc->write->handler = ngx_http_v2_run_request_handler;
     ngx_post_event(fc->write, &ngx_posted_events);
 
@@ -3479,6 +3510,63 @@
 
 
 static ngx_int_t
+ngx_http_v2_copy_header(ngx_http_request_t *r, ngx_table_elt_t *hdr)
+{
+    ngx_table_elt_t            *h;
+    ngx_http_header_t          *hh;
+    ngx_http_core_main_conf_t  *cmcf;
+
+    if (hdr == NULL) {
+        return NGX_OK;
+    }
+
+    h = ngx_list_push(&r->headers_in.headers);
+    if (h == NULL) {
+        return NGX_ERROR;
+    }
+
+    h->hash = hdr->hash;
+
+    h->key.len = hdr->key.len;
+
+    h->key.data = ngx_pnalloc(r->stream->pool, h->key.len + 1);
+    if (h->key.data == NULL) {
+        h->hash = 0;
+        return NGX_ERROR;
+    }
+
+    (void) ngx_cpystrn(h->key.data, hdr->key.data, h->key.len + 1);
+
+    h->value.len = hdr->value.len;
+
+    h->value.data = ngx_pnalloc(r->stream->pool, h->value.len + 1);
+    if (h->key.data == NULL) {
+        h->hash = 0;
+        return NGX_ERROR;
+    }
+
+    (void) ngx_cpystrn(h->value.data, hdr->value.data, h->value.len + 1);
+
+    h->lowcase_key = h->key.data;
+
+    cmcf = ngx_http_get_module_main_conf(r, ngx_http_core_module);
+
+    hh = ngx_hash_find(&cmcf->headers_in_hash, h->hash,
+                       h->lowcase_key, h->key.len);
+
+    if (hh == NULL) {
+        return NGX_ERROR;
+    }
+
+    if (hh->handler(r, h, hh->offset) != NGX_OK) {
+        return NGX_ERROR;
+    }
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
 ngx_http_v2_construct_request_line(ngx_http_request_t *r)
 {
     u_char  *p;
diff -r a49af443656f -r 4f7f42e6d13a src/http/v2/ngx_http_v2.h
--- a/src/http/v2/ngx_http_v2.h	Thu Feb 08 12:11:30 2018 +0300
+++ b/src/http/v2/ngx_http_v2.h	Thu Feb 08 16:57:12 2018 +0000
@@ -285,7 +285,7 @@
 
 ngx_int_t ngx_http_v2_push_stream(ngx_http_v2_connection_t *h2c,
     ngx_uint_t depend, size_t request_length, ngx_str_t *path,
-    ngx_str_t *authority);
+    ngx_str_t *authority, ngx_http_headers_in_t *headers_in);
 
 void ngx_http_v2_close_stream(ngx_http_v2_stream_t *stream, ngx_int_t rc);
 
diff -r a49af443656f -r 4f7f42e6d13a src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c	Thu Feb 08 12:11:30 2018 +0300
+++ b/src/http/v2/ngx_http_v2_filter_module.c	Thu Feb 08 16:57:12 2018 +0000
@@ -972,6 +972,34 @@
           + authority->len
           + 1;
 
+    if (r->headers_in.user_agent) {
+        len += 1 + NGX_HTTP_V2_INT_OCTETS + r->headers_in.user_agent->key.len
+                 + NGX_HTTP_V2_INT_OCTETS + r->headers_in.user_agent->value.len;
+    }
+
+#if (NGX_HTTP_HEADERS)
+    if (r->headers_in.accept) {
+        len += 1 + NGX_HTTP_V2_INT_OCTETS + r->headers_in.accept->key.len
+                 + NGX_HTTP_V2_INT_OCTETS + r->headers_in.accept->value.len;
+    }
+
+    if (r->headers_in.accept_language) {
+        len += 1 + NGX_HTTP_V2_INT_OCTETS
+                 + r->headers_in.accept_language->key.len
+                 + NGX_HTTP_V2_INT_OCTETS
+                 + r->headers_in.accept_language->value.len;
+    }
+#endif
+
+#if (NGX_HTTP_GZIP)
+    if (r->headers_in.accept_encoding) {
+        len += 1 + NGX_HTTP_V2_INT_OCTETS
+                 + r->headers_in.accept_encoding->key.len
+                 + NGX_HTTP_V2_INT_OCTETS
+                 + r->headers_in.accept_encoding->value.len;
+    }
+#endif
+
     tmp = ngx_palloc(r->pool, len);
     pos = ngx_pnalloc(r->pool, len);
 
@@ -1018,6 +1046,66 @@
         *pos++ = ngx_http_v2_indexed(NGX_HTTP_V2_SCHEME_HTTP_INDEX);
     }
 
+    if (r->headers_in.user_agent) {
+        *pos++ = 0;
+
+        pos = ngx_http_v2_write_name(pos,
+                                     r->headers_in.user_agent->key.data,
+                                     r->headers_in.user_agent->key.len,
+                                     tmp);
+
+        pos = ngx_http_v2_write_value(pos,
+                                      r->headers_in.user_agent->value.data,
+                                      r->headers_in.user_agent->value.len,
+                                      tmp);
+    }
+
+#if (NGX_HTTP_HEADERS)
+    if (r->headers_in.accept) {
+        *pos++ = 0;
+
+        pos = ngx_http_v2_write_name(pos,
+                                     r->headers_in.accept->key.data,
+                                     r->headers_in.accept->key.len,
+                                     tmp);
+
+        pos = ngx_http_v2_write_value(pos,
+                                      r->headers_in.accept->value.data,
+                                      r->headers_in.accept->value.len,
+                                      tmp);
+    }
+
+    if (r->headers_in.accept_language) {
+        *pos++ = 0;
+
+        pos = ngx_http_v2_write_name(pos,
+                                     r->headers_in.accept_language->key.data,
+                                     r->headers_in.accept_language->key.len,
+                                     tmp);
+
+        pos = ngx_http_v2_write_value(pos,
+                                      r->headers_in.accept_language->value.data,
+                                      r->headers_in.accept_language->value.len,
+                                      tmp);
+    }
+#endif
+
+#if (NGX_HTTP_GZIP)
+    if (r->headers_in.accept_encoding) {
+        *pos++ = 0;
+
+        pos = ngx_http_v2_write_name(pos,
+                                     r->headers_in.accept_encoding->key.data,
+                                     r->headers_in.accept_encoding->key.len,
+                                     tmp);
+
+        pos = ngx_http_v2_write_value(pos,
+                                      r->headers_in.accept_encoding->value.data,
+                                      r->headers_in.accept_encoding->value.len,
+                                      tmp);
+    }
+#endif
+
     frame = ngx_http_v2_create_push_frame(r, start, pos);
     if (frame == NULL) {
         return NGX_ERROR;
@@ -1028,7 +1116,7 @@
     stream->queued++;
 
     return ngx_http_v2_push_stream(h2c, stream->node->id, pos - start,
-                                   path, &host->value);
+                                   path, &host->value, &r->headers_in);
 }
 
 

From alessandro at ghedini.me  Thu Feb  8 17:07:49 2018
From: alessandro at ghedini.me (Alessandro Ghedini)
Date: Thu, 8 Feb 2018 17:07:49 +0000
Subject: [PATCH] HTTP/2: copy additional headers in the pushed requests
In-Reply-To: <4f7f42e6d13add2ab0c7.1518109498@mandy.ghedini.home>
References: <4f7f42e6d13add2ab0c7.1518109498@mandy.ghedini.home>
Message-ID: <20180208170749.GA3818@pinky>

On Thu, Feb 08, 2018 at 05:04:58PM +0000, Alessandro Ghedini wrote:
> # HG changeset patch
> # User Alessandro Ghedini <alessandro at cloudflare.com>
> # Date 1518109032 0
> #      Thu Feb 08 16:57:12 2018 +0000
> # Branch push-copy-headers
> # Node ID 4f7f42e6d13add2ab0c7a9654472bb74085181d1
> # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> HTTP/2: copy additional headers in the pushed requests.
> 
> To ensure pushed requests are processed consistently with the original
> client request, some headers need to be copied from the original request
> into the pushed one.
> 
> The headers currently copied are User-Agent, Accept, Accept-Language and
> Accept-Encoding.

So, I'm not quite sure if this is the correct way to go about doing this, but
I think the issue is real and worth fixing, so I'd be happy to implement this
differently if you have alternative ideas.

Cheers

From mdounin at mdounin.ru  Thu Feb  8 19:00:27 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 8 Feb 2018 22:00:27 +0300
Subject: [PATCH] HTTP/2: expose function to push single resource to modules
In-Reply-To: <1bb98b06d5536dfc80a4.1518108779@mandy.ghedini.home>
References: <1bb98b06d5536dfc80a4.1518108779@mandy.ghedini.home>
Message-ID: <20180208190027.GN24410@mdounin.ru>

Hello!

On Thu, Feb 08, 2018 at 04:52:59PM +0000, Alessandro Ghedini wrote:

> # HG changeset patch
> # User Alessandro Ghedini <alessandro at ghedini.me>
> # Date 1518108716 0
> #      Thu Feb 08 16:51:56 2018 +0000
> # Branch expose-push
> # Node ID 1bb98b06d5536dfc80a407aabd8d06f9309f8df6
> # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> HTTP/2: expose function to push single resource to modules.
> 
> This makes it possible for 3rd party modules to implement alternative
> methods for deciding which resources to push to clients on a per-request
> basis (e.g. by parsing HTML from the response body, by using a custom
> Link header parser, ...).
> 
> No functional changes.

Not sure this is a good idea.

You may consider exposing a variable to be used in http2_push 
instead.

-- 
Maxim Dounin
http://mdounin.ru/

From mdounin at mdounin.ru  Thu Feb  8 19:18:54 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 8 Feb 2018 22:18:54 +0300
Subject: [PATCH] HTTP/2: copy additional headers in the pushed requests
In-Reply-To: <20180208170749.GA3818@pinky>
References: <4f7f42e6d13add2ab0c7.1518109498@mandy.ghedini.home>
 <20180208170749.GA3818@pinky>
Message-ID: <20180208191853.GO24410@mdounin.ru>

Hello!

On Thu, Feb 08, 2018 at 05:07:49PM +0000, Alessandro Ghedini wrote:

> On Thu, Feb 08, 2018 at 05:04:58PM +0000, Alessandro Ghedini wrote:
> > # HG changeset patch
> > # User Alessandro Ghedini <alessandro at cloudflare.com>
> > # Date 1518109032 0
> > #      Thu Feb 08 16:57:12 2018 +0000
> > # Branch push-copy-headers
> > # Node ID 4f7f42e6d13add2ab0c7a9654472bb74085181d1
> > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > HTTP/2: copy additional headers in the pushed requests.
> > 
> > To ensure pushed requests are processed consistently with the original
> > client request, some headers need to be copied from the original request
> > into the pushed one.
> > 
> > The headers currently copied are User-Agent, Accept, Accept-Language and
> > Accept-Encoding.
> 
> So, I'm not quite sure if this is the correct way to go about doing this, but
> I think the issue is real and worth fixing, so I'd be happy to implement this
> differently if you have alternative ideas.

Could you please elaborate more on "the issue is real"?

AFAIR, use of content negotiation with HTTP/2 push isn't a good 
idea[1], and trying to copy some headers might only make things 
more complicated and harder to debug.

[1] https://lists.w3.org/Archives/Public/public-webapps-github/2017Sep/1492.html

-- 
Maxim Dounin
http://mdounin.ru/

From ru at nginx.com  Thu Feb  8 19:28:34 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 8 Feb 2018 22:28:34 +0300
Subject: [PATCH] HTTP/2: copy additional headers in the pushed requests
In-Reply-To: <20180208170749.GA3818@pinky>
References: <4f7f42e6d13add2ab0c7.1518109498@mandy.ghedini.home>
 <20180208170749.GA3818@pinky>
Message-ID: <20180208192834.GA75377@lo0.su>

On Thu, Feb 08, 2018 at 05:07:49PM +0000, Alessandro Ghedini wrote:
> On Thu, Feb 08, 2018 at 05:04:58PM +0000, Alessandro Ghedini wrote:
> > # HG changeset patch
> > # User Alessandro Ghedini <alessandro at cloudflare.com>
> > # Date 1518109032 0
> > #      Thu Feb 08 16:57:12 2018 +0000
> > # Branch push-copy-headers
> > # Node ID 4f7f42e6d13add2ab0c7a9654472bb74085181d1
> > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > HTTP/2: copy additional headers in the pushed requests.
> > 
> > To ensure pushed requests are processed consistently with the original
> > client request, some headers need to be copied from the original request
> > into the pushed one.
> > 
> > The headers currently copied are User-Agent, Accept, Accept-Language and
> > Accept-Encoding.
> 
> So, I'm not quite sure if this is the correct way to go about doing this, but
> I think the issue is real and worth fixing, so I'd be happy to implement this
> differently if you have alternative ideas.

I don't think this is a good idea.  Copying Accept is definitely
a bad idea.  If this is at all needed, this should be implemented
as an "http2_push_header <header> <value>" directive, but then it
could not be made dependent on the pushed resource.

For example, a simple HTML with "img src" results in the following
Accept values for HTML, and then for image (Chrome and Firefox):

http request line: "GET / HTTP/1.1"
http header: "Host: 127.0.0.1:8000"
http header: "Connection: keep-alive"
http header: "Cache-Control: max-age=0"
http header: "Upgrade-Insecure-Requests: 1"
http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36"
http header: "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"
http header: "Accept-Encoding: gzip, deflate, br"
http header: "Accept-Language: ru,en-US;q=0.9,en;q=0.8,zh-TW;q=0.7,zh;q=0.6"

http request line: "GET /foo.jpg HTTP/1.1"
http header: "Host: 127.0.0.1:8000"
http header: "Connection: keep-alive"
http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36"
http header: "Accept: image/webp,image/apng,image/*,*/*;q=0.8"
http header: "Referer: http://127.0.0.1:8000/"
http header: "Accept-Encoding: gzip, deflate, br"
http header: "Accept-Language: ru,en-US;q=0.9,en;q=0.8,zh-TW;q=0.7,zh;q=0.6"


http request line: "GET / HTTP/1.1"
http header: "Host: 127.0.0.1:8000"
http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Firefox/52.0"
http header: "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
http header: "Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3"
http header: "Accept-Encoding: gzip, deflate"
http header: "Connection: keep-alive"
http header: "Upgrade-Insecure-Requests: 1"

http request line: "GET /foo.jpg HTTP/1.1"
http header: "Host: 127.0.0.1:8000"
http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Firefox/52.0"
http header: "Accept: */*"
http header: "Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3"
http header: "Accept-Encoding: gzip, deflate"
http header: "Referer: http://127.0.0.1:8000/"
http header: "Connection: keep-alive"

From alessandro at ghedini.me  Thu Feb  8 19:48:25 2018
From: alessandro at ghedini.me (Alessandro Ghedini)
Date: Thu, 8 Feb 2018 19:48:25 +0000
Subject: [PATCH] HTTP/2: expose function to push single resource to modules
In-Reply-To: <20180208190027.GN24410@mdounin.ru>
References: <1bb98b06d5536dfc80a4.1518108779@mandy.ghedini.home>
 <20180208190027.GN24410@mdounin.ru>
Message-ID: <20180208194825.GA6873@pinky>

On Thu, Feb 08, 2018 at 10:00:27PM +0300, Maxim Dounin wrote:
> Hello!
> 
> On Thu, Feb 08, 2018 at 04:52:59PM +0000, Alessandro Ghedini wrote:
> 
> > # HG changeset patch
> > # User Alessandro Ghedini <alessandro at ghedini.me>
> > # Date 1518108716 0
> > #      Thu Feb 08 16:51:56 2018 +0000
> > # Branch expose-push
> > # Node ID 1bb98b06d5536dfc80a407aabd8d06f9309f8df6
> > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > HTTP/2: expose function to push single resource to modules.
> > 
> > This makes it possible for 3rd party modules to implement alternative
> > methods for deciding which resources to push to clients on a per-request
> > basis (e.g. by parsing HTML from the response body, by using a custom
> > Link header parser, ...).
> > 
> > No functional changes.
> 
> Not sure this is a good idea.
> 
> You may consider exposing a variable to be used in http2_push 
> instead.

Right, the problem is that as far as I can tell http2_push only supports a
single resource, even when a variable is used, so it wouldn't be possible to
push multiple resources without specifying multiple http2_push directives,
each with its own variable, and even then you'd only have a fixed number of
resources that can be pushed, which wouldn't work well when the number of
resources changes depending on each request/response.

So in the end exposing the internal functions to modules seemed better than
just trying to make http2_push support multiple resources per directive,
which would add complexity to NGINX itself rather than the external modules
(though I can do that if you think it would be a better solution).

Cheers

From alessandro at ghedini.me  Thu Feb  8 20:04:57 2018
From: alessandro at ghedini.me (Alessandro Ghedini)
Date: Thu, 8 Feb 2018 20:04:57 +0000
Subject: [PATCH] HTTP/2: copy additional headers in the pushed requests
In-Reply-To: <20180208191853.GO24410@mdounin.ru>
References: <4f7f42e6d13add2ab0c7.1518109498@mandy.ghedini.home>
 <20180208170749.GA3818@pinky> <20180208191853.GO24410@mdounin.ru>
Message-ID: <20180208200457.GA21737@pinky>

On Thu, Feb 08, 2018 at 10:18:54PM +0300, Maxim Dounin wrote:
> Hello!
> 
> On Thu, Feb 08, 2018 at 05:07:49PM +0000, Alessandro Ghedini wrote:
> 
> > On Thu, Feb 08, 2018 at 05:04:58PM +0000, Alessandro Ghedini wrote:
> > > # HG changeset patch
> > > # User Alessandro Ghedini <alessandro at cloudflare.com>
> > > # Date 1518109032 0
> > > #      Thu Feb 08 16:57:12 2018 +0000
> > > # Branch push-copy-headers
> > > # Node ID 4f7f42e6d13add2ab0c7a9654472bb74085181d1
> > > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > > HTTP/2: copy additional headers in the pushed requests.
> > > 
> > > To ensure pushed requests are processed consistently with the original
> > > client request, some headers need to be copied from the original request
> > > into the pushed one.
> > > 
> > > The headers currently copied are User-Agent, Accept, Accept-Language and
> > > Accept-Encoding.
> > 
> > So, I'm not quite sure if this is the correct way to go about doing this, but
> > I think the issue is real and worth fixing, so I'd be happy to implement this
> > differently if you have alternative ideas.
> 
> Could you please elaborate more on "the issue is real"?

Right, sorry. Essentially the problem is that origins may generate a different
responses dependning on these headers, and a pushed request may be processed
inconsistently from the original request if these headers are different (say,
stupid example, if the client requests a specific language, the main request
may get a response in one language, and the pushed one in another).

Additionally things like compression wouldn't work for pushed requests, which
is not optimal.

There are of course other headers that could influence the origin's response,
but this somewhat minimal list of standard headers seemed like a good start.

FWIW, other server push implementations like Apache's mod_http2, h2o (used
in production by Fastly) and Cloudflare's closed-source implementation, all
behave like this, and have done so for years, without any apparent issue.

Cheers

From alessandro at ghedini.me  Thu Feb  8 20:49:42 2018
From: alessandro at ghedini.me (Alessandro Ghedini)
Date: Thu, 8 Feb 2018 20:49:42 +0000
Subject: [PATCH] HTTP/2: copy additional headers in the pushed requests
In-Reply-To: <20180208192834.GA75377@lo0.su>
References: <4f7f42e6d13add2ab0c7.1518109498@mandy.ghedini.home>
 <20180208170749.GA3818@pinky> <20180208192834.GA75377@lo0.su>
Message-ID: <20180208204942.GA24351@pinky>

On Thu, Feb 08, 2018 at 10:28:34PM +0300, Ruslan Ermilov wrote:
> On Thu, Feb 08, 2018 at 05:07:49PM +0000, Alessandro Ghedini wrote:
> > On Thu, Feb 08, 2018 at 05:04:58PM +0000, Alessandro Ghedini wrote:
> > > # HG changeset patch
> > > # User Alessandro Ghedini <alessandro at cloudflare.com>
> > > # Date 1518109032 0
> > > #      Thu Feb 08 16:57:12 2018 +0000
> > > # Branch push-copy-headers
> > > # Node ID 4f7f42e6d13add2ab0c7a9654472bb74085181d1
> > > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > > HTTP/2: copy additional headers in the pushed requests.
> > > 
> > > To ensure pushed requests are processed consistently with the original
> > > client request, some headers need to be copied from the original request
> > > into the pushed one.
> > > 
> > > The headers currently copied are User-Agent, Accept, Accept-Language and
> > > Accept-Encoding.
> > 
> > So, I'm not quite sure if this is the correct way to go about doing this, but
> > I think the issue is real and worth fixing, so I'd be happy to implement this
> > differently if you have alternative ideas.
> 
> I don't think this is a good idea.  Copying Accept is definitely
> a bad idea.  If this is at all needed, this should be implemented
> as an "http2_push_header <header> <value>" directive, but then it
> could not be made dependent on the pushed resource.

So, I guess this could be implemented so you can have something like:

    http2_push_header User-Agent $http_user_agent
    ...

This would work for me, and indeed is much better than just having a static
list of headers like I did. I'll look into making a patch if this is something
you'd be happy with.

Cheers

From piotrsikora at google.com  Thu Feb  8 20:56:31 2018
From: piotrsikora at google.com (Piotr Sikora)
Date: Thu, 08 Feb 2018 20:56:31 +0000
Subject: [PATCH] HTTP/2: copy additional headers in the pushed requests
In-Reply-To: <20180208192834.GA75377@lo0.su>
References: <4f7f42e6d13add2ab0c7.1518109498@mandy.ghedini.home>
 <20180208170749.GA3818@pinky> <20180208192834.GA75377@lo0.su>
Message-ID: <CAF-CG+KrWXvuQ72DtuEtrsKOTM0HDbVh6M9F+ECCbGWm6Biayw@mail.gmail.com>

Hi Ruslan,

> http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36"
> http header: "Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"
> http header: "Accept-Encoding: gzip, deflate, br"
> http header: "Accept-Language:
ru,en-US;q=0.9,en;q=0.8,zh-TW;q=0.7,zh;q=0.6"
> (...)
> http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36"
> http header: "Accept: image/webp,image/apng,image/*,*/*;q=0.8"
> http header: "Accept-Encoding: gzip, deflate, br"
> http header: "Accept-Language:
ru,en-US;q=0.9,en;q=0.8,zh-TW;q=0.7,zh;q=0.6"

> http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13;
rv:52.0) Gecko/20100101 Firefox/52.0"
> http header: "Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
> http header: "Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3"
> http header: "Accept-Encoding: gzip, deflate"
> (...)
> http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13;
rv:52.0) Gecko/20100101 Firefox/52.0"
> http header: "Accept: */*"
> http header: "Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3"
> http header: "Accept-Encoding: gzip, deflate"

In both examples, User-Agent, Accept-Encoding and Accept-Language are the
same, and Accept is (an extension-specific?) subset of the "/" request,
which proves that this is in fact a good idea.

Best regards,
Piotr Sikora

From ru at nginx.com  Fri Feb  9 07:23:46 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Fri, 9 Feb 2018 10:23:46 +0300
Subject: [PATCH] HTTP/2: copy additional headers in the pushed requests
In-Reply-To: <20180208200457.GA21737@pinky>
References: <4f7f42e6d13add2ab0c7.1518109498@mandy.ghedini.home>
 <20180208170749.GA3818@pinky> <20180208191853.GO24410@mdounin.ru>
 <20180208200457.GA21737@pinky>
Message-ID: <20180209072346.GD75377@lo0.su>

On Thu, Feb 08, 2018 at 08:04:57PM +0000, Alessandro Ghedini wrote:
> On Thu, Feb 08, 2018 at 10:18:54PM +0300, Maxim Dounin wrote:
> > Hello!
> > 
> > On Thu, Feb 08, 2018 at 05:07:49PM +0000, Alessandro Ghedini wrote:
> > 
> > > On Thu, Feb 08, 2018 at 05:04:58PM +0000, Alessandro Ghedini wrote:
> > > > # HG changeset patch
> > > > # User Alessandro Ghedini <alessandro at cloudflare.com>
> > > > # Date 1518109032 0
> > > > #      Thu Feb 08 16:57:12 2018 +0000
> > > > # Branch push-copy-headers
> > > > # Node ID 4f7f42e6d13add2ab0c7a9654472bb74085181d1
> > > > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > > > HTTP/2: copy additional headers in the pushed requests.
> > > > 
> > > > To ensure pushed requests are processed consistently with the original
> > > > client request, some headers need to be copied from the original request
> > > > into the pushed one.
> > > > 
> > > > The headers currently copied are User-Agent, Accept, Accept-Language and
> > > > Accept-Encoding.
> > > 
> > > So, I'm not quite sure if this is the correct way to go about doing this, but
> > > I think the issue is real and worth fixing, so I'd be happy to implement this
> > > differently if you have alternative ideas.
> > 
> > Could you please elaborate more on "the issue is real"?
> 
> Right, sorry. Essentially the problem is that origins may generate a different
> responses dependning on these headers, and a pushed request may be processed
> inconsistently from the original request if these headers are different (say,
> stupid example, if the client requests a specific language, the main request
> may get a response in one language, and the pushed one in another).
> 
> Additionally things like compression wouldn't work for pushed requests, which
> is not optimal.
> 
> There are of course other headers that could influence the origin's response,
> but this somewhat minimal list of standard headers seemed like a good start.
> 
> FWIW, other server push implementations like Apache's mod_http2, h2o (used
> in production by Fastly) and Cloudflare's closed-source implementation, all
> behave like this, and have done so for years, without any apparent issue.

Akamai copies User-Agent and Accept-Encoding, but sets "Accept: */*",
and doesn't copy/set Accept-Language, on pushes requests:
https://http2.akamai.com

Pushing "Accept: */* looks redundant, as
https://tools.ietf.org/html/rfc7231#section-5.3.2
says that "A request without any Accept header field implies that the
user agent will accept any media type in response.".

h2o seems to blindly copy the headers you mentioned, including Accept
from the original request, which is obviously wrong:

: [  1.239] recv (stream_id=13) :method: GET
: [  1.239] recv (stream_id=13) :scheme: https
: [  1.239] recv (stream_id=13) :authority: h2o.examp1e.net
: [  1.239] recv (stream_id=13) :path: /search/jquery-1.9.1.min.js
: [  1.239] recv (stream_id=13) accept: text/html
: [  1.239] recv (stream_id=13) accept-encoding: [...]
: [  1.239] recv (stream_id=13) user-agent: [...]
: [  1.239] recv (stream_id=13) accept-language: [...]
: [  1.239] recv PUSH_PROMISE frame <length=207, flags=0x04, stream_id=13>
:           ; END_HEADERS
:           (padlen=0, promised_stream_id=2)

https://http2.golang.org/serverpush only sets the minimally required
fields, as we currently do.

Can't test IIS, but its API allows setting additional headers to push:
https://docs.microsoft.com/en-us/iis/get-started/whats-new-in-iis-10/http2-on-iis#what-about-push
https://msdn.microsoft.com/en-us/library/system.web.httpresponse.pushpromise(v=vs.110).aspx

Can't tell for Cloudflare as I can't find a working site on it that
uses HTTP/2 with push enabled, maybe due to this:
https://community.cloudflare.com/t/is-http-2-server-push-disabled/5577

And last but not least:
https://http2.github.io/faq/#how-can-i-use-http2-server-push
: To maximize the likelihood that a pushed resource will be accepted,
: content negotiation is best avoided.

From ru at nginx.com  Fri Feb  9 07:35:59 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Fri, 9 Feb 2018 10:35:59 +0300
Subject: [PATCH] HTTP/2: expose function to push single resource to modules
In-Reply-To: <20180208194825.GA6873@pinky>
References: <1bb98b06d5536dfc80a4.1518108779@mandy.ghedini.home>
 <20180208190027.GN24410@mdounin.ru> <20180208194825.GA6873@pinky>
Message-ID: <20180209073559.GE75377@lo0.su>

On Thu, Feb 08, 2018 at 07:48:25PM +0000, Alessandro Ghedini wrote:
> On Thu, Feb 08, 2018 at 10:00:27PM +0300, Maxim Dounin wrote:
> > On Thu, Feb 08, 2018 at 04:52:59PM +0000, Alessandro Ghedini wrote:
> > 
> > > # HG changeset patch
> > > # User Alessandro Ghedini <alessandro at ghedini.me>
> > > # Date 1518108716 0
> > > #      Thu Feb 08 16:51:56 2018 +0000
> > > # Branch expose-push
> > > # Node ID 1bb98b06d5536dfc80a407aabd8d06f9309f8df6
> > > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > > HTTP/2: expose function to push single resource to modules.
> > > 
> > > This makes it possible for 3rd party modules to implement alternative
> > > methods for deciding which resources to push to clients on a per-request
> > > basis (e.g. by parsing HTML from the response body, by using a custom
> > > Link header parser, ...).
> > > 
> > > No functional changes.
> > 
> > Not sure this is a good idea.
> > 
> > You may consider exposing a variable to be used in http2_push 
> > instead.
> 
> Right, the problem is that as far as I can tell http2_push only supports a
> single resource, even when a variable is used, so it wouldn't be possible to
> push multiple resources without specifying multiple http2_push directives,
> each with its own variable, and even then you'd only have a fixed number of
> resources that can be pushed, which wouldn't work well when the number of
> resources changes depending on each request/response.
> 
> So in the end exposing the internal functions to modules seemed better than
> just trying to make http2_push support multiple resources per directive,
> which would add complexity to NGINX itself rather than the external modules
> (though I can do that if you think it would be a better solution).

We've also considered adding support for the X-Accel-Push header, but
decided not to implement it at this time.  If implemented, there could
be multiple X-Accel-Push headers in the proxied response.

From mdounin at mdounin.ru  Fri Feb  9 15:47:48 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 9 Feb 2018 18:47:48 +0300
Subject: [PATCH] HTTP/2: copy additional headers in the pushed requests
In-Reply-To: <CAF-CG+KrWXvuQ72DtuEtrsKOTM0HDbVh6M9F+ECCbGWm6Biayw@mail.gmail.com>
References: <4f7f42e6d13add2ab0c7.1518109498@mandy.ghedini.home>
 <20180208170749.GA3818@pinky> <20180208192834.GA75377@lo0.su>
 <CAF-CG+KrWXvuQ72DtuEtrsKOTM0HDbVh6M9F+ECCbGWm6Biayw@mail.gmail.com>
Message-ID: <20180209154747.GU24410@mdounin.ru>

Hello!

On Thu, Feb 08, 2018 at 08:56:31PM +0000, Piotr Sikora via nginx-devel wrote:

> Hi Ruslan,
> 
> > http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36"
> > http header: "Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"
> > http header: "Accept-Encoding: gzip, deflate, br"
> > http header: "Accept-Language:
> ru,en-US;q=0.9,en;q=0.8,zh-TW;q=0.7,zh;q=0.6"
> > (...)
> > http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36"
> > http header: "Accept: image/webp,image/apng,image/*,*/*;q=0.8"
> > http header: "Accept-Encoding: gzip, deflate, br"
> > http header: "Accept-Language:
> ru,en-US;q=0.9,en;q=0.8,zh-TW;q=0.7,zh;q=0.6"
> 
> > http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13;
> rv:52.0) Gecko/20100101 Firefox/52.0"
> > http header: "Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
> > http header: "Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3"
> > http header: "Accept-Encoding: gzip, deflate"
> > (...)
> > http header: "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13;
> rv:52.0) Gecko/20100101 Firefox/52.0"
> > http header: "Accept: */*"
> > http header: "Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3"
> > http header: "Accept-Encoding: gzip, deflate"
> 
> In both examples, User-Agent, Accept-Encoding and Accept-Language are the
> same, and Accept is (an extension-specific?) subset of the "/" request,
> which proves that this is in fact a good idea.

In both examples Accept headers are different, and this can easily 
result in different responses if content negotiation is used.  I 
don't see how it proves anything but the fact that content 
negotiation should be avoided when using HTTP/2 push.

-- 
Maxim Dounin
http://mdounin.ru/

From xeioex at nginx.com  Fri Feb  9 16:17:49 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 09 Feb 2018 16:17:49 +0000
Subject: [njs] Using hg archive to make dist.
Message-ID: <hg.2f266ec441c3.1518193069.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/2f266ec441c3
branches:  
changeset: 435:2f266ec441c3
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Feb 09 19:16:18 2018 +0300
description:
Using hg archive to make dist.

diffstat:

 Makefile       |  14 ++++++--------
 njs/njscript.h |   2 ++
 2 files changed, 8 insertions(+), 8 deletions(-)

diffs (40 lines):

diff -r c69b48375b90 -r 2f266ec441c3 Makefile
--- a/Makefile	Wed Nov 22 20:38:10 2017 +0300
+++ b/Makefile	Fri Feb 09 19:16:18 2018 +0300
@@ -1,5 +1,3 @@
-
-NJS_VER =	0.1.15
 
 NXT_LIB =	nxt
 
@@ -100,12 +98,12 @@ clean:
 	rm -f $(NXT_LIB)/Makefile.conf $(NXT_LIB)/nxt_auto_config.h
 
 dist:
-	make clean
-	mkdir njs-$(NJS_VER)
-	cp -rp configure Makefile LICENSE README CHANGES $(NXT_LIB) njs nginx \
-		njs-$(NJS_VER)
-	tar czf njs-$(NJS_VER).tar.gz njs-$(NJS_VER)
-	rm -rf njs-$(NJS_VER)
+	NJS_VER=`grep NJS_VERSION njs/njscript.h | sed -e 's/.*"\(.*\)".*/\1/'`; \
+	rm -rf njs-$${NJS_VER} \
+	&& hg archive njs-$${NJS_VER}.tar.gz \
+		      -p njs-$${NJS_VER} \
+		      -X ".hg*" \
+	&& echo njs-$${NJS_VER}.tar.gz done
 
 $(NXT_LIB)/nxt_auto_config.h:
 	@echo
diff -r c69b48375b90 -r 2f266ec441c3 njs/njscript.h
--- a/njs/njscript.h	Wed Nov 22 20:38:10 2017 +0300
+++ b/njs/njscript.h	Fri Feb 09 19:16:18 2018 +0300
@@ -7,6 +7,8 @@
 #ifndef _NJSCRIPT_H_INCLUDED_
 #define _NJSCRIPT_H_INCLUDED_
 
+#define NJS_VERSION                 "0.1.15"
+
 
 typedef intptr_t                    njs_ret_t;
 typedef uintptr_t                   njs_index_t;

From xeioex at nginx.com  Fri Feb  9 16:17:49 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 09 Feb 2018 16:17:49 +0000
Subject: [njs] Reporting njs version by CLI.
Message-ID: <hg.624f79e326d5.1518193069.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/624f79e326d5
branches:  
changeset: 436:624f79e326d5
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Feb 09 19:16:18 2018 +0300
description:
Reporting njs version by CLI.

diffstat:

 njs/njs.c                    |  14 ++++++++++++--
 njs/test/njs_expect_test.exp |   5 ++---
 2 files changed, 14 insertions(+), 5 deletions(-)

diffs (67 lines):

diff -r 2f266ec441c3 -r 624f79e326d5 njs/njs.c
--- a/njs/njs.c	Fri Feb 09 19:16:18 2018 +0300
+++ b/njs/njs.c	Fri Feb 09 19:16:18 2018 +0300
@@ -41,6 +41,7 @@ typedef enum {
 
 typedef struct {
     char                    *file;
+    nxt_int_t               version;
     nxt_int_t               disassemble;
     nxt_int_t               interactive;
 } njs_opts_t;
@@ -137,6 +138,11 @@ main(int argc, char **argv)
         return (ret == NXT_DONE) ? EXIT_SUCCESS : EXIT_FAILURE;
     }
 
+    if (opts.version != 0) {
+        printf("%s\n", NJS_VERSION);
+        return EXIT_SUCCESS;
+    }
+
     mcp = nxt_mem_cache_pool_create(&njs_vm_mem_cache_pool_proto, NULL,
                                     NULL, 2 * nxt_pagesize(), 128, 512, 16);
     if (nxt_slow_path(mcp == NULL)) {
@@ -189,6 +195,10 @@ njs_get_options(njs_opts_t *opts, int ar
             opts->disassemble = 1;
             break;
 
+        case 'V':
+            opts->version = 1;
+            break;
+
         default:
             fprintf(stderr, "Unknown argument: \"%s\"\n", argv[i]);
             ret = NXT_ERROR;
@@ -197,7 +207,7 @@ njs_get_options(njs_opts_t *opts, int ar
 
         case 'h':
         case '?':
-            printf("Usage: %s [<file>|-] [-d]\n", argv[0]);
+            printf("Usage: %s [<file>|-] [-dV]\n", argv[0]);
             return ret;
         }
     }
@@ -254,7 +264,7 @@ njs_interactive_shell(njs_opts_t *opts, 
         return NXT_ERROR;
     }
 
-    printf("interactive njscript\n\n");
+    printf("interactive njscript %s\n\n", NJS_VERSION);
 
     printf("v.<Tab> -> the properties and prototype methods of v.\n");
     printf("type console.help() for more information\n\n");
diff -r 2f266ec441c3 -r 624f79e326d5 njs/test/njs_expect_test.exp
--- a/njs/test/njs_expect_test.exp	Fri Feb 09 19:16:18 2018 +0300
+++ b/njs/test/njs_expect_test.exp	Fri Feb 09 19:16:18 2018 +0300
@@ -5,9 +5,8 @@
 
 proc njs_test {body} {
     spawn  -nottycopy njs
-    expect "interactive njscript\r
-\r
-v.<Tab> -> the properties and prototype methods of v.\r
+    expect -re "interactive njscript \\d+\.\\d+\.\\d+\r\n\r"
+    expect "v.<Tab> -> the properties and prototype methods of v.\r
 type console.help() for more information\r
 \r
 >> "

From xeioex at nginx.com  Fri Feb  9 16:17:49 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 09 Feb 2018 16:17:49 +0000
Subject: [njs] Interactive shell: fixed non-ascii character support.
Message-ID: <hg.05605a9f0ad7.1518193069.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/05605a9f0ad7
branches:  
changeset: 437:05605a9f0ad7
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Feb 09 19:16:19 2018 +0300
description:
Interactive shell: fixed non-ascii character support.

diffstat:

 njs/njs.c                    |  3 +++
 njs/test/njs_expect_test.exp |  8 ++++++++
 2 files changed, 11 insertions(+), 0 deletions(-)

diffs (38 lines):

diff -r 624f79e326d5 -r 05605a9f0ad7 njs/njs.c
--- a/njs/njs.c	Fri Feb 09 19:16:18 2018 +0300
+++ b/njs/njs.c	Fri Feb 09 19:16:19 2018 +0300
@@ -11,6 +11,7 @@
 #include <fcntl.h>
 #include <stdlib.h>
 #include <sys/stat.h>
+#include <locale.h>
 
 #include <nxt_auto_config.h>
 #include <nxt_types.h>
@@ -459,6 +460,8 @@ njs_editline_init(njs_vm_t *vm)
     rl_attempted_completion_function = njs_completion_handler;
     rl_basic_word_break_characters = (char *) " \t\n\"\\'`@$><=;,|&{(";
 
+    setlocale(LC_ALL, "");
+
     njs_completion.completions = njs_vm_completions(vm, NULL);
     if (njs_completion.completions == NULL) {
         return NXT_ERROR;
diff -r 624f79e326d5 -r 05605a9f0ad7 njs/test/njs_expect_test.exp
--- a/njs/test/njs_expect_test.exp	Fri Feb 09 19:16:18 2018 +0300
+++ b/njs/test/njs_expect_test.exp	Fri Feb 09 19:16:19 2018 +0300
@@ -182,6 +182,14 @@ njs_test {
      "JSON.parse(Error()\r\nSyntaxError: Unexpected token \"\" in 1"}
 }
 
+# Non-ASCII characters
+njs_test {
+    {"'???'\r\n"
+     "???"}
+    {"var v = '?????????????';v[10]\r\n"
+     "?"}
+}
+
 # require('fs')
 
 set file [open njs_test_file w]

From jeppojeps at gmail.com  Fri Feb  9 16:31:18 2018
From: jeppojeps at gmail.com (Antonio Nappa)
Date: Fri, 9 Feb 2018 17:31:18 +0100
Subject: Permanent structure to accumulate data
Message-ID: <CAOJuYzVbZ4-SgyV3RHNB+kw6wz9fTozXaVh7PKpfdSkyjV14SA@mail.gmail.com>

Hello,

I am looking for a structure that I can use to store a counter in the
module, this structure should not be reset every time there is a new
request.

I have made tests with the module custom ctx and the module custom srv
configuration without success, which means every time there is a new
request the field becomes zero.

Thanks,
Antonio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180209/d08c9135/attachment.html>

From serg.brester at sebres.de  Fri Feb  9 16:57:11 2018
From: serg.brester at sebres.de (Sergey Brester)
Date: Fri, 09 Feb 2018 17:57:11 +0100
Subject: Permanent structure to accumulate data
In-Reply-To: <CAOJuYzVbZ4-SgyV3RHNB+kw6wz9fTozXaVh7PKpfdSkyjV14SA@mail.gmail.com>
References: <CAOJuYzVbZ4-SgyV3RHNB+kw6wz9fTozXaVh7PKpfdSkyjV14SA@mail.gmail.com>
Message-ID: <f07905e17ca73159f8418d7837c84b59@sebres.de>

 

Unfortunately, the question is ambiguous. 

You can put it into the structure, you initialized `mycf =
ngx_pcalloc(cf->pool, size)`, 
where cf is `ngx_conf_t *cf`, if you want to save it per location.
Then you can get it in the handler, using `mycf =
ngx_http_get_module_loc_conf(r, ngx_http_your_module);`...

There are many another places, where you can put it. 
E. g. just as static variable like `static int var`, so valid per
module-scope.

But you should provide more info what exactly you want do.

For example another issue may be also multiple-workers at all (that are
processes, not the threads in nginx), so 
if it should be exactly one counter across all workers, then you should
put it into the shared memory (or somewhere in shared pools, using
ngx_slab_alloc functions). The good example would be own nginx module
"ngx_http_limit_conn_module".
Because otherwise, each worker will just get its own counter.

Regards,
Serg.

Am 09.02.2018 17:31, schrieb Antonio Nappa: 

> Hello,
> 
> I am looking for a structure that I can use to store a counter in the module, this structure should not be reset every time there is a new request.
> 
> I have made tests with the module custom ctx and the module custom srv configuration without success, which means every time there is a new request the field becomes zero.
> 
> Thanks, Antonio 
> 
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel [1]
 

Links:
------
[1] http://mailman.nginx.org/mailman/listinfo/nginx-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180209/4ab2608f/attachment.html>

From xeioex at nginx.com  Fri Feb  9 17:14:05 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 09 Feb 2018 17:14:05 +0000
Subject: [njs] Fixed the calculation of the scope indices for variables.
Message-ID: <hg.03eebf0e08cc.1518196445.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/03eebf0e08cc
branches:  
changeset: 438:03eebf0e08cc
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Feb 09 20:11:17 2018 +0300
description:
Fixed the calculation of the scope indices for variables.

diffstat:

 njs/njs_parser.h         |  14 +++++---
 njs/njs_variable.c       |  80 +++++++++++++++++++++++++++++++++++++----------
 njs/test/njs_unit_test.c |  21 ++++++++++++
 3 files changed, 93 insertions(+), 22 deletions(-)

diffs (221 lines):

diff -r 05605a9f0ad7 -r 03eebf0e08cc njs/njs_parser.h
--- a/njs/njs_parser.h	Fri Feb 09 19:16:19 2018 +0300
+++ b/njs/njs_parser.h	Fri Feb 09 20:11:17 2018 +0300
@@ -235,11 +235,15 @@ struct njs_parser_scope_s {
     nxt_lvlhsh_t                    variables;
     nxt_lvlhsh_t                    references;
 
-    nxt_array_t                     *values[2]; /* Array of njs_value_t. */
+    /*
+     * 0: local scope index;
+     * 1: closure scope index.
+     */
+    nxt_array_t                     *values[2];  /* Array of njs_value_t. */
     njs_index_t                     next_index[2];
 
     njs_scope_t                     type:8;
-    uint8_t                         nesting;    /* 4 bits */
+    uint8_t                         nesting;     /* 4 bits */
     uint8_t                         argument_closures;
 };
 
@@ -248,9 +252,9 @@ typedef struct njs_parser_node_s    njs_
 
 struct njs_parser_node_s {
     njs_token_t                     token:16;
-    uint8_t                         ctor:1;     /* 1 bit  */
-    uint8_t                         temporary;  /* 1 bit  */
-    uint8_t                         reference;  /* 1 bit  */
+    uint8_t                         ctor:1;      /* 1 bit  */
+    uint8_t                         temporary;   /* 1 bit  */
+    uint8_t                         reference;   /* 1 bit  */
     uint32_t                        token_line;
     uint32_t                        variable_name_hash;
 
diff -r 05605a9f0ad7 -r 03eebf0e08cc njs/njs_variable.c
--- a/njs/njs_variable.c	Fri Feb 09 19:16:19 2018 +0300
+++ b/njs/njs_variable.c	Fri Feb 09 20:11:17 2018 +0300
@@ -223,15 +223,17 @@ njs_variable_reference(njs_vm_t *vm, njs
 }
 
 
-njs_ret_t
-njs_variables_scope_reference(njs_vm_t *vm, njs_parser_scope_t *scope)
+static njs_ret_t
+njs_variables_scope_resolve(njs_vm_t *vm, njs_parser_scope_t *scope,
+    nxt_bool_t local_scope)
 {
-    njs_ret_t           ret;
-    nxt_queue_t         *nested;
-    njs_variable_t      *var;
-    nxt_queue_link_t    *lnk;
-    njs_parser_node_t   *node;
-    nxt_lvlhsh_each_t   lhe;
+    njs_ret_t             ret;
+    nxt_queue_t           *nested;
+    njs_variable_t        *var;
+    nxt_queue_link_t      *lnk;
+    njs_parser_node_t     *node;
+    nxt_lvlhsh_each_t     lhe;
+    njs_variable_scope_t  vs;
 
     nested = &scope->nested;
 
@@ -241,7 +243,7 @@ njs_variables_scope_reference(njs_vm_t *
     {
         scope = nxt_queue_link_data(lnk, njs_parser_scope_t, link);
 
-        ret = njs_variables_scope_reference(vm, scope);
+        ret = njs_variables_scope_resolve(vm, scope, local_scope);
         if (nxt_slow_path(ret != NXT_OK)) {
             return NXT_ERROR;
         }
@@ -255,6 +257,25 @@ njs_variables_scope_reference(njs_vm_t *
                 break;
             }
 
+            if (!local_scope) {
+                ret = njs_variable_find(vm, node, &vs);
+                if (nxt_slow_path(ret != NXT_OK)) {
+                    continue;
+                }
+
+                if (vs.scope->type == NJS_SCOPE_GLOBAL) {
+                    continue;
+                }
+
+                if (node->scope->nesting == vs.scope->nesting) {
+                    /*
+                     * A variable is referenced locally here, but may be
+                     * referenced non-locally in other places, skipping.
+                     */
+                    continue;
+                }
+            }
+
             var = njs_variable_get(vm, node);
             if (nxt_slow_path(var == NULL)) {
                 return NXT_ERROR;
@@ -266,6 +287,31 @@ njs_variables_scope_reference(njs_vm_t *
 }
 
 
+njs_ret_t
+njs_variables_scope_reference(njs_vm_t *vm, njs_parser_scope_t *scope)
+{
+    njs_ret_t  ret;
+
+    /*
+     * Calculating proper scope types for variables.
+     * A variable is considered to be local variable if it is referenced
+     * only in the local scope (reference and definition nestings are the same).
+     */
+
+    ret = njs_variables_scope_resolve(vm, scope, 0);
+    if (nxt_slow_path(ret != NXT_OK)) {
+        return NXT_ERROR;
+    }
+
+    ret = njs_variables_scope_resolve(vm, scope, 1);
+    if (nxt_slow_path(ret != NXT_OK)) {
+        return NXT_ERROR;
+    }
+
+    return NXT_OK;
+}
+
+
 njs_index_t
 njs_variable_typeof(njs_vm_t *vm, njs_parser_node_t *node)
 {
@@ -309,7 +355,7 @@ njs_variable_t *
 njs_variable_get(njs_vm_t *vm, njs_parser_node_t *node)
 {
     nxt_int_t             ret;
-    nxt_uint_t            n;
+    nxt_uint_t            scope_index;
     nxt_array_t           *values;
     njs_index_t           index;
     njs_value_t           *value;
@@ -322,10 +368,10 @@ njs_variable_get(njs_vm_t *vm, njs_parse
         goto not_found;
     }
 
-    n = 0;
+    scope_index = 0;
 
     if (vs.scope->type > NJS_SCOPE_GLOBAL) {
-        n = (node->scope->nesting != vs.scope->nesting);
+        scope_index = (node->scope->nesting != vs.scope->nesting);
     }
 
     var = vs.variable;
@@ -333,7 +379,7 @@ njs_variable_get(njs_vm_t *vm, njs_parse
 
     if (index != NJS_INDEX_NONE) {
 
-        if (n == 0 || njs_scope_type(index) != NJS_SCOPE_ARGUMENTS) {
+        if (scope_index == 0 || njs_scope_type(index) != NJS_SCOPE_ARGUMENTS) {
             node->index = index;
 
             return var;
@@ -371,7 +417,7 @@ njs_variable_get(njs_vm_t *vm, njs_parse
         index = (njs_index_t) value;
 
     } else {
-        values = vs.scope->values[n];
+        values = vs.scope->values[scope_index];
 
         if (values == NULL) {
             values = nxt_array_create(4, sizeof(njs_value_t),
@@ -380,7 +426,7 @@ njs_variable_get(njs_vm_t *vm, njs_parse
                 return NULL;
             }
 
-            vs.scope->values[n] = values;
+            vs.scope->values[scope_index] = values;
         }
 
         value = nxt_array_add(values, &njs_array_mem_proto, vm->mem_cache_pool);
@@ -388,8 +434,8 @@ njs_variable_get(njs_vm_t *vm, njs_parse
             return NULL;
         }
 
-        index = vs.scope->next_index[n];
-        vs.scope->next_index[n] += sizeof(njs_value_t);
+        index = vs.scope->next_index[scope_index];
+        vs.scope->next_index[scope_index] += sizeof(njs_value_t);
     }
 
     if (njs_is_object(&var->value)) {
diff -r 05605a9f0ad7 -r 03eebf0e08cc njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Fri Feb 09 19:16:19 2018 +0300
+++ b/njs/test/njs_unit_test.c	Fri Feb 09 20:11:17 2018 +0300
@@ -4591,6 +4591,27 @@ static njs_unit_test_t  njs_test[] =
                  "function f() { function h() { x = 3; return y; } }"),
       nxt_string("undefined") },
 
+    { nxt_string("function f() {"
+                 "    var a = 'a';"
+                 "    if (0) { a = 'b' };"
+                 "    function f2() { return a };"
+                 "    return f2"
+                 "};"
+                 "f()()"),
+      nxt_string("a") },
+
+    { nxt_string("function f() {"
+                 "    var a = 'a'; "
+                 "    if (0) { if (0) {a = 'b'} };"
+                 "    function f2() { return a };"
+                 "    return f2"
+                 "};"
+                 "f()()"),
+      nxt_string("a") },
+
+    { nxt_string("function f() { var a = f2(); }"),
+      nxt_string("ReferenceError: \"f2\" is not defined in 1") },
+
     /* Recursive fibonacci. */
 
     { nxt_string("function fibo(n) {"

From xeioex at nginx.com  Fri Feb  9 17:14:05 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 09 Feb 2018 17:14:05 +0000
Subject: [njs] Fixed Object's methods for the undefined value.
Message-ID: <hg.0a9fbf9e925f.1518196445.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/0a9fbf9e925f
branches:  
changeset: 439:0a9fbf9e925f
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Feb 09 20:12:41 2018 +0300
description:
Fixed Object's methods for the undefined value.

diffstat:

 njs/njs_object.c         |  24 ++++++++++++------------
 njs/test/njs_unit_test.c |  40 ++++++++++++++++++++++++++++++++--------
 2 files changed, 44 insertions(+), 20 deletions(-)

diffs (166 lines):

diff -r 03eebf0e08cc -r 0a9fbf9e925f njs/njs_object.c
--- a/njs/njs_object.c	Fri Feb 09 20:11:17 2018 +0300
+++ b/njs/njs_object.c	Fri Feb 09 20:12:41 2018 +0300
@@ -741,8 +741,8 @@ njs_object_freeze(njs_vm_t *vm, njs_valu
     nxt_lvlhsh_each_t  lhe;
 
     if (nargs < 2 || !njs_is_object(&args[1])) {
-        njs_exception_type_error(vm, NULL, NULL);
-        return NXT_ERROR;
+        vm->retval = njs_value_void;
+        return NXT_OK;
     }
 
     object = args[1].data.u.object;
@@ -780,8 +780,8 @@ njs_object_is_frozen(njs_vm_t *vm, njs_v
     const njs_value_t  *retval;
 
     if (nargs < 2 || !njs_is_object(&args[1])) {
-        njs_exception_type_error(vm, NULL, NULL);
-        return NXT_ERROR;
+        vm->retval = njs_string_true;
+        return NXT_OK;
     }
 
     retval = &njs_string_false;
@@ -827,8 +827,8 @@ njs_object_seal(njs_vm_t *vm, njs_value_
     nxt_lvlhsh_each_t  lhe;
 
     if (nargs < 2 || !njs_is_object(&args[1])) {
-        njs_exception_type_error(vm, NULL, NULL);
-        return NXT_ERROR;
+        vm->retval = (nargs < 2) ? njs_value_void : args[1];
+        return NXT_OK;
     }
 
     object = args[1].data.u.object;
@@ -865,8 +865,8 @@ njs_object_is_sealed(njs_vm_t *vm, njs_v
     const njs_value_t  *retval;
 
     if (nargs < 2 || !njs_is_object(&args[1])) {
-        njs_exception_type_error(vm, NULL, NULL);
-        return NXT_ERROR;
+        vm->retval = njs_string_true;
+        return NXT_OK;
     }
 
     retval = &njs_string_false;
@@ -907,8 +907,8 @@ njs_object_prevent_extensions(njs_vm_t *
     njs_index_t unused)
 {
     if (nargs < 2 || !njs_is_object(&args[1])) {
-        njs_exception_type_error(vm, NULL, NULL);
-        return NXT_ERROR;
+        vm->retval = (nargs < 2) ? njs_value_void : args[1];
+        return NXT_OK;
     }
 
     args[1].data.u.object->extensible = 0;
@@ -926,8 +926,8 @@ njs_object_is_extensible(njs_vm_t *vm, n
     const njs_value_t  *retval;
 
     if (nargs < 2 || !njs_is_object(&args[1])) {
-        njs_exception_type_error(vm, NULL, NULL);
-        return NXT_ERROR;
+        vm->retval = njs_string_false;
+        return NXT_OK;
     }
 
     retval = args[1].data.u.object->extensible ? &njs_string_true
diff -r 03eebf0e08cc -r 0a9fbf9e925f njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Fri Feb 09 20:11:17 2018 +0300
+++ b/njs/test/njs_unit_test.c	Fri Feb 09 20:12:41 2018 +0300
@@ -6433,6 +6433,9 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("Object.defineProperties(Object.freeze({}), {b:{}})"),
       nxt_string("TypeError") },
 
+    { nxt_string("Object.freeze()"),
+      nxt_string("undefined") },
+
     { nxt_string("var o = Object.freeze({a:1}); o.a = 2; o.a"),
       nxt_string("1") },
 
@@ -6535,11 +6538,14 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("Object.isFrozen(new RegExp(''))"),
       nxt_string("false") },
 
+    { nxt_string("Object.isFrozen()"),
+      nxt_string("true") },
+
     { nxt_string("Object.isFrozen(1)"),
-      nxt_string("TypeError") },
+      nxt_string("true") },
 
     { nxt_string("Object.isFrozen('')"),
-      nxt_string("TypeError") },
+      nxt_string("true") },
 
     { nxt_string("Object.isFrozen(Object.defineProperties({}, {a:{value:1}}))"),
       nxt_string("false") },
@@ -6586,11 +6592,14 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("var o = Object.seal({a:{b:1}}); o.a.b = 2; o.a.b"),
       nxt_string("2") },
 
+    { nxt_string("Object.seal()"),
+      nxt_string("undefined") },
+
     { nxt_string("Object.seal(1)"),
-      nxt_string("TypeError") },
+      nxt_string("1") },
 
     { nxt_string("Object.seal('')"),
-      nxt_string("TypeError") },
+      nxt_string("") },
 
     { nxt_string("Object.isSealed({a:1})"),
       nxt_string("false") },
@@ -6607,11 +6616,14 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("Object.isSealed(new RegExp(''))"),
       nxt_string("false") },
 
+    { nxt_string("Object.isSealed()"),
+      nxt_string("true") },
+
     { nxt_string("Object.isSealed(1)"),
-      nxt_string("TypeError") },
+      nxt_string("true") },
 
     { nxt_string("Object.isSealed('')"),
-      nxt_string("TypeError") },
+      nxt_string("true") },
 
     { nxt_string("Object.isSealed(Object.defineProperties({}, {a:{value:1}}))"),
       nxt_string("false") },
@@ -6660,6 +6672,15 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("var o = Object.preventExtensions({a:1}); o.b = 1; o.b"),
       nxt_string("undefined") },
 
+    { nxt_string("Object.preventExtensions()"),
+      nxt_string("undefined") },
+
+    { nxt_string("Object.preventExtensions(1)"),
+      nxt_string("1") },
+
+    { nxt_string("Object.preventExtensions('')"),
+      nxt_string("") },
+
     { nxt_string("Object.isExtensible({})"),
       nxt_string("true") },
 
@@ -6675,11 +6696,14 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("Object.isExtensible(new RegExp(''))"),
       nxt_string("true") },
 
+    { nxt_string("Object.isExtensible()"),
+      nxt_string("false") },
+
     { nxt_string("Object.isExtensible(1)"),
-      nxt_string("TypeError") },
+      nxt_string("false") },
 
     { nxt_string("Object.isExtensible('')"),
-      nxt_string("TypeError") },
+      nxt_string("false") },
 
     { nxt_string("Object.isExtensible(Object.preventExtensions({}))"),
       nxt_string("false") },

From xeioex at nginx.com  Fri Feb  9 17:34:59 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Fri, 09 Feb 2018 17:34:59 +0000
Subject: [njs] Fixed expect tests for systems where echo -e is not available.
Message-ID: <hg.f3ed1e9fa8d2.1518197699.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/f3ed1e9fa8d2
branches:  
changeset: 440:f3ed1e9fa8d2
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Fri Feb 09 20:34:44 2018 +0300
description:
Fixed expect tests for systems where echo -e is not available.

diffstat:

 njs/test/njs_expect_test.exp |  6 ++----
 njs/test/non_utf8            |  1 +
 2 files changed, 3 insertions(+), 4 deletions(-)

diffs (35 lines):

diff -r 0a9fbf9e925f -r f3ed1e9fa8d2 njs/test/njs_expect_test.exp
--- a/njs/test/njs_expect_test.exp	Fri Feb 09 20:12:41 2018 +0300
+++ b/njs/test/njs_expect_test.exp	Fri Feb 09 20:34:44 2018 +0300
@@ -196,8 +196,6 @@ set file [open njs_test_file w]
 puts -nonewline $file "????Z??"
 flush $file
 
-exec /bin/echo -ne {\x80\x80} > njs_test_file_non_utf8
-
 njs_test {
     {"var fs = require('fs')\r\n"
      "undefined\r\n>> "}
@@ -277,14 +275,14 @@ njs_test {
 njs_test {
     {"var fs = require('fs')\r\n"
      "undefined\r\n>> "}
-    {"fs.readFileSync('njs_test_file_non_utf8').charCodeAt(1)\r\n"
+    {"fs.readFileSync('./njs/test/non_utf8').charCodeAt(1)\r\n"
      "128"}
 }
 
 njs_test {
     {"var fs = require('fs')\r\n"
      "undefined\r\n>> "}
-    {"fs.readFileSync('njs_test_file_non_utf8', 'utf8')\r\n"
+    {"fs.readFileSync('./njs/test/non_utf8', 'utf8')\r\n"
      "Error: Non-UTF8 file, convertion is not implemented"}
 }
 
diff -r 0a9fbf9e925f -r f3ed1e9fa8d2 njs/test/non_utf8
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/njs/test/non_utf8	Fri Feb 09 20:34:44 2018 +0300
@@ -0,0 +1,1 @@
+??
\ No newline at end of file

From ru at nginx.com  Mon Feb 12 09:15:49 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Mon, 12 Feb 2018 09:15:49 +0000
Subject: [nginx] HTTP/2: fixed null pointer dereference with server push.
Message-ID: <hg.8b0553239592.1518426949.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/8b0553239592
branches:  
changeset: 7203:8b0553239592
user:      Ruslan Ermilov <ru at nginx.com>
date:      Fri Feb 09 23:20:08 2018 +0300
description:
HTTP/2: fixed null pointer dereference with server push.

r->headers_in.host can be NULL in ngx_http_v2_push_resource().

This happens when a request is terminated with 400 before the :authority
or Host header is parsed, and either pushing is enabled on the server{}
level or error_page 400 redirects to a location with pushes configured.

Found by Coverity (CID 1429156).

diffstat:

 src/http/v2/ngx_http_v2_filter_module.c |  6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

diffs (16 lines):

diff -r a49af443656f -r 8b0553239592 src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c	Thu Feb 08 12:11:30 2018 +0300
+++ b/src/http/v2/ngx_http_v2_filter_module.c	Fri Feb 09 23:20:08 2018 +0300
@@ -946,7 +946,11 @@ ngx_http_v2_push_resource(ngx_http_reque
 
     host = r->headers_in.host;
 
-    if (authority->len == 0 && host) {
+    if (host == NULL) {
+        return NGX_ABORT;
+    }
+
+    if (authority->len == 0) {
 
         len = 1 + NGX_HTTP_V2_INT_OCTETS + host->value.len;
 

From xeioex at nginx.com  Mon Feb 12 11:56:32 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Mon, 12 Feb 2018 11:56:32 +0000
Subject: [njs] Fixed building by SunC.
Message-ID: <hg.700daa193d35.1518436592.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/700daa193d35
branches:  
changeset: 441:700daa193d35
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Feb 12 14:54:23 2018 +0300
description:
Fixed building by SunC.

diffstat:

 njs/njs_object.c |  9 +++++++--
 1 files changed, 7 insertions(+), 2 deletions(-)

diffs (31 lines):

diff -r f3ed1e9fa8d2 -r 700daa193d35 njs/njs_object.c
--- a/njs/njs_object.c	Fri Feb 09 20:34:44 2018 +0300
+++ b/njs/njs_object.c	Mon Feb 12 14:54:23 2018 +0300
@@ -823,11 +823,13 @@ njs_object_seal(njs_vm_t *vm, njs_value_
 {
     nxt_lvlhsh_t       *hash;
     njs_object_t       *object;
+    const njs_value_t  *retval;
     njs_object_prop_t  *prop;
     nxt_lvlhsh_each_t  lhe;
 
     if (nargs < 2 || !njs_is_object(&args[1])) {
-        vm->retval = (nargs < 2) ? njs_value_void : args[1];
+        retval = (nargs < 2) ? &njs_value_void : &args[1];
+        vm->retval = *retval;
         return NXT_OK;
     }
 
@@ -906,8 +908,11 @@ static njs_ret_t
 njs_object_prevent_extensions(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
     njs_index_t unused)
 {
+    const njs_value_t  *retval;
+
     if (nargs < 2 || !njs_is_object(&args[1])) {
-        vm->retval = (nargs < 2) ? njs_value_void : args[1];
+        retval = (nargs < 2) ? &njs_value_void : &args[1];
+        vm->retval = *retval;
         return NXT_OK;
     }
 

From xeioex at nginx.com  Mon Feb 12 11:56:32 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Mon, 12 Feb 2018 11:56:32 +0000
Subject: [njs] Adding textual description for type converting exceptions.
Message-ID: <hg.0daf5a5cd37a.1518436592.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/0daf5a5cd37a
branches:  
changeset: 442:0daf5a5cd37a
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Feb 12 14:54:24 2018 +0300
description:
Adding textual description for type converting exceptions.

diffstat:

 njs/njs_array.c                 |   13 +-
 njs/njs_boolean.c               |    6 +-
 njs/njs_date.c                  |    2 +-
 njs/njs_error.c                 |    2 +-
 njs/njs_function.c              |   17 +--
 njs/njs_number.c                |    6 +-
 njs/njs_object.c                |   49 ++++++++--
 njs/njs_regexp.c                |    6 +-
 njs/njs_string.c                |   12 +-
 njs/njs_vm.c                    |  179 ++++++++++++++++++++++++++++++++++++---
 njs/njs_vm.h                    |    2 +
 njs/test/njs_expect_test.exp    |    7 +-
 njs/test/njs_interactive_test.c |   18 ++--
 njs/test/njs_unit_test.c        |  108 ++++++++++++-----------
 14 files changed, 306 insertions(+), 121 deletions(-)

diffs (truncated from 1155 to 1000 lines):

diff -r 700daa193d35 -r 0daf5a5cd37a njs/njs_array.c
--- a/njs/njs_array.c	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/njs_array.c	Mon Feb 12 14:54:24 2018 +0300
@@ -1714,7 +1714,7 @@ njs_array_prototype_reduce(njs_vm_t *vm,
         n = njs_array_iterator_index(array, iter);
 
         if (n == NJS_ARRAY_INVALID_INDEX) {
-            njs_exception_type_error(vm, NULL, NULL);
+            njs_exception_type_error(vm, "invalid index", NULL);
             return NXT_ERROR;
         }
 
@@ -1775,7 +1775,7 @@ njs_array_iterator_args(njs_vm_t *vm, nj
         return NXT_OK;
     }
 
-    njs_exception_type_error(vm, NULL, NULL);
+    njs_exception_type_error(vm, "unexpected iterator arguments", NULL);
 
     return NXT_ERROR;
 }
@@ -1849,7 +1849,9 @@ njs_array_prototype_reduce_right(njs_vm_
         n = njs_array_reduce_right_index(array, iter);
 
         if (n == NJS_ARRAY_INVALID_INDEX) {
-            goto type_error;
+            njs_exception_type_error(vm, "invalid index", NULL);
+
+            return NXT_ERROR;
         }
 
         iter->retval = array->start[n];
@@ -1857,11 +1859,6 @@ njs_array_prototype_reduce_right(njs_vm_
 
     return njs_array_prototype_reduce_right_continuation(vm, args, nargs,
                                                          unused);
-type_error:
-
-    njs_exception_type_error(vm, NULL, NULL);
-
-    return NXT_ERROR;
 }
 
 
diff -r 700daa193d35 -r 0daf5a5cd37a njs/njs_boolean.c
--- a/njs/njs_boolean.c	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/njs_boolean.c	Mon Feb 12 14:54:24 2018 +0300
@@ -99,7 +99,8 @@ njs_boolean_prototype_value_of(njs_vm_t 
             value = &value->data.u.object_value->value;
 
         } else {
-            njs_exception_type_error(vm, NULL, NULL);
+            njs_exception_type_error(vm, "unexpected value type:%s",
+                                     njs_type_string(value->type));
             return NXT_ERROR;
         }
     }
@@ -124,7 +125,8 @@ njs_boolean_prototype_to_string(njs_vm_t
             value = &value->data.u.object_value->value;
 
         } else {
-            njs_exception_type_error(vm, NULL, NULL);
+            njs_exception_type_error(vm, "unexpected value type:%s",
+                                     njs_type_string(value->type));
             return NXT_ERROR;
         }
     }
diff -r 700daa193d35 -r 0daf5a5cd37a njs/njs_date.c
--- a/njs/njs_date.c	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/njs_date.c	Mon Feb 12 14:54:24 2018 +0300
@@ -1911,7 +1911,7 @@ njs_date_prototype_to_json(njs_vm_t *vm,
         }
     }
 
-    njs_exception_type_error(vm, NULL, NULL);
+    njs_exception_type_error(vm, "'this' argument is not an object", NULL);
 
     return NXT_ERROR;
 }
diff -r 700daa193d35 -r 0daf5a5cd37a njs/njs_error.c
--- a/njs/njs_error.c	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/njs_error.c	Mon Feb 12 14:54:24 2018 +0300
@@ -612,7 +612,7 @@ njs_error_prototype_to_string(njs_vm_t *
     static const njs_value_t  default_name = njs_string("Error");
 
     if (nargs < 1 || !njs_is_object(&args[0])) {
-        njs_exception_type_error(vm, NULL, NULL);
+        njs_exception_type_error(vm, "'this' argument is not an object", NULL);
         return NXT_ERROR;
     }
 
diff -r 700daa193d35 -r 0daf5a5cd37a njs/njs_function.c
--- a/njs/njs_function.c	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/njs_function.c	Mon Feb 12 14:54:24 2018 +0300
@@ -509,7 +509,7 @@ njs_function_prototype_call(njs_vm_t *vm
     njs_function_t  *function;
 
     if (!njs_is_function(&args[0])) {
-        njs_exception_type_error(vm, NULL, NULL);
+        njs_exception_type_error(vm, "'this' argument is not a function", NULL);
         return NXT_ERROR;
     }
 
@@ -537,7 +537,8 @@ njs_function_prototype_apply(njs_vm_t *v
     njs_function_t  *function;
 
     if (!njs_is_function(&args[0])) {
-        goto type_error;
+        njs_exception_type_error(vm, "'this' argument is not a function", NULL);
+        return NXT_ERROR;
     }
 
     function = args[0].data.u.function;
@@ -545,7 +546,9 @@ njs_function_prototype_apply(njs_vm_t *v
 
     if (nargs > 2) {
         if (!njs_is_array(&args[2])) {
-            goto type_error;
+            njs_exception_type_error(vm, "second argument is not an array",
+                                     NULL);
+            return NXT_ERROR;
         }
 
         array = args[2].data.u.array;
@@ -561,12 +564,6 @@ njs_function_prototype_apply(njs_vm_t *v
     }
 
     return njs_function_activate(vm, function, this, args, nargs, retval);
-
-type_error:
-
-    njs_exception_type_error(vm, NULL, NULL);
-
-    return NXT_ERROR;
 }
 
 
@@ -622,7 +619,7 @@ njs_function_prototype_bind(njs_vm_t *vm
     njs_function_t  *function;
 
     if (!njs_is_function(&args[0])) {
-        njs_exception_type_error(vm, NULL, NULL);
+        njs_exception_type_error(vm, "'this' argument is not a function", NULL);
         return NXT_ERROR;
     }
 
diff -r 700daa193d35 -r 0daf5a5cd37a njs/njs_number.c
--- a/njs/njs_number.c	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/njs_number.c	Mon Feb 12 14:54:24 2018 +0300
@@ -586,7 +586,8 @@ njs_number_prototype_value_of(njs_vm_t *
             value = &value->data.u.object_value->value;
 
         } else {
-            njs_exception_type_error(vm, NULL, NULL);
+            njs_exception_type_error(vm, "unexpected value type:%s",
+                                     njs_type_string(value->type));
             return NXT_ERROR;
         }
     }
@@ -612,7 +613,8 @@ njs_number_prototype_to_string(njs_vm_t 
             value = &value->data.u.object_value->value;
 
         } else {
-            njs_exception_type_error(vm, NULL, NULL);
+            njs_exception_type_error(vm, "unexpected value type:%s",
+                                     njs_type_string(value->type));
             return NXT_ERROR;
         }
     }
diff -r 700daa193d35 -r 0daf5a5cd37a njs/njs_object.c
--- a/njs/njs_object.c	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/njs_object.c	Mon Feb 12 14:54:24 2018 +0300
@@ -224,8 +224,6 @@ njs_object_property(njs_vm_t *vm, njs_ob
 
     } while (object != NULL);
 
-    njs_exception_type_error(vm, NULL, NULL);
-
     return NULL;
 }
 
@@ -264,7 +262,8 @@ njs_object_constructor(njs_vm_t *vm, njs
             type = njs_object_value_type(value->type);
 
         } else {
-            njs_exception_type_error(vm, NULL, NULL);
+            njs_exception_type_error(vm, "unexpected constructor argument:%s",
+                                     njs_type_string(value->type));
 
             return NXT_ERROR;
         }
@@ -312,7 +311,7 @@ njs_object_create(njs_vm_t *vm, njs_valu
         }
     }
 
-    njs_exception_type_error(vm, NULL, NULL);
+    njs_exception_type_error(vm, "too few arguments", NULL);
 
     return NXT_ERROR;
 }
@@ -325,7 +324,10 @@ njs_object_keys(njs_vm_t *vm, njs_value_
     njs_array_t  *keys;
 
     if (nargs < 2 || !njs_is_object(&args[1])) {
-        njs_exception_type_error(vm, NULL, NULL);
+        njs_exception_type_error(vm, "cannot convert %s argument to object",
+                                 (nargs >= 2) ? njs_type_string(args[1].type)
+                                              : "null");
+
         return NXT_ERROR;
     }
 
@@ -426,15 +428,24 @@ static njs_ret_t
 njs_object_define_property(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
     njs_index_t unused)
 {
-    nxt_int_t  ret;
+    nxt_int_t   ret;
+    const char  *type;
 
     if (nargs < 4 || !njs_is_object(&args[1]) || !njs_is_object(&args[3])) {
-        njs_exception_type_error(vm, NULL, NULL);
+        if (nargs < 2 || !njs_is_object(&args[1])) {
+            type = (nargs > 1) ? njs_type_string(args[1].type) : "null";
+            njs_exception_type_error(vm, "cannot convert %s argument to object",
+                                     type);
+
+        } else {
+            njs_exception_type_error(vm, "descriptor is not an object", NULL);
+        }
+
         return NXT_ERROR;
     }
 
     if (!args[1].data.u.object->extensible) {
-        njs_exception_type_error(vm, NULL, NULL);
+        njs_exception_type_error(vm, "object is not extensible", NULL);
         return NXT_ERROR;
     }
 
@@ -456,18 +467,27 @@ njs_object_define_properties(njs_vm_t *v
     njs_index_t unused)
 {
     nxt_int_t          ret;
+    const char         *type;
     nxt_lvlhsh_t       *hash;
     njs_object_t       *object;
     nxt_lvlhsh_each_t  lhe;
     njs_object_prop_t  *prop;
 
     if (nargs < 3 || !njs_is_object(&args[1]) || !njs_is_object(&args[2])) {
-        njs_exception_type_error(vm, NULL, NULL);
+        if (nargs < 2 || !njs_is_object(&args[1])) {
+            type = (nargs > 1) ? njs_type_string(args[1].type) : "null";
+            njs_exception_type_error(vm, "cannot convert %s argument to object",
+                                     type);
+
+        } else {
+            njs_exception_type_error(vm, "descriptor is not an object", NULL);
+        }
+
         return NXT_ERROR;
     }
 
     if (!args[1].data.u.object->extensible) {
-        njs_exception_type_error(vm, NULL, NULL);
+        njs_exception_type_error(vm, "object is not extensible", NULL);
         return NXT_ERROR;
     }
 
@@ -590,6 +610,7 @@ njs_object_get_own_property_descriptor(n
 {
     uint32_t            index;
     nxt_int_t           ret;
+    const char          *type;
     njs_array_t         *array;
     njs_object_t        *descriptor;
     njs_object_prop_t   *pr, *prop, array_prop;
@@ -597,7 +618,9 @@ njs_object_get_own_property_descriptor(n
     nxt_lvlhsh_query_t  lhq;
 
     if (nargs < 3 || !njs_is_object(&args[1])) {
-        njs_exception_type_error(vm, NULL, NULL);
+        type = (nargs > 1) ? njs_type_string(args[1].type) : "null";
+        njs_exception_type_error(vm, "cannot convert %s argument to object",
+                                 type);
         return NXT_ERROR;
     }
 
@@ -726,7 +749,9 @@ njs_object_get_prototype_of(njs_vm_t *vm
         return NXT_OK;
     }
 
-    njs_exception_type_error(vm, NULL, NULL);
+    njs_exception_type_error(vm, "cannot convert %s argument to object",
+                             (nargs > 1) ? njs_type_string(args[1].type)
+                                         : "null");
     return NXT_ERROR;
 }
 
diff -r 700daa193d35 -r 0daf5a5cd37a njs/njs_regexp.c
--- a/njs/njs_regexp.c	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/njs_regexp.c	Mon Feb 12 14:54:24 2018 +0300
@@ -573,7 +573,7 @@ njs_regexp_prototype_to_string(njs_vm_t 
         return njs_regexp_string_create(vm, &vm->retval, source, size, length);
     }
 
-    njs_exception_type_error(vm, NULL, NULL);
+    njs_exception_type_error(vm, "'this' argument is not a regexp", NULL);
 
     return NXT_ERROR;
 }
@@ -591,7 +591,7 @@ njs_regexp_prototype_test(njs_vm_t *vm, 
     njs_regexp_pattern_t  *pattern;
 
     if (!njs_is_regexp(&args[0])) {
-        njs_exception_type_error(vm, NULL, NULL);
+        njs_exception_type_error(vm, "'this' argument is not a regexp", NULL);
         return NXT_ERROR;
     }
 
@@ -641,7 +641,7 @@ njs_regexp_prototype_exec(njs_vm_t *vm, 
     nxt_regex_match_data_t  *match_data;
 
     if (!njs_is_regexp(&args[0])) {
-        njs_exception_type_error(vm, NULL, NULL);
+        njs_exception_type_error(vm, "'this' argument is not a regexp", NULL);
         return NXT_ERROR;
     }
 
diff -r 700daa193d35 -r 0daf5a5cd37a njs/njs_string.c
--- a/njs/njs_string.c	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/njs_string.c	Mon Feb 12 14:54:24 2018 +0300
@@ -553,7 +553,8 @@ njs_string_prototype_value_of(njs_vm_t *
             value = &value->data.u.object_value->value;
 
         } else {
-            njs_exception_type_error(vm, NULL, NULL);
+            njs_exception_type_error(vm, "unexpected value type:%s",
+                                     njs_type_string(value->type));
             return NXT_ERROR;
         }
     }
@@ -578,7 +579,8 @@ njs_string_prototype_concat(njs_vm_t *vm
     njs_string_prop_t  string;
 
     if (njs_is_null_or_void(&args[0])) {
-        njs_exception_type_error(vm, NULL, NULL);
+        njs_exception_type_error(vm, "'this' argument is null or undefined",
+                                 NULL);
         return NXT_ERROR;
     }
 
@@ -2539,7 +2541,8 @@ njs_string_replace_regexp_continuation(n
 
     nxt_regex_match_data_free(r->match_data, vm->regex_context);
 
-    njs_exception_type_error(vm, NULL, NULL);
+    njs_exception_internal_error(vm, "unexpected continuation retval type:%s",
+                                 njs_type_string(r->retval.type));
 
     return NXT_ERROR;
 }
@@ -2651,7 +2654,8 @@ njs_string_replace_search_continuation(n
         return njs_string_replace_join(vm, r);
     }
 
-    njs_exception_type_error(vm, NULL, NULL);
+    njs_exception_internal_error(vm, "unexpected continuation retval type:%s",
+                                 njs_type_string(r->retval.type));
 
     return NXT_ERROR;
 }
diff -r 700daa193d35 -r 0daf5a5cd37a njs/njs_vm.c
--- a/njs/njs_vm.c	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/njs_vm.c	Mon Feb 12 14:54:24 2018 +0300
@@ -674,7 +674,8 @@ njs_vmcode_property_set(njs_vm_t *vm, nj
     njs_vmcode_prop_set_t  *code;
 
     if (njs_is_primitive(object)) {
-        njs_exception_type_error(vm, NULL, NULL);
+        njs_exception_type_error(vm, "property set on primitive %s type",
+                                 njs_type_string(object->type));
         return NXT_ERROR;
     }
 
@@ -794,7 +795,7 @@ njs_vmcode_property_in(njs_vm_t *vm, njs
 
     case NJS_PRIMITIVE_VALUE:
     case NJS_STRING_VALUE:
-        njs_exception_type_error(vm, NULL, NULL);
+        njs_exception_type_error(vm, "property in on a primitive value", NULL);
 
         return NXT_ERROR;
 
@@ -1039,7 +1040,23 @@ njs_property_query(njs_vm_t *vm, njs_pro
         break;
 
     default:  /* NJS_VOID, NJS_NULL. */
-        njs_exception_type_error(vm, NULL, NULL);
+        if (nxt_fast_path(njs_is_primitive(property))) {
+
+            ret = njs_primitive_value_to_string(vm, &pq->value, property);
+
+            if (nxt_fast_path(ret == NXT_OK)) {
+                njs_string_get(&pq->value, &pq->lhq.key);
+                njs_exception_type_error(vm,
+                                      "cannot get property '%.*s' of undefined",
+                                      (int) pq->lhq.key.length,
+                                      pq->lhq.key.start);
+                return NXT_ERROR;
+            }
+        }
+
+        njs_exception_type_error(vm,
+                            "cannot get property 'unknown' of undefined", NULL);
+
         return NXT_ERROR;
     }
 
@@ -1322,7 +1339,7 @@ njs_vmcode_instance_of(njs_vm_t *vm, njs
     nxt_lvlhsh_query_t  lhq;
 
     if (!njs_is_function(constructor)) {
-        njs_exception_type_error(vm, NULL, NULL);
+        njs_exception_type_error(vm, "right argument is not a function", NULL);
         return NXT_ERROR;
     }
 
@@ -2245,7 +2262,7 @@ njs_function_frame_create(njs_vm_t *vm, 
         }
     }
 
-    njs_exception_type_error(vm, NULL, NULL);
+    njs_exception_type_error(vm, "object is not callable", NULL);
 
     return NXT_ERROR;
 }
@@ -2304,7 +2321,9 @@ njs_vmcode_method_frame(njs_vm_t *vm, nj
 
     pq.query = NJS_PROPERTY_QUERY_GET;
 
-    switch (njs_property_query(vm, &pq, object, name)) {
+    ret = njs_property_query(vm, &pq, object, name);
+
+    switch (ret) {
 
     case NXT_OK:
         prop = pq.lhq.value;
@@ -2326,13 +2345,20 @@ njs_vmcode_method_frame(njs_vm_t *vm, nj
         ret = nxt_lvlhsh_find(&ext->hash, &pq.lhq);
 
         if (nxt_slow_path(ret != NXT_OK)) {
-            goto type_error;
+            njs_exception_type_error(vm,
+                            "cannot find property '%.*s' of an external object",
+                            (int) pq.lhq.key.length, pq.lhq.key.start);
+            return NXT_ERROR;
+
         }
 
         ext = pq.lhq.value;
 
         if (nxt_slow_path(ext->type != NJS_EXTERN_METHOD)) {
-            goto type_error;
+            njs_exception_type_error(vm,
+                          "method '%.*s' of an external object is not callable",
+                          (int) pq.lhq.key.length, pq.lhq.key.start);
+            return NXT_ERROR;
         }
 
         this.data.u.data = vm->external[ext->object];
@@ -2342,7 +2368,10 @@ njs_vmcode_method_frame(njs_vm_t *vm, nj
         break;
 
     default:
-        goto type_error;
+        njs_exception_internal_error(vm, "method '%.*s' query failed:%d",
+                                     (int) pq.lhq.key.length, pq.lhq.key.start,
+                                     ret);
+        return NXT_ERROR;
     }
 
     if (nxt_fast_path(ret == NXT_OK)) {
@@ -2350,12 +2379,6 @@ njs_vmcode_method_frame(njs_vm_t *vm, nj
     }
 
     return ret;
-
-type_error:
-
-    njs_exception_type_error(vm, NULL, NULL);
-
-    return NXT_ERROR;
 }
 
 
@@ -2574,12 +2597,130 @@ trap:
 
 type_error:
 
-    njs_exception_type_error(vm, NULL, NULL);
+    njs_exception_type_error(vm, "cannot convert %s to %s",
+                             njs_type_string(args->type),
+                             njs_arg_type_string(*args_types));
 
     return NXT_ERROR;
 }
 
 
+const char *
+njs_type_string(njs_value_type_t type)
+{
+    switch (type) {
+    case NJS_NULL:
+        return "null";
+
+    case NJS_VOID:
+        return "void";
+
+    case NJS_BOOLEAN:
+        return "boolean";
+
+    case NJS_NUMBER:
+        return "number";
+
+    case NJS_STRING:
+        return "string";
+
+    case NJS_EXTERNAL:
+        return "external";
+
+    case NJS_INVALID:
+        return "invalid";
+
+    case NJS_OBJECT:
+        return "object";
+
+    case NJS_ARRAY:
+        return "array";
+
+    case NJS_OBJECT_BOOLEAN:
+        return "object boolean";
+
+    case NJS_OBJECT_NUMBER:
+        return "object number";
+
+    case NJS_OBJECT_STRING:
+        return "object string";
+
+    case NJS_FUNCTION:
+        return "function";
+
+    case NJS_REGEXP:
+        return "regexp";
+
+    case NJS_DATE:
+        return "date";
+
+    case NJS_OBJECT_ERROR:
+        return "error";
+
+    case NJS_OBJECT_EVAL_ERROR:
+        return "eval error";
+
+    case NJS_OBJECT_INTERNAL_ERROR:
+        return "internal error";
+
+    case NJS_OBJECT_RANGE_ERROR:
+        return "range error";
+
+    case NJS_OBJECT_REF_ERROR:
+        return "reference error";
+
+    case NJS_OBJECT_SYNTAX_ERROR:
+        return "syntax error";
+
+    case NJS_OBJECT_TYPE_ERROR:
+        return "type error";
+
+    case NJS_OBJECT_URI_ERROR:
+        return "uri error";
+
+    default:
+        return NULL;
+    }
+}
+
+
+const char *
+njs_arg_type_string(uint8_t arg)
+{
+    switch (arg) {
+    case NJS_SKIP_ARG:
+        return "skip";
+
+    case NJS_NUMBER_ARG:
+        return "number";
+
+    case NJS_INTEGER_ARG:
+        return "integer";
+
+    case NJS_STRING_ARG:
+        return "string";
+
+    case NJS_OBJECT_ARG:
+        return "object";
+
+    case NJS_STRING_OBJECT_ARG:
+        return "string object";
+
+    case NJS_FUNCTION_ARG:
+        return "function";
+
+    case NJS_REGEXP_ARG:
+        return "regexp";
+
+    case NJS_DATE_ARG:
+        return "regexp";
+
+    default:
+        return "unknown";
+    }
+}
+
+
 njs_ret_t
 njs_vmcode_return(njs_vm_t *vm, njs_value_t *invld, njs_value_t *retval)
 {
@@ -3173,7 +3314,6 @@ njs_primitive_value(njs_vm_t *vm, njs_va
         if (!njs_is_primitive(retval)) {
 
             for ( ;; ) {
-                njs_exception_type_error(vm, NULL, NULL);
                 ret = NXT_ERROR;
 
                 if (njs_is_object(value) && vm->top_frame->trap_tries < 2) {
@@ -3221,6 +3361,11 @@ njs_primitive_value(njs_vm_t *vm, njs_va
                     }
                 }
 
+                if (ret == NXT_ERROR) {
+                    njs_exception_type_error(vm, "cannot evaluate an object's "
+                                             "value", NULL);
+                }
+
                 return ret;
             }
         }
diff -r 700daa193d35 -r 0daf5a5cd37a njs/njs_vm.h
--- a/njs/njs_vm.h	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/njs_vm.h	Mon Feb 12 14:54:24 2018 +0300
@@ -1152,6 +1152,8 @@ nxt_bool_t njs_values_strict_equal(const
 
 njs_ret_t njs_normalize_args(njs_vm_t *vm, njs_value_t *args,
     uint8_t *args_types, nxt_uint_t nargs);
+const char *njs_type_string(njs_value_type_t type);
+const char *njs_arg_type_string(uint8_t arg);
 
 njs_ret_t njs_native_function_arguments(njs_vm_t *vm, njs_value_t *args,
     uint8_t *args_types, nxt_uint_t nargs);
diff -r 700daa193d35 -r 0daf5a5cd37a njs/test/njs_expect_test.exp
--- a/njs/test/njs_expect_test.exp	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/test/njs_expect_test.exp	Mon Feb 12 14:54:24 2018 +0300
@@ -166,12 +166,17 @@ njs_test {
      "console.help()\r\nVM built-in objects:"}
 }
 
+njs_test {
+    {"console.ll()\r\n"
+     "console.ll()\r\nTypeError: cannot find property 'll' of an external object"}
+}
+
 # Exception in njs_vm_retval()
 njs_test {
     {"var o = { toString: function() { return [1] } }\r\n"
      "undefined\r\n>> "}
     {"o\r\n"
-     "TypeError"}
+     "TypeError: cannot evaluate an object's value"}
 }
 
 # Backtraces are reset between invocations
diff -r 700daa193d35 -r 0daf5a5cd37a njs/test/njs_interactive_test.c
--- a/njs/test/njs_interactive_test.c	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/test/njs_interactive_test.c	Mon Feb 12 14:54:24 2018 +0300
@@ -120,7 +120,7 @@ static njs_interactive_test_t  njs_test[
     { nxt_string("function ff(o) {return o.a.a}" ENTER
                  "function f(o) {return ff(o)}" ENTER
                  "f({})" ENTER),
-      nxt_string("TypeError\n"
+      nxt_string("TypeError: cannot get property 'a' of undefined\n"
                  "    at ff (:1)\n"
                  "    at f (:1)\n"
                  "    at main (native)\n") },
@@ -129,27 +129,27 @@ static njs_interactive_test_t  njs_test[
                  "function f(o) {try {return ff(o)} "
                                  "finally {return o.a.a}}" ENTER
                  "f({})" ENTER),
-      nxt_string("TypeError\n"
+      nxt_string("TypeError: cannot get property 'a' of undefined\n"
                  "    at f (:1)\n"
                  "    at main (native)\n") },
 
     { nxt_string("function f(ff, o) {return ff(o)}" ENTER
                  "f(function (o) {return o.a.a}, {})" ENTER),
-      nxt_string("TypeError\n"
+      nxt_string("TypeError: cannot get property 'a' of undefined\n"
                  "    at anonymous (:1)\n"
                  "    at f (:1)\n"
                  "    at main (native)\n") },
 
     { nxt_string("'str'.replace(/t/g,"
                  "              function(m) {return m.a.a})" ENTER),
-      nxt_string("TypeError\n"
+      nxt_string("TypeError: cannot get property 'a' of undefined\n"
                  "    at anonymous (:1)\n"
                  "    at String.prototype.replace (native)\n"
                  "    at main (native)\n") },
 
     { nxt_string("function f(o) {return Object.keys(o)}" ENTER
                  "f()" ENTER),
-      nxt_string("TypeError\n"
+      nxt_string("TypeError: cannot convert void to object\n"
                  "    at Object.keys (native)\n"
                  "    at f (:1)\n"
                  "    at main (native)\n") },
@@ -160,20 +160,20 @@ static njs_interactive_test_t  njs_test[
                  "    at main (native)\n") },
 
     { nxt_string("Math.log({}.a.a)" ENTER),
-      nxt_string("TypeError\n"
+      nxt_string("TypeError: cannot get property 'a' of undefined\n"
                  "    at Math.log (native)\n"
                  "    at main (native)\n") },
 
     { nxt_string("function f(o) {function f_in(o) {return o.a.a};"
                  "               return f_in(o)}; f({})" ENTER),
-      nxt_string("TypeError\n"
+      nxt_string("TypeError: cannot get property 'a' of undefined\n"
                  "    at f_in (:1)\n"
                  "    at f (:1)\n"
                  "    at main (native)\n") },
 
     { nxt_string("function f(o) {var ff = function (o) {return o.a.a};"
                  "               return ff(o)}; f({})" ENTER),
-      nxt_string("TypeError\n"
+      nxt_string("TypeError: cannot get property 'a' of undefined\n"
                  "    at anonymous (:1)\n"
                  "    at f (:1)\n"
                  "    at main (native)\n") },
@@ -187,7 +187,7 @@ static njs_interactive_test_t  njs_test[
 
     { nxt_string("var o = { toString: function() { return [1] } }" ENTER
                  "o" ENTER),
-      nxt_string("TypeError\n"
+      nxt_string("TypeError: cannot evaluate an object's value\n"
                  "    at main (native)\n") },
 
 };
diff -r 700daa193d35 -r 0daf5a5cd37a njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Mon Feb 12 14:54:23 2018 +0300
+++ b/njs/test/njs_unit_test.c	Mon Feb 12 14:54:24 2018 +0300
@@ -2309,10 +2309,10 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("undefined") },
 
     { nxt_string("var a = {}; a.b.c"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: cannot get property 'c' of undefined") },
 
     { nxt_string("'a'.b = 1"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: property set on primitive string type") },
 
     { nxt_string("var a = {}; a.b = 1; a.b"),
       nxt_string("1") },
@@ -2342,16 +2342,16 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("3") },
 
     { nxt_string("var a = undefined; a.b++; a.b"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: cannot get property 'b' of undefined") },
 
     { nxt_string("var a = null; a.b++; a.b"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: cannot get property 'b' of undefined") },
 
     { nxt_string("var a = true; a.b++; a.b"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: property set on primitive boolean type") },
 
     { nxt_string("var a = 1; a.b++; a.b"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: property set on primitive number type") },
 
     { nxt_string("var n = 1, o = { p: n += 1 }; o.p"),
       nxt_string("2") },
@@ -2369,7 +2369,7 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("2 1") },
 
     { nxt_string("var a = 2; a.b = 1; var c = a.b++; a +' '+ a.b +' '+ c"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: property set on primitive number type") },
 
     { nxt_string("var x = { a: 1 }; x.a"),
       nxt_string("1") },
@@ -2408,7 +2408,7 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("SyntaxError: Unexpected token \";\" in 1") },
 
     { nxt_string("var x = { a: 1, b: x.a }"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: cannot get property 'a' of undefined") },
 
     { nxt_string("var a = { b: 2 }; a.b += 1"),
       nxt_string("3") },
@@ -2481,10 +2481,10 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("false") },
 
     { nxt_string("var a = 1; 1 in a"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: property in on a primitive value") },
 
     { nxt_string("var a = true; 1 in a"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: property in on a primitive value") },
 
     { nxt_string("var n = { toString: function() { return 'a' } };"
                  "var o = { a: 5 }; o[n]"),
@@ -3224,7 +3224,7 @@ static njs_unit_test_t  njs_test[] =
 
     { nxt_string("var a = [];"
                  "a.reduce(function(p, v, i, a) { return p + v })"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: invalid index") },
 
     { nxt_string("var a = [];"
                  "a.reduce(function(p, v, i, a) { return p + v }, 10)"),
@@ -3232,7 +3232,7 @@ static njs_unit_test_t  njs_test[] =
 
     { nxt_string("var a = [,,];"
                  "a.reduce(function(p, v, i, a) { return p + v })"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: invalid index") },
 
     { nxt_string("var a = [,,];"
                  "a.reduce(function(p, v, i, a) { return p + v }, 10)"),
@@ -3260,7 +3260,7 @@ static njs_unit_test_t  njs_test[] =
 
     { nxt_string("var a = [];"
                  "a.reduceRight(function(p, v, i, a) { return p + v })"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: invalid index") },
 
     { nxt_string("var a = [];"
                  "a.reduceRight(function(p, v, i, a) { return p + v }, 10)"),
@@ -3268,7 +3268,7 @@ static njs_unit_test_t  njs_test[] =
 
     { nxt_string("var a = [,,];"
                  "a.reduceRight(function(p, v, i, a) { return p + v })"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: invalid index") },
 
     { nxt_string("var a = [,,];"
                  "a.reduceRight(function(p, v, i, a) { return p + v }, 10)"),
@@ -3725,7 +3725,7 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("abc") },
 
     { nxt_string("String.prototype.toString.call(1)"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: unexpected value type:number") },
 
     { nxt_string("'abc'.valueOf()"),
       nxt_string("abc") },
@@ -4131,7 +4131,7 @@ static njs_unit_test_t  njs_test[] =
 
     { nxt_string("var r = { toString: function() { return /45/ } };"
                  "'123456'.search(r)"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: cannot evaluate an object's value") },
 
     { nxt_string("var r = { toString: function() { return /34/ },"
                  "          valueOf:  function() { return 45 } };"
@@ -4265,7 +4265,7 @@ static njs_unit_test_t  njs_test[] =
 
     { nxt_string("var r = { toString: function() { return /45/ } };"
                  "'123456'.match(r)"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: cannot evaluate an object's value") },
 
     { nxt_string("var r = { toString: function() { return /34/ },"
                  "          valueOf:  function() { return 45 } };"
@@ -4509,10 +4509,10 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("OKundefined") },
 
     { nxt_string("var a = 1; a()"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: object is not callable") },
 
     { nxt_string("var o = {a:1}; o.a()"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: object is not callable") },
 
     { nxt_string("(function(){})()"),
       nxt_string("undefined") },
@@ -4800,7 +4800,7 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("5") },
 
     { nxt_string("var f = function(a) { return this + a }; f.apply(5, 1)"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: second argument is not an array") },
 
     { nxt_string("var f = function(a, b) { return this + a + b };"
                  "f.apply(5, [1, 2])"),
@@ -4814,7 +4814,7 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("[object Function]") },
 
     { nxt_string("''.concat.call()"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: 'this' argument is null or undefined") },
 
     { nxt_string("''.concat.call('a', 'b', 'c')"),
       nxt_string("abc") },
@@ -4829,13 +4829,13 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("ab,cd") },
 
     { nxt_string("''.concat.apply()"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: 'this' argument is null or undefined") },
 
     { nxt_string("''.concat.apply('a')"),
       nxt_string("a") },
 
     { nxt_string("''.concat.apply('a', 'b')"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: second argument is not an array") },
 
     { nxt_string("''.concat.apply('a', [ 'b', 'c' ])"),
       nxt_string("abc") },
@@ -4853,10 +4853,10 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("1552553") },
 
     { nxt_string("[].join.call()"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: cannot convert void to object") },
 
     { nxt_string("[].slice.call()"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: cannot convert void to object") },
 
     { nxt_string("function f(a) {} ; var a = f; var b = f; a === b"),
       nxt_string("true") },
@@ -4900,7 +4900,7 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("01") },
 
     { nxt_string("var concat = ''.concat; concat(1,2,3)"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: 'this' argument is null or undefined") },
 
     { nxt_string("var concat = ''.concat; concat.call(1,2,3)"),
       nxt_string("123") },
@@ -4997,10 +4997,10 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("object") },
 
     { nxt_string("new decodeURI('%00')"),
-      nxt_string("TypeError")},
+      nxt_string("TypeError: object is not callable")},
 
     { nxt_string("new ''.toString"),
-      nxt_string("TypeError")},
+      nxt_string("TypeError: object is not callable")},
 
     { nxt_string("function F() { return Number }"
                  "var o = new (F())(5);"
@@ -5214,7 +5214,7 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("true") },
 
     { nxt_string("[0].map(RegExp().toString)"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: 'this' argument is not a regexp") },
 
     /* Non-standard ECMA-262 features. */
 
@@ -5565,10 +5565,10 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("o:OK") },
 
     { nxt_string("var o = { toString: function() { return [1] } }; o"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: cannot evaluate an object's value") },
 
     { nxt_string("var o = { toString: function() { return [1] } }; 'o:' + o"),
-      nxt_string("TypeError") },
+      nxt_string("TypeError: cannot evaluate an object's value") },
 
     { nxt_string("var a = { valueOf: function() { return '3' } };"
                  "var b = { toString: function() { return 10 - a + 'OK' } };"
@@ -5620,7 +5620,7 @@ static njs_unit_test_t  njs_test[] =
       nxt_string("true") },
 
     { nxt_string("[] instanceof []"),
-      nxt_string("TypeError") },

From xeioex at nginx.com  Mon Feb 12 12:15:01 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Mon, 12 Feb 2018 12:15:01 +0000
Subject: [njs] Fixed using of internal NJS headers in nginx modules.
Message-ID: <hg.703cc218b93c.1518437701.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/703cc218b93c
branches:  
changeset: 443:703cc218b93c
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Feb 12 14:57:24 2018 +0300
description:
Fixed using of internal NJS headers in nginx modules.

Public API is rectified to make it easier to work with the private
structure njs_value_t from the outside:
    1) njs_vm_retval() is split into njs_vm_retval() which now returns
    the njs_value_t * as a return value and njs_vm_value_to_ext_string()
    which stringifies an njs_value_t * passed as an argument.

    2) njs_value_*_set() methods are added.
    3) Similar public methods are grouped together.

diffstat:

 nginx/ngx_http_js_module.c      |   13 +-
 nginx/ngx_stream_js_module.c    |   33 +++----
 njs/njs.c                       |    4 +-
 njs/njs_array.c                 |   22 ++--
 njs/njs_date.c                  |   64 +++++++-------
 njs/njs_math.c                  |   70 ++++++++--------
 njs/njs_number.c                |    4 +-
 njs/njs_regexp.c                |    4 +-
 njs/njs_string.c                |   14 +-
 njs/njs_string.h                |    2 -
 njs/njs_vm.c                    |  171 ++++++++++++++++++++++++++++++---------
 njs/njs_vm.h                    |    5 +-
 njs/njscript.c                  |   88 +-------------------
 njs/njscript.h                  |   28 ++++--
 njs/test/njs_benchmark.c        |    2 +-
 njs/test/njs_expect_test.exp    |    2 +-
 njs/test/njs_interactive_test.c |    4 +-
 njs/test/njs_unit_test.c        |    7 +-
 18 files changed, 276 insertions(+), 261 deletions(-)

diffs (truncated from 1697 to 1000 lines):

diff -r 0daf5a5cd37a -r 703cc218b93c nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c	Mon Feb 12 14:54:24 2018 +0300
+++ b/nginx/ngx_http_js_module.c	Mon Feb 12 14:57:24 2018 +0300
@@ -20,8 +20,6 @@
 #include <nxt_mem_cache_pool.h>
 
 #include <njscript.h>
-#include <njs_vm.h>
-#include <njs_string.h>
 
 
 #define NGX_HTTP_JS_MCP_CLUSTER_SIZE    (2 * ngx_pagesize)
@@ -445,7 +443,7 @@ ngx_http_js_handler(ngx_http_request_t *
     }
 
     if (njs_vm_call(ctx->vm, func, ctx->args, 2) != NJS_OK) {
-        njs_vm_retval(ctx->vm, &exception);
+        njs_vm_retval_to_ext_string(ctx->vm, &exception);
 
         ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                       "js exception: %*s", exception.length, exception.start);
@@ -496,7 +494,7 @@ ngx_http_js_variable(ngx_http_request_t 
     }
 
     if (njs_vm_call(ctx->vm, func, ctx->args, 2) != NJS_OK) {
-        njs_vm_retval(ctx->vm, &exception);
+        njs_vm_retval_to_ext_string(ctx->vm, &exception);
 
         ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                       "js exception: %*s", exception.length, exception.start);
@@ -505,7 +503,7 @@ ngx_http_js_variable(ngx_http_request_t 
         return NGX_OK;
     }
 
-    if (njs_vm_retval(ctx->vm, &value) != NJS_OK) {
+    if (njs_vm_retval_to_ext_string(ctx->vm, &value) != NJS_OK) {
         return NGX_ERROR;
     }
 
@@ -1024,7 +1022,8 @@ ngx_http_js_ext_log(njs_vm_t *vm, njs_va
     r = njs_value_data(njs_argument(args, 0));
     c = r->connection;
 
-    if (njs_value_to_ext_string(vm, &msg, njs_argument(args, 1)) == NJS_ERROR)
+    if (njs_vm_value_to_ext_string(vm, &msg, njs_argument(args, 1), 0)
+        == NJS_ERROR)
     {
         return NJS_ERROR;
     }
@@ -1334,7 +1333,7 @@ ngx_http_js_include(ngx_conf_t *cf, ngx_
     rc = njs_vm_compile(jlcf->vm, &start, end);
 
     if (rc != NJS_OK) {
-        njs_vm_retval(jlcf->vm, &text);
+        njs_vm_retval_to_ext_string(jlcf->vm, &text);
 
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "%*s, included",
diff -r 0daf5a5cd37a -r 703cc218b93c nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c	Mon Feb 12 14:54:24 2018 +0300
+++ b/nginx/ngx_stream_js_module.c	Mon Feb 12 14:57:24 2018 +0300
@@ -20,8 +20,6 @@
 #include <nxt_mem_cache_pool.h>
 
 #include <njscript.h>
-#include <njs_vm.h>
-#include <njs_string.h>
 
 
 #define NGX_STREAM_JS_MCP_CLUSTER_SIZE    (2 * ngx_pagesize)
@@ -408,7 +406,7 @@ ngx_stream_js_phase_handler(ngx_stream_s
     }
 
     if (njs_vm_call(ctx->vm, func, ctx->arg, 1) != NJS_OK) {
-        njs_vm_retval(ctx->vm, &exception);
+        njs_vm_retval_to_ext_string(ctx->vm, &exception);
 
         ngx_log_error(NGX_LOG_ERR, c->log, 0, "js exception: %*s",
                       exception.length, exception.start);
@@ -416,11 +414,11 @@ ngx_stream_js_phase_handler(ngx_stream_s
         return NGX_ERROR;
     }
 
-    if (ctx->vm->retval.type == NJS_VOID) {
+    if (njs_value_is_void(njs_vm_retval(ctx->vm))) {
         return NGX_OK;
     }
 
-    if (njs_vm_retval(ctx->vm, &value) != NJS_OK) {
+    if (njs_vm_retval_to_ext_string(ctx->vm, &value) != NJS_OK) {
         return NGX_ERROR;
     }
 
@@ -495,7 +493,7 @@ ngx_stream_js_body_filter(ngx_stream_ses
         ctx->buf = in->buf;
 
         if (njs_vm_call(ctx->vm, func, ctx->arg, 1) != NJS_OK) {
-            njs_vm_retval(ctx->vm, &exception);
+            njs_vm_retval_to_ext_string(ctx->vm, &exception);
 
             ngx_log_error(NGX_LOG_ERR, c->log, 0, "js exception: %*s",
                           exception.length, exception.start);
@@ -503,8 +501,8 @@ ngx_stream_js_body_filter(ngx_stream_ses
             return NGX_ERROR;
         }
 
-        if (ctx->vm->retval.type != NJS_VOID) {
-            if (njs_vm_retval(ctx->vm, &value) != NJS_OK) {
+        if (!njs_value_is_void(njs_vm_retval(ctx->vm))) {
+            if (njs_vm_retval_to_ext_string(ctx->vm, &value) != NJS_OK) {
                 return NGX_ERROR;
             }
 
@@ -593,7 +591,7 @@ ngx_stream_js_variable(ngx_stream_sessio
     }
 
     if (njs_vm_call(ctx->vm, func, ctx->arg, 1) != NJS_OK) {
-        njs_vm_retval(ctx->vm, &exception);
+        njs_vm_retval_to_ext_string(ctx->vm, &exception);
 
         ngx_log_error(NGX_LOG_ERR, s->connection->log, 0,
                       "js exception: %*s", exception.length, exception.start);
@@ -602,7 +600,7 @@ ngx_stream_js_variable(ngx_stream_sessio
         return NGX_OK;
     }
 
-    if (njs_vm_retval(ctx->vm, &value) != NJS_OK) {
+    if (njs_vm_retval_to_ext_string(ctx->vm, &value) != NJS_OK) {
         return NGX_ERROR;
     }
 
@@ -746,7 +744,7 @@ ngx_stream_js_ext_get_eof(njs_vm_t *vm, 
 
     b = ctx->filter ? ctx->buf : c->buffer;
 
-    *value = (b && b->last_buf ? njs_value_true : njs_value_false);
+    njs_value_boolean_set(value, b && b->last_buf);
 
     return NJS_OK;
 }
@@ -762,7 +760,7 @@ ngx_stream_js_ext_get_from_upstream(njs_
     s = (ngx_stream_session_t *) obj;
     ctx = ngx_stream_get_module_ctx(s, ngx_stream_js_module);
 
-    *value = (ctx->from_upstream ? njs_value_true : njs_value_false);
+    njs_value_boolean_set(value, ctx->from_upstream);
 
     return NJS_OK;
 }
@@ -874,7 +872,9 @@ ngx_stream_js_ext_log(njs_vm_t *vm, njs_
     s = njs_value_data(njs_argument(args, 0));
     c = s->connection;
 
-    if (njs_value_to_ext_string(vm, &msg, njs_argument(args, 1)) == NJS_ERROR) {
+    if (njs_vm_value_to_ext_string(vm, &msg, njs_argument(args, 1), 0)
+        == NJS_ERROR)
+    {
         return NJS_ERROR;
     }
 
@@ -920,10 +920,7 @@ static njs_ret_t
 ngx_stream_js_ext_get_code(njs_vm_t *vm, njs_value_t *value, void *obj,
     uintptr_t data)
 {
-    ngx_memzero(value, sizeof(njs_value_t));
-
-    value->data.type = NJS_NUMBER;
-    value->data.u.number = data;
+    njs_value_number_set(value, (double) data);
 
     return NJS_OK;
 }
@@ -1044,7 +1041,7 @@ ngx_stream_js_include(ngx_conf_t *cf, ng
     rc = njs_vm_compile(jscf->vm, &start, end);
 
     if (rc != NJS_OK) {
-        njs_vm_retval(jscf->vm, &text);
+        njs_vm_retval_to_ext_string(jscf->vm, &text);
 
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "%*s, included",
diff -r 0daf5a5cd37a -r 703cc218b93c njs/njs.c
--- a/njs/njs.c	Mon Feb 12 14:54:24 2018 +0300
+++ b/njs/njs.c	Mon Feb 12 14:57:24 2018 +0300
@@ -445,7 +445,7 @@ njs_process_script(njs_vm_t *vm, njs_opt
         }
     }
 
-    if (njs_vm_retval(vm, out) != NXT_OK) {
+    if (njs_vm_retval_to_ext_string(vm, out) != NXT_OK) {
         return NXT_ERROR;
     }
 
@@ -617,7 +617,7 @@ njs_ext_console_log(njs_vm_t *vm, njs_va
     msg.start = NULL;
 
     if (nargs >= 2
-        && njs_value_to_ext_string(vm, &msg, njs_argument(args, 1))
+        && njs_vm_value_to_ext_string(vm, &msg, njs_argument(args, 1), 0)
            == NJS_ERROR)
     {
 
diff -r 0daf5a5cd37a -r 703cc218b93c njs/njs_array.c
--- a/njs/njs_array.c	Mon Feb 12 14:54:24 2018 +0300
+++ b/njs/njs_array.c	Mon Feb 12 14:57:24 2018 +0300
@@ -383,7 +383,7 @@ const njs_object_init_t  njs_array_const
 static njs_ret_t
 njs_array_prototype_length(njs_vm_t *vm, njs_value_t *array)
 {
-    njs_number_set(&vm->retval, array->data.u.array->length);
+    njs_value_number_set(&vm->retval, array->data.u.array->length);
 
     njs_release(vm, array);
 
@@ -499,7 +499,7 @@ njs_array_prototype_push(njs_vm_t *vm, n
             }
         }
 
-        njs_number_set(&vm->retval, array->length);
+        njs_value_number_set(&vm->retval, array->length);
     }
 
     return NXT_OK;
@@ -565,7 +565,7 @@ njs_array_prototype_unshift(njs_vm_t *vm
             } while (n > 1);
         }
 
-        njs_number_set(&vm->retval, array->length);
+        njs_value_number_set(&vm->retval, array->length);
     }
 
     return NXT_OK;
@@ -1074,7 +1074,7 @@ njs_array_prototype_index_of(njs_vm_t *v
 
 done:
 
-    njs_number_set(&vm->retval, index);
+    njs_value_number_set(&vm->retval, index);
 
     return NXT_OK;
 }
@@ -1133,7 +1133,7 @@ njs_array_prototype_last_index_of(njs_vm
 
 done:
 
-    njs_number_set(&vm->retval, index);
+    njs_value_number_set(&vm->retval, index);
 
     return NXT_OK;
 }
@@ -1574,7 +1574,7 @@ njs_array_prototype_find_index_continuat
         index = -1;
     }
 
-    njs_number_set(&vm->retval, index);
+    njs_value_number_set(&vm->retval, index);
 
     return NXT_OK;
 }
@@ -1602,7 +1602,7 @@ njs_array_prototype_find_apply(njs_vm_t 
 
     arguments[1] = *value;
 
-    njs_number_set(&arguments[2], n);
+    njs_value_number_set(&arguments[2], n);
 
     arguments[3] = args[0];
 
@@ -1751,7 +1751,7 @@ njs_array_prototype_reduce_continuation(
 
     arguments[2] = array->start[n];
 
-    njs_number_set(&arguments[3], n);
+    njs_value_number_set(&arguments[3], n);
 
     arguments[4] = args[0];
 
@@ -1815,7 +1815,7 @@ njs_array_iterator_apply(njs_vm_t *vm, n
     n = iter->index;
     arguments[1] = args[0].data.u.array->start[n];
 
-    njs_number_set(&arguments[2], n);
+    njs_value_number_set(&arguments[2], n);
 
     arguments[3] = args[0];
 
@@ -1888,7 +1888,7 @@ njs_array_prototype_reduce_right_continu
 
     arguments[2] = array->start[n];
 
-    njs_number_set(&arguments[3], n);
+    njs_value_number_set(&arguments[3], n);
 
     arguments[4] = args[0];
 
@@ -1934,7 +1934,7 @@ njs_array_string_sort(njs_vm_t *vm, njs_
 
     ret = njs_string_cmp(&args[1], &args[2]);
 
-    njs_number_set(&vm->retval, ret);
+    njs_value_number_set(&vm->retval, ret);
 
     return NXT_OK;
 }
diff -r 0daf5a5cd37a -r 703cc218b93c njs/njs_date.c
--- a/njs/njs_date.c	Mon Feb 12 14:54:24 2018 +0300
+++ b/njs/njs_date.c	Mon Feb 12 14:57:24 2018 +0300
@@ -219,7 +219,7 @@ njs_date_utc(njs_vm_t *vm, njs_value_t *
 
 done:
 
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -278,7 +278,7 @@ static njs_ret_t
 njs_date_now(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
     njs_index_t unused)
 {
-    njs_number_set(&vm->retval, njs_gettime());
+    njs_value_number_set(&vm->retval, njs_gettime());
 
     return NXT_OK;
 }
@@ -297,7 +297,7 @@ njs_date_parse(njs_vm_t *vm, njs_value_t
         time = NAN;
     }
 
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -942,7 +942,7 @@ static njs_ret_t
 njs_date_prototype_value_of(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
     njs_index_t unused)
 {
-    njs_number_set(&vm->retval, args[0].data.u.date->time);
+    njs_value_number_set(&vm->retval, args[0].data.u.date->time);
 
     return NXT_OK;
 }
@@ -1086,7 +1086,7 @@ njs_date_prototype_get_full_year(njs_vm_
         value = tm.tm_year + 1900;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1109,7 +1109,7 @@ njs_date_prototype_get_utc_full_year(njs
         value = tm.tm_year + 1900;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1132,7 +1132,7 @@ njs_date_prototype_get_month(njs_vm_t *v
         value = tm.tm_mon;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1156,7 +1156,7 @@ njs_date_prototype_get_utc_month(njs_vm_
         value = tm.tm_mon;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1179,7 +1179,7 @@ njs_date_prototype_get_date(njs_vm_t *vm
         value = tm.tm_mday;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1202,7 +1202,7 @@ njs_date_prototype_get_utc_date(njs_vm_t
         value = tm.tm_mday;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1225,7 +1225,7 @@ njs_date_prototype_get_day(njs_vm_t *vm,
         value = tm.tm_wday;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1248,7 +1248,7 @@ njs_date_prototype_get_utc_day(njs_vm_t 
         value = tm.tm_wday;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1272,7 +1272,7 @@ njs_date_prototype_get_hours(njs_vm_t *v
         value = tm.tm_hour;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1295,7 +1295,7 @@ njs_date_prototype_get_utc_hours(njs_vm_
         value = tm.tm_hour;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1319,7 +1319,7 @@ njs_date_prototype_get_minutes(njs_vm_t 
         value = tm.tm_min;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1342,7 +1342,7 @@ njs_date_prototype_get_utc_minutes(njs_v
         value = tm.tm_min;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1360,7 +1360,7 @@ njs_date_prototype_get_seconds(njs_vm_t 
         value = (int64_t) (value / 1000) % 60;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1378,7 +1378,7 @@ njs_date_prototype_get_milliseconds(njs_
         value = (int64_t) value % 1000;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1401,7 +1401,7 @@ njs_date_prototype_get_timezone_offset(n
         value = - nxt_timezone(&tm) / 60;
     }
 
-    njs_number_set(&vm->retval, value);
+    njs_value_number_set(&vm->retval, value);
 
     return NXT_OK;
 }
@@ -1426,7 +1426,7 @@ njs_date_prototype_set_time(njs_vm_t *vm
     }
 
     args[0].data.u.date->time = time;
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -1451,7 +1451,7 @@ njs_date_prototype_set_milliseconds(njs_
     }
 
     args[0].data.u.date->time = time;
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -1480,7 +1480,7 @@ njs_date_prototype_set_seconds(njs_vm_t 
     }
 
     args[0].data.u.date->time = time;
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -1519,7 +1519,7 @@ njs_date_prototype_set_minutes(njs_vm_t 
     }
 
     args[0].data.u.date->time = time;
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -1554,7 +1554,7 @@ njs_date_prototype_set_utc_minutes(njs_v
     }
 
     args[0].data.u.date->time = time;
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -1597,7 +1597,7 @@ njs_date_prototype_set_hours(njs_vm_t *v
     }
 
     args[0].data.u.date->time = time;
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -1633,7 +1633,7 @@ njs_date_prototype_set_utc_hours(njs_vm_
     }
 
     args[0].data.u.date->time = time;
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -1665,7 +1665,7 @@ njs_date_prototype_set_date(njs_vm_t *vm
     }
 
     args[0].data.u.date->time = time;
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -1697,7 +1697,7 @@ njs_date_prototype_set_utc_date(njs_vm_t
     }
 
     args[0].data.u.date->time = time;
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -1733,7 +1733,7 @@ njs_date_prototype_set_month(njs_vm_t *v
     }
 
     args[0].data.u.date->time = time;
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -1769,7 +1769,7 @@ njs_date_prototype_set_utc_month(njs_vm_
     }
 
     args[0].data.u.date->time = time;
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -1809,7 +1809,7 @@ njs_date_prototype_set_full_year(njs_vm_
     }
 
     args[0].data.u.date->time = time;
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
@@ -1849,7 +1849,7 @@ njs_date_prototype_set_utc_full_year(njs
     }
 
     args[0].data.u.date->time = time;
-    njs_number_set(&vm->retval, time);
+    njs_value_number_set(&vm->retval, time);
 
     return NXT_OK;
 }
diff -r 0daf5a5cd37a -r 703cc218b93c njs/njs_math.c
--- a/njs/njs_math.c	Mon Feb 12 14:54:24 2018 +0300
+++ b/njs/njs_math.c	Mon Feb 12 14:57:24 2018 +0300
@@ -34,7 +34,7 @@ njs_object_math_abs(njs_vm_t *vm, njs_va
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -62,7 +62,7 @@ njs_object_math_acos(njs_vm_t *vm, njs_v
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -81,7 +81,7 @@ njs_object_math_acosh(njs_vm_t *vm, njs_
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -109,7 +109,7 @@ njs_object_math_asin(njs_vm_t *vm, njs_v
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -128,7 +128,7 @@ njs_object_math_asinh(njs_vm_t *vm, njs_
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -147,7 +147,7 @@ njs_object_math_atan(njs_vm_t *vm, njs_v
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -169,7 +169,7 @@ njs_object_math_atan2(njs_vm_t *vm, njs_
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -188,7 +188,7 @@ njs_object_math_atanh(njs_vm_t *vm, njs_
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -207,7 +207,7 @@ njs_object_math_cbrt(njs_vm_t *vm, njs_v
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -226,7 +226,7 @@ njs_object_math_ceil(njs_vm_t *vm, njs_v
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -247,7 +247,7 @@ njs_object_math_clz32(njs_vm_t *vm, njs_
         num = 32;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -266,7 +266,7 @@ njs_object_math_cos(njs_vm_t *vm, njs_va
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -285,7 +285,7 @@ njs_object_math_cosh(njs_vm_t *vm, njs_v
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -304,7 +304,7 @@ njs_object_math_exp(njs_vm_t *vm, njs_va
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -323,7 +323,7 @@ njs_object_math_expm1(njs_vm_t *vm, njs_
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -342,7 +342,7 @@ njs_object_math_floor(njs_vm_t *vm, njs_
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -361,7 +361,7 @@ njs_object_math_fround(njs_vm_t *vm, njs
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -392,7 +392,7 @@ njs_object_math_hypot(njs_vm_t *vm, njs_
         }
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -415,7 +415,7 @@ njs_object_math_imul(njs_vm_t *vm, njs_v
         num = 0;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -434,7 +434,7 @@ njs_object_math_log(njs_vm_t *vm, njs_va
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -453,7 +453,7 @@ njs_object_math_log10(njs_vm_t *vm, njs_
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -472,7 +472,7 @@ njs_object_math_log1p(njs_vm_t *vm, njs_
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -500,7 +500,7 @@ njs_object_math_log2(njs_vm_t *vm, njs_v
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -531,7 +531,7 @@ njs_object_math_max(njs_vm_t *vm, njs_va
         num = -INFINITY;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -562,7 +562,7 @@ njs_object_math_min(njs_vm_t *vm, njs_va
         num = INFINITY;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -595,7 +595,7 @@ njs_object_math_pow(njs_vm_t *vm, njs_va
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -609,7 +609,7 @@ njs_object_math_random(njs_vm_t *vm, njs
 
     num = nxt_random(&vm->random) / 4294967296.0;
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -628,7 +628,7 @@ njs_object_math_round(njs_vm_t *vm, njs_
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -651,7 +651,7 @@ njs_object_math_sign(njs_vm_t *vm, njs_v
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -670,7 +670,7 @@ njs_object_math_sin(njs_vm_t *vm, njs_va
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -689,7 +689,7 @@ njs_object_math_sinh(njs_vm_t *vm, njs_v
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -708,7 +708,7 @@ njs_object_math_sqrt(njs_vm_t *vm, njs_v
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -727,7 +727,7 @@ njs_object_math_tan(njs_vm_t *vm, njs_va
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -746,7 +746,7 @@ njs_object_math_tanh(njs_vm_t *vm, njs_v
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -765,7 +765,7 @@ njs_object_math_trunc(njs_vm_t *vm, njs_
         num = NAN;
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
diff -r 0daf5a5cd37a -r 703cc218b93c njs/njs_number.c
--- a/njs/njs_number.c	Mon Feb 12 14:54:24 2018 +0300
+++ b/njs/njs_number.c	Mon Feb 12 14:57:24 2018 +0300
@@ -848,7 +848,7 @@ njs_number_parse_int(njs_vm_t *vm, njs_v
 
 done:
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -866,7 +866,7 @@ njs_number_parse_float(njs_vm_t *vm, njs
         num = njs_string_to_number(&args[1], 1);
     }
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
diff -r 0daf5a5cd37a -r 703cc218b93c njs/njs_regexp.c
--- a/njs/njs_regexp.c	Mon Feb 12 14:54:24 2018 +0300
+++ b/njs/njs_regexp.c	Mon Feb 12 14:57:24 2018 +0300
@@ -487,7 +487,7 @@ njs_regexp_prototype_last_index(njs_vm_t
     (void) njs_string_prop(&string, &regexp->string);
 
     index = njs_string_index(&string, regexp->last_index);
-    njs_number_set(&vm->retval, index);
+    njs_value_number_set(&vm->retval, index);
 
     return NXT_OK;
 }
@@ -752,7 +752,7 @@ njs_regexp_exec_result(njs_vm_t *vm, njs
 
     /* TODO: Non UTF-8 position */
 
-    njs_number_set(&prop->value, regexp->last_index + captures[0]);
+    njs_value_number_set(&prop->value, regexp->last_index + captures[0]);
 
     if (regexp->pattern->global) {
         regexp->last_index += captures[1];
diff -r 0daf5a5cd37a -r 703cc218b93c njs/njs_string.c
--- a/njs/njs_string.c	Mon Feb 12 14:54:24 2018 +0300
+++ b/njs/njs_string.c	Mon Feb 12 14:57:24 2018 +0300
@@ -461,7 +461,7 @@ njs_string_prototype_length(njs_vm_t *vm
         length = (length == 0) ? size : length;
     }
 
-    njs_number_set(&vm->retval, length);
+    njs_value_number_set(&vm->retval, length);
 
     njs_release(vm, value);
 
@@ -1117,7 +1117,7 @@ njs_string_prototype_char_code_at(njs_vm
 
 done:
 
-    njs_number_set(&vm->retval, num);
+    njs_value_number_set(&vm->retval, num);
 
     return NXT_OK;
 }
@@ -1240,7 +1240,7 @@ njs_string_prototype_index_of(njs_vm_t *
 
 done:
 
-    njs_number_set(&vm->retval, index);
+    njs_value_number_set(&vm->retval, index);
 
     return NXT_OK;
 }
@@ -1329,7 +1329,7 @@ njs_string_prototype_last_index_of(njs_v
 
 done:
 
-    njs_number_set(&vm->retval, index);
+    njs_value_number_set(&vm->retval, index);
 
     return NXT_OK;
 }
@@ -1931,7 +1931,7 @@ njs_string_prototype_search(njs_vm_t *vm
 
 done:
 
-    njs_number_set(&vm->retval, index);
+    njs_value_number_set(&vm->retval, index);

From alessandro at ghedini.me  Mon Feb 12 12:35:13 2018
From: alessandro at ghedini.me (Alessandro Ghedini)
Date: Mon, 12 Feb 2018 12:35:13 +0000
Subject: [PATCH] HTTP/2: added support for setting custom push request headers
Message-ID: <4eb0c9e8da0bc5206557.1518438913@mandy.ghedini.home>

# HG changeset patch
# User Alessandro Ghedini <alessandro at ghedini.me>
# Date 1518438578 0
#      Mon Feb 12 12:29:38 2018 +0000
# Branch http2-push-header
# Node ID 4eb0c9e8da0bc52065578e4ee78df1833617ac35
# Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
HTTP/2: added support for setting custom push request headers.

This implementa the http2_push_header configuration directive that makes
it possible to set custom request headers for pushed requests.

Complex values are evaluated in the context of the original request,
so it's possible to copy its headers into pushed requests.

Example usage:

    http2_push_header User-Agent $http_user_agent;

diff -r a49af443656f -r 4eb0c9e8da0b src/http/v2/ngx_http_v2.c
--- a/src/http/v2/ngx_http_v2.c	Thu Feb 08 12:11:30 2018 +0300
+++ b/src/http/v2/ngx_http_v2.c	Mon Feb 12 12:29:38 2018 +0000
@@ -2516,15 +2516,22 @@
 
 ngx_int_t
 ngx_http_v2_push_stream(ngx_http_v2_connection_t *h2c, ngx_uint_t depend,
-    size_t request_length, ngx_str_t *path, ngx_str_t *authority)
+    size_t request_length, ngx_str_t *path, ngx_str_t *authority,
+    ngx_http_headers_in_t *headers_in)
 {
     ngx_int_t              rc;
     ngx_str_t              value;
+    ngx_uint_t             i;
+    ngx_list_part_t       *part;
+    ngx_table_elt_t       *hdr, *h;
     ngx_connection_t      *fc;
+    ngx_http_header_t     *hh;
     ngx_http_request_t    *r;
     ngx_http_v2_node_t    *node;
     ngx_http_v2_stream_t  *stream;
 
+    ngx_http_core_main_conf_t  *cmcf;
+
     node = ngx_http_v2_get_node_by_id(h2c, h2c->last_push, 1);
 
     if (node == NULL) {
@@ -2605,6 +2612,57 @@
         goto error;
     }
 
+    cmcf = ngx_http_get_module_main_conf(r, ngx_http_core_module);
+
+    part = &headers_in->headers.part;
+    hdr = part->elts;
+
+    for (i = 0; /* void */; i++) {
+
+        if (i >= part->nelts) {
+            if (part->next == NULL) {
+                break;
+            }
+
+            part = part->next;
+            hdr = part->elts;
+            i = 0;
+        }
+
+        h = ngx_list_push(&r->headers_in.headers);
+        if (h == NULL) {
+            goto error;
+        }
+
+        h->key = hdr[i].key;
+
+        h->hash = hdr[i].hash;
+
+        h->value.len = hdr[i].value.len;
+
+        h->value.data = ngx_pnalloc(r->stream->pool, h->value.len + 1);
+        if (h->key.data == NULL) {
+            h->hash = 0;
+            goto error;
+        }
+
+        (void) ngx_cpystrn(h->value.data, hdr[i].value.data, h->value.len + 1);
+
+        h->lowcase_key = h->key.data;
+
+        hh = ngx_hash_find(&cmcf->headers_in_hash, h->hash,
+                           h->lowcase_key, h->key.len);
+
+        if (hh == NULL) {
+            continue;
+        }
+
+        rc = hh->handler(r, h, hh->offset);
+        if (rc != NGX_OK) {
+            goto error;
+        }
+    }
+
     fc->write->handler = ngx_http_v2_run_request_handler;
     ngx_post_event(fc->write, &ngx_posted_events);
 
diff -r a49af443656f -r 4eb0c9e8da0b src/http/v2/ngx_http_v2.h
--- a/src/http/v2/ngx_http_v2.h	Thu Feb 08 12:11:30 2018 +0300
+++ b/src/http/v2/ngx_http_v2.h	Mon Feb 12 12:29:38 2018 +0000
@@ -285,7 +285,7 @@
 
 ngx_int_t ngx_http_v2_push_stream(ngx_http_v2_connection_t *h2c,
     ngx_uint_t depend, size_t request_length, ngx_str_t *path,
-    ngx_str_t *authority);
+    ngx_str_t *authority, ngx_http_headers_in_t *headers_in);
 
 void ngx_http_v2_close_stream(ngx_http_v2_stream_t *stream, ngx_int_t rc);
 
diff -r a49af443656f -r 4eb0c9e8da0b src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c	Thu Feb 08 12:11:30 2018 +0300
+++ b/src/http/v2/ngx_http_v2_filter_module.c	Mon Feb 12 12:29:38 2018 +0000
@@ -907,13 +907,19 @@
 ngx_http_v2_push_resource(ngx_http_request_t *r, ngx_str_t *path,
     ngx_str_t *authority)
 {
-    u_char                    *start, *pos, *tmp;
-    size_t                     len;
-    ngx_table_elt_t           *host;
-    ngx_connection_t          *fc;
-    ngx_http_v2_stream_t      *stream;
-    ngx_http_v2_out_frame_t   *frame;
-    ngx_http_v2_connection_t  *h2c;
+    u_char                         *start, *pos, *tmp;
+    size_t                          len;
+    ngx_str_t                       value;
+    ngx_uint_t                      i;
+    ngx_list_part_t                *part;
+    ngx_table_elt_t                *host, *h;
+    ngx_connection_t               *fc;
+    ngx_http_headers_in_t           headers_in;
+    ngx_http_v2_stream_t           *stream;
+    ngx_http_v2_loc_conf_t         *h2lcf;
+    ngx_http_v2_out_frame_t        *frame;
+    ngx_http_v2_connection_t       *h2c;
+    ngx_http_v2_push_header_val_t  *hv;
 
     fc = r->connection;
 
@@ -972,6 +978,43 @@
           + authority->len
           + 1;
 
+    h2lcf = ngx_http_get_module_loc_conf(r, ngx_http_v2_module);
+
+    if ((h2lcf->push_headers != NULL) && (h2lcf->push_headers->nelts > 0)) {
+
+        if (ngx_list_init(&headers_in.headers, r->pool,
+                          h2lcf->push_headers->nelts,
+                          sizeof(ngx_table_elt_t))
+            != NGX_OK)
+        {
+            return NGX_ERROR;
+        }
+
+        hv = h2lcf->push_headers->elts;
+        for (i = 0; i < h2lcf->push_headers->nelts; i++) {
+
+            if (ngx_http_complex_value(r, &hv[i].value, &value) != NGX_OK) {
+                return NGX_ERROR;
+            }
+
+            if (value.len) {
+                h = ngx_list_push(&headers_in.headers);
+                if (h == NULL) {
+                    return NGX_ERROR;
+                }
+
+                h->key = hv[i].key;
+
+                h->hash = hv[i].hash;
+
+                h->value = value;
+
+                len += 1 + NGX_HTTP_V2_INT_OCTETS + h->key.len
+                         + NGX_HTTP_V2_INT_OCTETS + h->value.len;
+            }
+        }
+    }
+
     tmp = ngx_palloc(r->pool, len);
     pos = ngx_pnalloc(r->pool, len);
 
@@ -1018,6 +1061,26 @@
         *pos++ = ngx_http_v2_indexed(NGX_HTTP_V2_SCHEME_HTTP_INDEX);
     }
 
+    part = &headers_in.headers.part;
+    h = part->elts;
+
+    for (i = 0; /* void */; i++) {
+
+        if (i >= part->nelts) {
+            if (part->next == NULL) {
+                break;
+            }
+
+            part = part->next;
+            h = part->elts;
+            i = 0;
+        }
+
+        *pos++ = 0;
+        pos = ngx_http_v2_write_name(pos, h[i].key.data, h[i].key.len, tmp);
+        pos = ngx_http_v2_write_value(pos, h[i].value.data, h[i].value.len, tmp);
+    }
+
     frame = ngx_http_v2_create_push_frame(r, start, pos);
     if (frame == NULL) {
         return NGX_ERROR;
@@ -1028,7 +1091,7 @@
     stream->queued++;
 
     return ngx_http_v2_push_stream(h2c, stream->node->id, pos - start,
-                                   path, &host->value);
+                                   path, &host->value, &headers_in);
 }
 
 
diff -r a49af443656f -r 4eb0c9e8da0b src/http/v2/ngx_http_v2_module.c
--- a/src/http/v2/ngx_http_v2_module.c	Thu Feb 08 12:11:30 2018 +0300
+++ b/src/http/v2/ngx_http_v2_module.c	Mon Feb 12 12:29:38 2018 +0000
@@ -28,6 +28,8 @@
     void *child);
 
 static char *ngx_http_v2_push(ngx_conf_t *cf, ngx_command_t *cmd, void *conf);
+static char *ngx_http_v2_push_header(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf);
 
 static char *ngx_http_v2_recv_buffer_size(ngx_conf_t *cf, void *post,
     void *data);
@@ -152,6 +154,13 @@
       0,
       NULL },
 
+    { ngx_string("http2_push_header"),
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2,
+      ngx_http_v2_push_header,
+      NGX_HTTP_LOC_CONF_OFFSET,
+      0,
+      NULL },
+
     { ngx_string("spdy_recv_buffer_size"),
       NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1,
       ngx_http_v2_spdy_deprecated,
@@ -416,6 +425,7 @@
      * set by ngx_pcalloc():
      *
      *     h2lcf->pushes = NULL;
+     *     h2lcf->push_headers = NULL;
      */
 
     h2lcf->chunk_size = NGX_CONF_UNSET_SIZE;
@@ -441,6 +451,10 @@
         conf->pushes = prev->pushes;
     }
 
+    if (conf->push && conf->push_headers == NULL) {
+        conf->push_headers = prev->push_headers;
+    }
+
     ngx_conf_merge_value(conf->push_preload, prev->push_preload, 0);
 
     return NGX_CONF_OK;
@@ -506,6 +520,55 @@
 
 
 static char *
+ngx_http_v2_push_header(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
+{
+    ngx_http_v2_loc_conf_t *h2lcf = conf;
+
+    ngx_str_t                         *value;
+    ngx_http_v2_push_header_val_t     *hv;
+    ngx_http_compile_complex_value_t   ccv;
+
+    value = cf->args->elts;
+
+    if (h2lcf->push_headers == NULL) {
+        h2lcf->push_headers = ngx_array_create(cf->pool, 1,
+                                         sizeof(ngx_http_v2_push_header_val_t));
+        if (h2lcf->push_headers == NULL) {
+            return NGX_CONF_ERROR;
+        }
+    }
+
+    hv = ngx_array_push(h2lcf->push_headers);
+    if (hv == NULL) {
+        return NGX_CONF_ERROR;
+    }
+
+    hv->key = value[1];
+
+    ngx_strlow(hv->key.data, hv->key.data, hv->key.len);
+
+    hv->hash = ngx_hash_key(hv->key.data, hv->key.len);
+
+    if (value[2].len == 0) {
+        ngx_memzero(&hv->value, sizeof(ngx_http_complex_value_t));
+
+    } else {
+        ngx_memzero(&ccv, sizeof(ngx_http_compile_complex_value_t));
+
+        ccv.cf = cf;
+        ccv.value = &value[2];
+        ccv.complex_value = &hv->value;
+
+        if (ngx_http_compile_complex_value(&ccv) != NGX_OK) {
+            return NGX_CONF_ERROR;
+        }
+    }
+
+    return NGX_CONF_OK;
+}
+
+
+static char *
 ngx_http_v2_recv_buffer_size(ngx_conf_t *cf, void *post, void *data)
 {
     size_t *sp = data;
diff -r a49af443656f -r 4eb0c9e8da0b src/http/v2/ngx_http_v2_module.h
--- a/src/http/v2/ngx_http_v2_module.h	Thu Feb 08 12:11:30 2018 +0300
+++ b/src/http/v2/ngx_http_v2_module.h	Mon Feb 12 12:29:38 2018 +0000
@@ -41,9 +41,17 @@
 
     ngx_flag_t                      push;
     ngx_array_t                    *pushes;
+    ngx_array_t                    *push_headers;
 } ngx_http_v2_loc_conf_t;
 
 
+typedef struct {
+    ngx_http_complex_value_t   value;
+    ngx_str_t                  key;
+    ngx_uint_t                 hash;
+} ngx_http_v2_push_header_val_t;
+
+
 extern ngx_module_t  ngx_http_v2_module;
 
 

From alessandro at ghedini.me  Mon Feb 12 12:42:30 2018
From: alessandro at ghedini.me (Alessandro Ghedini)
Date: Mon, 12 Feb 2018 12:42:30 +0000
Subject: [PATCH] HTTP/2: copy additional headers in the pushed requests
In-Reply-To: <20180209072346.GD75377@lo0.su>
References: <4f7f42e6d13add2ab0c7.1518109498@mandy.ghedini.home>
 <20180208170749.GA3818@pinky> <20180208191853.GO24410@mdounin.ru>
 <20180208200457.GA21737@pinky> <20180209072346.GD75377@lo0.su>
Message-ID: <20180212124230.GA23407@mandy>

On Fri, Feb 09, 2018 at 10:23:46AM +0300, Ruslan Ermilov wrote:
> On Thu, Feb 08, 2018 at 08:04:57PM +0000, Alessandro Ghedini wrote:
> > On Thu, Feb 08, 2018 at 10:18:54PM +0300, Maxim Dounin wrote:
> > > Hello!
> > > 
> > > On Thu, Feb 08, 2018 at 05:07:49PM +0000, Alessandro Ghedini wrote:
> > > 
> > > > On Thu, Feb 08, 2018 at 05:04:58PM +0000, Alessandro Ghedini wrote:
> > > > > # HG changeset patch
> > > > > # User Alessandro Ghedini <alessandro at cloudflare.com>
> > > > > # Date 1518109032 0
> > > > > #      Thu Feb 08 16:57:12 2018 +0000
> > > > > # Branch push-copy-headers
> > > > > # Node ID 4f7f42e6d13add2ab0c7a9654472bb74085181d1
> > > > > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > > > > HTTP/2: copy additional headers in the pushed requests.
> > > > > 
> > > > > To ensure pushed requests are processed consistently with the original
> > > > > client request, some headers need to be copied from the original request
> > > > > into the pushed one.
> > > > > 
> > > > > The headers currently copied are User-Agent, Accept, Accept-Language and
> > > > > Accept-Encoding.
> > > > 
> > > > So, I'm not quite sure if this is the correct way to go about doing this, but
> > > > I think the issue is real and worth fixing, so I'd be happy to implement this
> > > > differently if you have alternative ideas.
> > > 
> > > Could you please elaborate more on "the issue is real"?
> > 
> > Right, sorry. Essentially the problem is that origins may generate a different
> > responses dependning on these headers, and a pushed request may be processed
> > inconsistently from the original request if these headers are different (say,
> > stupid example, if the client requests a specific language, the main request
> > may get a response in one language, and the pushed one in another).
> > 
> > Additionally things like compression wouldn't work for pushed requests, which
> > is not optimal.
> > 
> > There are of course other headers that could influence the origin's response,
> > but this somewhat minimal list of standard headers seemed like a good start.
> > 
> > FWIW, other server push implementations like Apache's mod_http2, h2o (used
> > in production by Fastly) and Cloudflare's closed-source implementation, all
> > behave like this, and have done so for years, without any apparent issue.
> 
> Akamai copies User-Agent and Accept-Encoding, but sets "Accept: */*",
> and doesn't copy/set Accept-Language, on pushes requests:
> https://http2.akamai.com
> 
> Pushing "Accept: */* looks redundant, as
> https://tools.ietf.org/html/rfc7231#section-5.3.2
> says that "A request without any Accept header field implies that the
> user agent will accept any media type in response.".
> 
> h2o seems to blindly copy the headers you mentioned, including Accept
> from the original request, which is obviously wrong:
> 
> : [  1.239] recv (stream_id=13) :method: GET
> : [  1.239] recv (stream_id=13) :scheme: https
> : [  1.239] recv (stream_id=13) :authority: h2o.examp1e.net
> : [  1.239] recv (stream_id=13) :path: /search/jquery-1.9.1.min.js
> : [  1.239] recv (stream_id=13) accept: text/html
> : [  1.239] recv (stream_id=13) accept-encoding: [...]
> : [  1.239] recv (stream_id=13) user-agent: [...]
> : [  1.239] recv (stream_id=13) accept-language: [...]
> : [  1.239] recv PUSH_PROMISE frame <length=207, flags=0x04, stream_id=13>
> :           ; END_HEADERS
> :           (padlen=0, promised_stream_id=2)
> 
> https://http2.golang.org/serverpush only sets the minimally required
> fields, as we currently do.
> 
> Can't test IIS, but its API allows setting additional headers to push:
> https://docs.microsoft.com/en-us/iis/get-started/whats-new-in-iis-10/http2-on-iis#what-about-push
> https://msdn.microsoft.com/en-us/library/system.web.httpresponse.pushpromise(v=vs.110).aspx
> 
> Can't tell for Cloudflare as I can't find a working site on it that
> uses HTTP/2 with push enabled, maybe due to this:
> https://community.cloudflare.com/t/is-http-2-server-push-disabled/5577

Server push is enabled by default for all Cloudflare sites that also have
HTTP/2 enabled. You can test with e.g. https://sigsegv.ninja : User-Agent,
Accept-Encoding and Accept-Language are copied, but not Accept.

Cheers

From xeioex at nginx.com  Mon Feb 12 12:52:10 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Mon, 12 Feb 2018 12:52:10 +0000
Subject: [njs] Fixed console.help() method return value.
Message-ID: <hg.727f8e98cce4.1518439930.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/727f8e98cce4
branches:  
changeset: 445:727f8e98cce4
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Feb 12 15:15:53 2018 +0300
description:
Fixed console.help() method return value.

diffstat:

 njs/njs.c |  2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diffs (11 lines):

diff -r feb725292e21 -r 727f8e98cce4 njs/njs.c
--- a/njs/njs.c	Mon Feb 12 15:15:52 2018 +0300
+++ b/njs/njs.c	Mon Feb 12 15:15:53 2018 +0300
@@ -659,5 +659,7 @@ njs_ext_console_help(njs_vm_t *vm, njs_v
 
     printf("\n");
 
+    vm->retval = njs_value_void;
+
     return NJS_OK;
 }

From xeioex at nginx.com  Mon Feb 12 12:52:10 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Mon, 12 Feb 2018 12:52:10 +0000
Subject: [njs] Fixed PATH shell variable for expect tests.
Message-ID: <hg.e12ed0068e57.1518439930.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/e12ed0068e57
branches:  
changeset: 446:e12ed0068e57
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Feb 12 15:15:53 2018 +0300
description:
Fixed PATH shell variable for expect tests.

To avoid interference with the already installed njs binary.

diffstat:

 nxt/auto/expect |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff -r 727f8e98cce4 -r e12ed0068e57 nxt/auto/expect
--- a/nxt/auto/expect	Mon Feb 12 15:15:53 2018 +0300
+++ b/nxt/auto/expect	Mon Feb 12 15:15:53 2018 +0300
@@ -21,7 +21,7 @@ if [ $nxt_found = yes -a $NXT_HAVE_LIBED
     cat << END >> $NXT_MAKEFILE_CONF
 
 njs_expect_test:	njs njs/test/njs_expect_test.exp
-	PATH=\$(PATH):\$(NXT_BUILDDIR) expect -f njs/test/njs_expect_test.exp
+	PATH=\$(NXT_BUILDDIR):\$(PATH) expect -f njs/test/njs_expect_test.exp
 END
 
 else

From xeioex at nginx.com  Mon Feb 12 12:52:10 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Mon, 12 Feb 2018 12:52:10 +0000
Subject: [njs] Fixed njs_string_create() prototype to reflect underlying types.
Message-ID: <hg.feb725292e21.1518439930.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/feb725292e21
branches:  
changeset: 444:feb725292e21
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Mon Feb 12 15:15:52 2018 +0300
description:
Fixed njs_string_create() prototype to reflect underlying types.

diffstat:

 njs/njs_string.c |  4 ++--
 njs/njscript.h   |  2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diffs (26 lines):

diff -r 703cc218b93c -r feb725292e21 njs/njs_string.c
--- a/njs/njs_string.c	Mon Feb 12 14:57:24 2018 +0300
+++ b/njs/njs_string.c	Mon Feb 12 15:15:52 2018 +0300
@@ -120,8 +120,8 @@ static njs_ret_t njs_string_decode(njs_v
 
 
 njs_ret_t
-njs_string_create(njs_vm_t *vm, njs_value_t *value, u_char *start, size_t size,
-    size_t length)
+njs_string_create(njs_vm_t *vm, njs_value_t *value, u_char *start,
+    uint32_t size, uint32_t length)
 {
     u_char        *dst, *src;
     njs_string_t  *string;
diff -r 703cc218b93c -r feb725292e21 njs/njscript.h
--- a/njs/njscript.h	Mon Feb 12 14:57:24 2018 +0300
+++ b/njs/njscript.h	Mon Feb 12 15:15:52 2018 +0300
@@ -119,7 +119,7 @@ NXT_EXPORT njs_value_t *njs_vm_retval(nj
 NXT_EXPORT u_char * njs_string_alloc(njs_vm_t *vm, njs_value_t *value,
     uint32_t size, uint32_t length);
 NXT_EXPORT njs_ret_t njs_string_create(njs_vm_t *vm, njs_value_t *value,
-    u_char *start, size_t size, size_t length);
+    u_char *start, uint32_t size, uint32_t length);
 
 NXT_EXPORT nxt_int_t njs_value_string_copy(njs_vm_t *vm, nxt_str_t *retval,
     njs_value_t *value, uintptr_t *next);

From ru at nginx.com  Mon Feb 12 14:11:55 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Mon, 12 Feb 2018 17:11:55 +0300
Subject: [PATCH] HTTP/2: added support for setting custom push request
 headers
In-Reply-To: <4eb0c9e8da0bc5206557.1518438913@mandy.ghedini.home>
References: <4eb0c9e8da0bc5206557.1518438913@mandy.ghedini.home>
Message-ID: <20180212141155.GB67691@lo0.su>

On Mon, Feb 12, 2018 at 12:35:13PM +0000, Alessandro Ghedini wrote:
> # HG changeset patch
> # User Alessandro Ghedini <alessandro at ghedini.me>
> # Date 1518438578 0
> #      Mon Feb 12 12:29:38 2018 +0000
> # Branch http2-push-header
> # Node ID 4eb0c9e8da0bc52065578e4ee78df1833617ac35
> # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> HTTP/2: added support for setting custom push request headers.
> 
> This implementa the http2_push_header configuration directive that makes
> it possible to set custom request headers for pushed requests.
> 
> Complex values are evaluated in the context of the original request,
> so it's possible to copy its headers into pushed requests.
> 
> Example usage:
> 
>     http2_push_header User-Agent $http_user_agent;

If I'm not mistaken, both the original patch and this one
do not send the copied headers in the PUSH_PROMISE frame.

Given the https://trac.nginx.org/nginx/ticket/1478, and some
research of how different implementations behave, I prepared
the following patch:

# HG changeset patch
# User Ruslan Ermilov <ru at nginx.com>
# Date 1518444403 -10800
#      Mon Feb 12 17:06:43 2018 +0300
# Node ID 4e72b3bfdeaa9917bbdc89a7fa75580c6bec41a7
# Parent  8b0553239592f5d0fd419e5116b9d343838685cf
HTTP/2: push additional headers (ticket #1478).

When pushing requests, copy and push Accept-Encoding, Accept-Language,
and User-Agent header fields from the original request.

diff --git a/src/http/v2/ngx_http_v2_filter_module.c b/src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c
+++ b/src/http/v2/ngx_http_v2_filter_module.c
@@ -50,20 +50,37 @@
 #define NGX_HTTP_V2_STATUS_404_INDEX      13
 #define NGX_HTTP_V2_STATUS_500_INDEX      14
 
+#define NGX_HTTP_V2_ACCEPT_ENCODING_INDEX 16
+#define NGX_HTTP_V2_ACCEPT_LANGUAGE_INDEX 17
 #define NGX_HTTP_V2_CONTENT_LENGTH_INDEX  28
 #define NGX_HTTP_V2_CONTENT_TYPE_INDEX    31
 #define NGX_HTTP_V2_DATE_INDEX            33
 #define NGX_HTTP_V2_LAST_MODIFIED_INDEX   44
 #define NGX_HTTP_V2_LOCATION_INDEX        46
 #define NGX_HTTP_V2_SERVER_INDEX          54
+#define NGX_HTTP_V2_USER_AGENT_INDEX      58
 #define NGX_HTTP_V2_VARY_INDEX            59
 
 #define NGX_HTTP_V2_NO_TRAILERS           (ngx_http_v2_out_frame_t *) -1
 
 
+typedef struct {
+    ngx_str_t                             authority;
+#if (NGX_HTTP_GZIP)
+    ngx_str_t                             accept_encoding;
+#endif
+#if (NGX_HTTP_HEADERS)
+    ngx_str_t                             accept_language;
+#endif
+    ngx_str_t                             user_agent;
+} ngx_http_v2_push_ctx_t;
+
+
 static ngx_int_t ngx_http_v2_push_resources(ngx_http_request_t *r);
 static ngx_int_t ngx_http_v2_push_resource(ngx_http_request_t *r,
-    ngx_str_t *path, ngx_str_t *authority);
+    ngx_str_t *path, ngx_http_v2_push_ctx_t *ctx);
+static ngx_int_t ngx_http_v2_indexed_header_encode(ngx_http_request_t *r,
+    u_char index, ngx_str_t *value, ngx_str_t *header);
 
 static u_char *ngx_http_v2_string_encode(u_char *dst, u_char *src, size_t len,
     u_char *tmp, ngx_uint_t lower);
@@ -685,16 +702,17 @@ ngx_http_v2_push_resources(ngx_http_requ
 {
     u_char                     *start, *end, *last;
     ngx_int_t                   rc;
-    ngx_str_t                   path, authority;
+    ngx_str_t                   path;
     ngx_uint_t                  i, push;
     ngx_table_elt_t           **h;
     ngx_http_v2_loc_conf_t     *h2lcf;
+    ngx_http_v2_push_ctx_t      ctx;
     ngx_http_complex_value_t   *pushes;
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                    "http2 push resources");
 
-    ngx_str_null(&authority);
+    ngx_memzero(&ctx, sizeof(ngx_http_v2_push_ctx_t));
 
     h2lcf = ngx_http_get_module_loc_conf(r, ngx_http_v2_module);
 
@@ -715,7 +733,7 @@ ngx_http_v2_push_resources(ngx_http_requ
                 continue;
             }
 
-            rc = ngx_http_v2_push_resource(r, &path, &authority);
+            rc = ngx_http_v2_push_resource(r, &path, &ctx);
 
             if (rc == NGX_ERROR) {
                 return NGX_ERROR;
@@ -880,7 +898,7 @@ ngx_http_v2_push_resources(ngx_http_requ
         if (push && path.len
             && !(path.len > 1 && path.data[0] == '/' && path.data[1] == '/'))
         {
-            rc = ngx_http_v2_push_resource(r, &path, &authority);
+            rc = ngx_http_v2_push_resource(r, &path, &ctx);
 
             if (rc == NGX_ERROR) {
                 return NGX_ERROR;
@@ -905,11 +923,17 @@ ngx_http_v2_push_resources(ngx_http_requ
 
 static ngx_int_t
 ngx_http_v2_push_resource(ngx_http_request_t *r, ngx_str_t *path,
-    ngx_str_t *authority)
+    ngx_http_v2_push_ctx_t *ctx)
 {
     u_char                    *start, *pos, *tmp;
     size_t                     len;
-    ngx_table_elt_t           *host;
+    ngx_table_elt_t           *host, *user_agent;
+#if (NGX_HTTP_GZIP)
+    ngx_table_elt_t           *accept_encoding;
+#endif
+#if (NGX_HTTP_HEADERS)
+    ngx_table_elt_t           *accept_language;
+#endif
     ngx_connection_t          *fc;
     ngx_http_v2_stream_t      *stream;
     ngx_http_v2_out_frame_t   *frame;
@@ -950,31 +974,69 @@ ngx_http_v2_push_resource(ngx_http_reque
         return NGX_ABORT;
     }
 
-    if (authority->len == 0) {
-
-        len = 1 + NGX_HTTP_V2_INT_OCTETS + host->value.len;
-
-        tmp = ngx_palloc(r->pool, len);
-        pos = ngx_pnalloc(r->pool, len);
-
-        if (pos == NULL || tmp == NULL) {
+#if (NGX_HTTP_GZIP)
+    accept_encoding = r->headers_in.accept_encoding;
+#endif
+
+#if (NGX_HTTP_HEADERS)
+    accept_language = r->headers_in.accept_language;
+#endif
+
+    user_agent = r->headers_in.user_agent;
+
+    if (ctx->authority.len == 0) {
+
+        if (ngx_http_v2_indexed_header_encode(r, NGX_HTTP_V2_AUTHORITY_INDEX,
+                                              &host->value, &ctx->authority)
+            != NGX_OK)
+        {
             return NGX_ERROR;
         }
 
-        authority->data = pos;
-
-        *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_AUTHORITY_INDEX);
-        pos = ngx_http_v2_write_value(pos, host->value.data, host->value.len,
-                                      tmp);
-
-        authority->len = pos - authority->data;
+#if (NGX_HTTP_GZIP)
+        if (accept_encoding
+            && ngx_http_v2_indexed_header_encode(r,
+                   NGX_HTTP_V2_ACCEPT_ENCODING_INDEX,
+                   &accept_encoding->value, &ctx->accept_encoding)
+               != NGX_OK)
+        {
+            return NGX_ERROR;
+        }
+#endif
+
+#if (NGX_HTTP_HEADERS)
+        if (accept_language
+            && ngx_http_v2_indexed_header_encode(r,
+                   NGX_HTTP_V2_ACCEPT_LANGUAGE_INDEX,
+                   &accept_language->value, &ctx->accept_language)
+               != NGX_OK)
+        {
+            return NGX_ERROR;
+        }
+#endif
+
+        if (user_agent
+            && ngx_http_v2_indexed_header_encode(r,
+                   NGX_HTTP_V2_USER_AGENT_INDEX,
+                   &user_agent->value, &ctx->user_agent)
+               != NGX_OK)
+        {
+            return NGX_ERROR;
+        }
     }
 
     len = (h2c->table_update ? 1 : 0)
           + 1
           + 1 + NGX_HTTP_V2_INT_OCTETS + path->len
-          + authority->len
-          + 1;
+          + ctx->authority.len
+          + 1
+#if (NGX_HTTP_GZIP)
+          + ctx->accept_encoding.len
+#endif
+#if (NGX_HTTP_HEADERS)
+          + ctx->accept_language.len
+#endif
+          + ctx->user_agent.len;
 
     tmp = ngx_palloc(r->pool, len);
     pos = ngx_pnalloc(r->pool, len);
@@ -1006,7 +1068,7 @@ ngx_http_v2_push_resource(ngx_http_reque
     ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
                    "http2 push header: \":authority: %V\"", &host->value);
 
-    pos = ngx_cpymem(pos, authority->data, authority->len);
+    pos = ngx_cpymem(pos, ctx->authority.data, ctx->authority.len);
 
 #if (NGX_HTTP_SSL)
     if (fc->ssl) {
@@ -1022,6 +1084,36 @@ ngx_http_v2_push_resource(ngx_http_reque
         *pos++ = ngx_http_v2_indexed(NGX_HTTP_V2_SCHEME_HTTP_INDEX);
     }
 
+#if (NGX_HTTP_GZIP)
+    if (accept_encoding) {
+        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 push header: \"accept-encoding: %V\"",
+                       &accept_encoding->value);
+
+        pos = ngx_cpymem(pos, ctx->accept_encoding.data,
+                         ctx->accept_encoding.len);
+    }
+#endif
+
+#if (NGX_HTTP_HEADERS)
+    if (accept_language) {
+        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 push header: \"accept-language: %V\"",
+                       &accept_language->value);
+
+        pos = ngx_cpymem(pos, ctx->accept_language.data,
+                         ctx->accept_language.len);
+    }
+#endif
+
+    if (user_agent) {
+        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 push header: \"user-agent: %V\"",
+                       &user_agent->value);
+
+        pos = ngx_cpymem(pos, ctx->user_agent.data, ctx->user_agent.len);
+    }
+
     frame = ngx_http_v2_create_push_frame(r, start, pos);
     if (frame == NULL) {
         return NGX_ERROR;
@@ -1036,6 +1128,33 @@ ngx_http_v2_push_resource(ngx_http_reque
 }
 
 
+static ngx_int_t
+ngx_http_v2_indexed_header_encode(ngx_http_request_t *r, u_char index,
+    ngx_str_t *value, ngx_str_t *header)
+{
+    size_t   len;
+    u_char  *pos, *tmp;
+
+    len = 1 + NGX_HTTP_V2_INT_OCTETS + value->len;
+
+    tmp = ngx_palloc(r->pool, len);
+    pos = ngx_pnalloc(r->pool, len);
+
+    if (pos == NULL || tmp == NULL) {
+        return NGX_ERROR;
+    }
+
+    header->data = pos;
+
+    *pos++ = ngx_http_v2_inc_indexed(index);
+    pos = ngx_http_v2_write_value(pos, value->data, value->len, tmp);
+
+    header->len = pos - header->data;
+
+    return NGX_OK;
+}
+
+
 static u_char *
 ngx_http_v2_string_encode(u_char *dst, u_char *src, size_t len, u_char *tmp,
     ngx_uint_t lower)


From alessandro at ghedini.me  Mon Feb 12 15:17:32 2018
From: alessandro at ghedini.me (Alessandro Ghedini)
Date: Mon, 12 Feb 2018 15:17:32 +0000
Subject: [PATCH] HTTP/2: added support for setting custom push request
 headers
In-Reply-To: <20180212141155.GB67691@lo0.su>
References: <4eb0c9e8da0bc5206557.1518438913@mandy.ghedini.home>
 <20180212141155.GB67691@lo0.su>
Message-ID: <20180212151732.GA365@mandy>

On Mon, Feb 12, 2018 at 05:11:55PM +0300, Ruslan Ermilov wrote:
> On Mon, Feb 12, 2018 at 12:35:13PM +0000, Alessandro Ghedini wrote:
> > # HG changeset patch
> > # User Alessandro Ghedini <alessandro at ghedini.me>
> > # Date 1518438578 0
> > #      Mon Feb 12 12:29:38 2018 +0000
> > # Branch http2-push-header
> > # Node ID 4eb0c9e8da0bc52065578e4ee78df1833617ac35
> > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > HTTP/2: added support for setting custom push request headers.
> > 
> > This implementa the http2_push_header configuration directive that makes
> > it possible to set custom request headers for pushed requests.
> > 
> > Complex values are evaluated in the context of the original request,
> > so it's possible to copy its headers into pushed requests.
> > 
> > Example usage:
> > 
> >     http2_push_header User-Agent $http_user_agent;
> 
> If I'm not mistaken, both the original patch and this one
> do not send the copied headers in the PUSH_PROMISE frame.

They both do actually AFAICT, see the changes in ngx_http_v2_push_resource.
Though they don't use HPACK.

Cheers

From alessandro at ghedini.me  Mon Feb 12 15:30:21 2018
From: alessandro at ghedini.me (Alessandro Ghedini)
Date: Mon, 12 Feb 2018 15:30:21 +0000
Subject: [PATCH] HTTP/2: added support for setting custom push request
 headers
In-Reply-To: <20180212141155.GB67691@lo0.su>
References: <4eb0c9e8da0bc5206557.1518438913@mandy.ghedini.home>
 <20180212141155.GB67691@lo0.su>
Message-ID: <20180212153021.GC365@mandy>

On Mon, Feb 12, 2018 at 05:11:55PM +0300, Ruslan Ermilov wrote:
> On Mon, Feb 12, 2018 at 12:35:13PM +0000, Alessandro Ghedini wrote:
> > # HG changeset patch
> > # User Alessandro Ghedini <alessandro at ghedini.me>
> > # Date 1518438578 0
> > #      Mon Feb 12 12:29:38 2018 +0000
> > # Branch http2-push-header
> > # Node ID 4eb0c9e8da0bc52065578e4ee78df1833617ac35
> > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > HTTP/2: added support for setting custom push request headers.
> > 
> > This implementa the http2_push_header configuration directive that makes
> > it possible to set custom request headers for pushed requests.
> > 
> > Complex values are evaluated in the context of the original request,
> > so it's possible to copy its headers into pushed requests.
> > 
> > Example usage:
> > 
> >     http2_push_header User-Agent $http_user_agent;
> 
> If I'm not mistaken, both the original patch and this one
> do not send the copied headers in the PUSH_PROMISE frame.
> 
> Given the https://trac.nginx.org/nginx/ticket/1478, and some
> research of how different implementations behave, I prepared
> the following patch:

Also, your patch doesn't seem to work completely. You don't to copy the request
headers in the request's headers_in, which means they don't get processed by
NGINX, if the pushed resource is handled by NGINX itself.

E.g.

  daemon off;
  worker_processes  1;
  master_process off;

  events {
    worker_connections  1024;
  }

  http {
    access_log /dev/stdout;
    error_log  logs/error.log debug;

    server {
        listen       4433 ssl http2;

        server_name  localhost;

        ssl_certificate      cert.crt;
        ssl_certificate_key  cert.key;

        http2_max_concurrent_pushes   200;
        http2_push_preload            on;


        gzip on;
        gzip_types text/plain;
        gzip_min_length 1;

        location /hello {
            http2_push /a;

            root   html;
        }

        location /a {
            return 200 "a";
        }
    }
  }

 % nghttp https://127.0.0.1:4433/hello -vn  
[  0.001] Connected
[WARNING] Certificate verification failed: Hostname mismatch
The negotiated protocol: h2
[  0.003] send SETTINGS frame <length=12, flags=0x00, stream_id=0>
          (niv=2)
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
[  0.003] send PRIORITY frame <length=5, flags=0x00, stream_id=3>
          (dep_stream_id=0, weight=201, exclusive=0)
[  0.003] send PRIORITY frame <length=5, flags=0x00, stream_id=5>
          (dep_stream_id=0, weight=101, exclusive=0)
[  0.003] send PRIORITY frame <length=5, flags=0x00, stream_id=7>
          (dep_stream_id=0, weight=1, exclusive=0)
[  0.003] send PRIORITY frame <length=5, flags=0x00, stream_id=9>
          (dep_stream_id=7, weight=1, exclusive=0)
[  0.003] send PRIORITY frame <length=5, flags=0x00, stream_id=11>
          (dep_stream_id=3, weight=1, exclusive=0)
[  0.003] send HEADERS frame <length=44, flags=0x25, stream_id=13>
          ; END_STREAM | END_HEADERS | PRIORITY
          (padlen=0, dep_stream_id=11, weight=16, exclusive=0)
          ; Open new stream
          :method: GET
          :path: /hello
          :scheme: https
          :authority: 127.0.0.1:4433
          accept: */*
          accept-encoding: gzip, deflate
          user-agent: nghttp2/1.29.0
[  0.004] recv SETTINGS frame <length=18, flags=0x00, stream_id=0>
          (niv=3)
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):128]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65536]
          [SETTINGS_MAX_FRAME_SIZE(0x05):16777215]
[  0.004] recv WINDOW_UPDATE frame <length=4, flags=0x00, stream_id=0>
          (window_size_increment=2147418112)
[  0.004] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
          ; ACK
          (niv=0)
[  0.004] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
          ; ACK
          (niv=0)
[  0.004] recv (stream_id=13) :method: GET
[  0.004] recv (stream_id=13) :path: /a
[  0.004] recv (stream_id=13) :authority: 127.0.0.1:4433
[  0.004] recv (stream_id=13) :scheme: https
[  0.004] recv (stream_id=13) accept-encoding: gzip, deflate
[  0.004] recv (stream_id=13) user-agent: nghttp2/1.29.0
[  0.004] recv PUSH_PROMISE frame <length=47, flags=0x04, stream_id=13>
          ; END_HEADERS
          (padlen=0, promised_stream_id=2)
[  0.004] recv (stream_id=13) :status: 200
[  0.004] recv (stream_id=13) server: nginx/1.13.9
[  0.004] recv (stream_id=13) date: Mon, 12 Feb 2018 15:18:39 GMT
[  0.004] recv (stream_id=13) content-type: text/plain
[  0.004] recv (stream_id=13) last-modified: Thu, 08 Feb 2018 15:43:05 GMT
[  0.004] recv (stream_id=13) etag: W/"5a7c7009-1f5"
[  0.004] recv (stream_id=13) content-encoding: gzip
[  0.004] recv HEADERS frame <length=105, flags=0x04, stream_id=13>
          ; END_HEADERS
          (padlen=0)
          ; First response header
[  0.004] recv DATA frame <length=32, flags=0x01, stream_id=13>
          ; END_STREAM
[  0.004] recv (stream_id=2) :status: 200
[  0.004] recv (stream_id=2) server: nginx/1.13.9
[  0.004] recv (stream_id=2) date: Mon, 12 Feb 2018 15:18:39 GMT
[  0.004] recv (stream_id=2) content-type: text/plain
[  0.004] recv (stream_id=2) content-length: 1
[  0.004] recv HEADERS frame <length=48, flags=0x04, stream_id=2>
          ; END_HEADERS
          (padlen=0)
          ; First push response header
[  0.004] recv DATA frame <length=1, flags=0x01, stream_id=2>
          ; END_STREAM
[  0.004] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
          (last_stream_id=2, error_code=NO_ERROR(0x00), opaque_data(0)=[])

Notice how the response of the pushed request doesn't actually get compressed.
Additionally from the access logs I see:

  127.0.0.1 - - [12/Feb/2018:15:18:39 +0000] "GET /hello HTTP/2.0" 200 32 "-" "nghttp2/1.29.0"
  127.0.0.1 - - [12/Feb/2018:15:18:39 +0000] "GET /a HTTP/2.0" 200 1 "-" "-"

The second one is the pushed request, notice how it doesn't have a user agent
unlike the first request (which is the one coming from the client directly).

With my patch (the one with the custom config directive, but this mostly applies
to the other one as well) however I get this:

 % nghttp https://127.0.0.1:4433/hello -vn
[  0.002] Connected
[WARNING] Certificate verification failed: Hostname mismatch
The negotiated protocol: h2
[  0.004] send SETTINGS frame <length=12, flags=0x00, stream_id=0>
          (niv=2)
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
[  0.004] send PRIORITY frame <length=5, flags=0x00, stream_id=3>
          (dep_stream_id=0, weight=201, exclusive=0)
[  0.004] send PRIORITY frame <length=5, flags=0x00, stream_id=5>
          (dep_stream_id=0, weight=101, exclusive=0)
[  0.004] send PRIORITY frame <length=5, flags=0x00, stream_id=7>
          (dep_stream_id=0, weight=1, exclusive=0)
[  0.004] send PRIORITY frame <length=5, flags=0x00, stream_id=9>
          (dep_stream_id=7, weight=1, exclusive=0)
[  0.004] send PRIORITY frame <length=5, flags=0x00, stream_id=11>
          (dep_stream_id=3, weight=1, exclusive=0)
[  0.004] send HEADERS frame <length=44, flags=0x25, stream_id=13>
          ; END_STREAM | END_HEADERS | PRIORITY
          (padlen=0, dep_stream_id=11, weight=16, exclusive=0)
          ; Open new stream
          :method: GET
          :path: /hello
          :scheme: https
          :authority: 127.0.0.1:4433
          accept: */*
          accept-encoding: gzip, deflate
          user-agent: nghttp2/1.29.0
[  0.004] recv SETTINGS frame <length=18, flags=0x00, stream_id=0>
          (niv=3)
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):128]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65536]
          [SETTINGS_MAX_FRAME_SIZE(0x05):16777215]
[  0.004] recv WINDOW_UPDATE frame <length=4, flags=0x00, stream_id=0>
          (window_size_increment=2147418112)
[  0.004] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
          ; ACK
          (niv=0)
[  0.005] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
          ; ACK
          (niv=0)
[  0.005] recv (stream_id=13) :method: GET
[  0.005] recv (stream_id=13) :path: /a
[  0.005] recv (stream_id=13) :authority: 127.0.0.1:4433
[  0.005] recv (stream_id=13) :scheme: https
[  0.005] recv (stream_id=13) accept-encoding: gzip, deflate
[  0.005] recv (stream_id=13) user-agent: nghttp2/1.29.0
[  0.005] recv PUSH_PROMISE frame <length=67, flags=0x04, stream_id=13>
          ; END_HEADERS
          (padlen=0, promised_stream_id=2)
[  0.005] recv (stream_id=13) :status: 200
[  0.005] recv (stream_id=13) server: nginx/1.13.9
[  0.005] recv (stream_id=13) date: Mon, 12 Feb 2018 15:27:14 GMT
[  0.005] recv (stream_id=13) content-type: text/plain
[  0.005] recv (stream_id=13) last-modified: Thu, 08 Feb 2018 15:43:05 GMT
[  0.005] recv (stream_id=13) etag: W/"5a7c7009-1f5"
[  0.005] recv (stream_id=13) content-encoding: gzip
[  0.005] recv HEADERS frame <length=105, flags=0x04, stream_id=13>
          ; END_HEADERS
          (padlen=0)
          ; First response header
[  0.005] recv DATA frame <length=32, flags=0x01, stream_id=13>
          ; END_STREAM
[  0.006] recv (stream_id=2) :status: 200
[  0.006] recv (stream_id=2) server: nginx/1.13.9
[  0.006] recv (stream_id=2) date: Mon, 12 Feb 2018 15:27:14 GMT
[  0.006] recv (stream_id=2) content-type: text/plain
[  0.006] recv (stream_id=2) content-encoding: gzip
[  0.006] recv HEADERS frame <length=62, flags=0x04, stream_id=2>
          ; END_HEADERS
          (padlen=0)
          ; First push response header
[  0.006] recv DATA frame <length=21, flags=0x01, stream_id=2>
          ; END_STREAM
[  0.006] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
          (last_stream_id=2, error_code=NO_ERROR(0x00), opaque_data(0)=[])

The second response is compressed and the logs look like this:

  127.0.0.1 - - [12/Feb/2018:15:27:14 +0000] "GET /hello HTTP/2.0" 200 32 "-" "nghttp2/1.29.0"
  127.0.0.1 - - [12/Feb/2018:15:27:14 +0000] "GET /a HTTP/2.0" 200 21 "-" "nghttp2/1.29.0"

Cheers

From matthew.marangoni at gmail.com  Mon Feb 12 20:18:34 2018
From: matthew.marangoni at gmail.com (Matthew Marangoni)
Date: Mon, 12 Feb 2018 15:18:34 -0500
Subject: [PATCH 2 of 2] Updated __ORDER_LITTLE_ENDIAN__ to match NGX
 standards and changed to uint32_t data type
In-Reply-To: <1777243.zDY3EbAnj5@vbart-laptop>
References: <e84b174c99ca39155e80.1517675994@localhost.localdomain>
 <4fe15c57e1d5c7fa4e2d.1517675995@localhost.localdomain>
 <1777243.zDY3EbAnj5@vbart-laptop>
Message-ID: <CAPE-YcFTAooH=TseJno_Z2S835VfN8S9a_wCsL6r4n8N1dxZ8Q@mail.gmail.com>

Are there any existing macro's that determine if the platform has alignment
resolution issues?

On Sat, Feb 3, 2018 at 8:21 PM, Valentin V. Bartenev <vbart at nginx.com>
wrote:

> On Saturday, 3 February 2018 19:39:55 MSK Matthew Marangoni wrote:
> > # HG changeset patch
> > # User Matthew Marangoni <matthew.marangoni at gmail.com>
> > # Date 1517675878 28800
> > #      Sat Feb 03 08:37:58 2018 -0800
> > # Node ID 4fe15c57e1d5c7fa4e2daa365f82fd00d0a1f6f4
> > # Parent  e84b174c99ca39155e80dbb7458181e20190b70b
> > Updated __ORDER_LITTLE_ENDIAN__ to match NGX standards and changed to
> uint32_t data type
> >
> > diff -r e84b174c99ca -r 4fe15c57e1d5 src/core/ngx_murmurhash.c
> > --- a/src/core/ngx_murmurhash.c       Mon Jan 22 13:15:25 2018 -0800
> > +++ b/src/core/ngx_murmurhash.c       Sat Feb 03 08:37:58 2018 -0800
> > @@ -16,8 +16,8 @@
> >      h = 0 ^ len;
> >
> >      while (len >= 4) {
> > -#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
> > -        k = *(u_int32_t*)data;
> > +#if (NGX_HAVE_LITTLE_ENDIAN)
> > +        k = *(uint32_t*)data;
> >  #else
> >          k  = data[0];
> >          k |= data[1] << 8;
>
>
> Your patch will break nginx on platforms that have alignment requirements.
>
>   wbr, Valentin V. Bartenev
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180212/59c62772/attachment.html>

From yar at nginx.com  Tue Feb 13 11:02:30 2018
From: yar at nginx.com (Yaroslav Zhuravlev)
Date: Tue, 13 Feb 2018 14:02:30 +0300
Subject: nginx undocumented directives
In-Reply-To: <f8be7289-9327-13c2-7cc1-759f0db72c0a@csdoc.com>
References: <418fda51-227c-221f-804f-aa7e88322a53@csdoc.com>
 <20171228122533.GC15734@lo0.su> <20171228131735.GJ34136@mdounin.ru>
 <f8be7289-9327-13c2-7cc1-759f0db72c0a@csdoc.com>
Message-ID: <C83D2180-4550-4C20-846B-49469B930BE2@nginx.com>

Hello!

[?]

> 
> directives smtp_client_buffer and smtp_greeting_delay
> 
> still not documented.

[...]

Documentation added for smtp_client_buffer and smtp_greeting_delay:

http://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_client_buffer
http://nginx.org/en/docs/mail/ngx_mail_smtp_module.html#smtp_greeting_delay


Best regards,
yar

From alessandro at ghedini.me  Tue Feb 13 11:58:40 2018
From: alessandro at ghedini.me (Alessandro Ghedini)
Date: Tue, 13 Feb 2018 11:58:40 +0000
Subject: [PATCH] HTTP/2: added support for setting custom push request
 headers
In-Reply-To: <20180212153021.GC365@mandy>
References: <4eb0c9e8da0bc5206557.1518438913@mandy.ghedini.home>
 <20180212141155.GB67691@lo0.su> <20180212153021.GC365@mandy>
Message-ID: <20180213115840.GA15739@mandy>

On Mon, Feb 12, 2018 at 03:30:21PM +0000, Alessandro Ghedini wrote:
> On Mon, Feb 12, 2018 at 05:11:55PM +0300, Ruslan Ermilov wrote:
> > On Mon, Feb 12, 2018 at 12:35:13PM +0000, Alessandro Ghedini wrote:
> > > # HG changeset patch
> > > # User Alessandro Ghedini <alessandro at ghedini.me>
> > > # Date 1518438578 0
> > > #      Mon Feb 12 12:29:38 2018 +0000
> > > # Branch http2-push-header
> > > # Node ID 4eb0c9e8da0bc52065578e4ee78df1833617ac35
> > > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > > HTTP/2: added support for setting custom push request headers.
> > > 
> > > This implementa the http2_push_header configuration directive that makes
> > > it possible to set custom request headers for pushed requests.
> > > 
> > > Complex values are evaluated in the context of the original request,
> > > so it's possible to copy its headers into pushed requests.
> > > 
> > > Example usage:
> > > 
> > >     http2_push_header User-Agent $http_user_agent;
> > 
> > If I'm not mistaken, both the original patch and this one
> > do not send the copied headers in the PUSH_PROMISE frame.
> > 
> > Given the https://trac.nginx.org/nginx/ticket/1478, and some
> > research of how different implementations behave, I prepared
> > the following patch:
> 
> Also, your patch doesn't seem to work completely. You don't to copy the request
> headers in the request's headers_in, which means they don't get processed by
> NGINX, if the pushed resource is handled by NGINX itself.

If it's of any help, I merged your patch and mine into one, which copies the
headers (excluding Accept) into PUSH_PROMISE and r->headers_in like my original
patch did, as well as HPACK encode them into PUSH_PROMISE instead of writing
them as literal strings, as your patch did. This fixes the problems I mentioned
in my previous email.

# HG changeset patch
# User Alessandro Ghedini <alessandro at ghedini.me>
# Date 1518522249 0
#      Tue Feb 13 11:44:09 2018 +0000
# Branch push-headers
# Node ID bfd3019a0f63cf00364d44bab74b7864808abbd7
# Parent  8b0553239592f5d0fd419e5116b9d343838685cf
HTTP/2: push additional headers (ticket #1478).

When pushing requests, copy and push Accept-Encoding, Accept-Language,
and User-Agent header fields from the original request.

Includes changes made by Ruslan Ermilov.

diff -r 8b0553239592 -r bfd3019a0f63 src/http/v2/ngx_http_v2.c
--- a/src/http/v2/ngx_http_v2.c	Fri Feb 09 23:20:08 2018 +0300
+++ b/src/http/v2/ngx_http_v2.c	Tue Feb 13 11:44:09 2018 +0000
@@ -156,6 +156,8 @@
     ngx_str_t *value);
 static ngx_int_t ngx_http_v2_parse_authority(ngx_http_request_t *r,
     ngx_str_t *value);
+static ngx_int_t ngx_http_v2_copy_header(ngx_http_request_t *r,
+    ngx_table_elt_t *hdr);
 static ngx_int_t ngx_http_v2_construct_request_line(ngx_http_request_t *r);
 static ngx_int_t ngx_http_v2_cookie(ngx_http_request_t *r,
     ngx_http_v2_header_t *header);
@@ -2516,7 +2518,8 @@
 
 ngx_int_t
 ngx_http_v2_push_stream(ngx_http_v2_connection_t *h2c, ngx_uint_t depend,
-    size_t request_length, ngx_str_t *path, ngx_str_t *authority)
+    size_t request_length, ngx_str_t *path, ngx_str_t *authority,
+    ngx_http_headers_in_t *headers_in)
 {
     ngx_int_t              rc;
     ngx_str_t              value;
@@ -2605,6 +2608,28 @@
         goto error;
     }
 
+    rc = ngx_http_v2_copy_header(r, headers_in->user_agent);
+
+    if (rc != NGX_OK) {
+        goto error;
+    }
+
+#if (NGX_HTTP_GZIP)
+    rc = ngx_http_v2_copy_header(r, headers_in->accept_encoding);
+
+    if (rc != NGX_OK) {
+        goto error;
+    }
+#endif
+
+#if (NGX_HTTP_HEADERS)
+    rc = ngx_http_v2_copy_header(r, headers_in->accept_language);
+
+    if (rc != NGX_OK) {
+        goto error;
+    }
+#endif
+
     fc->write->handler = ngx_http_v2_run_request_handler;
     ngx_post_event(fc->write, &ngx_posted_events);
 
@@ -3479,6 +3504,63 @@
 
 
 static ngx_int_t
+ngx_http_v2_copy_header(ngx_http_request_t *r, ngx_table_elt_t *hdr)
+{
+    ngx_table_elt_t            *h;
+    ngx_http_header_t          *hh;
+    ngx_http_core_main_conf_t  *cmcf;
+
+    if (hdr == NULL) {
+        return NGX_OK;
+    }
+
+    h = ngx_list_push(&r->headers_in.headers);
+    if (h == NULL) {
+        return NGX_ERROR;
+    }
+
+    h->hash = hdr->hash;
+
+    h->key.len = hdr->key.len;
+
+    h->key.data = ngx_pnalloc(r->stream->pool, h->key.len + 1);
+    if (h->key.data == NULL) {
+        h->hash = 0;
+        return NGX_ERROR;
+    }
+
+    (void) ngx_cpystrn(h->key.data, hdr->key.data, h->key.len + 1);
+
+    h->value.len = hdr->value.len;
+
+    h->value.data = ngx_pnalloc(r->stream->pool, h->value.len + 1);
+    if (h->key.data == NULL) {
+        h->hash = 0;
+        return NGX_ERROR;
+    }
+
+    (void) ngx_cpystrn(h->value.data, hdr->value.data, h->value.len + 1);
+
+    h->lowcase_key = h->key.data;
+
+    cmcf = ngx_http_get_module_main_conf(r, ngx_http_core_module);
+
+    hh = ngx_hash_find(&cmcf->headers_in_hash, h->hash,
+                       h->lowcase_key, h->key.len);
+
+    if (hh == NULL) {
+        return NGX_ERROR;
+    }
+
+    if (hh->handler(r, h, hh->offset) != NGX_OK) {
+        return NGX_ERROR;
+    }
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
 ngx_http_v2_construct_request_line(ngx_http_request_t *r)
 {
     u_char  *p;
diff -r 8b0553239592 -r bfd3019a0f63 src/http/v2/ngx_http_v2.h
--- a/src/http/v2/ngx_http_v2.h	Fri Feb 09 23:20:08 2018 +0300
+++ b/src/http/v2/ngx_http_v2.h	Tue Feb 13 11:44:09 2018 +0000
@@ -285,7 +285,7 @@
 
 ngx_int_t ngx_http_v2_push_stream(ngx_http_v2_connection_t *h2c,
     ngx_uint_t depend, size_t request_length, ngx_str_t *path,
-    ngx_str_t *authority);
+    ngx_str_t *authority, ngx_http_headers_in_t *headers_in);
 
 void ngx_http_v2_close_stream(ngx_http_v2_stream_t *stream, ngx_int_t rc);
 
diff -r 8b0553239592 -r bfd3019a0f63 src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c	Fri Feb 09 23:20:08 2018 +0300
+++ b/src/http/v2/ngx_http_v2_filter_module.c	Tue Feb 13 11:44:09 2018 +0000
@@ -50,20 +50,37 @@
 #define NGX_HTTP_V2_STATUS_404_INDEX      13
 #define NGX_HTTP_V2_STATUS_500_INDEX      14
 
+#define NGX_HTTP_V2_ACCEPT_ENCODING_INDEX 16
+#define NGX_HTTP_V2_ACCEPT_LANGUAGE_INDEX 17
 #define NGX_HTTP_V2_CONTENT_LENGTH_INDEX  28
 #define NGX_HTTP_V2_CONTENT_TYPE_INDEX    31
 #define NGX_HTTP_V2_DATE_INDEX            33
 #define NGX_HTTP_V2_LAST_MODIFIED_INDEX   44
 #define NGX_HTTP_V2_LOCATION_INDEX        46
 #define NGX_HTTP_V2_SERVER_INDEX          54
+#define NGX_HTTP_V2_USER_AGENT_INDEX      58
 #define NGX_HTTP_V2_VARY_INDEX            59
 
 #define NGX_HTTP_V2_NO_TRAILERS           (ngx_http_v2_out_frame_t *) -1
 
 
+typedef struct {
+    ngx_str_t                             authority;
+#if (NGX_HTTP_GZIP)
+    ngx_str_t                             accept_encoding;
+#endif
+#if (NGX_HTTP_HEADERS)
+    ngx_str_t                             accept_language;
+#endif
+    ngx_str_t                             user_agent;
+} ngx_http_v2_push_ctx_t;
+
+
 static ngx_int_t ngx_http_v2_push_resources(ngx_http_request_t *r);
 static ngx_int_t ngx_http_v2_push_resource(ngx_http_request_t *r,
-    ngx_str_t *path, ngx_str_t *authority);
+    ngx_str_t *path, ngx_http_v2_push_ctx_t *ctx);
+static ngx_int_t ngx_http_v2_indexed_header_encode(ngx_http_request_t *r,
+    u_char index, ngx_str_t *value, ngx_str_t *header);
 
 static u_char *ngx_http_v2_string_encode(u_char *dst, u_char *src, size_t len,
     u_char *tmp, ngx_uint_t lower);
@@ -685,16 +702,17 @@
 {
     u_char                     *start, *end, *last;
     ngx_int_t                   rc;
-    ngx_str_t                   path, authority;
+    ngx_str_t                   path;
     ngx_uint_t                  i, push;
     ngx_table_elt_t           **h;
     ngx_http_v2_loc_conf_t     *h2lcf;
+    ngx_http_v2_push_ctx_t      ctx;
     ngx_http_complex_value_t   *pushes;
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                    "http2 push resources");
 
-    ngx_str_null(&authority);
+    ngx_memzero(&ctx, sizeof(ngx_http_v2_push_ctx_t));
 
     h2lcf = ngx_http_get_module_loc_conf(r, ngx_http_v2_module);
 
@@ -715,7 +733,7 @@
                 continue;
             }
 
-            rc = ngx_http_v2_push_resource(r, &path, &authority);
+            rc = ngx_http_v2_push_resource(r, &path, &ctx);
 
             if (rc == NGX_ERROR) {
                 return NGX_ERROR;
@@ -880,7 +898,7 @@
         if (push && path.len
             && !(path.len > 1 && path.data[0] == '/' && path.data[1] == '/'))
         {
-            rc = ngx_http_v2_push_resource(r, &path, &authority);
+            rc = ngx_http_v2_push_resource(r, &path, &ctx);
 
             if (rc == NGX_ERROR) {
                 return NGX_ERROR;
@@ -905,11 +923,17 @@
 
 static ngx_int_t
 ngx_http_v2_push_resource(ngx_http_request_t *r, ngx_str_t *path,
-    ngx_str_t *authority)
+    ngx_http_v2_push_ctx_t *ctx)
 {
     u_char                    *start, *pos, *tmp;
     size_t                     len;
-    ngx_table_elt_t           *host;
+    ngx_table_elt_t           *host, *user_agent;
+#if (NGX_HTTP_GZIP)
+    ngx_table_elt_t           *accept_encoding;
+#endif
+#if (NGX_HTTP_HEADERS)
+    ngx_table_elt_t           *accept_language;
+#endif
     ngx_connection_t          *fc;
     ngx_http_v2_stream_t      *stream;
     ngx_http_v2_out_frame_t   *frame;
@@ -950,31 +974,69 @@
         return NGX_ABORT;
     }
 
-    if (authority->len == 0) {
-
-        len = 1 + NGX_HTTP_V2_INT_OCTETS + host->value.len;
-
-        tmp = ngx_palloc(r->pool, len);
-        pos = ngx_pnalloc(r->pool, len);
-
-        if (pos == NULL || tmp == NULL) {
+#if (NGX_HTTP_GZIP)
+    accept_encoding = r->headers_in.accept_encoding;
+#endif
+
+#if (NGX_HTTP_HEADERS)
+    accept_language = r->headers_in.accept_language;
+#endif
+
+    user_agent = r->headers_in.user_agent;
+
+    if (ctx->authority.len == 0) {
+
+        if (ngx_http_v2_indexed_header_encode(r, NGX_HTTP_V2_AUTHORITY_INDEX,
+                                              &host->value, &ctx->authority)
+            != NGX_OK)
+        {
             return NGX_ERROR;
         }
 
-        authority->data = pos;
-
-        *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_AUTHORITY_INDEX);
-        pos = ngx_http_v2_write_value(pos, host->value.data, host->value.len,
-                                      tmp);
-
-        authority->len = pos - authority->data;
+#if (NGX_HTTP_GZIP)
+        if (accept_encoding
+            && ngx_http_v2_indexed_header_encode(r,
+                   NGX_HTTP_V2_ACCEPT_ENCODING_INDEX,
+                   &accept_encoding->value, &ctx->accept_encoding)
+               != NGX_OK)
+        {
+            return NGX_ERROR;
+        }
+#endif
+
+#if (NGX_HTTP_HEADERS)
+        if (accept_language
+            && ngx_http_v2_indexed_header_encode(r,
+                   NGX_HTTP_V2_ACCEPT_LANGUAGE_INDEX,
+                   &accept_language->value, &ctx->accept_language)
+               != NGX_OK)
+        {
+            return NGX_ERROR;
+        }
+#endif
+
+        if (user_agent
+            && ngx_http_v2_indexed_header_encode(r,
+                   NGX_HTTP_V2_USER_AGENT_INDEX,
+                   &user_agent->value, &ctx->user_agent)
+               != NGX_OK)
+        {
+            return NGX_ERROR;
+        }
     }
 
     len = (h2c->table_update ? 1 : 0)
           + 1
           + 1 + NGX_HTTP_V2_INT_OCTETS + path->len
-          + authority->len
-          + 1;
+          + ctx->authority.len
+          + 1
+#if (NGX_HTTP_GZIP)
+          + ctx->accept_encoding.len
+#endif
+#if (NGX_HTTP_HEADERS)
+          + ctx->accept_language.len
+#endif
+          + ctx->user_agent.len;
 
     tmp = ngx_palloc(r->pool, len);
     pos = ngx_pnalloc(r->pool, len);
@@ -1006,7 +1068,7 @@
     ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
                    "http2 push header: \":authority: %V\"", &host->value);
 
-    pos = ngx_cpymem(pos, authority->data, authority->len);
+    pos = ngx_cpymem(pos, ctx->authority.data, ctx->authority.len);
 
 #if (NGX_HTTP_SSL)
     if (fc->ssl) {
@@ -1022,6 +1084,36 @@
         *pos++ = ngx_http_v2_indexed(NGX_HTTP_V2_SCHEME_HTTP_INDEX);
     }
 
+#if (NGX_HTTP_GZIP)
+    if (accept_encoding) {
+        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 push header: \"accept-encoding: %V\"",
+                       &accept_encoding->value);
+
+        pos = ngx_cpymem(pos, ctx->accept_encoding.data,
+                         ctx->accept_encoding.len);
+    }
+#endif
+
+#if (NGX_HTTP_HEADERS)
+    if (accept_language) {
+        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 push header: \"accept-language: %V\"",
+                       &accept_language->value);
+
+        pos = ngx_cpymem(pos, ctx->accept_language.data,
+                         ctx->accept_language.len);
+    }
+#endif
+
+    if (user_agent) {
+        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 push header: \"user-agent: %V\"",
+                       &user_agent->value);
+
+        pos = ngx_cpymem(pos, ctx->user_agent.data, ctx->user_agent.len);
+    }
+
     frame = ngx_http_v2_create_push_frame(r, start, pos);
     if (frame == NULL) {
         return NGX_ERROR;
@@ -1032,7 +1124,34 @@
     stream->queued++;
 
     return ngx_http_v2_push_stream(h2c, stream->node->id, pos - start,
-                                   path, &host->value);
+                                   path, &host->value, &r->headers_in);
+}
+
+
+static ngx_int_t
+ngx_http_v2_indexed_header_encode(ngx_http_request_t *r, u_char index,
+    ngx_str_t *value, ngx_str_t *header)
+{
+    size_t   len;
+    u_char  *pos, *tmp;
+
+    len = 1 + NGX_HTTP_V2_INT_OCTETS + value->len;
+
+    tmp = ngx_palloc(r->pool, len);
+    pos = ngx_pnalloc(r->pool, len);
+
+    if (pos == NULL || tmp == NULL) {
+        return NGX_ERROR;
+    }
+
+    header->data = pos;
+
+    *pos++ = ngx_http_v2_inc_indexed(index);
+    pos = ngx_http_v2_write_value(pos, value->data, value->len, tmp);
+
+    header->len = pos - header->data;
+
+    return NGX_OK;
 }
 
 



From leon at darkk.net.ru  Tue Feb 13 12:08:53 2018
From: leon at darkk.net.ru (Leonid Evdokimov)
Date: Tue, 13 Feb 2018 15:08:53 +0300
Subject: [patch]: document SHA-2 support in glibc crypt()
In-Reply-To: <20171010172943.GR75166@mdounin.ru>
References: <20171009165402.GB5231@darkk-ya-laptop>
 <20171009165816.GA5388@darkk-ya-laptop>
 <20171009174407.GB5549@darkk-ya-laptop> <20171009194411.GH75166@mdounin.ru>
 <20171009201847.GA8279@darkk-ya-laptop> <20171010172943.GR75166@mdounin.ru>
Message-ID: <CAB-U=tAEZCeNh5VHcD6Qd1s7Ci6P_+RH1m5T3NL9=Cu9jop+Zg@mail.gmail.com>

On Tue, Oct 10, 2017 at 8:29 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> The paragraph in question is expected to say that nginx uses the
> crypt() function as provided by system libraries.  If it is not
> clear, we can consider improving the wording,

Yep, I think that it would be perfect to mention libc somewhere,
because `crypt()` is really loaded with possible interpretations.

Maybe something like "the crypt() function from libc", "system crypt()
function" or "native crypt() function" will be a bit more explicit.
Probably, system-specific examples are not useful in this case.

-- 
WBRBW, Leonid Evdokimov, xmpp:leon at darkk.net.ru http://darkk.net.ru
tel:+79816800702
PGP: 6691 DE6B 4CCD C1C1 76A0  0D4A E1F2 A980 7F50 FAB2

From alessandro at ghedini.me  Tue Feb 13 12:21:36 2018
From: alessandro at ghedini.me (Alessandro Ghedini)
Date: Tue, 13 Feb 2018 12:21:36 +0000
Subject: [PATCH] HTTP/2: expose function to push single resource to modules
In-Reply-To: <20180209073559.GE75377@lo0.su>
References: <1bb98b06d5536dfc80a4.1518108779@mandy.ghedini.home>
 <20180208190027.GN24410@mdounin.ru> <20180208194825.GA6873@pinky>
 <20180209073559.GE75377@lo0.su>
Message-ID: <20180213122136.GA20751@mandy>

On Fri, Feb 09, 2018 at 10:35:59AM +0300, Ruslan Ermilov wrote:
> On Thu, Feb 08, 2018 at 07:48:25PM +0000, Alessandro Ghedini wrote:
> > On Thu, Feb 08, 2018 at 10:00:27PM +0300, Maxim Dounin wrote:
> > > On Thu, Feb 08, 2018 at 04:52:59PM +0000, Alessandro Ghedini wrote:
> > > 
> > > > # HG changeset patch
> > > > # User Alessandro Ghedini <alessandro at ghedini.me>
> > > > # Date 1518108716 0
> > > > #      Thu Feb 08 16:51:56 2018 +0000
> > > > # Branch expose-push
> > > > # Node ID 1bb98b06d5536dfc80a407aabd8d06f9309f8df6
> > > > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > > > HTTP/2: expose function to push single resource to modules.
> > > > 
> > > > This makes it possible for 3rd party modules to implement alternative
> > > > methods for deciding which resources to push to clients on a per-request
> > > > basis (e.g. by parsing HTML from the response body, by using a custom
> > > > Link header parser, ...).
> > > > 
> > > > No functional changes.
> > > 
> > > Not sure this is a good idea.
> > > 
> > > You may consider exposing a variable to be used in http2_push 
> > > instead.
> > 
> > Right, the problem is that as far as I can tell http2_push only supports a
> > single resource, even when a variable is used, so it wouldn't be possible to
> > push multiple resources without specifying multiple http2_push directives,
> > each with its own variable, and even then you'd only have a fixed number of
> > resources that can be pushed, which wouldn't work well when the number of
> > resources changes depending on each request/response.
> > 
> > So in the end exposing the internal functions to modules seemed better than
> > just trying to make http2_push support multiple resources per directive,
> > which would add complexity to NGINX itself rather than the external modules
> > (though I can do that if you think it would be a better solution).
> 
> We've also considered adding support for the X-Accel-Push header, but
> decided not to implement it at this time.  If implemented, there could
> be multiple X-Accel-Push headers in the proxied response.

That might work for us, but it's a somewhat awkward interface to use from
inside a module, so I'd still prefer something more direct, and as I said,
exposing the function to modules seemed the least invasive change to NGINX.

Could you please expand a bit on why you think this might be a bad idea?

In any case I can look into implementing X-Accel-Push support if you don't
plan on doing it yourself.

Cheers

From mdounin at mdounin.ru  Tue Feb 13 12:30:32 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 13 Feb 2018 15:30:32 +0300
Subject: [PATCH 2 of 2] Updated __ORDER_LITTLE_ENDIAN__ to match NGX
 standards and changed to uint32_t data type
In-Reply-To: <CAPE-YcFTAooH=TseJno_Z2S835VfN8S9a_wCsL6r4n8N1dxZ8Q@mail.gmail.com>
References: <e84b174c99ca39155e80.1517675994@localhost.localdomain>
 <4fe15c57e1d5c7fa4e2d.1517675995@localhost.localdomain>
 <1777243.zDY3EbAnj5@vbart-laptop>
 <CAPE-YcFTAooH=TseJno_Z2S835VfN8S9a_wCsL6r4n8N1dxZ8Q@mail.gmail.com>
Message-ID: <20180213123032.GH24410@mdounin.ru>

Hello!

On Mon, Feb 12, 2018 at 03:18:34PM -0500, Matthew Marangoni wrote:

> Are there any existing macro's that determine if the platform has alignment
> resolution issues?

There is, usual check is "#if (NGX_HAVE_LITTLE_ENDIAN && 
NGX_HAVE_NONALIGNED)", take a look at src/core/ngx_md5.c or for an 
example.

On the other hand, I would rather recommend to avoid such 
micro-optimizations unless you have a really good reason for them 
(read: unless you have a real-world use case where the change 
results in a statistically significant performance improvement).

[...]

-- 
Maxim Dounin
http://mdounin.ru/

From mdounin at mdounin.ru  Tue Feb 13 13:31:17 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 13 Feb 2018 16:31:17 +0300
Subject: [PATCH] HTTP/2: expose function to push single resource to modules
In-Reply-To: <20180213122136.GA20751@mandy>
References: <1bb98b06d5536dfc80a4.1518108779@mandy.ghedini.home>
 <20180208190027.GN24410@mdounin.ru> <20180208194825.GA6873@pinky>
 <20180209073559.GE75377@lo0.su> <20180213122136.GA20751@mandy>
Message-ID: <20180213133117.GK24410@mdounin.ru>

Hello!

On Tue, Feb 13, 2018 at 12:21:36PM +0000, Alessandro Ghedini wrote:

> On Fri, Feb 09, 2018 at 10:35:59AM +0300, Ruslan Ermilov wrote:
> > On Thu, Feb 08, 2018 at 07:48:25PM +0000, Alessandro Ghedini wrote:
> > > On Thu, Feb 08, 2018 at 10:00:27PM +0300, Maxim Dounin wrote:
> > > > On Thu, Feb 08, 2018 at 04:52:59PM +0000, Alessandro Ghedini wrote:
> > > > 
> > > > > # HG changeset patch
> > > > > # User Alessandro Ghedini <alessandro at ghedini.me>
> > > > > # Date 1518108716 0
> > > > > #      Thu Feb 08 16:51:56 2018 +0000
> > > > > # Branch expose-push
> > > > > # Node ID 1bb98b06d5536dfc80a407aabd8d06f9309f8df6
> > > > > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > > > > HTTP/2: expose function to push single resource to modules.
> > > > > 
> > > > > This makes it possible for 3rd party modules to implement alternative
> > > > > methods for deciding which resources to push to clients on a per-request
> > > > > basis (e.g. by parsing HTML from the response body, by using a custom
> > > > > Link header parser, ...).
> > > > > 
> > > > > No functional changes.
> > > > 
> > > > Not sure this is a good idea.
> > > > 
> > > > You may consider exposing a variable to be used in http2_push 
> > > > instead.
> > > 
> > > Right, the problem is that as far as I can tell http2_push only supports a
> > > single resource, even when a variable is used, so it wouldn't be possible to
> > > push multiple resources without specifying multiple http2_push directives,
> > > each with its own variable, and even then you'd only have a fixed number of
> > > resources that can be pushed, which wouldn't work well when the number of
> > > resources changes depending on each request/response.
> > > 
> > > So in the end exposing the internal functions to modules seemed better than
> > > just trying to make http2_push support multiple resources per directive,
> > > which would add complexity to NGINX itself rather than the external modules
> > > (though I can do that if you think it would be a better solution).
> > 
> > We've also considered adding support for the X-Accel-Push header, but
> > decided not to implement it at this time.  If implemented, there could
> > be multiple X-Accel-Push headers in the proxied response.
> 
> That might work for us, but it's a somewhat awkward interface to use from
> inside a module, so I'd still prefer something more direct, and as I said,
> exposing the function to modules seemed the least invasive change to NGINX.
> 
> Could you please expand a bit on why you think this might be a bad idea?
> 
> In any case I can look into implementing X-Accel-Push support if you don't
> plan on doing it yourself.

I presonally think that X-Accel-Push is not needed and should not 
be added given we already have http2_push.  Also, HTTP/2 push as a 
technology has enough design problems to introduce additional 
interfaces.

If you want to use HTTP2 pushes from a module, consider either 
using http2_push with variables as previously suggested (right now 
you can use multiple http2_push directives if you want to push 
multiple resources), or adding "Link: rel=preload" headers and 
using http2_push_preload. 

-- 
Maxim Dounin
http://mdounin.ru/

From ru at nginx.com  Wed Feb 14 19:47:43 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Wed, 14 Feb 2018 22:47:43 +0300
Subject: [PATCH] HTTP/2: added support for setting custom push request
 headers
In-Reply-To: <20180213115840.GA15739@mandy>
References: <4eb0c9e8da0bc5206557.1518438913@mandy.ghedini.home>
 <20180212141155.GB67691@lo0.su> <20180212153021.GC365@mandy>
 <20180213115840.GA15739@mandy>
Message-ID: <20180214194743.GL67691@lo0.su>

On Tue, Feb 13, 2018 at 11:58:40AM +0000, Alessandro Ghedini wrote:
> If it's of any help, I merged your patch and mine into one, which copies the
> headers (excluding Accept) into PUSH_PROMISE and r->headers_in like my original
> patch did, as well as HPACK encode them into PUSH_PROMISE instead of writing
> them as literal strings, as your patch did. This fixes the problems I mentioned
> in my previous email.

To keep you updated, we're cultivating these patches:

# HG changeset patch
# User Ruslan Ermilov <ru at nginx.com>
# Date 1518609549 -10800
#      Wed Feb 14 14:59:09 2018 +0300
# Node ID 9cd08f096c366771ffbebd2872883be04903d425
# Parent  8b0553239592f5d0fd419e5116b9d343838685cf
Expose more headers with NGX_HTTP_HEADERS.

diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c
+++ b/src/http/ngx_http_request.c
@@ -140,7 +140,7 @@ ngx_http_header_t  ngx_http_headers_in[]
                  offsetof(ngx_http_headers_in_t, upgrade),
                  ngx_http_process_header_line },
 
-#if (NGX_HTTP_GZIP)
+#if (NGX_HTTP_GZIP || NGX_HTTP_HEADERS)
     { ngx_string("Accept-Encoding"),
                  offsetof(ngx_http_headers_in_t, accept_encoding),
                  ngx_http_process_header_line },
diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -200,7 +200,7 @@ typedef struct {
     ngx_table_elt_t                  *expect;
     ngx_table_elt_t                  *upgrade;
 
-#if (NGX_HTTP_GZIP)
+#if (NGX_HTTP_GZIP || NGX_HTTP_HEADERS)
     ngx_table_elt_t                  *accept_encoding;
     ngx_table_elt_t                  *via;
 #endif
# HG changeset patch
# User Ruslan Ermilov <ru at nginx.com>
# Date 1518634812 -10800
#      Wed Feb 14 22:00:12 2018 +0300
# Node ID cd54884b375b70ecd358358e5e78f43e70b6163a
# Parent  9cd08f096c366771ffbebd2872883be04903d425
HTTP/2: push additional request headers (closes #1478).

The Accept-Encoding, Accept-Language, and User-Agent header fields
are now copied from the original request to pushed requests.

diff --git a/auto/modules b/auto/modules
--- a/auto/modules
+++ b/auto/modules
@@ -420,6 +420,7 @@ if [ $HTTP = YES ]; then
 
     if [ $HTTP_V2 = YES ]; then
         have=NGX_HTTP_V2 . auto/have
+        have=NGX_HTTP_HEADERS . auto/have
 
         ngx_module_name=ngx_http_v2_module
         ngx_module_incs=src/http/v2
diff --git a/src/http/v2/ngx_http_v2.c b/src/http/v2/ngx_http_v2.c
--- a/src/http/v2/ngx_http_v2.c
+++ b/src/http/v2/ngx_http_v2.c
@@ -11,6 +11,14 @@
 #include <ngx_http_v2_module.h>
 
 
+typedef struct {
+    ngx_str_t           name;
+    ngx_uint_t          offset;
+    ngx_uint_t          hash;
+    ngx_http_header_t  *hh;
+} ngx_http_v2_parse_header_t;
+
+
 /* errors */
 #define NGX_HTTP_V2_NO_ERROR                     0x0
 #define NGX_HTTP_V2_PROTOCOL_ERROR               0x1
@@ -156,6 +164,8 @@ static ngx_int_t ngx_http_v2_parse_schem
     ngx_str_t *value);
 static ngx_int_t ngx_http_v2_parse_authority(ngx_http_request_t *r,
     ngx_str_t *value);
+static ngx_int_t ngx_http_v2_parse_header(ngx_http_request_t *r,
+    ngx_http_v2_parse_header_t *header, ngx_str_t *value);
 static ngx_int_t ngx_http_v2_construct_request_line(ngx_http_request_t *r);
 static ngx_int_t ngx_http_v2_cookie(ngx_http_request_t *r,
     ngx_http_v2_header_t *header);
@@ -201,6 +211,23 @@ static ngx_http_v2_handler_pt ngx_http_v
     (sizeof(ngx_http_v2_frame_states) / sizeof(ngx_http_v2_handler_pt))
 
 
+static ngx_http_v2_parse_header_t  ngx_http_v2_parse_headers[] = {
+    { ngx_string("host"),
+      offsetof(ngx_http_headers_in_t, host), 0, NULL },
+
+    { ngx_string("accept-encoding"),
+      offsetof(ngx_http_headers_in_t, accept_encoding), 0, NULL },
+
+    { ngx_string("accept-language"),
+      offsetof(ngx_http_headers_in_t, accept_language), 0, NULL },
+
+    { ngx_string("user-agent"),
+      offsetof(ngx_http_headers_in_t, user_agent), 0, NULL },
+
+    { ngx_null_string, 0, 0, NULL }
+};
+
+
 void
 ngx_http_v2_init(ngx_event_t *rev)
 {
@@ -2514,21 +2541,25 @@ ngx_http_v2_parse_int(ngx_http_v2_connec
 }
 
 
-ngx_int_t
-ngx_http_v2_push_stream(ngx_http_v2_connection_t *h2c, ngx_uint_t depend,
-    size_t request_length, ngx_str_t *path, ngx_str_t *authority)
+ngx_http_v2_stream_t *
+ngx_http_v2_push_stream(ngx_http_v2_stream_t *parent, ngx_str_t *path)
 {
-    ngx_int_t              rc;
-    ngx_str_t              value;
-    ngx_connection_t      *fc;
-    ngx_http_request_t    *r;
-    ngx_http_v2_node_t    *node;
-    ngx_http_v2_stream_t  *stream;
+    ngx_int_t                     rc;
+    ngx_str_t                     value;
+    ngx_table_elt_t             **h;
+    ngx_connection_t             *fc;
+    ngx_http_request_t           *r;
+    ngx_http_v2_node_t           *node;
+    ngx_http_v2_stream_t         *stream;
+    ngx_http_v2_connection_t     *h2c;
+    ngx_http_v2_parse_header_t   *header;
+
+    h2c = parent->connection;
 
     node = ngx_http_v2_get_node_by_id(h2c, h2c->last_push, 1);
 
     if (node == NULL) {
-        return NGX_ERROR;
+        return NULL;
     }
 
     if (node->parent) {
@@ -2538,19 +2569,17 @@ ngx_http_v2_push_stream(ngx_http_v2_conn
 
     stream = ngx_http_v2_create_stream(h2c, 1);
     if (stream == NULL) {
-        return NGX_ERROR;
+        return NULL;
     }
 
     stream->pool = ngx_create_pool(1024, h2c->connection->log);
     if (stream->pool == NULL) {
-        return NGX_ERROR;
+        return NULL;
     }
 
     r = stream->request;
     fc = r->connection;
 
-    r->request_length = request_length;
-
     stream->in_closed = 1;
     stream->node = node;
 
@@ -2559,10 +2588,10 @@ ngx_http_v2_push_stream(ngx_http_v2_conn
     ngx_log_debug2(NGX_LOG_DEBUG_HTTP, h2c->connection->log, 0,
                    "http2 push stream sid:%ui "
                    "depends on %ui excl:0 weight:16",
-                   h2c->last_push, depend);
+                   h2c->last_push, parent->node->id);
 
     node->weight = NGX_HTTP_V2_DEFAULT_WEIGHT;
-    ngx_http_v2_set_dependency(h2c, node, depend, 0);
+    ngx_http_v2_set_dependency(h2c, node, parent->node->id, 0);
 
     r->method_name = ngx_http_core_get_method;
     r->method = NGX_HTTP_GET;
@@ -2579,51 +2608,64 @@ ngx_http_v2_push_stream(ngx_http_v2_conn
         r->schema_end = r->schema_start + 4;
     }
 
-    value.len = authority->len;
-
-    value.data = ngx_pstrdup(stream->pool, authority);
+    value.data = ngx_pstrdup(stream->pool, path);
     if (value.data == NULL) {
-        return NGX_ERROR;
-    }
-
-    rc = ngx_http_v2_parse_authority(r, &value);
-
-    if (rc != NGX_OK) {
-        goto error;
+        return NULL;
     }
 
     value.len = path->len;
 
-    value.data = ngx_pstrdup(stream->pool, path);
-    if (value.data == NULL) {
-        return NGX_ERROR;
-    }
-
     rc = ngx_http_v2_parse_path(r, &value);
 
     if (rc != NGX_OK) {
         goto error;
     }
 
+    for (header = ngx_http_v2_parse_headers; header->name.len; header++) {
+        h = (ngx_table_elt_t **)
+                ((char *) &parent->request->headers_in + header->offset);
+
+        if (*h == NULL) {
+            continue;
+        }
+
+        value.len = (*h)->value.len;
+
+        value.data = ngx_pnalloc(stream->pool, value.len + 1);
+        if (value.data == NULL) {
+            return NULL;
+        }
+
+        ngx_memcpy(value.data, (*h)->value.data, value.len);
+        value.data[value.len] = '\0';
+
+        rc = ngx_http_v2_parse_header(r, header, &value);
+
+        if (rc != NGX_OK) {
+            goto error;
+        }
+    }
+
     fc->write->handler = ngx_http_v2_run_request_handler;
     ngx_post_event(fc->write, &ngx_posted_events);
 
-    return NGX_OK;
+    return stream;
 
 error:
 
     if (rc == NGX_ABORT) {
-        return NGX_ERROR;
+        /* header handler has already finalized request */
+        return NULL;
     }
 
     if (rc == NGX_DECLINED) {
         ngx_http_finalize_request(r, NGX_HTTP_BAD_REQUEST);
-        return NGX_ERROR;
+        return NULL;
     }
 
     (void) ngx_http_v2_connection_error(h2c, NGX_HTTP_V2_INTERNAL_ERROR);
 
-    return NGX_ERROR;
+    return NULL;
 }
 
 
@@ -3436,41 +3478,45 @@ ngx_http_v2_parse_scheme(ngx_http_reques
 static ngx_int_t
 ngx_http_v2_parse_authority(ngx_http_request_t *r, ngx_str_t *value)
 {
+    return ngx_http_v2_parse_header(r, &ngx_http_v2_parse_headers[0], value);
+}
+
+
+static ngx_int_t
+ngx_http_v2_parse_header(ngx_http_request_t *r,
+    ngx_http_v2_parse_header_t *header, ngx_str_t *value)
+{
     ngx_table_elt_t            *h;
-    ngx_http_header_t          *hh;
     ngx_http_core_main_conf_t  *cmcf;
 
-    static ngx_str_t host = ngx_string("host");
-
     h = ngx_list_push(&r->headers_in.headers);
     if (h == NULL) {
         return NGX_ERROR;
     }
 
-    h->hash = ngx_hash_key(host.data, host.len);
-
-    h->key.len = host.len;
-    h->key.data = host.data;
+    h->key.len = header->name.len;
+    h->key.data = header->name.data;
+    h->lowcase_key = header->name.data;
+
+    if (header->hh == NULL) {
+        header->hash = ngx_hash_key(header->name.data, header->name.len);
+
+        cmcf = ngx_http_get_module_main_conf(r, ngx_http_core_module);
+
+        header->hh = ngx_hash_find(&cmcf->headers_in_hash, header->hash,
+                                   h->lowcase_key, h->key.len);
+        if (header->hh == NULL) {
+            return NGX_ERROR;
+        }
+    }
+
+    h->hash = header->hash;
 
     h->value.len = value->len;
     h->value.data = value->data;
 
-    h->lowcase_key = host.data;
-
-    cmcf = ngx_http_get_module_main_conf(r, ngx_http_core_module);
-
-    hh = ngx_hash_find(&cmcf->headers_in_hash, h->hash,
-                       h->lowcase_key, h->key.len);
-
-    if (hh == NULL) {
-        return NGX_ERROR;
-    }
-
-    if (hh->handler(r, h, hh->offset) != NGX_OK) {
-        /*
-         * request has been finalized already
-         * in ngx_http_process_host()
-         */
+    if (header->hh->handler(r, h, header->hh->offset) != NGX_OK) {
+        /* header handler has already finalized request */
         return NGX_ABORT;
     }
 
diff --git a/src/http/v2/ngx_http_v2.h b/src/http/v2/ngx_http_v2.h
--- a/src/http/v2/ngx_http_v2.h
+++ b/src/http/v2/ngx_http_v2.h
@@ -283,9 +283,8 @@ void ngx_http_v2_init(ngx_event_t *rev);
 ngx_int_t ngx_http_v2_read_request_body(ngx_http_request_t *r);
 ngx_int_t ngx_http_v2_read_unbuffered_request_body(ngx_http_request_t *r);
 
-ngx_int_t ngx_http_v2_push_stream(ngx_http_v2_connection_t *h2c,
-    ngx_uint_t depend, size_t request_length, ngx_str_t *path,
-    ngx_str_t *authority);
+ngx_http_v2_stream_t *ngx_http_v2_push_stream(ngx_http_v2_stream_t *parent,
+    ngx_str_t *path);
 
 void ngx_http_v2_close_stream(ngx_http_v2_stream_t *stream, ngx_int_t rc);
 
diff --git a/src/http/v2/ngx_http_v2_filter_module.c b/src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c
+++ b/src/http/v2/ngx_http_v2_filter_module.c
@@ -50,20 +50,48 @@
 #define NGX_HTTP_V2_STATUS_404_INDEX      13
 #define NGX_HTTP_V2_STATUS_500_INDEX      14
 
+#define NGX_HTTP_V2_ACCEPT_ENCODING_INDEX 16
+#define NGX_HTTP_V2_ACCEPT_LANGUAGE_INDEX 17
 #define NGX_HTTP_V2_CONTENT_LENGTH_INDEX  28
 #define NGX_HTTP_V2_CONTENT_TYPE_INDEX    31
 #define NGX_HTTP_V2_DATE_INDEX            33
 #define NGX_HTTP_V2_LAST_MODIFIED_INDEX   44
 #define NGX_HTTP_V2_LOCATION_INDEX        46
 #define NGX_HTTP_V2_SERVER_INDEX          54
+#define NGX_HTTP_V2_USER_AGENT_INDEX      58
 #define NGX_HTTP_V2_VARY_INDEX            59
 
 #define NGX_HTTP_V2_NO_TRAILERS           (ngx_http_v2_out_frame_t *) -1
 
 
+typedef struct {
+    ngx_str_t      name;
+    u_char         index;
+    ngx_uint_t     offset;
+} ngx_http_v2_push_header_t;
+
+
+static ngx_http_v2_push_header_t  ngx_http_v2_push_headers[] = {
+    { ngx_string(":authority"), NGX_HTTP_V2_AUTHORITY_INDEX,
+      offsetof(ngx_http_headers_in_t, host) },
+
+    { ngx_string("accept-encoding"), NGX_HTTP_V2_ACCEPT_ENCODING_INDEX,
+      offsetof(ngx_http_headers_in_t, accept_encoding) },
+
+    { ngx_string("accept-language"), NGX_HTTP_V2_ACCEPT_LANGUAGE_INDEX,
+      offsetof(ngx_http_headers_in_t, accept_language) },
+
+    { ngx_string("user-agent"), NGX_HTTP_V2_USER_AGENT_INDEX,
+      offsetof(ngx_http_headers_in_t, user_agent) },
+};
+
+#define NGX_HTTP_V2_PUSH_HEADERS                                              \
+    (sizeof(ngx_http_v2_push_headers) / sizeof(ngx_http_v2_push_header_t))
+
+
 static ngx_int_t ngx_http_v2_push_resources(ngx_http_request_t *r);
 static ngx_int_t ngx_http_v2_push_resource(ngx_http_request_t *r,
-    ngx_str_t *path, ngx_str_t *authority);
+    ngx_str_t *path, ngx_str_t *binary);
 
 static u_char *ngx_http_v2_string_encode(u_char *dst, u_char *src, size_t len,
     u_char *tmp, ngx_uint_t lower);
@@ -685,16 +713,17 @@ ngx_http_v2_push_resources(ngx_http_requ
 {
     u_char                     *start, *end, *last;
     ngx_int_t                   rc;
-    ngx_str_t                   path, authority;
+    ngx_str_t                   path;
     ngx_uint_t                  i, push;
     ngx_table_elt_t           **h;
     ngx_http_v2_loc_conf_t     *h2lcf;
     ngx_http_complex_value_t   *pushes;
+    ngx_str_t                   binary[NGX_HTTP_V2_PUSH_HEADERS];
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                    "http2 push resources");
 
-    ngx_str_null(&authority);
+    ngx_memzero(binary, NGX_HTTP_V2_PUSH_HEADERS * sizeof(ngx_str_t));
 
     h2lcf = ngx_http_get_module_loc_conf(r, ngx_http_v2_module);
 
@@ -715,7 +744,7 @@ ngx_http_v2_push_resources(ngx_http_requ
                 continue;
             }
 
-            rc = ngx_http_v2_push_resource(r, &path, &authority);
+            rc = ngx_http_v2_push_resource(r, &path, binary);
 
             if (rc == NGX_ERROR) {
                 return NGX_ERROR;
@@ -880,7 +909,7 @@ ngx_http_v2_push_resources(ngx_http_requ
         if (push && path.len
             && !(path.len > 1 && path.data[0] == '/' && path.data[1] == '/'))
         {
-            rc = ngx_http_v2_push_resource(r, &path, &authority);
+            rc = ngx_http_v2_push_resource(r, &path, binary);
 
             if (rc == NGX_ERROR) {
                 return NGX_ERROR;
@@ -905,15 +934,18 @@ ngx_http_v2_push_resources(ngx_http_requ
 
 static ngx_int_t
 ngx_http_v2_push_resource(ngx_http_request_t *r, ngx_str_t *path,
-    ngx_str_t *authority)
+    ngx_str_t *binary)
 {
-    u_char                    *start, *pos, *tmp;
-    size_t                     len;
-    ngx_table_elt_t           *host;
-    ngx_connection_t          *fc;
-    ngx_http_v2_stream_t      *stream;
-    ngx_http_v2_out_frame_t   *frame;
-    ngx_http_v2_connection_t  *h2c;
+    u_char                      *start, *pos, *tmp;
+    size_t                       len;
+    ngx_str_t                   *value;
+    ngx_uint_t                   i;
+    ngx_table_elt_t            **h;
+    ngx_connection_t            *fc;
+    ngx_http_v2_stream_t        *stream;
+    ngx_http_v2_out_frame_t     *frame;
+    ngx_http_v2_connection_t    *h2c;
+    ngx_http_v2_push_header_t   *ph;
 
     fc = r->connection;
 
@@ -944,42 +976,70 @@ ngx_http_v2_push_resource(ngx_http_reque
         return NGX_DECLINED;
     }
 
-    host = r->headers_in.host;
-
-    if (host == NULL) {
+    if (r->headers_in.host == NULL) {
         return NGX_ABORT;
     }
 
-    if (authority->len == 0) {
-
-        len = 1 + NGX_HTTP_V2_INT_OCTETS + host->value.len;
+    ph = ngx_http_v2_push_headers;
+
+    if (binary[0].len) {
+        tmp = ngx_palloc(r->pool, path->len);
+        if (tmp == NULL) {
+            return NGX_ERROR;
+        }
+
+    } else {
+        len = path->len;
+
+        for (i = 0; i < NGX_HTTP_V2_PUSH_HEADERS; i++) {
+            h = (ngx_table_elt_t **) ((char *) &r->headers_in + ph[i].offset);
+
+            if (*h) {
+                len = ngx_max(len, (*h)->value.len);
+            }
+        }
 
         tmp = ngx_palloc(r->pool, len);
-        pos = ngx_pnalloc(r->pool, len);
-
-        if (pos == NULL || tmp == NULL) {
+        if (tmp == NULL) {
             return NGX_ERROR;
         }
 
-        authority->data = pos;
-
-        *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_AUTHORITY_INDEX);
-        pos = ngx_http_v2_write_value(pos, host->value.data, host->value.len,
-                                      tmp);
-
-        authority->len = pos - authority->data;
+        for (i = 0; i < NGX_HTTP_V2_PUSH_HEADERS; i++) {
+            h = (ngx_table_elt_t **) ((char *) &r->headers_in + ph[i].offset);
+
+            if (*h == NULL) {
+                continue;
+            }
+
+            value = &(*h)->value;
+
+            len = 1 + NGX_HTTP_V2_INT_OCTETS + value->len;
+
+            pos = ngx_pnalloc(r->pool, len);
+            if (pos == NULL) {
+                return NGX_ERROR;
+            }
+
+            binary[i].data = pos;
+
+            *pos++ = ngx_http_v2_inc_indexed(ph[i].index);
+            pos = ngx_http_v2_write_value(pos, value->data, value->len, tmp);
+
+            binary[i].len = pos - binary[i].data;
+        }
     }
 
     len = (h2c->table_update ? 1 : 0)
           + 1
           + 1 + NGX_HTTP_V2_INT_OCTETS + path->len
-          + authority->len
           + 1;
 
-    tmp = ngx_palloc(r->pool, len);
+    for (i = 0; i < NGX_HTTP_V2_PUSH_HEADERS; i++) {
+        len += binary[i].len;
+    }
+
     pos = ngx_pnalloc(r->pool, len);
-
-    if (pos == NULL || tmp == NULL) {
+    if (pos == NULL) {
         return NGX_ERROR;
     }
 
@@ -1003,11 +1063,6 @@ ngx_http_v2_push_resource(ngx_http_reque
     *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_PATH_INDEX);
     pos = ngx_http_v2_write_value(pos, path->data, path->len, tmp);
 
-    ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
-                   "http2 push header: \":authority: %V\"", &host->value);
-
-    pos = ngx_cpymem(pos, authority->data, authority->len);
-
 #if (NGX_HTTP_SSL)
     if (fc->ssl) {
         ngx_log_debug0(NGX_LOG_DEBUG_HTTP, fc->log, 0,
@@ -1022,6 +1077,20 @@ ngx_http_v2_push_resource(ngx_http_reque
         *pos++ = ngx_http_v2_indexed(NGX_HTTP_V2_SCHEME_HTTP_INDEX);
     }
 
+    for (i = 0; i < NGX_HTTP_V2_PUSH_HEADERS; i++) {
+        h = (ngx_table_elt_t **) ((char *) &r->headers_in + ph[i].offset);
+
+        if (*h == NULL) {
+            continue;
+        }
+
+        ngx_log_debug2(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 push header: \"%V: %V\"",
+                       &ph[i].name, &(*h)->value);
+
+        pos = ngx_cpymem(pos, binary[i].data, binary[i].len);
+    }
+
     frame = ngx_http_v2_create_push_frame(r, start, pos);
     if (frame == NULL) {
         return NGX_ERROR;
@@ -1031,8 +1100,14 @@ ngx_http_v2_push_resource(ngx_http_reque
 
     stream->queued++;
 
-    return ngx_http_v2_push_stream(h2c, stream->node->id, pos - start,
-                                   path, &host->value);
+    stream = ngx_http_v2_push_stream(stream, path);
+
+    if (stream) {
+        stream->request->request_length = pos - start;
+        return NGX_OK;
+    }
+
+    return NGX_ERROR;
 }
 
 
# HG changeset patch
# User Ruslan Ermilov <ru at nginx.com>
# Date 1518634817 -10800
#      Wed Feb 14 22:00:17 2018 +0300
# Node ID 49ff9d543ccd8cb60c6fc5246e49a02d86332ec0
# Parent  cd54884b375b70ecd358358e5e78f43e70b6163a
HTTP/2: fixed ngx_http_v2_push_stream() allocation error handling.

In particular, if a stream object allocation failed, and a client sent
the PRIORITY frame for this stream, ngx_http_v2_set_dependency() could
dereference a null pointer while trying to re-parent a dependency node.

diff --git a/src/http/v2/ngx_http_v2.c b/src/http/v2/ngx_http_v2.c
--- a/src/http/v2/ngx_http_v2.c
+++ b/src/http/v2/ngx_http_v2.c
@@ -2546,6 +2546,7 @@ ngx_http_v2_push_stream(ngx_http_v2_stre
 {
     ngx_int_t                     rc;
     ngx_str_t                     value;
+    ngx_pool_t                   *pool;
     ngx_table_elt_t             **h;
     ngx_connection_t             *fc;
     ngx_http_request_t           *r;
@@ -2556,10 +2557,28 @@ ngx_http_v2_push_stream(ngx_http_v2_stre
 
     h2c = parent->connection;
 
+    pool = ngx_create_pool(1024, h2c->connection->log);
+    if (pool == NULL) {
+        goto rst_stream;
+    }
+
     node = ngx_http_v2_get_node_by_id(h2c, h2c->last_push, 1);
 
     if (node == NULL) {
-        return NULL;
+        ngx_destroy_pool(pool);
+        goto rst_stream;
+    }
+
+    stream = ngx_http_v2_create_stream(h2c, 1);
+    if (stream == NULL) {
+
+        if (node->parent == NULL) {
+            ngx_queue_insert_tail(&h2c->closed, &node->reuse);
+            h2c->closed_nodes++;
+        }
+
+        ngx_destroy_pool(pool);
+        goto rst_stream;
     }
 
     if (node->parent) {
@@ -2567,15 +2586,7 @@ ngx_http_v2_push_stream(ngx_http_v2_stre
         h2c->closed_nodes--;
     }
 
-    stream = ngx_http_v2_create_stream(h2c, 1);
-    if (stream == NULL) {
-        return NULL;
-    }
-
-    stream->pool = ngx_create_pool(1024, h2c->connection->log);
-    if (stream->pool == NULL) {
-        return NULL;
-    }
+    stream->pool = pool;
 
     r = stream->request;
     fc = r->connection;
@@ -2608,9 +2619,9 @@ ngx_http_v2_push_stream(ngx_http_v2_stre
         r->schema_end = r->schema_start + 4;
     }
 
-    value.data = ngx_pstrdup(stream->pool, path);
+    value.data = ngx_pstrdup(pool, path);
     if (value.data == NULL) {
-        return NULL;
+        goto close;
     }
 
     value.len = path->len;
@@ -2631,9 +2642,9 @@ ngx_http_v2_push_stream(ngx_http_v2_stre
 
         value.len = (*h)->value.len;
 
-        value.data = ngx_pnalloc(stream->pool, value.len + 1);
+        value.data = ngx_pnalloc(pool, value.len + 1);
         if (value.data == NULL) {
-            return NULL;
+            goto close;
         }
 
         ngx_memcpy(value.data, (*h)->value.data, value.len);
@@ -2663,7 +2674,20 @@ error:
         return NULL;
     }
 
-    (void) ngx_http_v2_connection_error(h2c, NGX_HTTP_V2_INTERNAL_ERROR);
+close:
+
+    ngx_http_v2_close_stream(stream, NGX_HTTP_INTERNAL_SERVER_ERROR);
+
+    return NULL;
+
+rst_stream:
+
+    if (ngx_http_v2_send_rst_stream(h2c, h2c->last_push,
+                                    NGX_HTTP_INTERNAL_SERVER_ERROR)
+        != NGX_OK)
+    {
+        h2c->connection->error = 1;
+    }
 
     return NULL;
 }

From ru at nginx.com  Thu Feb 15 03:39:55 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 15 Feb 2018 06:39:55 +0300
Subject: [PATCH] HTTP/2: expose function to push single resource to modules
In-Reply-To: <20180213133117.GK24410@mdounin.ru>
References: <1bb98b06d5536dfc80a4.1518108779@mandy.ghedini.home>
 <20180208190027.GN24410@mdounin.ru> <20180208194825.GA6873@pinky>
 <20180209073559.GE75377@lo0.su> <20180213122136.GA20751@mandy>
 <20180213133117.GK24410@mdounin.ru>
Message-ID: <20180215033955.GP67691@lo0.su>

On Tue, Feb 13, 2018 at 04:31:17PM +0300, Maxim Dounin wrote:
> Hello!
> 
> On Tue, Feb 13, 2018 at 12:21:36PM +0000, Alessandro Ghedini wrote:
> 
> > On Fri, Feb 09, 2018 at 10:35:59AM +0300, Ruslan Ermilov wrote:
> > > On Thu, Feb 08, 2018 at 07:48:25PM +0000, Alessandro Ghedini wrote:
> > > > On Thu, Feb 08, 2018 at 10:00:27PM +0300, Maxim Dounin wrote:
> > > > > On Thu, Feb 08, 2018 at 04:52:59PM +0000, Alessandro Ghedini wrote:
> > > > > 
> > > > > > # HG changeset patch
> > > > > > # User Alessandro Ghedini <alessandro at ghedini.me>
> > > > > > # Date 1518108716 0
> > > > > > #      Thu Feb 08 16:51:56 2018 +0000
> > > > > > # Branch expose-push
> > > > > > # Node ID 1bb98b06d5536dfc80a407aabd8d06f9309f8df6
> > > > > > # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> > > > > > HTTP/2: expose function to push single resource to modules.
> > > > > > 
> > > > > > This makes it possible for 3rd party modules to implement alternative
> > > > > > methods for deciding which resources to push to clients on a per-request
> > > > > > basis (e.g. by parsing HTML from the response body, by using a custom
> > > > > > Link header parser, ...).
> > > > > > 
> > > > > > No functional changes.
> > > > > 
> > > > > Not sure this is a good idea.
> > > > > 
> > > > > You may consider exposing a variable to be used in http2_push 
> > > > > instead.
> > > > 
> > > > Right, the problem is that as far as I can tell http2_push only supports a
> > > > single resource, even when a variable is used, so it wouldn't be possible to
> > > > push multiple resources without specifying multiple http2_push directives,
> > > > each with its own variable, and even then you'd only have a fixed number of
> > > > resources that can be pushed, which wouldn't work well when the number of
> > > > resources changes depending on each request/response.
> > > > 
> > > > So in the end exposing the internal functions to modules seemed better than
> > > > just trying to make http2_push support multiple resources per directive,
> > > > which would add complexity to NGINX itself rather than the external modules
> > > > (though I can do that if you think it would be a better solution).
> > > 
> > > We've also considered adding support for the X-Accel-Push header, but
> > > decided not to implement it at this time.  If implemented, there could
> > > be multiple X-Accel-Push headers in the proxied response.
> > 
> > That might work for us, but it's a somewhat awkward interface to use from
> > inside a module, so I'd still prefer something more direct, and as I said,
> > exposing the function to modules seemed the least invasive change to NGINX.
> > 
> > Could you please expand a bit on why you think this might be a bad idea?
> > 
> > In any case I can look into implementing X-Accel-Push support if you don't
> > plan on doing it yourself.
> 
> I presonally think that X-Accel-Push is not needed and should not 
> be added given we already have http2_push.  Also, HTTP/2 push as a 
> technology has enough design problems to introduce additional 
> interfaces.
> 
> If you want to use HTTP2 pushes from a module, consider either 
> using http2_push with variables as previously suggested (right now 
> you can use multiple http2_push directives if you want to push 
> multiple resources), or adding "Link: rel=preload" headers and 
> using http2_push_preload. 

The latter was made programmatically available in 6ba68ad8b24c,
"Basic support of the Link response header."

From pluknet at nginx.com  Thu Feb 15 10:01:31 2018
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Thu, 15 Feb 2018 10:01:31 +0000
Subject: [nginx] HTTP/2: style.
Message-ID: <hg.e44c297a6b95.1518688891.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/e44c297a6b95
branches:  
changeset: 7204:e44c297a6b95
user:      Sergey Kandaurov <pluknet at nginx.com>
date:      Thu Feb 15 02:34:16 2018 +0300
description:
HTTP/2: style.

diffstat:

 src/http/v2/ngx_http_v2.c               |  4 ++--
 src/http/v2/ngx_http_v2_filter_module.c |  4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diffs (28 lines):

diff -r 8b0553239592 -r e44c297a6b95 src/http/v2/ngx_http_v2.c
--- a/src/http/v2/ngx_http_v2.c	Fri Feb 09 23:20:08 2018 +0300
+++ b/src/http/v2/ngx_http_v2.c	Thu Feb 15 02:34:16 2018 +0300
@@ -3862,8 +3862,8 @@ ngx_http_v2_process_request_body(ngx_htt
         } else {
             if (size > (size_t) (buf->end - buf->last)) {
                 ngx_log_error(NGX_LOG_INFO, fc->log, 0,
-                                "client intended to send body data "
-                                "larger than declared");
+                              "client intended to send body data "
+                              "larger than declared");
 
                 return NGX_HTTP_BAD_REQUEST;
             }
diff -r 8b0553239592 -r e44c297a6b95 src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c	Fri Feb 09 23:20:08 2018 +0300
+++ b/src/http/v2/ngx_http_v2_filter_module.c	Thu Feb 15 02:34:16 2018 +0300
@@ -929,8 +929,8 @@ ngx_http_v2_push_resource(ngx_http_reque
     }
 
     ngx_log_debug2(NGX_LOG_DEBUG_HTTP, h2c->connection->log, 0,
-                  "http2 pushing:%ui limit:%ui",
-                  h2c->pushing, h2c->concurrent_pushes);
+                   "http2 pushing:%ui limit:%ui",
+                   h2c->pushing, h2c->concurrent_pushes);
 
     if (h2c->pushing >= h2c->concurrent_pushes) {
         return NGX_ABORT;

From vl at nginx.com  Thu Feb 15 13:46:16 2018
From: vl at nginx.com (Vladimir Homutov)
Date: Thu, 15 Feb 2018 13:46:16 +0000
Subject: [nginx] Core: added a stub for additional zone configuration.
Message-ID: <hg.fc4d82c13c04.1518702376.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/fc4d82c13c04
branches:  
changeset: 7205:fc4d82c13c04
user:      Vladimir Homutov <vl at nginx.com>
date:      Thu Feb 15 16:08:05 2018 +0300
description:
Core: added a stub for additional zone configuration.

diffstat:

 src/core/ngx_cycle.h |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (11 lines):

diff -r e44c297a6b95 -r fc4d82c13c04 src/core/ngx_cycle.h
--- a/src/core/ngx_cycle.h	Thu Feb 15 02:34:16 2018 +0300
+++ b/src/core/ngx_cycle.h	Thu Feb 15 16:08:05 2018 +0300
@@ -31,6 +31,7 @@ struct ngx_shm_zone_s {
     ngx_shm_t                 shm;
     ngx_shm_zone_init_pt      init;
     void                     *tag;
+    void                     *sync;
     ngx_uint_t                noreuse;  /* unsigned  noreuse:1; */
 };
 

From ru at nginx.com  Thu Feb 15 14:55:00 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 15 Feb 2018 14:55:00 +0000
Subject: [nginx] Expose more headers with NGX_HTTP_HEADERS.
Message-ID: <hg.33edea74bd58.1518706500.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/33edea74bd58
branches:  
changeset: 7206:33edea74bd58
user:      Ruslan Ermilov <ru at nginx.com>
date:      Thu Feb 15 17:51:26 2018 +0300
description:
Expose more headers with NGX_HTTP_HEADERS.

diffstat:

 src/http/ngx_http_request.c |  2 +-
 src/http/ngx_http_request.h |  2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diffs (24 lines):

diff -r fc4d82c13c04 -r 33edea74bd58 src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c	Thu Feb 15 16:08:05 2018 +0300
+++ b/src/http/ngx_http_request.c	Thu Feb 15 17:51:26 2018 +0300
@@ -140,7 +140,7 @@ ngx_http_header_t  ngx_http_headers_in[]
                  offsetof(ngx_http_headers_in_t, upgrade),
                  ngx_http_process_header_line },
 
-#if (NGX_HTTP_GZIP)
+#if (NGX_HTTP_GZIP || NGX_HTTP_HEADERS)
     { ngx_string("Accept-Encoding"),
                  offsetof(ngx_http_headers_in_t, accept_encoding),
                  ngx_http_process_header_line },
diff -r fc4d82c13c04 -r 33edea74bd58 src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h	Thu Feb 15 16:08:05 2018 +0300
+++ b/src/http/ngx_http_request.h	Thu Feb 15 17:51:26 2018 +0300
@@ -200,7 +200,7 @@ typedef struct {
     ngx_table_elt_t                  *expect;
     ngx_table_elt_t                  *upgrade;
 
-#if (NGX_HTTP_GZIP)
+#if (NGX_HTTP_GZIP || NGX_HTTP_HEADERS)
     ngx_table_elt_t                  *accept_encoding;
     ngx_table_elt_t                  *via;
 #endif

From ru at nginx.com  Thu Feb 15 14:55:01 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 15 Feb 2018 14:55:01 +0000
Subject: [nginx] HTTP/2: push additional request headers (closes #1478).
Message-ID: <hg.3d2b0b02bd3d.1518706501.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/3d2b0b02bd3d
branches:  
changeset: 7207:3d2b0b02bd3d
user:      Ruslan Ermilov <ru at nginx.com>
date:      Thu Feb 15 17:51:32 2018 +0300
description:
HTTP/2: push additional request headers (closes #1478).

The Accept-Encoding, Accept-Language, and User-Agent header fields
are now copied from the original request to pushed requests.

diffstat:

 auto/modules                            |    1 +
 src/http/v2/ngx_http_v2.c               |  162 ++++++++++++++++++++-----------
 src/http/v2/ngx_http_v2.h               |    5 +-
 src/http/v2/ngx_http_v2_filter_module.c |  155 ++++++++++++++++++++++-------
 4 files changed, 222 insertions(+), 101 deletions(-)

diffs (560 lines):

diff -r 33edea74bd58 -r 3d2b0b02bd3d auto/modules
--- a/auto/modules	Thu Feb 15 17:51:26 2018 +0300
+++ b/auto/modules	Thu Feb 15 17:51:32 2018 +0300
@@ -420,6 +420,7 @@ if [ $HTTP = YES ]; then
 
     if [ $HTTP_V2 = YES ]; then
         have=NGX_HTTP_V2 . auto/have
+        have=NGX_HTTP_HEADERS . auto/have
 
         ngx_module_name=ngx_http_v2_module
         ngx_module_incs=src/http/v2
diff -r 33edea74bd58 -r 3d2b0b02bd3d src/http/v2/ngx_http_v2.c
--- a/src/http/v2/ngx_http_v2.c	Thu Feb 15 17:51:26 2018 +0300
+++ b/src/http/v2/ngx_http_v2.c	Thu Feb 15 17:51:32 2018 +0300
@@ -11,6 +11,14 @@
 #include <ngx_http_v2_module.h>
 
 
+typedef struct {
+    ngx_str_t           name;
+    ngx_uint_t          offset;
+    ngx_uint_t          hash;
+    ngx_http_header_t  *hh;
+} ngx_http_v2_parse_header_t;
+
+
 /* errors */
 #define NGX_HTTP_V2_NO_ERROR                     0x0
 #define NGX_HTTP_V2_PROTOCOL_ERROR               0x1
@@ -156,6 +164,8 @@ static ngx_int_t ngx_http_v2_parse_schem
     ngx_str_t *value);
 static ngx_int_t ngx_http_v2_parse_authority(ngx_http_request_t *r,
     ngx_str_t *value);
+static ngx_int_t ngx_http_v2_parse_header(ngx_http_request_t *r,
+    ngx_http_v2_parse_header_t *header, ngx_str_t *value);
 static ngx_int_t ngx_http_v2_construct_request_line(ngx_http_request_t *r);
 static ngx_int_t ngx_http_v2_cookie(ngx_http_request_t *r,
     ngx_http_v2_header_t *header);
@@ -201,6 +211,23 @@ static ngx_http_v2_handler_pt ngx_http_v
     (sizeof(ngx_http_v2_frame_states) / sizeof(ngx_http_v2_handler_pt))
 
 
+static ngx_http_v2_parse_header_t  ngx_http_v2_parse_headers[] = {
+    { ngx_string("host"),
+      offsetof(ngx_http_headers_in_t, host), 0, NULL },
+
+    { ngx_string("accept-encoding"),
+      offsetof(ngx_http_headers_in_t, accept_encoding), 0, NULL },
+
+    { ngx_string("accept-language"),
+      offsetof(ngx_http_headers_in_t, accept_language), 0, NULL },
+
+    { ngx_string("user-agent"),
+      offsetof(ngx_http_headers_in_t, user_agent), 0, NULL },
+
+    { ngx_null_string, 0, 0, NULL }
+};
+
+
 void
 ngx_http_v2_init(ngx_event_t *rev)
 {
@@ -2514,21 +2541,25 @@ ngx_http_v2_parse_int(ngx_http_v2_connec
 }
 
 
-ngx_int_t
-ngx_http_v2_push_stream(ngx_http_v2_connection_t *h2c, ngx_uint_t depend,
-    size_t request_length, ngx_str_t *path, ngx_str_t *authority)
+ngx_http_v2_stream_t *
+ngx_http_v2_push_stream(ngx_http_v2_stream_t *parent, ngx_str_t *path)
 {
-    ngx_int_t              rc;
-    ngx_str_t              value;
-    ngx_connection_t      *fc;
-    ngx_http_request_t    *r;
-    ngx_http_v2_node_t    *node;
-    ngx_http_v2_stream_t  *stream;
+    ngx_int_t                     rc;
+    ngx_str_t                     value;
+    ngx_table_elt_t             **h;
+    ngx_connection_t             *fc;
+    ngx_http_request_t           *r;
+    ngx_http_v2_node_t           *node;
+    ngx_http_v2_stream_t         *stream;
+    ngx_http_v2_connection_t     *h2c;
+    ngx_http_v2_parse_header_t   *header;
+
+    h2c = parent->connection;
 
     node = ngx_http_v2_get_node_by_id(h2c, h2c->last_push, 1);
 
     if (node == NULL) {
-        return NGX_ERROR;
+        return NULL;
     }
 
     if (node->parent) {
@@ -2538,19 +2569,17 @@ ngx_http_v2_push_stream(ngx_http_v2_conn
 
     stream = ngx_http_v2_create_stream(h2c, 1);
     if (stream == NULL) {
-        return NGX_ERROR;
+        return NULL;
     }
 
     stream->pool = ngx_create_pool(1024, h2c->connection->log);
     if (stream->pool == NULL) {
-        return NGX_ERROR;
+        return NULL;
     }
 
     r = stream->request;
     fc = r->connection;
 
-    r->request_length = request_length;
-
     stream->in_closed = 1;
     stream->node = node;
 
@@ -2559,10 +2588,10 @@ ngx_http_v2_push_stream(ngx_http_v2_conn
     ngx_log_debug2(NGX_LOG_DEBUG_HTTP, h2c->connection->log, 0,
                    "http2 push stream sid:%ui "
                    "depends on %ui excl:0 weight:16",
-                   h2c->last_push, depend);
+                   h2c->last_push, parent->node->id);
 
     node->weight = NGX_HTTP_V2_DEFAULT_WEIGHT;
-    ngx_http_v2_set_dependency(h2c, node, depend, 0);
+    ngx_http_v2_set_dependency(h2c, node, parent->node->id, 0);
 
     r->method_name = ngx_http_core_get_method;
     r->method = NGX_HTTP_GET;
@@ -2579,51 +2608,64 @@ ngx_http_v2_push_stream(ngx_http_v2_conn
         r->schema_end = r->schema_start + 4;
     }
 
-    value.len = authority->len;
-
-    value.data = ngx_pstrdup(stream->pool, authority);
+    value.data = ngx_pstrdup(stream->pool, path);
     if (value.data == NULL) {
-        return NGX_ERROR;
-    }
-
-    rc = ngx_http_v2_parse_authority(r, &value);
-
-    if (rc != NGX_OK) {
-        goto error;
+        return NULL;
     }
 
     value.len = path->len;
 
-    value.data = ngx_pstrdup(stream->pool, path);
-    if (value.data == NULL) {
-        return NGX_ERROR;
-    }
-
     rc = ngx_http_v2_parse_path(r, &value);
 
     if (rc != NGX_OK) {
         goto error;
     }
 
+    for (header = ngx_http_v2_parse_headers; header->name.len; header++) {
+        h = (ngx_table_elt_t **)
+                ((char *) &parent->request->headers_in + header->offset);
+
+        if (*h == NULL) {
+            continue;
+        }
+
+        value.len = (*h)->value.len;
+
+        value.data = ngx_pnalloc(stream->pool, value.len + 1);
+        if (value.data == NULL) {
+            return NULL;
+        }
+
+        ngx_memcpy(value.data, (*h)->value.data, value.len);
+        value.data[value.len] = '\0';
+
+        rc = ngx_http_v2_parse_header(r, header, &value);
+
+        if (rc != NGX_OK) {
+            goto error;
+        }
+    }
+
     fc->write->handler = ngx_http_v2_run_request_handler;
     ngx_post_event(fc->write, &ngx_posted_events);
 
-    return NGX_OK;
+    return stream;
 
 error:
 
     if (rc == NGX_ABORT) {
-        return NGX_ERROR;
+        /* header handler has already finalized request */
+        return NULL;
     }
 
     if (rc == NGX_DECLINED) {
         ngx_http_finalize_request(r, NGX_HTTP_BAD_REQUEST);
-        return NGX_ERROR;
+        return NULL;
     }
 
     (void) ngx_http_v2_connection_error(h2c, NGX_HTTP_V2_INTERNAL_ERROR);
 
-    return NGX_ERROR;
+    return NULL;
 }
 
 
@@ -3436,41 +3478,45 @@ ngx_http_v2_parse_scheme(ngx_http_reques
 static ngx_int_t
 ngx_http_v2_parse_authority(ngx_http_request_t *r, ngx_str_t *value)
 {
+    return ngx_http_v2_parse_header(r, &ngx_http_v2_parse_headers[0], value);
+}
+
+
+static ngx_int_t
+ngx_http_v2_parse_header(ngx_http_request_t *r,
+    ngx_http_v2_parse_header_t *header, ngx_str_t *value)
+{
     ngx_table_elt_t            *h;
-    ngx_http_header_t          *hh;
     ngx_http_core_main_conf_t  *cmcf;
 
-    static ngx_str_t host = ngx_string("host");
-
     h = ngx_list_push(&r->headers_in.headers);
     if (h == NULL) {
         return NGX_ERROR;
     }
 
-    h->hash = ngx_hash_key(host.data, host.len);
-
-    h->key.len = host.len;
-    h->key.data = host.data;
+    h->key.len = header->name.len;
+    h->key.data = header->name.data;
+    h->lowcase_key = header->name.data;
+
+    if (header->hh == NULL) {
+        header->hash = ngx_hash_key(header->name.data, header->name.len);
+
+        cmcf = ngx_http_get_module_main_conf(r, ngx_http_core_module);
+
+        header->hh = ngx_hash_find(&cmcf->headers_in_hash, header->hash,
+                                   h->lowcase_key, h->key.len);
+        if (header->hh == NULL) {
+            return NGX_ERROR;
+        }
+    }
+
+    h->hash = header->hash;
 
     h->value.len = value->len;
     h->value.data = value->data;
 
-    h->lowcase_key = host.data;
-
-    cmcf = ngx_http_get_module_main_conf(r, ngx_http_core_module);
-
-    hh = ngx_hash_find(&cmcf->headers_in_hash, h->hash,
-                       h->lowcase_key, h->key.len);
-
-    if (hh == NULL) {
-        return NGX_ERROR;
-    }
-
-    if (hh->handler(r, h, hh->offset) != NGX_OK) {
-        /*
-         * request has been finalized already
-         * in ngx_http_process_host()
-         */
+    if (header->hh->handler(r, h, header->hh->offset) != NGX_OK) {
+        /* header handler has already finalized request */
         return NGX_ABORT;
     }
 
diff -r 33edea74bd58 -r 3d2b0b02bd3d src/http/v2/ngx_http_v2.h
--- a/src/http/v2/ngx_http_v2.h	Thu Feb 15 17:51:26 2018 +0300
+++ b/src/http/v2/ngx_http_v2.h	Thu Feb 15 17:51:32 2018 +0300
@@ -283,9 +283,8 @@ void ngx_http_v2_init(ngx_event_t *rev);
 ngx_int_t ngx_http_v2_read_request_body(ngx_http_request_t *r);
 ngx_int_t ngx_http_v2_read_unbuffered_request_body(ngx_http_request_t *r);
 
-ngx_int_t ngx_http_v2_push_stream(ngx_http_v2_connection_t *h2c,
-    ngx_uint_t depend, size_t request_length, ngx_str_t *path,
-    ngx_str_t *authority);
+ngx_http_v2_stream_t *ngx_http_v2_push_stream(ngx_http_v2_stream_t *parent,
+    ngx_str_t *path);
 
 void ngx_http_v2_close_stream(ngx_http_v2_stream_t *stream, ngx_int_t rc);
 
diff -r 33edea74bd58 -r 3d2b0b02bd3d src/http/v2/ngx_http_v2_filter_module.c
--- a/src/http/v2/ngx_http_v2_filter_module.c	Thu Feb 15 17:51:26 2018 +0300
+++ b/src/http/v2/ngx_http_v2_filter_module.c	Thu Feb 15 17:51:32 2018 +0300
@@ -50,20 +50,48 @@
 #define NGX_HTTP_V2_STATUS_404_INDEX      13
 #define NGX_HTTP_V2_STATUS_500_INDEX      14
 
+#define NGX_HTTP_V2_ACCEPT_ENCODING_INDEX 16
+#define NGX_HTTP_V2_ACCEPT_LANGUAGE_INDEX 17
 #define NGX_HTTP_V2_CONTENT_LENGTH_INDEX  28
 #define NGX_HTTP_V2_CONTENT_TYPE_INDEX    31
 #define NGX_HTTP_V2_DATE_INDEX            33
 #define NGX_HTTP_V2_LAST_MODIFIED_INDEX   44
 #define NGX_HTTP_V2_LOCATION_INDEX        46
 #define NGX_HTTP_V2_SERVER_INDEX          54
+#define NGX_HTTP_V2_USER_AGENT_INDEX      58
 #define NGX_HTTP_V2_VARY_INDEX            59
 
 #define NGX_HTTP_V2_NO_TRAILERS           (ngx_http_v2_out_frame_t *) -1
 
 
+typedef struct {
+    ngx_str_t      name;
+    u_char         index;
+    ngx_uint_t     offset;
+} ngx_http_v2_push_header_t;
+
+
+static ngx_http_v2_push_header_t  ngx_http_v2_push_headers[] = {
+    { ngx_string(":authority"), NGX_HTTP_V2_AUTHORITY_INDEX,
+      offsetof(ngx_http_headers_in_t, host) },
+
+    { ngx_string("accept-encoding"), NGX_HTTP_V2_ACCEPT_ENCODING_INDEX,
+      offsetof(ngx_http_headers_in_t, accept_encoding) },
+
+    { ngx_string("accept-language"), NGX_HTTP_V2_ACCEPT_LANGUAGE_INDEX,
+      offsetof(ngx_http_headers_in_t, accept_language) },
+
+    { ngx_string("user-agent"), NGX_HTTP_V2_USER_AGENT_INDEX,
+      offsetof(ngx_http_headers_in_t, user_agent) },
+};
+
+#define NGX_HTTP_V2_PUSH_HEADERS                                              \
+    (sizeof(ngx_http_v2_push_headers) / sizeof(ngx_http_v2_push_header_t))
+
+
 static ngx_int_t ngx_http_v2_push_resources(ngx_http_request_t *r);
 static ngx_int_t ngx_http_v2_push_resource(ngx_http_request_t *r,
-    ngx_str_t *path, ngx_str_t *authority);
+    ngx_str_t *path, ngx_str_t *binary);
 
 static u_char *ngx_http_v2_string_encode(u_char *dst, u_char *src, size_t len,
     u_char *tmp, ngx_uint_t lower);
@@ -685,16 +713,17 @@ ngx_http_v2_push_resources(ngx_http_requ
 {
     u_char                     *start, *end, *last;
     ngx_int_t                   rc;
-    ngx_str_t                   path, authority;
+    ngx_str_t                   path;
     ngx_uint_t                  i, push;
     ngx_table_elt_t           **h;
     ngx_http_v2_loc_conf_t     *h2lcf;
     ngx_http_complex_value_t   *pushes;
+    ngx_str_t                   binary[NGX_HTTP_V2_PUSH_HEADERS];
 
     ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                    "http2 push resources");
 
-    ngx_str_null(&authority);
+    ngx_memzero(binary, NGX_HTTP_V2_PUSH_HEADERS * sizeof(ngx_str_t));
 
     h2lcf = ngx_http_get_module_loc_conf(r, ngx_http_v2_module);
 
@@ -715,7 +744,7 @@ ngx_http_v2_push_resources(ngx_http_requ
                 continue;
             }
 
-            rc = ngx_http_v2_push_resource(r, &path, &authority);
+            rc = ngx_http_v2_push_resource(r, &path, binary);
 
             if (rc == NGX_ERROR) {
                 return NGX_ERROR;
@@ -880,7 +909,7 @@ ngx_http_v2_push_resources(ngx_http_requ
         if (push && path.len
             && !(path.len > 1 && path.data[0] == '/' && path.data[1] == '/'))
         {
-            rc = ngx_http_v2_push_resource(r, &path, &authority);
+            rc = ngx_http_v2_push_resource(r, &path, binary);
 
             if (rc == NGX_ERROR) {
                 return NGX_ERROR;
@@ -905,15 +934,18 @@ ngx_http_v2_push_resources(ngx_http_requ
 
 static ngx_int_t
 ngx_http_v2_push_resource(ngx_http_request_t *r, ngx_str_t *path,
-    ngx_str_t *authority)
+    ngx_str_t *binary)
 {
-    u_char                    *start, *pos, *tmp;
-    size_t                     len;
-    ngx_table_elt_t           *host;
-    ngx_connection_t          *fc;
-    ngx_http_v2_stream_t      *stream;
-    ngx_http_v2_out_frame_t   *frame;
-    ngx_http_v2_connection_t  *h2c;
+    u_char                      *start, *pos, *tmp;
+    size_t                       len;
+    ngx_str_t                   *value;
+    ngx_uint_t                   i;
+    ngx_table_elt_t            **h;
+    ngx_connection_t            *fc;
+    ngx_http_v2_stream_t        *stream;
+    ngx_http_v2_out_frame_t     *frame;
+    ngx_http_v2_connection_t    *h2c;
+    ngx_http_v2_push_header_t   *ph;
 
     fc = r->connection;
 
@@ -944,42 +976,70 @@ ngx_http_v2_push_resource(ngx_http_reque
         return NGX_DECLINED;
     }
 
-    host = r->headers_in.host;
-
-    if (host == NULL) {
+    if (r->headers_in.host == NULL) {
         return NGX_ABORT;
     }
 
-    if (authority->len == 0) {
-
-        len = 1 + NGX_HTTP_V2_INT_OCTETS + host->value.len;
+    ph = ngx_http_v2_push_headers;
+
+    if (binary[0].len) {
+        tmp = ngx_palloc(r->pool, path->len);
+        if (tmp == NULL) {
+            return NGX_ERROR;
+        }
+
+    } else {
+        len = path->len;
+
+        for (i = 0; i < NGX_HTTP_V2_PUSH_HEADERS; i++) {
+            h = (ngx_table_elt_t **) ((char *) &r->headers_in + ph[i].offset);
+
+            if (*h) {
+                len = ngx_max(len, (*h)->value.len);
+            }
+        }
 
         tmp = ngx_palloc(r->pool, len);
-        pos = ngx_pnalloc(r->pool, len);
-
-        if (pos == NULL || tmp == NULL) {
+        if (tmp == NULL) {
             return NGX_ERROR;
         }
 
-        authority->data = pos;
-
-        *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_AUTHORITY_INDEX);
-        pos = ngx_http_v2_write_value(pos, host->value.data, host->value.len,
-                                      tmp);
-
-        authority->len = pos - authority->data;
+        for (i = 0; i < NGX_HTTP_V2_PUSH_HEADERS; i++) {
+            h = (ngx_table_elt_t **) ((char *) &r->headers_in + ph[i].offset);
+
+            if (*h == NULL) {
+                continue;
+            }
+
+            value = &(*h)->value;
+
+            len = 1 + NGX_HTTP_V2_INT_OCTETS + value->len;
+
+            pos = ngx_pnalloc(r->pool, len);
+            if (pos == NULL) {
+                return NGX_ERROR;
+            }
+
+            binary[i].data = pos;
+
+            *pos++ = ngx_http_v2_inc_indexed(ph[i].index);
+            pos = ngx_http_v2_write_value(pos, value->data, value->len, tmp);
+
+            binary[i].len = pos - binary[i].data;
+        }
     }
 
     len = (h2c->table_update ? 1 : 0)
           + 1
           + 1 + NGX_HTTP_V2_INT_OCTETS + path->len
-          + authority->len
           + 1;
 
-    tmp = ngx_palloc(r->pool, len);
+    for (i = 0; i < NGX_HTTP_V2_PUSH_HEADERS; i++) {
+        len += binary[i].len;
+    }
+
     pos = ngx_pnalloc(r->pool, len);
-
-    if (pos == NULL || tmp == NULL) {
+    if (pos == NULL) {
         return NGX_ERROR;
     }
 
@@ -1003,11 +1063,6 @@ ngx_http_v2_push_resource(ngx_http_reque
     *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_PATH_INDEX);
     pos = ngx_http_v2_write_value(pos, path->data, path->len, tmp);
 
-    ngx_log_debug1(NGX_LOG_DEBUG_HTTP, fc->log, 0,
-                   "http2 push header: \":authority: %V\"", &host->value);
-
-    pos = ngx_cpymem(pos, authority->data, authority->len);
-
 #if (NGX_HTTP_SSL)
     if (fc->ssl) {
         ngx_log_debug0(NGX_LOG_DEBUG_HTTP, fc->log, 0,
@@ -1022,6 +1077,20 @@ ngx_http_v2_push_resource(ngx_http_reque
         *pos++ = ngx_http_v2_indexed(NGX_HTTP_V2_SCHEME_HTTP_INDEX);
     }
 
+    for (i = 0; i < NGX_HTTP_V2_PUSH_HEADERS; i++) {
+        h = (ngx_table_elt_t **) ((char *) &r->headers_in + ph[i].offset);
+
+        if (*h == NULL) {
+            continue;
+        }
+
+        ngx_log_debug2(NGX_LOG_DEBUG_HTTP, fc->log, 0,
+                       "http2 push header: \"%V: %V\"",
+                       &ph[i].name, &(*h)->value);
+
+        pos = ngx_cpymem(pos, binary[i].data, binary[i].len);
+    }
+
     frame = ngx_http_v2_create_push_frame(r, start, pos);
     if (frame == NULL) {
         return NGX_ERROR;
@@ -1031,8 +1100,14 @@ ngx_http_v2_push_resource(ngx_http_reque
 
     stream->queued++;
 
-    return ngx_http_v2_push_stream(h2c, stream->node->id, pos - start,
-                                   path, &host->value);
+    stream = ngx_http_v2_push_stream(stream, path);
+
+    if (stream) {
+        stream->request->request_length = pos - start;
+        return NGX_OK;
+    }
+
+    return NGX_ERROR;
 }
 
 

From ru at nginx.com  Thu Feb 15 14:55:03 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 15 Feb 2018 14:55:03 +0000
Subject: [nginx] HTTP/2: fixed ngx_http_v2_push_stream() allocation error
 handling.
Message-ID: <hg.affeb6ef732c.1518706503.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/affeb6ef732c
branches:  
changeset: 7208:affeb6ef732c
user:      Ruslan Ermilov <ru at nginx.com>
date:      Thu Feb 15 17:51:37 2018 +0300
description:
HTTP/2: fixed ngx_http_v2_push_stream() allocation error handling.

In particular, if a stream object allocation failed, and a client sent
the PRIORITY frame for this stream, ngx_http_v2_set_dependency() could
dereference a null pointer while trying to re-parent a dependency node.

diffstat:

 src/http/v2/ngx_http_v2.c |  62 +++++++++++++++++++++++++++++++++++-----------
 1 files changed, 47 insertions(+), 15 deletions(-)

diffs (115 lines):

diff -r 3d2b0b02bd3d -r affeb6ef732c src/http/v2/ngx_http_v2.c
--- a/src/http/v2/ngx_http_v2.c	Thu Feb 15 17:51:32 2018 +0300
+++ b/src/http/v2/ngx_http_v2.c	Thu Feb 15 17:51:37 2018 +0300
@@ -2546,20 +2546,47 @@ ngx_http_v2_push_stream(ngx_http_v2_stre
 {
     ngx_int_t                     rc;
     ngx_str_t                     value;
+    ngx_pool_t                   *pool;
+    ngx_uint_t                    index;
     ngx_table_elt_t             **h;
     ngx_connection_t             *fc;
     ngx_http_request_t           *r;
     ngx_http_v2_node_t           *node;
     ngx_http_v2_stream_t         *stream;
+    ngx_http_v2_srv_conf_t       *h2scf;
     ngx_http_v2_connection_t     *h2c;
     ngx_http_v2_parse_header_t   *header;
 
     h2c = parent->connection;
 
+    pool = ngx_create_pool(1024, h2c->connection->log);
+    if (pool == NULL) {
+        goto rst_stream;
+    }
+
     node = ngx_http_v2_get_node_by_id(h2c, h2c->last_push, 1);
 
     if (node == NULL) {
-        return NULL;
+        ngx_destroy_pool(pool);
+        goto rst_stream;
+    }
+
+    stream = ngx_http_v2_create_stream(h2c, 1);
+    if (stream == NULL) {
+
+        if (node->parent == NULL) {
+            h2scf = ngx_http_get_module_srv_conf(h2c->http_connection->conf_ctx,
+                                                 ngx_http_v2_module);
+
+            index = ngx_http_v2_index(h2scf, h2c->last_push);
+            h2c->streams_index[index] = node->index;
+
+            ngx_queue_insert_tail(&h2c->closed, &node->reuse);
+            h2c->closed_nodes++;
+        }
+
+        ngx_destroy_pool(pool);
+        goto rst_stream;
     }
 
     if (node->parent) {
@@ -2567,15 +2594,7 @@ ngx_http_v2_push_stream(ngx_http_v2_stre
         h2c->closed_nodes--;
     }
 
-    stream = ngx_http_v2_create_stream(h2c, 1);
-    if (stream == NULL) {
-        return NULL;
-    }
-
-    stream->pool = ngx_create_pool(1024, h2c->connection->log);
-    if (stream->pool == NULL) {
-        return NULL;
-    }
+    stream->pool = pool;
 
     r = stream->request;
     fc = r->connection;
@@ -2608,9 +2627,9 @@ ngx_http_v2_push_stream(ngx_http_v2_stre
         r->schema_end = r->schema_start + 4;
     }
 
-    value.data = ngx_pstrdup(stream->pool, path);
+    value.data = ngx_pstrdup(pool, path);
     if (value.data == NULL) {
-        return NULL;
+        goto close;
     }
 
     value.len = path->len;
@@ -2631,9 +2650,9 @@ ngx_http_v2_push_stream(ngx_http_v2_stre
 
         value.len = (*h)->value.len;
 
-        value.data = ngx_pnalloc(stream->pool, value.len + 1);
+        value.data = ngx_pnalloc(pool, value.len + 1);
         if (value.data == NULL) {
-            return NULL;
+            goto close;
         }
 
         ngx_memcpy(value.data, (*h)->value.data, value.len);
@@ -2663,7 +2682,20 @@ error:
         return NULL;
     }
 
-    (void) ngx_http_v2_connection_error(h2c, NGX_HTTP_V2_INTERNAL_ERROR);
+close:
+
+    ngx_http_v2_close_stream(stream, NGX_HTTP_INTERNAL_SERVER_ERROR);
+
+    return NULL;
+
+rst_stream:
+
+    if (ngx_http_v2_send_rst_stream(h2c, h2c->last_push,
+                                    NGX_HTTP_INTERNAL_SERVER_ERROR)
+        != NGX_OK)
+    {
+        h2c->connection->error = 1;
+    }
 
     return NULL;
 }

From mdounin at mdounin.ru  Thu Feb 15 18:16:37 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 15 Feb 2018 18:16:37 +0000
Subject: [nginx] HTTP/2: precalculate hash for "Cookie".
Message-ID: <hg.3dfe9444324b.1518718597.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/3dfe9444324b
branches:  
changeset: 7209:3dfe9444324b
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Feb 15 19:06:22 2018 +0300
description:
HTTP/2: precalculate hash for "Cookie".

There is no need to calculate hashes of static strings at runtime.  The
ngx_hash() macro can be used to do it during compilation instead, similarly
to how it is done in ngx_http_proxy_module.c for "Server" and "Date" headers.

diffstat:

 src/http/v2/ngx_http_v2.c |  3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diffs (13 lines):

diff --git a/src/http/v2/ngx_http_v2.c b/src/http/v2/ngx_http_v2.c
--- a/src/http/v2/ngx_http_v2.c
+++ b/src/http/v2/ngx_http_v2.c
@@ -3698,7 +3698,8 @@ ngx_http_v2_construct_cookie_header(ngx_
         return NGX_ERROR;
     }
 
-    h->hash = ngx_hash_key(cookie.data, cookie.len);
+    h->hash = ngx_hash(ngx_hash(ngx_hash(ngx_hash(
+                                    ngx_hash('c', 'o'), 'o'), 'k'), 'i'), 'e');
 
     h->key.len = cookie.len;
     h->key.data = cookie.data;

From rep.dot.nop at gmail.com  Fri Feb 16 14:57:05 2018
From: rep.dot.nop at gmail.com (Bernhard Reutner-Fischer)
Date: Fri, 16 Feb 2018 15:57:05 +0100
Subject: [PATCH] Limit req: r/h and r/d support
Message-ID: <CAC1BbcTBsqGMeTnmT=dJiee-2krNRbPVutrPdxsreONFS6wbfA@mail.gmail.com>

Hi!

Attached is a patch to add r/h and r/d limit_req support.

There are at least two feature requests that i found asking for this:
- https://trac.nginx.org/nginx/ticket/68
- https://trac.nginx.org/nginx/ticket/1060

Ok?

Please consider applying.

thanks and cheers,
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ngx_http_limit_req_module.00.patch
Type: text/x-patch
Size: 5276 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180216/995d0a9a/attachment.bin>

From rep.dot.nop at gmail.com  Fri Feb 16 15:01:52 2018
From: rep.dot.nop at gmail.com (Bernhard Reutner-Fischer)
Date: Fri, 16 Feb 2018 16:01:52 +0100
Subject: [PATCH] Limit req: r/h and r/d support
In-Reply-To: <CAC1BbcTBsqGMeTnmT=dJiee-2krNRbPVutrPdxsreONFS6wbfA@mail.gmail.com>
References: <CAC1BbcTBsqGMeTnmT=dJiee-2krNRbPVutrPdxsreONFS6wbfA@mail.gmail.com>
Message-ID: <CAC1BbcS+QCbFKg2Ldptg-imzw=XJt_qQnot-VXVAPMeVQJ7Pnw@mail.gmail.com>

On 16 February 2018 at 15:57, Bernhard Reutner-Fischer
<rep.dot.nop at gmail.com> wrote:
> Hi!
>
> Attached is a patch to add r/h and r/d limit_req support.
>
> There are at least two feature requests that i found asking for this:
> - https://trac.nginx.org/nginx/ticket/68
> - https://trac.nginx.org/nginx/ticket/1060
>
I forgot to mention that it regression-tests cleanly for nginx-tests
and seems to work as intended.

> Ok?
>
> Please consider applying.
>
> thanks and cheers,

From mdounin at mdounin.ru  Fri Feb 16 16:52:25 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 16 Feb 2018 19:52:25 +0300
Subject: [PATCH] Limit req: r/h and r/d support
In-Reply-To: <CAC1BbcTBsqGMeTnmT=dJiee-2krNRbPVutrPdxsreONFS6wbfA@mail.gmail.com>
References: <CAC1BbcTBsqGMeTnmT=dJiee-2krNRbPVutrPdxsreONFS6wbfA@mail.gmail.com>
Message-ID: <20180216165224.GB24410@mdounin.ru>

Hello!

On Fri, Feb 16, 2018 at 03:57:05PM +0100, Bernhard Reutner-Fischer wrote:

> Attached is a patch to add r/h and r/d limit_req support.
> 
> There are at least two feature requests that i found asking for this:
> - https://trac.nginx.org/nginx/ticket/68
> - https://trac.nginx.org/nginx/ticket/1060
> 
> Ok?
> 
> Please consider applying.

Do you use it yourself, or your patch is based on the feature 
requests in question?  If yes, what is your use case?

Some obvious problems with the patch outlined below.

> # HG changeset patch
> # User Bernhard Reutner-Fischer <rep.dot.nop at gmail.com>
> # Date 1518790589 -3600
> #      Fri Feb 16 15:16:29 2018 +0100
> # Node ID 4f5981016b6d3b4b9745f90ddf9521fa7a0d375f
> # Parent  a49af443656f2b65ca5de9d8cad5594f44e18ff7
> Limit req: r/h and r/d support

Style: please add a trailing dot.

> 
> diff -r a49af443656f -r 4f5981016b6d src/http/modules/ngx_http_limit_req_module.c
> --- a/src/http/modules/ngx_http_limit_req_module.c	Thu Feb 08 12:11:30 2018 +0300
> +++ b/src/http/modules/ngx_http_limit_req_module.c	Fri Feb 16 15:16:29 2018 +0100
> @@ -16,7 +16,7 @@
>      u_short                      len;
>      ngx_queue_t                  queue;
>      ngx_msec_t                   last;
> -    /* integer value, 1 corresponds to 0.001 r/s */
> +    /* integer value, 1 corresponds to 0.00001 r/s */
>      ngx_uint_t                   excess;
>      ngx_uint_t                   count;
>      u_char                       data[1];
> @@ -33,7 +33,7 @@
>  typedef struct {
>      ngx_http_limit_req_shctx_t  *sh;
>      ngx_slab_pool_t             *shpool;
> -    /* integer value, 1 corresponds to 0.001 r/s */
> +    /* integer value, 1 corresponds to 0.00001 r/s */
>      ngx_uint_t                   rate;

On 32-bit platforms this means that the maximum supported rate 
would be about ~40kr/s.  And I suspect this can hurt real setups.

[...]

> @@ -213,7 +213,7 @@
>  
>          ngx_log_debug4(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
>                         "limit_req[%ui]: %i %ui.%03ui",
> -                       n, rc, excess / 1000, excess % 1000);
> +                       n, rc, excess / 100000, excess % 100000);

The "%ui.%03ui" format needs to be adjusted as well, here and in 
all other places.

[...]

> @@ -398,8 +398,9 @@
>              ngx_queue_insert_head(&ctx->sh->queue, &lr->queue);
>  
>              ms = (ngx_msec_int_t) (now - lr->last);
> +            ms = ngx_abs(ms);
>  
> -            excess = lr->excess - ctx->rate * ngx_abs(ms) / 1000 + 1000;
> +            excess = lr->excess - ctx->rate * ms * 100 / 100000 + 100000;

Just keeping "/ 1000" might be a better option, given it 
corresponds to milliseconds in the "ms" variable.

[...]

> @@ -527,7 +529,7 @@
>              continue;
>          }
>  
> -        delay = excess * 1000 / ctx->rate;
> +        delay = excess * 100000 / ctx->rate;
>  
>          if (delay > max_delay) {
>              max_delay = delay;
> @@ -535,7 +537,7 @@
>              *limit = &limits[n];
>          }
>      }
> -
> +    max_delay /= 100;
>      return max_delay;

It is not clear why calculate delay in different units instead of 
milliseconds as specified by the type used, and then convert it 
back to milliseconds.

[...]

> @@ -825,7 +835,11 @@
>          return NGX_CONF_ERROR;
>      }
>  
> -    ctx->rate = rate * 1000 / scale;
> +    ctx->rate = rate * 100000 / scale;
> +    if (ctx->rate < 1) {
> +        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid rate \"%ui\"", rate);
> +        return NGX_CONF_ERROR;
> +    }

The ctx->rate is unsigned, and the ctx->rate check will only fail 
if ctx->rate is 0.  While this might happen in some cases, it 
certainly won't catch all overflows (which are quite likely in 
real configurations on 32-bit platforms, see above).

Also, the "rate" type is ngx_int_t, so (rate * 100000) will cause 
integer overflow, which is not defined.  It might be a better idea 
to avoid overflows at all.

Also, the "rate" type is ngx_int_t, not ngx_uint_t, so proper 
format would be "%i". 

In general, it might be a better idea to move the check to 
ngx_atoi(), and test something like NGX_MAX_INT_T_VALUE / 100000.  
This will ensure no overflow can happen during calculations below, 
and will allow logging of the original value.  This will further 
limit maximum rate to about 20kr/s on 32-bit platforms though.

Note that I have no good solution for 32-bit platforms and the 
40kr/s (20kr/s) limit.

-- 
Maxim Dounin
http://mdounin.ru/

From rep.dot.nop at gmail.com  Sat Feb 17 21:07:27 2018
From: rep.dot.nop at gmail.com (Bernhard Reutner-Fischer)
Date: Sat, 17 Feb 2018 22:07:27 +0100
Subject: [PATCH] Limit req: r/h and r/d support
In-Reply-To: <20180216165224.GB24410@mdounin.ru>
References: <CAC1BbcTBsqGMeTnmT=dJiee-2krNRbPVutrPdxsreONFS6wbfA@mail.gmail.com>
 <20180216165224.GB24410@mdounin.ru>
Message-ID: <CAC1BbcRP=gtV=AqvfrBvR=UU0ZmuO+a-+uzVQ8OonBKGawWqkw@mail.gmail.com>

On 16 February 2018 at 17:52, Maxim Dounin <mdounin at mdounin.ru> wrote:

>> Limit req: r/h and r/d support

> Do you use it yourself, or your patch is based on the feature
> requests in question?  If yes, what is your use case?

I have to limit certain use-cases to a handful of requests per day (don't ask).
Mentioned feature-requests just show that this oddish corner case is a
real knock-out criterion in the real world so i mentioned them to
emphasis that it wasn't just /me but that you're losing users due to
this. Often this is a hard requirement. You don't seem to offer any
sensible, off the shelve (or documented) way to handle "long" request
limits anyway -- think shm -- at least none that i was able to find?

> Style: please add a trailing dot.

Sure, please excuse my sloppy reading of previous logs, i honestly
failed to notice that, added.

> On 32-bit platforms this means that the maximum supported rate
> would be about ~40kr/s.  And I suspect this can hurt real setups.

agreed. Not sure about current typical HTTP server rates, admittedly.
Getting esoteric, but current servers usually satisfy millions of
requests but i'm not sure if you'd usually limit those to let's say
100k/s, but of course i see your point.
[]
> The ctx->rate is unsigned, and the ctx->rate check will only fail
> if ctx->rate is 0.  While this might happen in some cases, it
> certainly won't catch all overflows (which are quite likely in
> real configurations on 32-bit platforms, see above).
>
> Also, the "rate" type is ngx_int_t, so (rate * 100000) will cause
> integer overflow, which is not defined.  It might be a better idea
> to avoid overflows at all.

yea, see below.
>
> Also, the "rate" type is ngx_int_t, not ngx_uint_t, so proper
> format would be "%i".

right.

> In general, it might be a better idea to move the check to
> ngx_atoi(), and test something like NGX_MAX_INT_T_VALUE / 100000.
> This will ensure no overflow can happen during calculations below,
> and will allow logging of the original value.  This will further
> limit maximum rate to about 20kr/s on 32-bit platforms though.
>
> Note that I have no good solution for 32-bit platforms and the
> 40kr/s (20kr/s) limit.

my notes read:
disentangle rate. Should have rate(per unit) and unit(in seconds?
minutes? calc!) where excess would have same base as rate; common
timer conversion helper for rate/unit?

This has much more impact and hence is a much larger patch but would
certainly be way easier to grok compared to the current handling.
Would you be willing to fix the existing rate mess^whandling?

cheers,

From mdounin at mdounin.ru  Mon Feb 19 14:12:29 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 19 Feb 2018 17:12:29 +0300
Subject: [PATCH] Limit req: r/h and r/d support
In-Reply-To: <CAC1BbcRP=gtV=AqvfrBvR=UU0ZmuO+a-+uzVQ8OonBKGawWqkw@mail.gmail.com>
References: <CAC1BbcTBsqGMeTnmT=dJiee-2krNRbPVutrPdxsreONFS6wbfA@mail.gmail.com>
 <20180216165224.GB24410@mdounin.ru>
 <CAC1BbcRP=gtV=AqvfrBvR=UU0ZmuO+a-+uzVQ8OonBKGawWqkw@mail.gmail.com>
Message-ID: <20180219141229.GD24410@mdounin.ru>

Hello!

On Sat, Feb 17, 2018 at 10:07:27PM +0100, Bernhard Reutner-Fischer wrote:

> On 16 February 2018 at 17:52, Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> >> Limit req: r/h and r/d support
> 
> > Do you use it yourself, or your patch is based on the feature
> > requests in question?  If yes, what is your use case?
> 
> I have to limit certain use-cases to a handful of requests per day (don't ask).
> Mentioned feature-requests just show that this oddish corner case is a
> real knock-out criterion in the real world so i mentioned them to
> emphasis that it wasn't just /me but that you're losing users due to
> this. Often this is a hard requirement. You don't seem to offer any
> sensible, off the shelve (or documented) way to handle "long" request
> limits anyway -- think shm -- at least none that i was able to find?

The limit_req module was created mostly to limit high rates - in 
particular, to limit requests to costly services to fight DoS, and 
to limit requests to password checking to fight bruteforce 
attempts.  It was never meant to limit normal user activity, and 
hence the data format choosen does not support rates less than 
1r/m.  Yet it works effectively with high rates the module was 
created for.

While it is understood that some people might want to use longer 
limits, it's not something the module tries to provide now.  Also, 
it might not actually be a good idea to provide such limits in the 
module, as any nginx restart / upgrade will mean that limits won't 
work as expected for significant time.  If you want to limits 
users to several requests per day, it might be a better idea to 
keep these limits outside of nginx.

> > Style: please add a trailing dot.
> 
> Sure, please excuse my sloppy reading of previous logs, i honestly
> failed to notice that, added.
> 
> > On 32-bit platforms this means that the maximum supported rate
> > would be about ~40kr/s.  And I suspect this can hurt real setups.
> 
> agreed. Not sure about current typical HTTP server rates, admittedly.
> Getting esoteric, but current servers usually satisfy millions of
> requests but i'm not sure if you'd usually limit those to let's say
> 100k/s, but of course i see your point.

I doubt rates like 100kr/s are needed in practice (especially on 
32-bit platforms), but I'm pretty sure that configurations with 
something like this are real.  In particular, googling "limit_req 
rate site:nginx.org/pipermail/" shows examples like 
"rate=999999r/s" pretty quickly.

[...]

> > Note that I have no good solution for 32-bit platforms and the
> > 40kr/s (20kr/s) limit.
> 
> my notes read:
> disentangle rate. Should have rate(per unit) and unit(in seconds?
> minutes? calc!) where excess would have same base as rate; common
> timer conversion helper for rate/unit?
> 
> This has much more impact and hence is a much larger patch but would
> certainly be way easier to grok compared to the current handling.
> Would you be willing to fix the existing rate mess^whandling?

Current handling is pretty clear and simple: all calculations are 
done in requests per millisecond, and this requires just one value 
in various structures.  We can tolerate switching to something 
different, though it would be a good idea to keep code logic close 
to the current implementation.

-- 
Maxim Dounin
http://mdounin.ru/

From mdounin at mdounin.ru  Tue Feb 20 14:17:14 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 20 Feb 2018 14:17:14 +0000
Subject: [nginx] nginx-1.13.9-RELEASE
Message-ID: <hg.fb1212c7eca4.1519136234.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/fb1212c7eca4
branches:  
changeset: 7210:fb1212c7eca4
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Feb 20 17:08:48 2018 +0300
description:
nginx-1.13.9-RELEASE

diffstat:

 docs/xml/nginx/changes.xml |  60 ++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 60 insertions(+), 0 deletions(-)

diffs (70 lines):

diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml
--- a/docs/xml/nginx/changes.xml
+++ b/docs/xml/nginx/changes.xml
@@ -5,6 +5,66 @@
 <change_log title="nginx">
 
 
+<changes ver="1.13.9" date="2018-02-20">
+
+<change type="feature">
+<para lang="ru">
+????????? HTTP/2 server push;
+????????? http2_push ? http2_push_preload.
+</para>
+<para lang="en">
+HTTP/2 server push support;
+the "http2_push" and "http2_push_preload" directives.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ????????????? ????
+? ????? ????? ?????????? ????????? "header already sent";
+?????? ????????? ? 1.9.13.
+</para>
+<para lang="en">
+"header already sent" alerts might appear in logs
+when using cache;
+the bug had appeared in 1.9.13.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ????????????? ????????? ssl_verify_client
+? ??????? ???????? ??? ????????? segmentation fault,
+???? ? ??????????? ??????? ?? ??? ?????? SSL-??????????.
+</para>
+<para lang="en">
+a segmentation fault might occur in a worker process
+if the "ssl_verify_client" directive was used
+and no SSL certificate was specified in a virtual server.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ?????? ngx_http_v2_module.
+</para>
+<para lang="en">
+in the ngx_http_v2_module.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ?????? ngx_http_dav_module.
+</para>
+<para lang="en">
+in the ngx_http_dav_module.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.13.8" date="2017-12-26">
 
 <change type="feature">

From mdounin at mdounin.ru  Tue Feb 20 14:17:16 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 20 Feb 2018 14:17:16 +0000
Subject: [nginx] release-1.13.9 tag
Message-ID: <hg.c91356fdd802.1519136236.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/c91356fdd802
branches:  
changeset: 7211:c91356fdd802
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Feb 20 17:08:49 2018 +0300
description:
release-1.13.9 tag

diffstat:

 .hgtags |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (8 lines):

diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -421,3 +421,4 @@ 0d45b4cf7c2e4e626a5a16e1fe604402ace1cea5
 f87da7d9ca02b8ced4caa6c5eb9013ccd47b0117 release-1.13.6
 47cca243d0ed39bf5dcb9859184affc958b79b6f release-1.13.7
 20ca4bcff108d3e66977f4d97508637093492287 release-1.13.8
+fb1212c7eca4c5328fe17d6cd95b010c67336aac release-1.13.9

From xeioex at nginx.com  Tue Feb 20 16:19:28 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 20 Feb 2018 16:19:28 +0000
Subject: [njs] Hiding the unit tests' output under the verbose mode.
Message-ID: <hg.18e0275576a3.1519143568.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/18e0275576a3
branches:  
changeset: 449:18e0275576a3
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Feb 20 19:12:53 2018 +0300
description:
Hiding the unit tests' output under the verbose mode.

diffstat:

 Makefile                        |   2 +-
 njs/test/njs_interactive_test.c |  26 ++++++++++++++++++++++----
 njs/test/njs_unit_test.c        |  25 ++++++++++++++++---------
 3 files changed, 39 insertions(+), 14 deletions(-)

diffs (129 lines):

diff -r e7bc9d328a20 -r 18e0275576a3 Makefile
--- a/Makefile	Tue Feb 20 19:12:53 2018 +0300
+++ b/Makefile	Tue Feb 20 19:12:53 2018 +0300
@@ -91,7 +91,7 @@ test:	njs_interactive_test \
 	$(NXT_BUILDDIR)/njs_unit_test \
 	$(NXT_BUILDDIR)/njs_benchmark \
 
-	$(NXT_BUILDDIR)/njs_unit_test d
+	$(NXT_BUILDDIR)/njs_unit_test
 
 clean:
 	rm -rf $(NXT_BUILDDIR)
diff -r e7bc9d328a20 -r 18e0275576a3 njs/test/njs_interactive_test.c
--- a/njs/test/njs_interactive_test.c	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/test/njs_interactive_test.c	Tue Feb 20 19:12:53 2018 +0300
@@ -192,7 +192,7 @@ static njs_interactive_test_t  njs_test[
 
 
 static nxt_int_t
-njs_interactive_test(void)
+njs_interactive_test(nxt_bool_t verbose)
 {
     u_char                  *start, *last, *end;
     njs_vm_t                *vm;
@@ -210,8 +210,10 @@ njs_interactive_test(void)
 
         test = &njs_test[i];
 
-        printf("\"%.*s\"\n", (int) test->script.length, test->script.start);
-        fflush(stdout);
+        if (verbose) {
+            printf("\"%.*s\"\n", (int) test->script.length, test->script.start);
+            fflush(stdout);
+        }
 
         memset(&options, 0, sizeof(njs_vm_opt_t));
 
@@ -279,5 +281,21 @@ done:
 int nxt_cdecl
 main(int argc, char **argv)
 {
-    return njs_interactive_test();
+    nxt_bool_t  verbose;
+
+    verbose = 0;
+
+    if (argc > 1) {
+        switch (argv[1][0]) {
+
+        case 'v':
+            verbose = 1;
+            break;
+
+        default:
+            break;
+        }
+    }
+
+    return njs_interactive_test(verbose);
 }
diff -r e7bc9d328a20 -r 18e0275576a3 njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/test/njs_unit_test.c	Tue Feb 20 19:12:53 2018 +0300
@@ -9376,7 +9376,7 @@ njs_externals_init(njs_vm_t *vm)
 
 
 static nxt_int_t
-njs_unit_test(nxt_bool_t disassemble)
+njs_unit_test(nxt_bool_t disassemble, nxt_bool_t verbose)
 {
     u_char        *start;
     njs_vm_t      *vm, *nvm;
@@ -9401,9 +9401,11 @@ njs_unit_test(nxt_bool_t disassemble)
 
     for (i = 0; i < nxt_nitems(njs_test); i++) {
 
-        printf("\"%.*s\"\n",
-               (int) njs_test[i].script.length, njs_test[i].script.start);
-        fflush(stdout);
+        if (verbose) {
+            printf("\"%.*s\"\n",
+                   (int) njs_test[i].script.length, njs_test[i].script.start);
+            fflush(stdout);
+        }
 
         memset(&options, 0, sizeof(njs_vm_opt_t));
 
@@ -9462,10 +9464,10 @@ njs_unit_test(nxt_bool_t disassemble)
             continue;
         }
 
-        printf("njs(\"%.*s\") failed: \"%.*s\" vs \"%.*s\"\n",
+        printf("njs(\"%.*s\")\nexpected: \"%.*s\"\n     got: \"%.*s\"\n",
                (int) njs_test[i].script.length, njs_test[i].script.start,
-               (int) njs_test[i].ret.length, njs_test[i].ret.start,
-               (int) s.length, s.start);
+               (int) s.length, s.start, (int) njs_test[i].ret.length,
+               njs_test[i].ret.start);
 
         goto done;
     }
@@ -9493,9 +9495,10 @@ done:
 int nxt_cdecl
 main(int argc, char **argv)
 {
-    nxt_bool_t  disassemble;
+    nxt_bool_t  disassemble, verbose;
 
     disassemble = 0;
+    verbose = 0;
 
     if (argc > 1) {
         switch (argv[1][0]) {
@@ -9504,10 +9507,14 @@ main(int argc, char **argv)
             disassemble = 1;
             break;
 
+        case 'v':
+            verbose = 1;
+            break;
+
         default:
             break;
         }
     }
 
-    return njs_unit_test(disassemble);
+    return njs_unit_test(disassemble, verbose);
 }

From xeioex at nginx.com  Tue Feb 20 16:19:28 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 20 Feb 2018 16:19:28 +0000
Subject: [njs] Fixed the names of global functions in backtraces.
Message-ID: <hg.757271547b56.1519143568.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/757271547b56
branches:  
changeset: 450:757271547b56
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Feb 20 19:12:55 2018 +0300
description:
Fixed the names of global functions in backtraces.

Previously, they were reported as 'native (native)'.

diffstat:

 njs/njs_builtin.c               |   92 ++++++++++++++++++++---------------
 njs/njs_extern.c                |  102 ++++++++++++++++++++++++++++++++++++++++
 njs/njs_extern.h                |    4 +
 njs/njs_function.c              |    2 +-
 njs/njs_module.c                |    7 ++
 njs/njs_module.h                |    1 +
 njs/njs_number.c                |   28 ++++++++++
 njs/njs_number.h                |    5 +
 njs/njs_string.c                |   35 +++++++++++++
 njs/njs_string.h                |    6 ++
 njs/njs_vm.c                    |   12 +++-
 njs/test/njs_expect_test.exp    |    6 ++
 njs/test/njs_interactive_test.c |   10 +++
 13 files changed, 268 insertions(+), 42 deletions(-)

diffs (463 lines):

diff -r 18e0275576a3 -r 757271547b56 njs/njs_builtin.c
--- a/njs/njs_builtin.c	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/njs_builtin.c	Tue Feb 20 19:12:55 2018 +0300
@@ -102,6 +102,38 @@ const njs_object_init_t    *njs_construc
 };
 
 
+const njs_object_init_t    *njs_function_init[] = {
+    &njs_eval_function_init,
+    &njs_to_string_function_init,
+    &njs_is_nan_function_init,
+    &njs_is_finite_function_init,
+    &njs_parse_int_function_init,
+    &njs_parse_float_function_init,
+    &njs_encode_uri_function_init,
+    &njs_encode_uri_component_function_init,
+    &njs_decode_uri_function_init,
+    &njs_decode_uri_component_function_init,
+    &njs_require_function_init
+};
+
+
+const njs_function_init_t  njs_native_functions[] = {
+    /* SunC does not allow empty array initialization. */
+    { njs_eval_function,               { 0 } },
+    { njs_object_prototype_to_string,  { 0 } },
+    { njs_number_global_is_nan,        { NJS_SKIP_ARG, NJS_NUMBER_ARG } },
+    { njs_number_is_finite,            { NJS_SKIP_ARG, NJS_NUMBER_ARG } },
+    { njs_number_parse_int,
+      { NJS_SKIP_ARG, NJS_STRING_ARG, NJS_INTEGER_ARG } },
+    { njs_number_parse_float,          { NJS_SKIP_ARG, NJS_STRING_ARG } },
+    { njs_string_encode_uri,           { NJS_SKIP_ARG, NJS_STRING_ARG } },
+    { njs_string_encode_uri_component, { NJS_SKIP_ARG, NJS_STRING_ARG } },
+    { njs_string_decode_uri,           { NJS_SKIP_ARG, NJS_STRING_ARG } },
+    { njs_string_decode_uri_component, { NJS_SKIP_ARG, NJS_STRING_ARG } },
+    { njs_module_require,              { NJS_SKIP_ARG, NJS_STRING_ARG } },
+};
+
+
 static njs_ret_t
 njs_prototype_function(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
     njs_index_t unused)
@@ -190,36 +222,6 @@ njs_builtin_objects_create(njs_vm_t *vm)
         { njs_memory_error_constructor,  { NJS_SKIP_ARG, NJS_STRING_ARG } },
     };
 
-    static const njs_object_init_t    *function_init[] = {
-        &njs_eval_function_init,      /* eval               */
-        NULL,                         /* toString           */
-        NULL,                         /* isNaN              */
-        NULL,                         /* isFinite           */
-        NULL,                         /* parseInt           */
-        NULL,                         /* parseFloat         */
-        NULL,                         /* encodeURI          */
-        NULL,                         /* encodeURIComponent */
-        NULL,                         /* decodeURI          */
-        NULL,                         /* decodeURIComponent */
-        NULL,                         /* require */
-    };
-
-    static const njs_function_init_t  native_functions[] = {
-        /* SunC does not allow empty array initialization. */
-        { njs_eval_function,               { 0 } },
-        { njs_object_prototype_to_string,  { 0 } },
-        { njs_number_global_is_nan,        { NJS_SKIP_ARG, NJS_NUMBER_ARG } },
-        { njs_number_is_finite,            { NJS_SKIP_ARG, NJS_NUMBER_ARG } },
-        { njs_number_parse_int,
-          { NJS_SKIP_ARG, NJS_STRING_ARG, NJS_INTEGER_ARG } },
-        { njs_number_parse_float,          { NJS_SKIP_ARG, NJS_STRING_ARG } },
-        { njs_string_encode_uri,           { NJS_SKIP_ARG, NJS_STRING_ARG } },
-        { njs_string_encode_uri_component, { NJS_SKIP_ARG, NJS_STRING_ARG } },
-        { njs_string_decode_uri,           { NJS_SKIP_ARG, NJS_STRING_ARG } },
-        { njs_string_decode_uri_component, { NJS_SKIP_ARG, NJS_STRING_ARG } },
-        { njs_module_require,              { NJS_SKIP_ARG, NJS_STRING_ARG } },
-    };
-
     static const njs_object_prop_t    null_proto_property = {
         .type = NJS_WHITEOUT,
         .name = njs_string("__proto__"),
@@ -293,10 +295,10 @@ njs_builtin_objects_create(njs_vm_t *vm)
     functions = vm->shared->functions;
 
     for (i = NJS_FUNCTION_EVAL; i < NJS_FUNCTION_MAX; i++) {
-        if (function_init[i] != NULL) {
+        if (njs_function_init[i]->items != 0) {
             ret = njs_object_hash_create(vm, &functions[i].object.shared_hash,
-                                         function_init[i]->properties,
-                                         function_init[i]->items);
+                                         njs_function_init[i]->properties,
+                                         njs_function_init[i]->items);
             if (nxt_slow_path(ret != NXT_OK)) {
                 return NXT_ERROR;
             }
@@ -306,12 +308,12 @@ njs_builtin_objects_create(njs_vm_t *vm)
         functions[i].object.extensible = 1;
         functions[i].native = 1;
         functions[i].args_offset = 1;
-        functions[i].u.native = native_functions[i].native;
-        functions[i].args_types[0] = native_functions[i].args_types[0];
-        functions[i].args_types[1] = native_functions[i].args_types[1];
-        functions[i].args_types[2] = native_functions[i].args_types[2];
-        functions[i].args_types[3] = native_functions[i].args_types[3];
-        functions[i].args_types[4] = native_functions[i].args_types[4];
+        functions[i].u.native = njs_native_functions[i].native;
+        functions[i].args_types[0] = njs_native_functions[i].args_types[0];
+        functions[i].args_types[1] = njs_native_functions[i].args_types[1];
+        functions[i].args_types[2] = njs_native_functions[i].args_types[2];
+        functions[i].args_types[3] = njs_native_functions[i].args_types[3];
+        functions[i].args_types[4] = njs_native_functions[i].args_types[4];
     }
 
     prototypes = vm->shared->prototypes;
@@ -1022,6 +1024,18 @@ njs_builtin_match_native_function(njs_vm
         }
     }
 
+    for (i = NJS_FUNCTION_EVAL; i < NJS_FUNCTION_MAX; i++) {
+        if (njs_function_init[i] == NULL) {
+            continue;
+        }
+
+        if (function->u.native == njs_native_functions[i].native) {
+            *name = njs_function_init[i]->name;
+
+            return NXT_OK;
+        }
+    }
+
     nxt_lvlhsh_each_init(&lhe, &njs_modules_hash_proto);
 
     for ( ;; ) {
diff -r 18e0275576a3 -r 757271547b56 njs/njs_extern.c
--- a/njs/njs_extern.c	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/njs_extern.c	Tue Feb 20 19:12:55 2018 +0300
@@ -22,6 +22,15 @@
 #include <njs_variable.h>
 #include <njs_parser.h>
 #include <string.h>
+#include <stdio.h>
+
+
+typedef struct njs_extern_part_s  njs_extern_part_t;
+
+struct njs_extern_part_s {
+    njs_extern_part_t  *next;
+    nxt_str_t          str;
+};
 
 
 static nxt_int_t
@@ -242,3 +251,96 @@ njs_parser_external(njs_vm_t *vm, njs_pa
 
     return NULL;
 }
+
+
+static nxt_int_t
+njs_external_match(njs_vm_t *vm, njs_function_native_t func, njs_extern_t *ext,
+    nxt_str_t *name, njs_extern_part_t *head, njs_extern_part_t *ppart)
+{
+    char               *buf, *p;
+    size_t             len;
+    nxt_int_t          ret;
+    njs_extern_t       *prop;
+    njs_extern_part_t  part, *pr;
+    nxt_lvlhsh_each_t  lhe;
+
+    ppart->next = &part;
+
+    nxt_lvlhsh_each_init(&lhe, &njs_extern_hash_proto);
+
+    for ( ;; ) {
+        prop = nxt_lvlhsh_each(&ext->hash, &lhe);
+        if (prop == NULL) {
+            break;
+        }
+
+        part.next = NULL;
+        part.str = prop->name;
+
+        if (prop->function && prop->function->u.native == func) {
+            goto found;
+        }
+
+        ret = njs_external_match(vm, func, prop, name, head, &part);
+        if (ret != NXT_DECLINED) {
+            return ret;
+        }
+    }
+
+    return NXT_DECLINED;
+
+found:
+
+    len = 0;
+
+    for (pr = head; pr != NULL; pr = pr->next) {
+        len += pr->str.length + sizeof(".") - 1;
+    }
+
+    buf = nxt_mem_cache_zalloc(vm->mem_cache_pool, len);
+    if (buf == NULL) {
+        return NXT_ERROR;
+    }
+
+    p = buf;
+
+    for (pr = head; pr != NULL; pr = pr->next) {
+        p += snprintf(p, buf + len - p, "%.*s.", (int) pr->str.length,
+                      pr->str.start);
+    }
+
+    name->start = (u_char *) buf;
+    name->length = len;
+
+    return NXT_OK;
+}
+
+
+nxt_int_t
+njs_external_match_native_function(njs_vm_t *vm, njs_function_native_t func,
+    nxt_str_t *name)
+{
+    nxt_int_t          ret;
+    njs_extern_t       *ext;
+    njs_extern_part_t  part;
+    nxt_lvlhsh_each_t  lhe;
+
+    nxt_lvlhsh_each_init(&lhe, &njs_extern_hash_proto);
+
+    for ( ;; ) {
+        ext = nxt_lvlhsh_each(&vm->external_prototypes_hash, &lhe);
+        if (ext == NULL) {
+            break;
+        }
+
+        part.next = NULL;
+        part.str = ext->name;
+
+        ret = njs_external_match(vm, func, ext, name, &part, &part);
+        if (ret != NXT_DECLINED) {
+            return ret;
+        }
+    }
+
+    return NXT_DECLINED;
+}
diff -r 18e0275576a3 -r 757271547b56 njs/njs_extern.h
--- a/njs/njs_extern.h	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/njs_extern.h	Tue Feb 20 19:12:55 2018 +0300
@@ -38,6 +38,10 @@ typedef struct {
 } njs_extern_value_t;
 
 
+nxt_int_t njs_external_match_native_function(njs_vm_t *vm,
+    njs_function_native_t func, nxt_str_t *name);
+
+
 extern const nxt_lvlhsh_proto_t  njs_extern_hash_proto;
 extern const nxt_lvlhsh_proto_t  njs_extern_value_hash_proto;
 
diff -r 18e0275576a3 -r 757271547b56 njs/njs_function.c
--- a/njs/njs_function.c	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/njs_function.c	Tue Feb 20 19:12:55 2018 +0300
@@ -723,7 +723,7 @@ static const njs_object_prop_t  njs_eval
 
 
 const njs_object_init_t  njs_eval_function_init = {
-    nxt_string("Function"),
+    nxt_string("eval"),
     njs_eval_function_properties,
     nxt_nitems(njs_eval_function_properties),
 };
diff -r 18e0275576a3 -r 757271547b56 njs/njs_module.c
--- a/njs/njs_module.c	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/njs_module.c	Tue Feb 20 19:12:55 2018 +0300
@@ -83,3 +83,10 @@ njs_ret_t njs_module_require(njs_vm_t *v
 
     return NJS_ERROR;
 }
+
+
+const njs_object_init_t  njs_require_function_init = {
+    nxt_string("require"),
+    NULL,
+    0,
+};
diff -r 18e0275576a3 -r 757271547b56 njs/njs_module.h
--- a/njs/njs_module.h	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/njs_module.h	Tue Feb 20 19:12:55 2018 +0300
@@ -18,5 +18,6 @@ njs_ret_t njs_module_require(njs_vm_t *v
 
 extern const nxt_lvlhsh_proto_t  njs_modules_hash_proto;
 
+extern const njs_object_init_t  njs_require_function_init;
 
 #endif /* _NJS_MODULE_H_INCLUDED_ */
diff -r 18e0275576a3 -r 757271547b56 njs/njs_number.c
--- a/njs/njs_number.c	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/njs_number.c	Tue Feb 20 19:12:55 2018 +0300
@@ -896,3 +896,31 @@ njs_number_to_integer(double num)
 
     return (uint32_t) i64;
 }
+
+
+const njs_object_init_t  njs_is_nan_function_init = {
+    nxt_string("isNaN"),
+    NULL,
+    0,
+};
+
+
+const njs_object_init_t  njs_is_finite_function_init = {
+    nxt_string("isFinite"),
+    NULL,
+    0,
+};
+
+
+const njs_object_init_t  njs_parse_int_function_init = {
+    nxt_string("parseInt"),
+    NULL,
+    0,
+};
+
+
+const njs_object_init_t  njs_parse_float_function_init = {
+    nxt_string("parseFloat"),
+    NULL,
+    0,
+};
diff -r 18e0275576a3 -r 757271547b56 njs/njs_number.h
--- a/njs/njs_number.h	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/njs_number.h	Tue Feb 20 19:12:55 2018 +0300
@@ -35,5 +35,10 @@ nxt_noinline uint32_t njs_number_to_inte
 extern const njs_object_init_t  njs_number_constructor_init;
 extern const njs_object_init_t  njs_number_prototype_init;
 
+extern const njs_object_init_t  njs_is_nan_function_init;
+extern const njs_object_init_t  njs_is_finite_function_init;
+extern const njs_object_init_t  njs_parse_int_function_init;
+extern const njs_object_init_t  njs_parse_float_function_init;
+
 
 #endif /* _NJS_NUMBER_H_INCLUDED_ */
diff -r 18e0275576a3 -r 757271547b56 njs/njs_string.c
--- a/njs/njs_string.c	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/njs_string.c	Tue Feb 20 19:12:55 2018 +0300
@@ -3791,3 +3791,38 @@ njs_value_index(njs_vm_t *vm, njs_parser
 
     return (njs_index_t) value;
 }
+
+
+const njs_object_init_t  njs_to_string_function_init = {
+    nxt_string("toString"),
+    NULL,
+    0,
+};
+
+
+const njs_object_init_t  njs_encode_uri_function_init = {
+    nxt_string("encodeURI"),
+    NULL,
+    0,
+};
+
+
+const njs_object_init_t  njs_encode_uri_component_function_init = {
+    nxt_string("encodeURIComponent"),
+    NULL,
+    0,
+};
+
+
+const njs_object_init_t  njs_decode_uri_function_init = {
+    nxt_string("decodeURI"),
+    NULL,
+    0,
+};
+
+
+const njs_object_init_t  njs_decode_uri_component_function_init = {
+    nxt_string("decodeURIComponent"),
+    NULL,
+    0,
+};
diff -r 18e0275576a3 -r 757271547b56 njs/njs_string.h
--- a/njs/njs_string.h	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/njs_string.h	Tue Feb 20 19:12:55 2018 +0300
@@ -158,5 +158,11 @@ njs_index_t njs_value_index(njs_vm_t *vm
 extern const njs_object_init_t  njs_string_constructor_init;
 extern const njs_object_init_t  njs_string_prototype_init;
 
+extern const njs_object_init_t  njs_to_string_function_init;
+extern const njs_object_init_t  njs_encode_uri_function_init;
+extern const njs_object_init_t  njs_encode_uri_component_function_init;
+extern const njs_object_init_t  njs_decode_uri_function_init;
+extern const njs_object_init_t  njs_decode_uri_component_function_init;
+
 
 #endif /* _NJS_STRING_H_INCLUDED_ */
diff -r 18e0275576a3 -r 757271547b56 njs/njs_vm.c
--- a/njs/njs_vm.c	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/njs_vm.c	Tue Feb 20 19:12:55 2018 +0300
@@ -3744,10 +3744,18 @@ njs_vm_add_backtrace_entry(njs_vm_t *vm,
 
     if (function->native) {
         ret = njs_builtin_match_native_function(vm, function, &be->name);
-        if (ret != NXT_OK) {
-            be->name = entry_native;
+        if (ret == NXT_OK) {
+            return NXT_OK;
         }
 
+        ret = njs_external_match_native_function(vm, function->u.native,
+                                                 &be->name);
+        if (ret == NXT_OK) {
+            return NXT_OK;
+        }
+
+        be->name = entry_native;
+
         return NXT_OK;
     }
 
diff -r 18e0275576a3 -r 757271547b56 njs/test/njs_expect_test.exp
--- a/njs/test/njs_expect_test.exp	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/test/njs_expect_test.exp	Tue Feb 20 19:12:55 2018 +0300
@@ -171,6 +171,12 @@ njs_test {
      "console.ll()\r\nTypeError: cannot find property 'll' of an external object"}
 }
 
+# Backtraces for external objects
+njs_test {
+    {"console.log(console)\r\n"
+     "console.log(console)\r\nTypeError:*at console.log (native)"}
+}
+
 # Exception in njs_vm_retval_to_ext_string()
 njs_test {
     {"var o = { toString: function() { return [1] } }\r\n"
diff -r 18e0275576a3 -r 757271547b56 njs/test/njs_interactive_test.c
--- a/njs/test/njs_interactive_test.c	Tue Feb 20 19:12:53 2018 +0300
+++ b/njs/test/njs_interactive_test.c	Tue Feb 20 19:12:55 2018 +0300
@@ -162,6 +162,16 @@ static njs_interactive_test_t  njs_test[
                  "    at Math.log (native)\n"
                  "    at main (native)\n") },
 
+    { nxt_string("eval()" ENTER),
+      nxt_string("InternalError: Not implemented\n"
+                 "    at eval (native)\n"
+                 "    at main (native)\n") },
+
+    { nxt_string("require()" ENTER),
+      nxt_string("TypeError: missing path\n"
+                 "    at require (native)\n"
+                 "    at main (native)\n") },
+
     { nxt_string("function f(o) {function f_in(o) {return o.a.a};"
                  "               return f_in(o)}; f({})" ENTER),
       nxt_string("TypeError: cannot get property 'a' of undefined\n"

From xeioex at nginx.com  Tue Feb 20 16:19:28 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 20 Feb 2018 16:19:28 +0000
Subject: [njs] Moving long_string fields of njs_value_t into a separate struct.
Message-ID: <hg.0a3645d22d22.1519143568.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/0a3645d22d22
branches:  
changeset: 447:0a3645d22d22
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Feb 20 19:12:52 2018 +0300
description:
Moving long_string fields of njs_value_t into a separate struct.

diffstat:

 njs/njs_object.c |   4 +-
 njs/njs_string.c |  83 ++++++++++++++++++++++++++++---------------------------
 njs/njs_vm.c     |  34 ++++++++++++----------
 njs/njs_vm.h     |  63 +++++++++++++++++++++++------------------
 4 files changed, 97 insertions(+), 87 deletions(-)

diffs (447 lines):

diff -r e12ed0068e57 -r 0a3645d22d22 njs/njs_object.c
--- a/njs/njs_object.c	Mon Feb 12 15:15:53 2018 +0300
+++ b/njs/njs_object.c	Tue Feb 20 19:12:52 2018 +0300
@@ -159,11 +159,11 @@ njs_object_hash_test(nxt_lvlhsh_query_t 
         start = prop->name.short_string.start;
 
     } else {
-        if (lhq->key.length != prop->name.data.string_size) {
+        if (lhq->key.length != prop->name.long_string.size) {
             return NXT_DECLINED;
         }
 
-        start = prop->name.data.u.string->start;
+        start = prop->name.long_string.data->start;
     }
 
     if (memcmp(start, lhq->key.start, lhq->key.length) == 0) {
diff -r e12ed0068e57 -r 0a3645d22d22 njs/njs_string.c
--- a/njs/njs_string.c	Mon Feb 12 15:15:53 2018 +0300
+++ b/njs/njs_string.c	Tue Feb 20 19:12:52 2018 +0300
@@ -151,15 +151,15 @@ njs_string_create(njs_vm_t *vm, njs_valu
          */
         value->short_string.size = NJS_STRING_LONG;
         value->short_string.length = 0;
-        value->data.external0 = 0xff;
-        value->data.string_size = size;
+        value->long_string.external = 0xff;
+        value->long_string.size = size;
 
         string = nxt_mem_cache_alloc(vm->mem_cache_pool, sizeof(njs_string_t));
         if (nxt_slow_path(string == NULL)) {
             return NXT_ERROR;
         }
 
-        value->data.u.string = string;
+        value->long_string.data = string;
 
         string->start = start;
         string->length = length;
@@ -210,8 +210,8 @@ njs_string_alloc(njs_vm_t *vm, njs_value
      */
     value->short_string.size = NJS_STRING_LONG;
     value->short_string.length = 0;
-    value->data.external0 = 0;
-    value->data.string_size = size;
+    value->long_string.external = 0;
+    value->long_string.size = size;
 
     if (size != length && length > NJS_STRING_MAP_STRIDE) {
         map_offset = njs_string_map_offset(size);
@@ -226,7 +226,7 @@ njs_string_alloc(njs_vm_t *vm, njs_value
                                  sizeof(njs_string_t) + total);
 
     if (nxt_fast_path(string != NULL)) {
-        value->data.u.string = string;
+        value->long_string.data = string;
 
         string->start = (u_char *) string + sizeof(njs_string_t);
         string->length = length;
@@ -284,9 +284,9 @@ njs_string_validate(njs_vm_t *vm, njs_st
         }
 
     } else {
-        string->start = value->data.u.string->start;
-        size = value->data.string_size;
-        length = value->data.u.string->length;
+        string->start = value->long_string.data->start;
+        size = value->long_string.size;
+        length = value->long_string.data->length;
 
         if (length == 0 && length != size) {
             length = nxt_utf8_length(string->start, size);
@@ -312,14 +312,14 @@ njs_string_validate(njs_vm_t *vm, njs_st
 
                     memcpy(start, string->start, size);
                     string->start = start;
-                    value->data.u.string->start = start;
+                    value->long_string.data->start = start;
 
                     map = (uint32_t *) (start + map_offset);
                     map[0] = 0;
                 }
             }
 
-            value->data.u.string->length = length;
+            value->long_string.data->length = length;
         }
     }
 
@@ -343,9 +343,9 @@ njs_string_prop(njs_string_prop_t *strin
         length = value->short_string.length;
 
     } else {
-        string->start = value->data.u.string->start;
-        size = value->data.string_size;
-        length = value->data.u.string->length;
+        string->start = value->long_string.data->start;
+        size = value->long_string.size;
+        length = value->long_string.data->length;
     }
 
     string->size = size;
@@ -454,8 +454,8 @@ njs_string_prototype_length(njs_vm_t *vm
         length = value->short_string.length;
 
         if (size == NJS_STRING_LONG) {
-            size = value->data.string_size;
-            length = value->data.u.string->length;
+            size = value->long_string.size;
+            length = value->long_string.data->length;
         }
 
         length = (length == 0) ? size : length;
@@ -486,14 +486,14 @@ njs_string_eq(const njs_value_t *v1, con
         start2 = v2->short_string.start;
 
     } else {
-        size = v1->data.string_size;
-
-        if (size != v2->data.string_size) {
+        size = v1->long_string.size;
+
+        if (size != v2->long_string.size) {
             return 0;
         }
 
-        start1 = v1->data.u.string->start;
-        start2 = v2->data.u.string->start;
+        start1 = v1->long_string.data->start;
+        start2 = v2->long_string.data->start;
     }
 
     return (memcmp(start1, start2, size) == 0);
@@ -513,8 +513,8 @@ njs_string_cmp(const njs_value_t *v1, co
         start1 = v1->short_string.start;
 
     } else {
-        size1 = v1->data.string_size;
-        start1 = v1->data.u.string->start;
+        size1 = v1->long_string.size;
+        start1 = v1->long_string.data->start;
     }
 
     size2 = v2->short_string.size;
@@ -523,8 +523,8 @@ njs_string_cmp(const njs_value_t *v1, co
         start2 = v2->short_string.start;
 
     } else {
-        size2 = v2->data.string_size;
-        start2 = v2->data.u.string->start;
+        size2 = v2->long_string.size;
+        start2 = v2->long_string.data->start;
     }
 
     size = nxt_min(size1, size2);
@@ -2903,8 +2903,8 @@ njs_string_replacement_copy(njs_string_r
         string->start = NULL;
 
     } else {
-        string->start = value->data.u.string->start;
-        size = value->data.string_size;
+        string->start = value->long_string.data->start;
+        size = value->long_string.size;
     }
 
     string->size = size;
@@ -2965,8 +2965,8 @@ njs_string_to_number(njs_value_t *value,
         p = value->short_string.start;
 
     } else {
-        size = value->data.string_size;
-        p = value->data.u.string->start;
+        size = value->long_string.size;
+        p = value->long_string.data->start;
     }
 
     end = p + size;
@@ -3044,8 +3044,8 @@ njs_string_to_index(njs_value_t *value)
         p = value->short_string.start;
 
     } else {
-        size = value->data.string_size;
-        p = value->data.u.string->start;
+        size = value->long_string.size;
+        p = value->long_string.data->start;
     }
 
     if (size == 0) {
@@ -3091,8 +3091,8 @@ njs_string_to_c_string(njs_vm_t *vm, njs
         }
 
     } else {
-        start = value->data.u.string->start;
-        size = value->data.string_size;
+        start = value->long_string.data->start;
+        size = value->long_string.size;
 
         if (start[size] == '\0') {
             return start;
@@ -3643,7 +3643,7 @@ njs_string_decode(njs_vm_t *vm, njs_valu
             vm->retval.short_string.length = length;
 
         } else {
-            vm->retval.data.u.string->length = length;
+            vm->retval.long_string.data->length = length;
         }
     }
 
@@ -3671,8 +3671,9 @@ njs_values_hash_test(nxt_lvlhsh_query_t 
     }
 
     if (njs_is_string(value)
-        && value->data.string_size == lhq->key.length
-        && memcmp(value->data.u.string->start, lhq->key.start, lhq->key.length)
+        && value->long_string.size == lhq->key.length
+        && memcmp(value->long_string.data->start, lhq->key.start,
+                  lhq->key.length)
            == 0)
     {
         return NXT_OK;
@@ -3714,8 +3715,8 @@ njs_value_index(njs_vm_t *vm, njs_parser
         start = (u_char *) src;
 
     } else {
-        size = src->data.string_size;
-        start = src->data.u.string->start;
+        size = src->long_string.size;
+        start = src->long_string.data->start;
     }
 
     lhq.key_hash = nxt_djb_hash(start, size);
@@ -3738,7 +3739,7 @@ njs_value_index(njs_vm_t *vm, njs_parser
             /* Long string value is allocated together with string. */
             value_size = sizeof(njs_value_t) + sizeof(njs_string_t);
 
-            length = src->data.u.string->length;
+            length = src->long_string.data->length;
 
             if (size != length && length > NJS_STRING_MAP_STRIDE) {
                 size = njs_string_map_offset(size)
@@ -3756,10 +3757,10 @@ njs_value_index(njs_vm_t *vm, njs_parser
 
         if (start != (u_char *) src) {
             string = (njs_string_t *) ((u_char *) value + sizeof(njs_value_t));
-            value->data.u.string = string;
+            value->long_string.data = string;
 
             string->start = (u_char *) string + sizeof(njs_string_t);
-            string->length = src->data.u.string->length;
+            string->length = src->long_string.data->length;
             string->retain = 0xffff;
 
             memcpy(string->start, start, size);
diff -r e12ed0068e57 -r 0a3645d22d22 njs/njs_vm.c
--- a/njs/njs_vm.c	Mon Feb 12 15:15:53 2018 +0300
+++ b/njs/njs_vm.c	Tue Feb 20 19:12:52 2018 +0300
@@ -303,11 +303,11 @@ njs_value_retain(njs_value_t *value)
 
     if (njs_is_string(value)) {
 
-        if (value->data.external0 != 0xff) {
-            string = value->data.u.string;
+        if (value->long_string.external != 0xff) {
+            string = value->long_string.data;
 
             nxt_thread_log_debug("retain:%uxD \"%*s\"", string->retain,
-                                 value->data.string_size, string->start);
+                                 value->long_string.size, string->start);
 
             if (string->retain != 0xffff) {
                 string->retain++;
@@ -324,11 +324,11 @@ njs_value_release(njs_vm_t *vm, njs_valu
 
     if (njs_is_string(value)) {
 
-        if (value->data.external0 != 0xff) {
-            string = value->data.u.string;
+        if (value->long_string.external != 0xff) {
+            string = value->long_string.data;
 
             nxt_thread_log_debug("release:%uxD \"%*s\"", string->retain,
-                                 value->data.string_size, string->start);
+                                 value->long_string.size, string->start);
 
             if (string->retain != 0xffff) {
                 string->retain--;
@@ -2132,18 +2132,20 @@ njs_values_strict_equal(const njs_value_
             start2 = val2->short_string.start;
 
         } else {
-            size = val1->data.string_size;
-
-            if (size != val2->data.string_size) {
+            size = val1->long_string.size;
+
+            if (size != val2->long_string.size) {
                 return 0;
             }
 
-            if (val1->data.u.string->length != val2->data.u.string->length) {
+            if (val1->long_string.data->length
+                != val2->long_string.data->length)
+            {
                 return 0;
             }
 
-            start1 = val1->data.u.string->start;
-            start2 = val2->data.u.string->start;
+            start1 = val1->long_string.data->start;
+            start2 = val2->long_string.data->start;
         }
 
         return (memcmp(start1, start2, size) == 0);
@@ -3466,8 +3468,8 @@ again:
                 memcpy(start, value.short_string.start, size);
 
             } else {
-                size = value.data.string_size;
-                start = value.data.u.string->start;
+                size = value.long_string.size;
+                start = value.long_string.data->start;
             }
 
             dst->length = size;
@@ -3784,8 +3786,8 @@ njs_debug(njs_index_t index, njs_value_t
             p = value->short_string.start;
 
         } else {
-            length = value->data.string_size;
-            p = value->data.u.string->start;
+            length = value->long_string.size;
+            p = value->long_string.data->start;
         }
 
         nxt_thread_log_debug("%p [\"%*s\"]", index, length, p);
diff -r e12ed0068e57 -r 0a3645d22d22 njs/njs_vm.h
--- a/njs/njs_vm.h	Mon Feb 12 15:15:53 2018 +0300
+++ b/njs/njs_vm.h	Tue Feb 20 19:12:52 2018 +0300
@@ -143,8 +143,8 @@ union njs_value_s {
      * to provide 4 bits to encode scope in njs_index_t.  This space is
      * used to store short strings.  The maximum size of a short string
      * is 14 (NJS_STRING_SHORT).  If the short_string.size field is 15
-     * (NJS_STRING_LONG) then the size is in the data.string_size field
-     * and the data.u.string field points to a long string.
+     * (NJS_STRING_LONG) then the size is in the long_string.size field
+     * and the long_string.data field points to a long string.
      *
      * The number of the string types is limited to 2 types to minimize
      * overhead of processing string fields.  It is also possible to add
@@ -163,28 +163,23 @@ union njs_value_s {
          */
         uint8_t                       truth;
 
-        /* 0xff if u.data.string is external string. */
-        uint8_t                       external0;
-        uint8_t                       _spare;
-
-        /* A long string size. */
-        uint32_t                      string_size;
+        uint16_t                      _spare1;
+        uint32_t                      _spare2;
 
         union {
-            double                     number;
-            njs_string_t               *string;
-            njs_object_t               *object;
-            njs_array_t                *array;
-            njs_object_value_t         *object_value;
-            njs_function_t             *function;
-            njs_function_lambda_t      *lambda;
-            njs_regexp_t               *regexp;
-            njs_date_t                 *date;
-            njs_getter_t               getter;
-            njs_extern_t               *external;
-            njs_value_t                *value;
-            njs_property_next_t        *next;
-            void                       *data;
+            double                    number;
+            njs_object_t              *object;
+            njs_array_t               *array;
+            njs_object_value_t        *object_value;
+            njs_function_t            *function;
+            njs_function_lambda_t     *lambda;
+            njs_regexp_t              *regexp;
+            njs_date_t                *date;
+            njs_getter_t              getter;
+            njs_extern_t              *external;
+            njs_value_t               *value;
+            njs_property_next_t       *next;
+            void                      *data;
         } u;
     } data;
 
@@ -200,6 +195,18 @@ union njs_value_s {
         u_char                        start[NJS_STRING_SHORT];
     } short_string;
 
+    struct {
+        njs_value_type_t              type:8;  /* 5 bits */
+        uint8_t                       truth;
+
+        /* 0xff if data is external string. */
+        uint8_t                       external;
+        uint8_t                       _spare;
+
+        uint32_t                      size;
+        njs_string_t                  *data;
+    } long_string;
+
     njs_value_type_t                  type:8;  /* 5 bits */
 };
 
@@ -326,11 +333,11 @@ typedef union {
 /* NJS_STRING_LONG is set for both big and little endian platforms. */
 
 #define njs_long_string(s) {                                                  \
-    .data = {                                                                 \
+    .long_string = {                                                          \
         .type = NJS_STRING,                                                   \
         .truth = (NJS_STRING_LONG << 4) | NJS_STRING_LONG,                    \
-        .string_size = sizeof(s) - 1,                                         \
-        .u.string = & (njs_string_t) {                                        \
+        .size = sizeof(s) - 1,                                                \
+        .data = & (njs_string_t) {                                            \
             .start = (u_char *) s,                                            \
             .length = sizeof(s) - 1,                                          \
         }                                                                     \
@@ -430,8 +437,8 @@ typedef njs_ret_t (*njs_vmcode_operation
             (str)->start = (u_char *) (value)->short_string.start;            \
                                                                               \
         } else {                                                              \
-            (str)->length = (value)->data.string_size;                        \
-            (str)->start = (u_char *) (value)->data.u.string->start;          \
+            (str)->length = (value)->long_string.size;                        \
+            (str)->start = (u_char *) (value)->long_string.data->start;       \
         }                                                                     \
     } while (0)
 
@@ -455,7 +462,7 @@ typedef njs_ret_t (*njs_vmcode_operation
             (value)->short_string.length = length;                            \
                                                                               \
         } else {                                                              \
-            (value)->data.u.string->length = length;                          \
+            (value)->long_string.data->length = length;                       \
         }                                                                     \
     } while (0)
 

From xeioex at nginx.com  Tue Feb 20 16:19:28 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 20 Feb 2018 16:19:28 +0000
Subject: [njs] Externals refactored.
Message-ID: <hg.e7bc9d328a20.1519143568.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/e7bc9d328a20
branches:  
changeset: 448:e7bc9d328a20
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Feb 20 19:12:53 2018 +0300
description:
Externals refactored.

Public API is rectified to allow the creation of external objects in
runtime.
    1) njs_vm_external_add() is replaced with njs_vm_external_prototype().
    The later functions returns a pointer to a prototype object which can
    be used to create a value with such a prototype in runtime.

    2) njs_vm_external() is split into njs_vm_external_create() and
    njs_vm_external_bind(). The former creates a variable with a specified
    prototype and associates it with an external pointer. The latter binds
    a variable to a name in the global namespace.

diffstat:

 nginx/ngx_http_js_module.c      |  212 ++++++---------------
 nginx/ngx_stream_js_module.c    |  176 ++++-------------
 njs/njs.c                       |   75 +++----
 njs/njs_builtin.c               |   26 +-
 njs/njs_extern.c                |  209 +++++++++++++-------
 njs/njs_extern.h                |   14 +-
 njs/njs_parser.c                |    6 +-
 njs/njs_parser.h                |    3 +-
 njs/njs_vm.c                    |  168 ++++++++++-------
 njs/njs_vm.h                    |   19 +-
 njs/njscript.c                  |   71 ++++--
 njs/njscript.h                  |   22 +-
 njs/test/njs_benchmark.c        |  114 ++++-------
 njs/test/njs_interactive_test.c |   27 +-
 njs/test/njs_unit_test.c        |  381 +++++++++++++++++++++++++++------------
 nxt/nxt_array.h                 |    4 +
 nxt/nxt_lvlhsh.c                |    4 +-
 nxt/nxt_lvlhsh.h                |    6 +-
 18 files changed, 800 insertions(+), 737 deletions(-)

diffs (truncated from 2773 to 1000 lines):

diff -r 0a3645d22d22 -r e7bc9d328a20 nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c	Tue Feb 20 19:12:52 2018 +0300
+++ b/nginx/ngx_http_js_module.c	Tue Feb 20 19:12:53 2018 +0300
@@ -15,37 +15,21 @@
 #include <nxt_string.h>
 #include <nxt_stub.h>
 #include <nxt_array.h>
-#include <nxt_random.h>
-#include <nxt_lvlhsh.h>
-#include <nxt_mem_cache_pool.h>
 
 #include <njscript.h>
 
 
-#define NGX_HTTP_JS_MCP_CLUSTER_SIZE    (2 * ngx_pagesize)
-#define NGX_HTTP_JS_MCP_PAGE_ALIGNMENT  128
-#define NGX_HTTP_JS_MCP_PAGE_SIZE       512
-#define NGX_HTTP_JS_MCP_MIN_CHUNK_SIZE  16
-
-
-#define ngx_http_js_create_mem_cache_pool()                                   \
-    nxt_mem_cache_pool_create(&ngx_http_js_mem_cache_pool_proto, NULL, NULL,  \
-                              NGX_HTTP_JS_MCP_CLUSTER_SIZE,                   \
-                              NGX_HTTP_JS_MCP_PAGE_ALIGNMENT,                 \
-                              NGX_HTTP_JS_MCP_PAGE_SIZE,                      \
-                              NGX_HTTP_JS_MCP_MIN_CHUNK_SIZE)
+typedef struct {
+    njs_vm_t            *vm;
+    ngx_str_t            content;
+    const njs_extern_t  *req_proto;
+    const njs_extern_t  *res_proto;
+} ngx_http_js_loc_conf_t;
 
 
 typedef struct {
     njs_vm_t            *vm;
     njs_opaque_value_t   args[2];
-    ngx_str_t            content;
-} ngx_http_js_loc_conf_t;
-
-
-typedef struct {
-    njs_vm_t            *vm;
-    njs_opaque_value_t  *args;
 } ngx_http_js_ctx_t;
 
 
@@ -59,12 +43,7 @@ static ngx_int_t ngx_http_js_handler(ngx
 static ngx_int_t ngx_http_js_variable(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_http_js_init_vm(ngx_http_request_t *r);
-static void ngx_http_js_cleanup_mem_cache_pool(void *data);
-
-static void *ngx_http_js_alloc(void *mem, size_t size);
-static void *ngx_http_js_calloc(void *mem, size_t size);
-static void *ngx_http_js_memalign(void *mem, size_t alignment, size_t size);
-static void ngx_http_js_free(void *mem, void *p);
+static void ngx_http_js_cleanup_vm(void *data);
 
 static njs_ret_t ngx_http_js_ext_get_string(njs_vm_t *vm, njs_value_t *value,
     void *obj, uintptr_t data);
@@ -183,17 +162,6 @@ ngx_module_t  ngx_http_js_module = {
 };
 
 
-static const nxt_mem_proto_t  ngx_http_js_mem_cache_pool_proto = {
-    ngx_http_js_alloc,
-    ngx_http_js_calloc,
-    ngx_http_js_memalign,
-    NULL,
-    ngx_http_js_free,
-    NULL,
-    NULL,
-};
-
-
 static njs_external_t  ngx_http_js_ext_response[] = {
 
     { nxt_string("headers"),
@@ -284,18 +252,6 @@ static njs_external_t  ngx_http_js_ext_r
 
 static njs_external_t  ngx_http_js_ext_request[] = {
 
-    { nxt_string("response"),
-      NJS_EXTERN_OBJECT,
-      ngx_http_js_ext_response,
-      nxt_nitems(ngx_http_js_ext_response),
-      NULL,
-      NULL,
-      NULL,
-      NULL,
-      NULL,
-      NULL,
-      0 },
-
     { nxt_string("log"),
       NJS_EXTERN_METHOD,
       NULL,
@@ -396,7 +352,7 @@ static njs_external_t  ngx_http_js_ext_r
 
 static njs_external_t  ngx_http_js_externals[] = {
 
-    { nxt_string("$r"),
+    { nxt_string("request"),
       NJS_EXTERN_OBJECT,
       ngx_http_js_ext_request,
       nxt_nitems(ngx_http_js_ext_request),
@@ -407,6 +363,18 @@ static njs_external_t  ngx_http_js_exter
       NULL,
       NULL,
       0 },
+
+    { nxt_string("response"),
+      NJS_EXTERN_OBJECT,
+      ngx_http_js_ext_response,
+      nxt_nitems(ngx_http_js_ext_response),
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      0 },
 };
 
 
@@ -520,11 +488,10 @@ ngx_http_js_variable(ngx_http_request_t 
 static ngx_int_t
 ngx_http_js_init_vm(ngx_http_request_t *r)
 {
-    void                    **ext;
-    ngx_http_js_ctx_t        *ctx;
-    ngx_pool_cleanup_t       *cln;
-    nxt_mem_cache_pool_t     *mcp;
-    ngx_http_js_loc_conf_t   *jlcf;
+    nxt_int_t                rc;
+    ngx_http_js_ctx_t       *ctx;
+    ngx_pool_cleanup_t      *cln;
+    ngx_http_js_loc_conf_t  *jlcf;
 
     jlcf = ngx_http_get_module_loc_conf(r, ngx_http_js_module);
     if (jlcf->vm == NULL) {
@@ -546,8 +513,8 @@ ngx_http_js_init_vm(ngx_http_request_t *
         return NGX_OK;
     }
 
-    mcp = ngx_http_js_create_mem_cache_pool();
-    if (mcp == NULL) {
+    ctx->vm = njs_vm_clone(jlcf->vm, r);
+    if (ctx->vm == NULL) {
         return NGX_ERROR;
     }
 
@@ -556,65 +523,33 @@ ngx_http_js_init_vm(ngx_http_request_t *
         return NGX_ERROR;
     }
 
-    cln->handler = ngx_http_js_cleanup_mem_cache_pool;
-    cln->data = mcp;
-
-    ext = ngx_palloc(r->pool, sizeof(void *));
-    if (ext == NULL) {
-        return NGX_ERROR;
-    }
-
-    *ext = r;
-
-    ctx->vm = njs_vm_clone(jlcf->vm, mcp, ext);
-    if (ctx->vm == NULL) {
-        return NGX_ERROR;
-    }
+    cln->handler = ngx_http_js_cleanup_vm;
+    cln->data = ctx->vm;
 
     if (njs_vm_run(ctx->vm) != NJS_OK) {
         return NGX_ERROR;
     }
 
-    ctx->args = &jlcf->args[0];
+    rc = njs_vm_external_create(ctx->vm, &ctx->args[0], jlcf->req_proto, r);
+    if (rc != NXT_OK) {
+        return NGX_ERROR;
+    }
+
+    rc = njs_vm_external_create(ctx->vm, &ctx->args[1], jlcf->res_proto, r);
+    if (rc != NXT_OK) {
+        return NGX_ERROR;
+    }
 
     return NGX_OK;
 }
 
 
 static void
-ngx_http_js_cleanup_mem_cache_pool(void *data)
+ngx_http_js_cleanup_vm(void *data)
 {
-    nxt_mem_cache_pool_t *mcp = data;
-
-    nxt_mem_cache_pool_destroy(mcp);
-}
-
-
-static void *
-ngx_http_js_alloc(void *mem, size_t size)
-{
-    return ngx_alloc(size, ngx_cycle->log);
-}
-
+    njs_vm_t *vm = data;
 
-static void *
-ngx_http_js_calloc(void *mem, size_t size)
-{
-    return ngx_calloc(size, ngx_cycle->log);
-}
-
-
-static void *
-ngx_http_js_memalign(void *mem, size_t alignment, size_t size)
-{
-    return ngx_memalign(alignment, size, ngx_cycle->log);
-}
-
-
-static void
-ngx_http_js_free(void *mem, void *p)
-{
-    ngx_free(p);
+    njs_vm_destroy(vm);
 }
 
 
@@ -1229,12 +1164,10 @@ ngx_http_js_include(ngx_conf_t *cf, ngx_
     ngx_fd_t               fd;
     ngx_str_t             *value, file;
     nxt_int_t              rc;
-    nxt_str_t              text, ext;
+    nxt_str_t              text;
     njs_vm_opt_t           options;
-    nxt_lvlhsh_t           externals;
     ngx_file_info_t        fi;
     ngx_pool_cleanup_t    *cln;
-    nxt_mem_cache_pool_t  *mcp;
 
     if (jlcf->vm) {
         return "is duplicate";
@@ -1295,8 +1228,13 @@ ngx_http_js_include(ngx_conf_t *cf, ngx_
 
     end = start + size;
 
-    mcp = ngx_http_js_create_mem_cache_pool();
-    if (mcp == NULL) {
+    ngx_memzero(&options, sizeof(njs_vm_opt_t));
+
+    options.backtrace = 1;
+
+    jlcf->vm = njs_vm_create(&options);
+    if (jlcf->vm == NULL) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to create JS VM");
         return NGX_CONF_ERROR;
     }
 
@@ -1305,28 +1243,21 @@ ngx_http_js_include(ngx_conf_t *cf, ngx_
         return NGX_CONF_ERROR;
     }
 
-    cln->handler = ngx_http_js_cleanup_mem_cache_pool;
-    cln->data = mcp;
-
-    nxt_lvlhsh_init(&externals);
+    cln->handler = ngx_http_js_cleanup_vm;
+    cln->data = jlcf->vm;
 
-    if (njs_vm_external_add(&externals, mcp, 0, ngx_http_js_externals,
-                            nxt_nitems(ngx_http_js_externals))
-        != NJS_OK)
-    {
-        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "could not add js externals");
+    jlcf->req_proto = njs_vm_external_prototype(jlcf->vm,
+                                                &ngx_http_js_externals[0]);
+    if (jlcf->req_proto == NULL) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to add request proto");
         return NGX_CONF_ERROR;
     }
 
-    ngx_memzero(&options, sizeof(njs_vm_opt_t));
-
-    options.mcp = mcp;
-    options.backtrace = 1;
-    options.externals_hash = &externals;
-
-    jlcf->vm = njs_vm_create(&options);
-    if (jlcf->vm == NULL) {
-        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to create JS VM");
+    jlcf->res_proto = njs_vm_external_prototype(jlcf->vm,
+                                                &ngx_http_js_externals[1]);
+    if (jlcf->res_proto == NULL) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "failed to add response proto");
         return NGX_CONF_ERROR;
     }
 
@@ -1348,25 +1279,6 @@ ngx_http_js_include(ngx_conf_t *cf, ngx_
         return NGX_CONF_ERROR;
     }
 
-    ext = nxt_string_value("$r");
-
-    if (njs_vm_external(jlcf->vm, NULL, &ext, &jlcf->args[0]) != NJS_OK) {
-        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                           "js external \"%*s\" not found",
-                           ext.length, ext.start);
-        return NGX_CONF_ERROR;
-    }
-
-    ext = nxt_string_value("response");
-
-    rc = njs_vm_external(jlcf->vm, &jlcf->args[0], &ext, &jlcf->args[1]);
-    if (rc != NXT_OK) {
-        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                           "js external \"$r.%*s\" not found",
-                           ext.length, ext.start);
-        return NGX_CONF_ERROR;
-    }
-
     return NGX_CONF_OK;
 }
 
@@ -1443,6 +1355,8 @@ ngx_http_js_create_loc_conf(ngx_conf_t *
      * set by ngx_pcalloc():
      *
      *     conf->vm = NULL;
+     *     conf->req_proto = NULL;
+     *     conf->res_proto = NULL;
      */
 
     return conf;
@@ -1457,8 +1371,8 @@ ngx_http_js_merge_loc_conf(ngx_conf_t *c
 
     if (conf->vm == NULL) {
         conf->vm = prev->vm;
-        conf->args[0] = prev->args[0];
-        conf->args[1] = prev->args[1];
+        conf->req_proto = prev->req_proto;
+        conf->res_proto = prev->res_proto;
     }
 
     return NGX_CONF_OK;
diff -r 0a3645d22d22 -r e7bc9d328a20 nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c	Tue Feb 20 19:12:52 2018 +0300
+++ b/nginx/ngx_stream_js_module.c	Tue Feb 20 19:12:53 2018 +0300
@@ -15,39 +15,22 @@
 #include <nxt_string.h>
 #include <nxt_stub.h>
 #include <nxt_array.h>
-#include <nxt_random.h>
-#include <nxt_lvlhsh.h>
-#include <nxt_mem_cache_pool.h>
 
 #include <njscript.h>
 
 
-#define NGX_STREAM_JS_MCP_CLUSTER_SIZE    (2 * ngx_pagesize)
-#define NGX_STREAM_JS_MCP_PAGE_ALIGNMENT  128
-#define NGX_STREAM_JS_MCP_PAGE_SIZE       512
-#define NGX_STREAM_JS_MCP_MIN_CHUNK_SIZE  16
-
-
-#define ngx_stream_js_create_mem_cache_pool()                                 \
-    nxt_mem_cache_pool_create(&ngx_stream_js_mem_cache_pool_proto, NULL, NULL,\
-                              NGX_STREAM_JS_MCP_CLUSTER_SIZE,                 \
-                              NGX_STREAM_JS_MCP_PAGE_ALIGNMENT,               \
-                              NGX_STREAM_JS_MCP_PAGE_SIZE,                    \
-                              NGX_STREAM_JS_MCP_MIN_CHUNK_SIZE)
+typedef struct {
+    njs_vm_t              *vm;
+    ngx_str_t              access;
+    ngx_str_t              preread;
+    ngx_str_t              filter;
+    const njs_extern_t    *proto;
+} ngx_stream_js_srv_conf_t;
 
 
 typedef struct {
     njs_vm_t              *vm;
     njs_opaque_value_t     arg;
-    ngx_str_t              access;
-    ngx_str_t              preread;
-    ngx_str_t              filter;
-} ngx_stream_js_srv_conf_t;
-
-
-typedef struct {
-    njs_vm_t              *vm;
-    njs_opaque_value_t    *arg;
     ngx_buf_t             *buf;
     ngx_chain_t           *free;
     ngx_chain_t           *busy;
@@ -65,13 +48,8 @@ static ngx_int_t ngx_stream_js_body_filt
     ngx_chain_t *in, ngx_uint_t from_upstream);
 static ngx_int_t ngx_stream_js_variable(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
-static void ngx_stream_js_cleanup_mem_cache_pool(void *data);
 static ngx_int_t ngx_stream_js_init_vm(ngx_stream_session_t *s);
-
-static void *ngx_stream_js_alloc(void *mem, size_t size);
-static void *ngx_stream_js_calloc(void *mem, size_t size);
-static void *ngx_stream_js_memalign(void *mem, size_t alignment, size_t size);
-static void ngx_stream_js_free(void *mem, void *p);
+static void ngx_stream_js_cleanup_vm(void *data);
 
 static njs_ret_t ngx_stream_js_ext_get_remote_address(njs_vm_t *vm,
     njs_value_t *value, void *obj, uintptr_t data);
@@ -169,17 +147,6 @@ ngx_module_t  ngx_stream_js_module = {
 };
 
 
-static const nxt_mem_proto_t  ngx_stream_js_mem_cache_pool_proto = {
-    ngx_stream_js_alloc,
-    ngx_stream_js_calloc,
-    ngx_stream_js_memalign,
-    NULL,
-    ngx_stream_js_free,
-    NULL,
-    NULL,
-};
-
-
 static njs_external_t  ngx_stream_js_ext_session[] = {
 
     { nxt_string("remoteAddress"),
@@ -318,7 +285,7 @@ static njs_external_t  ngx_stream_js_ext
 
 static njs_external_t  ngx_stream_js_externals[] = {
 
-    { nxt_string("$s"),
+    { nxt_string("stream"),
       NJS_EXTERN_OBJECT,
       ngx_stream_js_ext_session,
       nxt_nitems(ngx_stream_js_ext_session),
@@ -405,7 +372,7 @@ ngx_stream_js_phase_handler(ngx_stream_s
         return NGX_ERROR;
     }
 
-    if (njs_vm_call(ctx->vm, func, ctx->arg, 1) != NJS_OK) {
+    if (njs_vm_call(ctx->vm, func, &ctx->arg, 1) != NJS_OK) {
         njs_vm_retval_to_ext_string(ctx->vm, &exception);
 
         ngx_log_error(NGX_LOG_ERR, c->log, 0, "js exception: %*s",
@@ -492,7 +459,7 @@ ngx_stream_js_body_filter(ngx_stream_ses
     while (in) {
         ctx->buf = in->buf;
 
-        if (njs_vm_call(ctx->vm, func, ctx->arg, 1) != NJS_OK) {
+        if (njs_vm_call(ctx->vm, func, &ctx->arg, 1) != NJS_OK) {
             njs_vm_retval_to_ext_string(ctx->vm, &exception);
 
             ngx_log_error(NGX_LOG_ERR, c->log, 0, "js exception: %*s",
@@ -590,7 +557,7 @@ ngx_stream_js_variable(ngx_stream_sessio
         return NGX_OK;
     }
 
-    if (njs_vm_call(ctx->vm, func, ctx->arg, 1) != NJS_OK) {
+    if (njs_vm_call(ctx->vm, func, &ctx->arg, 1) != NJS_OK) {
         njs_vm_retval_to_ext_string(ctx->vm, &exception);
 
         ngx_log_error(NGX_LOG_ERR, s->connection->log, 0,
@@ -617,11 +584,10 @@ ngx_stream_js_variable(ngx_stream_sessio
 static ngx_int_t
 ngx_stream_js_init_vm(ngx_stream_session_t *s)
 {
-    void                      **ext;
-    ngx_pool_cleanup_t         *cln;
-    nxt_mem_cache_pool_t       *mcp;
-    ngx_stream_js_ctx_t        *ctx;
-    ngx_stream_js_srv_conf_t   *jscf;
+    nxt_int_t                  rc;
+    ngx_pool_cleanup_t        *cln;
+    ngx_stream_js_ctx_t       *ctx;
+    ngx_stream_js_srv_conf_t  *jscf;
 
     jscf = ngx_stream_get_module_srv_conf(s, ngx_stream_js_module);
     if (jscf->vm == NULL) {
@@ -643,8 +609,8 @@ ngx_stream_js_init_vm(ngx_stream_session
         return NGX_OK;
     }
 
-    mcp = ngx_stream_js_create_mem_cache_pool();
-    if (mcp == NULL) {
+    ctx->vm = njs_vm_clone(jscf->vm, s);
+    if (ctx->vm == NULL) {
         return NGX_ERROR;
     }
 
@@ -653,65 +619,28 @@ ngx_stream_js_init_vm(ngx_stream_session
         return NGX_ERROR;
     }
 
-    cln->handler = ngx_stream_js_cleanup_mem_cache_pool;
-    cln->data = mcp;
-
-    ext = ngx_palloc(s->connection->pool, sizeof(void *));
-    if (ext == NULL) {
-        return NGX_ERROR;
-    }
-
-    *ext = s;
-
-    ctx->vm = njs_vm_clone(jscf->vm, mcp, ext);
-    if (ctx->vm == NULL) {
-        return NGX_ERROR;
-    }
+    cln->handler = ngx_stream_js_cleanup_vm;
+    cln->data = ctx->vm;
 
     if (njs_vm_run(ctx->vm) != NJS_OK) {
         return NGX_ERROR;
     }
 
-    ctx->arg = &jscf->arg;
+    rc = njs_vm_external_create(ctx->vm, &ctx->arg, jscf->proto, s);
+    if (rc != NXT_OK) {
+        return NGX_ERROR;
+    }
 
     return NGX_OK;
 }
 
 
 static void
-ngx_stream_js_cleanup_mem_cache_pool(void *data)
+ngx_stream_js_cleanup_vm(void *data)
 {
-    nxt_mem_cache_pool_t *mcp = data;
-
-    nxt_mem_cache_pool_destroy(mcp);
-}
-
-
-static void *
-ngx_stream_js_alloc(void *mem, size_t size)
-{
-    return ngx_alloc(size, ngx_cycle->log);
-}
-
+    njs_vm_t *vm = data;
 
-static void *
-ngx_stream_js_calloc(void *mem, size_t size)
-{
-    return ngx_calloc(size, ngx_cycle->log);
-}
-
-
-static void *
-ngx_stream_js_memalign(void *mem, size_t alignment, size_t size)
-{
-    return ngx_memalign(alignment, size, ngx_cycle->log);
-}
-
-
-static void
-ngx_stream_js_free(void *mem, void *p)
-{
-    ngx_free(p);
+    njs_vm_destroy(vm);
 }
 
 
@@ -937,12 +866,10 @@ ngx_stream_js_include(ngx_conf_t *cf, ng
     ngx_fd_t               fd;
     ngx_str_t             *value, file;
     nxt_int_t              rc;
-    nxt_str_t              text, ext;
+    nxt_str_t              text;
     njs_vm_opt_t           options;
-    nxt_lvlhsh_t           externals;
     ngx_file_info_t        fi;
     ngx_pool_cleanup_t    *cln;
-    nxt_mem_cache_pool_t  *mcp;
 
     if (jscf->vm) {
         return "is duplicate";
@@ -1003,8 +930,13 @@ ngx_stream_js_include(ngx_conf_t *cf, ng
 
     end = start + size;
 
-    mcp = ngx_stream_js_create_mem_cache_pool();
-    if (mcp == NULL) {
+    ngx_memzero(&options, sizeof(njs_vm_opt_t));
+
+    options.backtrace = 1;
+
+    jscf->vm = njs_vm_create(&options);
+    if (jscf->vm == NULL) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to create JS VM");
         return NGX_CONF_ERROR;
     }
 
@@ -1013,28 +945,14 @@ ngx_stream_js_include(ngx_conf_t *cf, ng
         return NGX_CONF_ERROR;
     }
 
-    cln->handler = ngx_stream_js_cleanup_mem_cache_pool;
-    cln->data = mcp;
-
-    nxt_lvlhsh_init(&externals);
+    cln->handler = ngx_stream_js_cleanup_vm;
+    cln->data = jscf->vm;
 
-    if (njs_vm_external_add(&externals, mcp, 0, ngx_stream_js_externals,
-                            nxt_nitems(ngx_stream_js_externals))
-        != NJS_OK)
-    {
-        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "could not add js externals");
-        return NGX_CONF_ERROR;
-    }
+    jscf->proto = njs_vm_external_prototype(jscf->vm,
+                                            &ngx_stream_js_externals[0]);
 
-    ngx_memzero(&options, sizeof(njs_vm_opt_t));
-
-    options.mcp = mcp;
-    options.backtrace = 1;
-    options.externals_hash = &externals;
-
-    jscf->vm = njs_vm_create(&options);
-    if (jscf->vm == NULL) {
-        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to create JS VM");
+    if (jscf->proto == NULL) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to add stream proto");
         return NGX_CONF_ERROR;
     }
 
@@ -1056,14 +974,6 @@ ngx_stream_js_include(ngx_conf_t *cf, ng
         return NGX_CONF_ERROR;
     }
 
-    ext = nxt_string_value("$s");
-
-    if (njs_vm_external(jscf->vm, NULL, &ext, &jscf->arg) != NJS_OK) {
-        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
-                       "js external \"%*s\" not found", ext.length, ext.start);
-        return NGX_CONF_ERROR;
-    }
-
     return NGX_CONF_OK;
 }
 
@@ -1118,7 +1028,7 @@ ngx_stream_js_create_srv_conf(ngx_conf_t
      * set by ngx_pcalloc():
      *
      *     conf->vm = NULL;
-     *     conf->arg = NULL;
+     *     conf->proto = NULL;
      *     conf->access = { 0, NULL };
      *     conf->preread = { 0, NULL };
      *     conf->filter = { 0, NULL };
@@ -1136,7 +1046,7 @@ ngx_stream_js_merge_srv_conf(ngx_conf_t 
 
     if (conf->vm == NULL) {
         conf->vm = prev->vm;
-        conf->arg = prev->arg;
+        conf->proto = prev->proto;
     }
 
     ngx_conf_merge_str_value(conf->access, prev->access, "");
diff -r 0a3645d22d22 -r e7bc9d328a20 njs/njs.c
--- a/njs/njs.c	Tue Feb 20 19:12:52 2018 +0300
+++ b/njs/njs.c	Tue Feb 20 19:12:53 2018 +0300
@@ -60,7 +60,7 @@ typedef struct {
 
 
 static nxt_int_t njs_get_options(njs_opts_t *opts, int argc, char **argv);
-static nxt_int_t njs_externals_init(njs_opts_t *opts, njs_vm_opt_t *vm_options);
+static nxt_int_t njs_externals_init(njs_vm_t *vm);
 static nxt_int_t njs_interactive_shell(njs_opts_t *opts,
     njs_vm_opt_t *vm_options);
 static nxt_int_t njs_process_file(njs_opts_t *opts, njs_vm_opt_t *vm_options);
@@ -119,17 +119,15 @@ static njs_external_t  njs_externals[] =
 };
 
 
-static nxt_lvlhsh_t      njs_externals_hash;
 static njs_completion_t  njs_completion;
 
 
 int
 main(int argc, char **argv)
 {
-    nxt_int_t             ret;
-    njs_opts_t            opts;
-    njs_vm_opt_t          vm_options;
-    nxt_mem_cache_pool_t  *mcp;
+    nxt_int_t     ret;
+    njs_opts_t    opts;
+    njs_vm_opt_t  vm_options;
 
     memset(&opts, 0, sizeof(njs_opts_t));
     opts.interactive = 1;
@@ -144,22 +142,11 @@ main(int argc, char **argv)
         return EXIT_SUCCESS;
     }
 
-    mcp = nxt_mem_cache_pool_create(&njs_vm_mem_cache_pool_proto, NULL,
-                                    NULL, 2 * nxt_pagesize(), 128, 512, 16);
-    if (nxt_slow_path(mcp == NULL)) {
-        return EXIT_FAILURE;
-    }
-
     memset(&vm_options, 0, sizeof(njs_vm_opt_t));
 
-    vm_options.mcp = mcp;
     vm_options.accumulative = 1;
     vm_options.backtrace = 1;
 
-    if (njs_externals_init(&opts, &vm_options) != NXT_OK) {
-        return EXIT_FAILURE;
-    }
-
     if (opts.interactive) {
         ret = njs_interactive_shell(&opts, &vm_options);
 
@@ -218,30 +205,35 @@ njs_get_options(njs_opts_t *opts, int ar
 
 
 static nxt_int_t
-njs_externals_init(njs_opts_t *opts, njs_vm_opt_t *vm_options)
+njs_externals_init(njs_vm_t *vm)
 {
-    void        **ext;
-    nxt_uint_t  i;
-
-    nxt_lvlhsh_init(&njs_externals_hash);
+    nxt_uint_t          ret;
+    const njs_extern_t  *proto;
+    njs_opaque_value_t  *value;
 
-    for (i = 0; i < nxt_nitems(njs_externals); i++) {
-        if (njs_vm_external_add(&njs_externals_hash, vm_options->mcp,
-                                (uintptr_t) i, &njs_externals[i], 1)
-            != NXT_OK)
-        {
-            fprintf(stderr, "could not add external objects\n");
-            return NXT_ERROR;
-        }
-    }
+    static const nxt_str_t name = nxt_string_value("console");
 
-    ext = nxt_mem_cache_zalloc(vm_options->mcp, sizeof(void *) * i);
-    if (ext == NULL) {
+    proto = njs_vm_external_prototype(vm, &njs_externals[0]);
+    if (proto == NULL) {
+        fprintf(stderr, "failed to add console proto\n");
         return NXT_ERROR;
     }
 
-    vm_options->external = ext;
-    vm_options->externals_hash = &njs_externals_hash;
+    value = nxt_mem_cache_zalloc(vm->mem_cache_pool,
+                                 sizeof(njs_opaque_value_t));
+    if (value == NULL) {
+        return NXT_ERROR;
+    }
+
+    ret = njs_vm_external_create(vm, value, proto, NULL);
+    if (ret != NXT_OK) {
+        return NXT_ERROR;
+    }
+
+    ret = njs_vm_external_bind(vm, &name, value);
+    if (ret != NXT_OK) {
+        return NXT_ERROR;
+    }
 
     return NXT_OK;
 }
@@ -260,6 +252,10 @@ njs_interactive_shell(njs_opts_t *opts, 
         return NXT_ERROR;
     }
 
+    if (njs_externals_init(vm) != NXT_OK) {
+        return NXT_ERROR;
+    }
+
     if (njs_editline_init(vm) != NXT_OK) {
         fprintf(stderr, "failed to init completions\n");
         return NXT_ERROR;
@@ -393,6 +389,10 @@ njs_process_file(njs_opts_t *opts, njs_v
         goto done;
     }
 
+    if (njs_externals_init(vm) != NXT_OK) {
+        return NXT_ERROR;
+    }
+
     ret = njs_process_script(vm, opts, &script, &out);
     if (ret != NXT_OK) {
         fprintf(stderr, "failed to get retval from VM\n");
@@ -652,10 +652,7 @@ njs_ext_console_help(njs_vm_t *vm, njs_v
     }
 
     printf("\nEmbedded objects:\n");
-    for (i = 0; i < nxt_nitems(njs_externals); i++) {
-        printf("  %.*s\n", (int) njs_externals[i].name.length,
-               njs_externals[i].name.start);
-    }
+    printf("  console\n");
 
     printf("\n");
 
diff -r 0a3645d22d22 -r e7bc9d328a20 njs/njs_builtin.c
--- a/njs/njs_builtin.c	Tue Feb 20 19:12:52 2018 +0300
+++ b/njs/njs_builtin.c	Tue Feb 20 19:12:53 2018 +0300
@@ -513,12 +513,13 @@ njs_builtin_completions(njs_vm_t *vm, si
     size_t                  n, len;
     nxt_str_t               string;
     nxt_uint_t              i, k;
-    njs_extern_t            *ext_object, *ext_prop;
     njs_object_t            *objects;
     njs_keyword_t           *keyword;
     njs_function_t          *constructors;
     njs_object_prop_t       *prop;
     nxt_lvlhsh_each_t       lhe, lhe_prop;
+    njs_extern_value_t      *ev;
+    const njs_extern_t      *ext_proto, *ext_prop;
     njs_object_prototype_t  *prototypes;
 
     n = 0;
@@ -652,26 +653,27 @@ njs_builtin_completions(njs_vm_t *vm, si
         }
     }
 
-    nxt_lvlhsh_each_init(&lhe, &njs_extern_hash_proto);
+    nxt_lvlhsh_each_init(&lhe, &njs_extern_value_hash_proto);
 
     for ( ;; ) {
-        ext_object = nxt_lvlhsh_each(&vm->externals_hash, &lhe);
+        ev = nxt_lvlhsh_each(&vm->externals_hash, &lhe);
 
-        if (ext_object == NULL) {
+        if (ev == NULL) {
             break;
         }
 
+        ext_proto = ev->value->external.proto;
+
         nxt_lvlhsh_each_init(&lhe_prop, &njs_extern_hash_proto);
 
         if (completions != NULL) {
-            len = ext_object->name.length + 1;
+            len = ev->name.length + 1;
             compl = nxt_mem_cache_zalloc(vm->mem_cache_pool, len);
             if (compl == NULL) {
                 return NXT_ERROR;
             }
 
-            snprintf(compl, len, "%.*s",
-                     (int) ext_object->name.length, ext_object->name.start);
+            snprintf(compl, len, "%.*s", (int) ev->name.length, ev->name.start);
 
             completions[n].length = len;
             completions[n++].start = (u_char *) compl;
@@ -681,22 +683,22 @@ njs_builtin_completions(njs_vm_t *vm, si
         }
 
         for ( ;; ) {
-            ext_prop = nxt_lvlhsh_each(&ext_object->hash, &lhe_prop);
+            ext_prop = nxt_lvlhsh_each(&ext_proto->hash, &lhe_prop);
 
             if (ext_prop == NULL) {
                 break;
             }
 
             if (completions != NULL) {
-                len = ext_object->name.length + ext_prop->name.length + 2;
+                len = ev->name.length + ev->name.length + 2;
                 compl = nxt_mem_cache_zalloc(vm->mem_cache_pool, len);
                 if (compl == NULL) {
                     return NXT_ERROR;
                 }
 
-                snprintf(compl, len, "%.*s.%.*s",
-                         (int) ext_object->name.length, ext_object->name.start,
-                         (int) ext_prop->name.length, ext_prop->name.start);
+                snprintf(compl, len, "%.*s.%.*s", (int) ev->name.length,
+                         ev->name.start, (int) ext_prop->name.length,
+                         ext_prop->name.start);
 
                 completions[n].length = len;
                 completions[n++].start = (u_char *) compl;
diff -r 0a3645d22d22 -r e7bc9d328a20 njs/njs_extern.c
--- a/njs/njs_extern.c	Tue Feb 20 19:12:52 2018 +0300
+++ b/njs/njs_extern.c	Tue Feb 20 19:12:53 2018 +0300
@@ -29,11 +29,24 @@ njs_extern_hash_test(nxt_lvlhsh_query_t 
 {
     njs_extern_t  *ext;
 
-    ext = data;
+    ext = (njs_extern_t *) data;
+
+    if (nxt_strstr_eq(&lhq->key, &ext->name)) {
+        return NXT_OK;
+    }
+
+    return NXT_DECLINED;
+}
 
-// STUB
-//    if (nxt_strcasestr_eq(&lhq->key, &ext->name)) {
-    if (nxt_strstr_eq(&lhq->key, &ext->name)) {
+
+static nxt_int_t
+njs_extern_value_hash_test(nxt_lvlhsh_query_t *lhq, void *data)
+{
+    njs_extern_value_t  *ev;
+
+    ev = (njs_extern_value_t *) data;
+
+    if (nxt_strstr_eq(&lhq->key, &ev->name)) {
         return NXT_OK;
     }
 
@@ -52,71 +65,78 @@ const nxt_lvlhsh_proto_t  njs_extern_has
 };
 
 
-nxt_int_t
-njs_vm_external_add(nxt_lvlhsh_t *hash, nxt_mem_cache_pool_t *mcp,
-    uintptr_t object, njs_external_t *external, nxt_uint_t n)
+const nxt_lvlhsh_proto_t  njs_extern_value_hash_proto
+    nxt_aligned(64) =
+{
+    NXT_LVLHSH_DEFAULT,
+    NXT_LVLHSH_BATCH_ALLOC,
+    njs_extern_value_hash_test,
+    njs_lvlhsh_alloc,
+    njs_lvlhsh_free,
+};
+
+
+static njs_extern_t *
+njs_vm_external_add(njs_vm_t *vm, nxt_lvlhsh_t *hash, njs_external_t *external,
+    nxt_uint_t n)
 {
     nxt_int_t           ret;
-    njs_extern_t        *ext;
+    njs_extern_t        *ext, *child;
     nxt_lvlhsh_query_t  lhq;
 
     do {
-        ext = nxt_mem_cache_align(mcp, sizeof(njs_value_t),
-                                 sizeof(njs_extern_t));
+        ext = nxt_mem_cache_alloc(vm->mem_cache_pool, sizeof(njs_extern_t));
         if (nxt_slow_path(ext == NULL)) {
-            return NXT_ERROR;
-        }
-
-        ext->name.length = external->name.length;
-        ext->name.start = nxt_mem_cache_alloc(mcp, external->name.length);
-        if (nxt_slow_path(ext->name.start == NULL)) {
-            return NXT_ERROR;
+            return NULL;
         }
 
-        memcpy(ext->name.start, external->name.start, external->name.length);
-
-        ext->value.type = NJS_EXTERNAL;
-        ext->value.data.truth = 1;
-        ext->value.data.u.external = ext;
-
-        if (external->method != NULL) {
-            ext->function = nxt_mem_cache_zalloc(mcp, sizeof(njs_function_t));
-            if (nxt_slow_path(ext->function == NULL)) {
-                return NXT_ERROR;
-            }
-
-            ext->function->native = 1;
-            ext->function->args_offset = 1;
-            ext->function->u.native = external->method;
-        }
+        ext->name = external->name;
 
         nxt_lvlhsh_init(&ext->hash);
+
         ext->type = external->type;
         ext->get = external->get;
         ext->set = external->set;
         ext->find = external->find;
         ext->foreach = external->foreach;
         ext->next = external->next;
-        ext->object = object;
         ext->data = external->data;
 
-        lhq.key_hash = nxt_djb_hash(external->name.start, external->name.length);
-        lhq.key = ext->name;
-        lhq.replace = 0;
-        lhq.value = ext;
-        lhq.pool = mcp;

From pgnet.dev at gmail.com  Tue Feb 20 19:41:53 2018
From: pgnet.dev at gmail.com (PGNet Dev)
Date: Tue, 20 Feb 2018 11:41:53 -0800
Subject: nginx/1.13.9 + njs/head build : ngx_http_js_module.so: undefined
 symbol
Message-ID: <3d0a7041-a803-a36b-166d-4deeafe095c1@gmail.com>

Upgrading nginx 1.13.8 -> 1.13.9 with usual

	./configure \
	... \
	--add-dynamic-module=/usr/local/src/njs/nginx

and

	cd /usr/local/src/njs/nginx
	hg log | head
		changeset:   450:757271547b56
		tag:         tip
		user:        Dmitry Volyntsev <xeioex at nginx.com>
		date:        Tue Feb 20 19:12:55 2018 +0300
		summary:     Fixed the names of global functions in backtraces.

	ls -al /usr/local/src/njs/nginx
		total 76K
		drwxr-xr-x 2 root root 4.0K Feb 20 11:31 ./
		drwxr-xr-x 6 root root 4.0K Feb 20 11:31 ../
		-rw-r--r-- 1 root root  749 Feb 20 11:31 config
		-rw-r--r-- 1 root root  260 Feb 20 11:31 config.make
		-rw-r--r-- 1 root root  32K Feb 20 11:31 ngx_http_js_module.c
		-rw-r--r-- 1 root root  26K Feb 20 11:31 ngx_stream_js_module.c

No errors on build, but on conf check

	/usr/local/sbin/nginx -t -c /usr/local/etc/nginx/nginx.conf
		nginx: [emerg] dlopen() "/usr/local/nginx-modules/ngx_http_js_module.so" failed (/usr/local/nginx-modules/ngx_http_js_module.so: undefined symbol: njs_vm_value_to_ext_string) in /usr/local/etc/nginx/nginx.conf:34
		nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed

where, line 34's

	load_module /usr/local/nginx-modules/ngx_http_js_module.so;

and

	ls -al /usr/local/nginx-modules/ngx_http_js_module.so
		-rwxr-xr-x 1 root root 1.3M Feb 20 11:26 /usr/local/nginx-modules/ngx_http_js_module.so*
	ldd /usr/local/nginx-modules/ngx_http_js_module.so
        linux-vdso.so.1 (0x00007ffef8db5000)
        libssl.so.1.1 => /usr/local/openssl11/lib64/libssl.so.1.1 (0x00007f8822d18000)
        libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1 (0x00007f882286e000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f882266a000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f8822453000)
        libpcre.so.1 => /usr/local/lib64/libpcre.so.1 (0x00007f88221dc000)
        libpcrecpp.so.0 => /usr/local/lib64/libpcrecpp.so.0 (0x00007f8821fd2000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f8821cd5000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f8821934000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f8821717000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f88231d7000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f8821390000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f8821179000)

	nm /usr/local/nginx-modules/ngx_http_js_module.so | grep "U njs"
		U njs_vm_external_create
		U njs_vm_external_prototype
		U njs_vm_retval_to_ext_string
		U njs_vm_value_to_ext_string

From felix.ruess at gmail.com  Tue Feb 20 19:51:08 2018
From: felix.ruess at gmail.com (Felix Ruess)
Date: Tue, 20 Feb 2018 20:51:08 +0100
Subject: patch for fixing timouts #189
Message-ID: <CAEc+GFeVubN8dpGkXu_xZ+EqGuCu2RVK9R0qJ1OoUipmF3eEHQ@mail.gmail.com>

Hi all,

quite a while ago someone else already found the problem that the timeouts
use system time rather than a monotonic clock:
https://trac.nginx.org/nginx/ticket/189

I updated/fixed the patch [1] there a while ago already and have this
successfully deployed on ~50 devices since then.

Any chance to get this included now? Or is there still something missing?

Cheers,
Felix

[1] https://trac.nginx.org/nginx/attachment/ticket/189/
monotonic_timers.2.patch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180220/bb717018/attachment.html>

From pgnet.dev at gmail.com  Tue Feb 20 20:29:06 2018
From: pgnet.dev at gmail.com (PGNet Dev)
Date: Tue, 20 Feb 2018 12:29:06 -0800
Subject: nginx/1.13.9 + njs/head build : ngx_http_js_module.so: undefined
 symbol
In-Reply-To: <3d0a7041-a803-a36b-166d-4deeafe095c1@gmail.com>
References: <3d0a7041-a803-a36b-166d-4deeafe095c1@gmail.com>
Message-ID: <a71d378c-0804-9cf2-9b60-44a22cb2b9af@gmail.com>

On 2/20/18 11:41 AM, PGNet Dev wrote:
> 	cd /usr/local/src/njs/nginx
> 	hg log | head
> 		changeset:   450:757271547b56
> 		tag:         tip
> 		user:        Dmitry Volyntsev <xeioex at nginx.com>
> 		date:        Tue Feb 20 19:12:55 2018 +0300
> 		summary:     Fixed the names of global functions in backtraces.

fyi, simple revert to

     hg revert -r 446 --all

fixes the config check problem, and subsequent nginx build execs as usual





From xeioex at nginx.com  Tue Feb 20 21:26:21 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 21 Feb 2018 00:26:21 +0300
Subject: nginx/1.13.9 + njs/head build : ngx_http_js_module.so: undefined
 symbol
In-Reply-To: <3d0a7041-a803-a36b-166d-4deeafe095c1@gmail.com>
References: <3d0a7041-a803-a36b-166d-4deeafe095c1@gmail.com>
Message-ID: <50CD132A-AA6C-42D3-976A-002F4E5AE33A@nginx.com>

Thank you for reporting the problem. 

Please, make sure that you do 'make clean? in njs directory after hg update.


> On 20 Feb 2018, at 22:41, PGNet Dev <pgnet.dev at gmail.com> wrote:
> 
> Upgrading nginx 1.13.8 -> 1.13.9 with usual
> 
> 	./configure \
> 	... \
> 	--add-dynamic-module=/usr/local/src/njs/nginx
> 
> and
> 
> 	cd /usr/local/src/njs/nginx
> 	hg log | head
> 		changeset:   450:757271547b56
> 		tag:         tip
> 		user:        Dmitry Volyntsev <xeioex at nginx.com>
> 		date:        Tue Feb 20 19:12:55 2018 +0300
> 		summary:     Fixed the names of global functions in backtraces.
> 
> 	ls -al /usr/local/src/njs/nginx
> 		total 76K
> 		drwxr-xr-x 2 root root 4.0K Feb 20 11:31 ./
> 		drwxr-xr-x 6 root root 4.0K Feb 20 11:31 ../
> 		-rw-r--r-- 1 root root  749 Feb 20 11:31 config
> 		-rw-r--r-- 1 root root  260 Feb 20 11:31 config.make
> 		-rw-r--r-- 1 root root  32K Feb 20 11:31 ngx_http_js_module.c
> 		-rw-r--r-- 1 root root  26K Feb 20 11:31 ngx_stream_js_module.c
> 
> No errors on build, but on conf check
> 
> 	/usr/local/sbin/nginx -t -c /usr/local/etc/nginx/nginx.conf
> 		nginx: [emerg] dlopen() "/usr/local/nginx-modules/ngx_http_js_module.so" failed (/usr/local/nginx-modules/ngx_http_js_module.so: undefined symbol: njs_vm_value_to_ext_string) in /usr/local/etc/nginx/nginx.conf:34
> 		nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed
> 
> where, line 34's
> 
> 	load_module /usr/local/nginx-modules/ngx_http_js_module.so;
> 
> and
> 
> 	ls -al /usr/local/nginx-modules/ngx_http_js_module.so
> 		-rwxr-xr-x 1 root root 1.3M Feb 20 11:26 /usr/local/nginx-modules/ngx_http_js_module.so*
> 	ldd /usr/local/nginx-modules/ngx_http_js_module.so
>        linux-vdso.so.1 (0x00007ffef8db5000)
>        libssl.so.1.1 => /usr/local/openssl11/lib64/libssl.so.1.1 (0x00007f8822d18000)
>        libcrypto.so.1.1 => /usr/local/openssl11/lib64/libcrypto.so.1.1 (0x00007f882286e000)
>        libdl.so.2 => /lib64/libdl.so.2 (0x00007f882266a000)
>        libz.so.1 => /lib64/libz.so.1 (0x00007f8822453000)
>        libpcre.so.1 => /usr/local/lib64/libpcre.so.1 (0x00007f88221dc000)
>        libpcrecpp.so.0 => /usr/local/lib64/libpcrecpp.so.0 (0x00007f8821fd2000)
>        libm.so.6 => /lib64/libm.so.6 (0x00007f8821cd5000)
>        libc.so.6 => /lib64/libc.so.6 (0x00007f8821934000)
>        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f8821717000)
>        /lib64/ld-linux-x86-64.so.2 (0x00007f88231d7000)
>        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f8821390000)
>        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f8821179000)
> 
> 	nm /usr/local/nginx-modules/ngx_http_js_module.so | grep "U njs"
> 		U njs_vm_external_create
> 		U njs_vm_external_prototype
> 		U njs_vm_retval_to_ext_string
> 		U njs_vm_value_to_ext_string


From pgnet.dev at gmail.com  Tue Feb 20 21:32:13 2018
From: pgnet.dev at gmail.com (PGNet Dev)
Date: Tue, 20 Feb 2018 13:32:13 -0800
Subject: nginx/1.13.9 + njs/head build : ngx_http_js_module.so: undefined
 symbol
In-Reply-To: <50CD132A-AA6C-42D3-976A-002F4E5AE33A@nginx.com>
References: <3d0a7041-a803-a36b-166d-4deeafe095c1@gmail.com>
 <50CD132A-AA6C-42D3-976A-002F4E5AE33A@nginx.com>
Message-ID: <9ce03225-8671-9c8e-7a33-ed19d15abde9@gmail.com>

On 2/20/18 1:26 PM, Dmitry Volyntsev wrote:
> Thank you for reporting the problem.
> 
> Please, make sure that you do 'make clean? in njs directory after hg update.
> 


Of course.  I'm always

   hg update --clean tip

r450 is reproducibly causing the error.

r446 is fine atm.

From xeioex at nginx.com  Wed Feb 21 06:22:21 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 21 Feb 2018 09:22:21 +0300
Subject: nginx/1.13.9 + njs/head build : ngx_http_js_module.so: undefined
 symbol
In-Reply-To: <9ce03225-8671-9c8e-7a33-ed19d15abde9@gmail.com>
References: <3d0a7041-a803-a36b-166d-4deeafe095c1@gmail.com>
 <50CD132A-AA6C-42D3-976A-002F4E5AE33A@nginx.com>
 <9ce03225-8671-9c8e-7a33-ed19d15abde9@gmail.com>
Message-ID: <92494F49-889D-49F3-8096-F55F02AA1A68@nginx.com>



> On 21 Feb 2018, at 00:32, PGNet Dev <pgnet.dev at gmail.com> wrote:
> 
> On 2/20/18 1:26 PM, Dmitry Volyntsev wrote:
>> Thank you for reporting the problem.
>> Please, make sure that you do 'make clean? in njs directory after hg update.
> 
> 
> Of course.  I'm always
> 
>  hg update --clean tip
> 
> r450 is reproducibly causing the error.
> 
> r446 is fine atm.

I cannot reproduce it with a clean build (ubuntu 14.04, macOS 10.13).

Can you reproduce the issue from scratch (downloading nginx tip & njs tip in a separate directory)?

Somehow your ngx_http_js_module.so is linked against outdated libnjs.a.

On r450 you should see the following:

objdump -t ../njs/build/libnjs.a  | grep -E 'g.*njs_vm_'
000000000000021e g     F .text  000000000000026d njs_vm_create
0000000000000040 g     O .data.rel.ro.local     0000000000000038 .hidden njs_vm_mem_cache_pool_proto
000000000000048b g     F .text  0000000000000015 njs_vm_destroy
00000000000004a0 g     F .text  000000000000015c njs_vm_compile
00000000000005fc g     F .text  00000000000001c8 njs_vm_clone
00000000000007c4 g     F .text  0000000000000072 njs_vm_call
0000000000000836 g     F .text  0000000000000051 njs_vm_run
0000000000000887 g     F .text  0000000000000004 njs_vm_retval
000000000000088b g     F .text  000000000000000f njs_vm_retval_set
000000000000089a g     F .text  0000000000000035 njs_vm_retval_to_ext_string
0000000000002c0c g     F .text  000000000000001c .hidden njs_vm_backtrace
0000000000002c28 g     F .text  00000000000002fb njs_vm_value_to_ext_string
0000000000000384 g     F .text  000000000000001d njs_vm_external_prototype
00000000000003a1 g     F .text  0000000000000077 njs_vm_external_create
0000000000000418 g     F .text  00000000000000a6 njs_vm_external_bind
0000000000000803 g     F .text  0000000000000083 njs_vm_function
0000000000000b8d g     F .text  00000000000004c8 njs_vm_completions

nm ./objs/ngx_http_js_module.so | grep -E 'T.*njs_vm_'
000000000000b1f9 T njs_vm_call
000000000000b031 T njs_vm_clone
000000000000aed5 T njs_vm_compile
0000000000019c4d T njs_vm_completions
000000000000ac53 T njs_vm_create
000000000000aec0 T njs_vm_destroy
0000000000018698 T njs_vm_external_bind
0000000000018621 T njs_vm_external_create
0000000000018604 T njs_vm_external_prototype
000000000001903d T njs_vm_function
000000000000b2bc T njs_vm_retval
000000000000b2c0 T njs_vm_retval_set
000000000000b2cf T njs_vm_retval_to_ext_string
000000000000b26b T njs_vm_run
000000000000df38 T njs_vm_value_to_ext_string


From ru at nginx.com  Wed Feb 21 12:51:46 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Wed, 21 Feb 2018 12:51:46 +0000
Subject: [nginx] Version bump.
Message-ID: <hg.0237af43d409.1519217506.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/0237af43d409
branches:  
changeset: 7212:0237af43d409
user:      Ruslan Ermilov <ru at nginx.com>
date:      Wed Feb 21 15:50:35 2018 +0300
description:
Version bump.

diffstat:

 src/core/nginx.h |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff -r c91356fdd802 -r 0237af43d409 src/core/nginx.h
--- a/src/core/nginx.h	Tue Feb 20 17:08:49 2018 +0300
+++ b/src/core/nginx.h	Wed Feb 21 15:50:35 2018 +0300
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1013009
-#define NGINX_VERSION      "1.13.9"
+#define nginx_version      1013010
+#define NGINX_VERSION      "1.13.10"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #ifdef NGX_BUILD

From ru at nginx.com  Wed Feb 21 12:51:48 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Wed, 21 Feb 2018 12:51:48 +0000
Subject: [nginx] Geo: fixed memory allocation error handling (closes #1482).
Message-ID: <hg.c69c13f10502.1519217508.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/c69c13f10502
branches:  
changeset: 7213:c69c13f10502
user:      Ruslan Ermilov <ru at nginx.com>
date:      Wed Feb 21 15:50:42 2018 +0300
description:
Geo: fixed memory allocation error handling (closes #1482).

If during configuration parsing of the geo directive the memory
allocation has failed, pool used to parse configuration inside
the block, and sometimes the temporary pool were not destroyed.

diffstat:

 src/http/modules/ngx_http_geo_module.c |  27 ++++++++++++++++-----------
 src/stream/ngx_stream_geo_module.c     |  27 ++++++++++++++++-----------
 2 files changed, 32 insertions(+), 22 deletions(-)

diffs (170 lines):

diff -r 0237af43d409 -r c69c13f10502 src/http/modules/ngx_http_geo_module.c
--- a/src/http/modules/ngx_http_geo_module.c	Wed Feb 21 15:50:35 2018 +0300
+++ b/src/http/modules/ngx_http_geo_module.c	Wed Feb 21 15:50:42 2018 +0300
@@ -439,6 +439,7 @@ ngx_http_geo_block(ngx_conf_t *cf, ngx_c
 
     ctx.temp_pool = ngx_create_pool(NGX_DEFAULT_POOL_SIZE, cf->log);
     if (ctx.temp_pool == NULL) {
+        ngx_destroy_pool(pool);
         return NGX_CONF_ERROR;
     }
 
@@ -482,7 +483,7 @@ ngx_http_geo_block(ngx_conf_t *cf, ngx_c
 
                 ctx.high.low[i] = ngx_palloc(cf->pool, len + sizeof(void *));
                 if (ctx.high.low[i] == NULL) {
-                    return NGX_CONF_ERROR;
+                    goto failed;
                 }
 
                 ngx_memcpy(ctx.high.low[i], a->elts, len);
@@ -508,14 +509,11 @@ ngx_http_geo_block(ngx_conf_t *cf, ngx_c
         var->get_handler = ngx_http_geo_range_variable;
         var->data = (uintptr_t) geo;
 
-        ngx_destroy_pool(ctx.temp_pool);
-        ngx_destroy_pool(pool);
-
     } else {
         if (ctx.tree == NULL) {
             ctx.tree = ngx_radix_tree_create(cf->pool, -1);
             if (ctx.tree == NULL) {
-                return NGX_CONF_ERROR;
+                goto failed;
             }
         }
 
@@ -525,7 +523,7 @@ ngx_http_geo_block(ngx_conf_t *cf, ngx_c
         if (ctx.tree6 == NULL) {
             ctx.tree6 = ngx_radix_tree_create(cf->pool, -1);
             if (ctx.tree6 == NULL) {
-                return NGX_CONF_ERROR;
+                goto failed;
             }
         }
 
@@ -535,14 +533,11 @@ ngx_http_geo_block(ngx_conf_t *cf, ngx_c
         var->get_handler = ngx_http_geo_cidr_variable;
         var->data = (uintptr_t) geo;
 
-        ngx_destroy_pool(ctx.temp_pool);
-        ngx_destroy_pool(pool);
-
         if (ngx_radix32tree_insert(ctx.tree, 0, 0,
                                    (uintptr_t) &ngx_http_variable_null_value)
             == NGX_ERROR)
         {
-            return NGX_CONF_ERROR;
+            goto failed;
         }
 
         /* NGX_BUSY is okay (default was set explicitly) */
@@ -552,12 +547,22 @@ ngx_http_geo_block(ngx_conf_t *cf, ngx_c
                                     (uintptr_t) &ngx_http_variable_null_value)
             == NGX_ERROR)
         {
-            return NGX_CONF_ERROR;
+            goto failed;
         }
 #endif
     }
 
+    ngx_destroy_pool(ctx.temp_pool);
+    ngx_destroy_pool(pool);
+
     return rv;
+
+failed:
+
+    ngx_destroy_pool(ctx.temp_pool);
+    ngx_destroy_pool(pool);
+
+    return NGX_CONF_ERROR;
 }
 
 
diff -r 0237af43d409 -r c69c13f10502 src/stream/ngx_stream_geo_module.c
--- a/src/stream/ngx_stream_geo_module.c	Wed Feb 21 15:50:35 2018 +0300
+++ b/src/stream/ngx_stream_geo_module.c	Wed Feb 21 15:50:42 2018 +0300
@@ -409,6 +409,7 @@ ngx_stream_geo_block(ngx_conf_t *cf, ngx
 
     ctx.temp_pool = ngx_create_pool(NGX_DEFAULT_POOL_SIZE, cf->log);
     if (ctx.temp_pool == NULL) {
+        ngx_destroy_pool(pool);
         return NGX_CONF_ERROR;
     }
 
@@ -449,7 +450,7 @@ ngx_stream_geo_block(ngx_conf_t *cf, ngx
 
                 ctx.high.low[i] = ngx_palloc(cf->pool, len + sizeof(void *));
                 if (ctx.high.low[i] == NULL) {
-                    return NGX_CONF_ERROR;
+                    goto failed;
                 }
 
                 ngx_memcpy(ctx.high.low[i], a->elts, len);
@@ -475,14 +476,11 @@ ngx_stream_geo_block(ngx_conf_t *cf, ngx
         var->get_handler = ngx_stream_geo_range_variable;
         var->data = (uintptr_t) geo;
 
-        ngx_destroy_pool(ctx.temp_pool);
-        ngx_destroy_pool(pool);
-
     } else {
         if (ctx.tree == NULL) {
             ctx.tree = ngx_radix_tree_create(cf->pool, -1);
             if (ctx.tree == NULL) {
-                return NGX_CONF_ERROR;
+                goto failed;
             }
         }
 
@@ -492,7 +490,7 @@ ngx_stream_geo_block(ngx_conf_t *cf, ngx
         if (ctx.tree6 == NULL) {
             ctx.tree6 = ngx_radix_tree_create(cf->pool, -1);
             if (ctx.tree6 == NULL) {
-                return NGX_CONF_ERROR;
+                goto failed;
             }
         }
 
@@ -502,14 +500,11 @@ ngx_stream_geo_block(ngx_conf_t *cf, ngx
         var->get_handler = ngx_stream_geo_cidr_variable;
         var->data = (uintptr_t) geo;
 
-        ngx_destroy_pool(ctx.temp_pool);
-        ngx_destroy_pool(pool);
-
         if (ngx_radix32tree_insert(ctx.tree, 0, 0,
                                    (uintptr_t) &ngx_stream_variable_null_value)
             == NGX_ERROR)
         {
-            return NGX_CONF_ERROR;
+            goto failed;
         }
 
         /* NGX_BUSY is okay (default was set explicitly) */
@@ -519,12 +514,22 @@ ngx_stream_geo_block(ngx_conf_t *cf, ngx
                                     (uintptr_t) &ngx_stream_variable_null_value)
             == NGX_ERROR)
         {
-            return NGX_CONF_ERROR;
+            goto failed;
         }
 #endif
     }
 
+    ngx_destroy_pool(ctx.temp_pool);
+    ngx_destroy_pool(pool);
+
     return rv;
+
+failed:
+
+    ngx_destroy_pool(ctx.temp_pool);
+    ngx_destroy_pool(pool);
+
+    return NGX_CONF_ERROR;
 }
 
 

From ru at nginx.com  Wed Feb 21 12:51:49 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Wed, 21 Feb 2018 12:51:49 +0000
Subject: [nginx] Geo: optimized configuration parser.
Message-ID: <hg.88aad69eccef.1519217509.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/88aad69eccef
branches:  
changeset: 7214:88aad69eccef
user:      Ruslan Ermilov <ru at nginx.com>
date:      Wed Feb 21 15:50:43 2018 +0300
description:
Geo: optimized configuration parser.

If the geo block parser has failed, doing more things is pointless.

diffstat:

 src/http/modules/ngx_http_geo_module.c |  6 +++++-
 src/stream/ngx_stream_geo_module.c     |  6 +++++-
 2 files changed, 10 insertions(+), 2 deletions(-)

diffs (46 lines):

diff -r c69c13f10502 -r 88aad69eccef src/http/modules/ngx_http_geo_module.c
--- a/src/http/modules/ngx_http_geo_module.c	Wed Feb 21 15:50:42 2018 +0300
+++ b/src/http/modules/ngx_http_geo_module.c	Wed Feb 21 15:50:43 2018 +0300
@@ -461,6 +461,10 @@ ngx_http_geo_block(ngx_conf_t *cf, ngx_c
 
     *cf = save;
 
+    if (rv != NGX_CONF_OK) {
+        goto failed;
+    }
+
     geo->proxies = ctx.proxies;
     geo->proxy_recursive = ctx.proxy_recursive;
 
@@ -555,7 +559,7 @@ ngx_http_geo_block(ngx_conf_t *cf, ngx_c
     ngx_destroy_pool(ctx.temp_pool);
     ngx_destroy_pool(pool);
 
-    return rv;
+    return NGX_CONF_OK;
 
 failed:
 
diff -r c69c13f10502 -r 88aad69eccef src/stream/ngx_stream_geo_module.c
--- a/src/stream/ngx_stream_geo_module.c	Wed Feb 21 15:50:42 2018 +0300
+++ b/src/stream/ngx_stream_geo_module.c	Wed Feb 21 15:50:43 2018 +0300
@@ -431,6 +431,10 @@ ngx_stream_geo_block(ngx_conf_t *cf, ngx
 
     *cf = save;
 
+    if (rv != NGX_CONF_OK) {
+        goto failed;
+    }
+
     if (ctx.ranges) {
 
         if (ctx.high.low && !ctx.binary_include) {
@@ -522,7 +526,7 @@ ngx_stream_geo_block(ngx_conf_t *cf, ngx
     ngx_destroy_pool(ctx.temp_pool);
     ngx_destroy_pool(pool);
 
-    return rv;
+    return NGX_CONF_OK;
 
 failed:
 

From pluknet at nginx.com  Wed Feb 21 14:27:17 2018
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Wed, 21 Feb 2018 14:27:17 +0000
Subject: [nginx] Geo: fixed indentation.
Message-ID: <hg.2dc837d16099.1519223237.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/2dc837d16099
branches:  
changeset: 7215:2dc837d16099
user:      Sergey Kandaurov <pluknet at nginx.com>
date:      Wed Feb 21 17:26:00 2018 +0300
description:
Geo: fixed indentation.

diffstat:

 src/stream/ngx_stream_geo_module.c |  22 +++++++++++-----------
 1 files changed, 11 insertions(+), 11 deletions(-)

diffs (33 lines):

diff -r 88aad69eccef -r 2dc837d16099 src/stream/ngx_stream_geo_module.c
--- a/src/stream/ngx_stream_geo_module.c	Wed Feb 21 15:50:43 2018 +0300
+++ b/src/stream/ngx_stream_geo_module.c	Wed Feb 21 17:26:00 2018 +0300
@@ -341,18 +341,18 @@ ngx_stream_geo_addr(ngx_stream_session_t
 static char *
 ngx_stream_geo_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
 {
-    char                         *rv;
-    size_t                        len;
-    ngx_str_t                    *value, name;
-    ngx_uint_t                    i;
-    ngx_conf_t                    save;
-    ngx_pool_t                   *pool;
-    ngx_array_t                  *a;
-    ngx_stream_variable_t        *var;
-    ngx_stream_geo_ctx_t         *geo;
-    ngx_stream_geo_conf_ctx_t     ctx;
+    char                       *rv;
+    size_t                      len;
+    ngx_str_t                  *value, name;
+    ngx_uint_t                  i;
+    ngx_conf_t                  save;
+    ngx_pool_t                 *pool;
+    ngx_array_t                *a;
+    ngx_stream_variable_t      *var;
+    ngx_stream_geo_ctx_t       *geo;
+    ngx_stream_geo_conf_ctx_t   ctx;
 #if (NGX_HAVE_INET6)
-    static struct in6_addr        zero;
+    static struct in6_addr      zero;
 #endif
 
     value = cf->args->elts;

From mdounin at mdounin.ru  Wed Feb 21 17:45:54 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 21 Feb 2018 20:45:54 +0300
Subject: patch for fixing timouts #189
In-Reply-To: <CAEc+GFeVubN8dpGkXu_xZ+EqGuCu2RVK9R0qJ1OoUipmF3eEHQ@mail.gmail.com>
References: <CAEc+GFeVubN8dpGkXu_xZ+EqGuCu2RVK9R0qJ1OoUipmF3eEHQ@mail.gmail.com>
Message-ID: <20180221174554.GV24410@mdounin.ru>

Hello!

On Tue, Feb 20, 2018 at 08:51:08PM +0100, Felix Ruess wrote:

> quite a while ago someone else already found the problem that the timeouts
> use system time rather than a monotonic clock:
> https://trac.nginx.org/nginx/ticket/189
> 
> I updated/fixed the patch [1] there a while ago already and have this
> successfully deployed on ~50 devices since then.
> 
> Any chance to get this included now? Or is there still something missing?

The patch in question was never submitted to the mailing list[1], 
and hence was never properly reviewed and/or considered for 
inclusion.

Overall, I think the patch is somewhat overcomplicated.  Also, the 
"monotonic_timers" configuration directive introduced by the patch 
seems to behave wrong if changed during a reconfiguration: 
ngx_current_msec values can be saved into various shared memory 
zones (e.g., limit_req will do it), leading to incorrect results 
if the directive is changed.

Rather, we may want to use monotonic timers unconditionally, at 
least if a fast variant is available.  I'll take a look as time 
permits.

[1] http://nginx.org/en/docs/contributing_changes.html

-- 
Maxim Dounin
http://mdounin.ru/

From lstewart at netflix.com  Thu Feb 22 00:53:14 2018
From: lstewart at netflix.com (Lawrence Stewart)
Date: Thu, 22 Feb 2018 11:53:14 +1100
Subject: Sub-requests poisoning r->variables cache
Message-ID: <CAOToG0RtQi9vUbR-qQbKz3E95ziuuX8DAQz-WyGbMctVsAo=PQ@mail.gmail.com>

Greetings nginx developers,

The root cause analysis for a bug I observed during development of some
custom nginx module code turns out to be an unfortunate interaction between
sub-requests and indexed variable lookups using
ngx_http_get_indexed_variable(). We're based on nginx 1.13.6.

The sequence of events is:

- Main request arrives and triggers an access-phase sub-request. Main
request includes request variable "x=blah", but sub-request does not.

- Sub-request arrives at custom module's header filter

- Custom module looks up variable "x" with ngx_http_get_indexed_variable(),
and it is not found in the sub-request. This causes the "not_found" flag to
be set in the sr->variables var cache for variable "x", but
because ngx_http_subrequest() shallow copies r->variables from parent
request to sub-request, the not_found flag change affects the main
request's r->variables too.

- Sub-request completes, main request is allowed to progress and arrives at
custom module's header filter

- Custom module again looks up variable "x"
with ngx_http_get_indexed_variable(), and even though "x" is present in
r->args and I've confirmed can be found by
ngx_http_arg(), ngx_http_get_indexed_variable() sees the not_found flag,
does not refresh the variable cache and returns the poisoned
ngx_http_variable_value_t.

There are ways to deal with this in our custom code, but it seems like a
fundamental problem in the current design of the sub-request mechanism and
I'm thinking that it may be appropriate to address this in the nginx core
code e.g. by deep copying r->variables in ngx_http_subrequest(), or by
somehow tracking when to invalidate the ngx_http_variable_value_t flags and
refresh the r->variables cache.

Thoughts?

Cheers,
Lawrence
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20180222/3ef5f5a9/attachment.html>

From ru at nginx.com  Thu Feb 22 10:48:43 2018
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 22 Feb 2018 10:48:43 +0000
Subject: [nginx] HTTP/2: style.
Message-ID: <hg.aa60f5799a4c.1519296523.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/aa60f5799a4c
branches:  
changeset: 7216:aa60f5799a4c
user:      Ruslan Ermilov <ru at nginx.com>
date:      Thu Feb 22 12:42:29 2018 +0300
description:
HTTP/2: style.

Unified the style of validity checks in ngx_http_v2_validate_header().

diffstat:

 src/http/v2/ngx_http_v2.c |  21 ++++-----------------
 1 files changed, 4 insertions(+), 17 deletions(-)

diffs (38 lines):

diff -r 2dc837d16099 -r aa60f5799a4c src/http/v2/ngx_http_v2.c
--- a/src/http/v2/ngx_http_v2.c	Wed Feb 21 17:26:00 2018 +0300
+++ b/src/http/v2/ngx_http_v2.c	Thu Feb 22 12:42:29 2018 +0300
@@ -3257,19 +3257,9 @@ ngx_http_v2_validate_header(ngx_http_req
             continue;
         }
 
-        switch (ch) {
-        case '\0':
-        case LF:
-        case CR:
-        case ':':
-            ngx_log_error(NGX_LOG_INFO, r->connection->log, 0,
-                          "client sent invalid header name: \"%V\"",
-                          &header->name);
-
-            return NGX_ERROR;
-        }
-
-        if (ch >= 'A' && ch <= 'Z') {
+        if (ch == '\0' || ch == LF || ch == CR || ch == ':'
+            || (ch >= 'A' && ch <= 'Z'))
+        {
             ngx_log_error(NGX_LOG_INFO, r->connection->log, 0,
                           "client sent invalid header name: \"%V\"",
                           &header->name);
@@ -3283,10 +3273,7 @@ ngx_http_v2_validate_header(ngx_http_req
     for (i = 0; i != header->value.len; i++) {
         ch = header->value.data[i];
 
-        switch (ch) {
-        case '\0':
-        case LF:
-        case CR:
+        if (ch == '\0' || ch == LF || ch == CR) {
             ngx_log_error(NGX_LOG_INFO, r->connection->log, 0,
                           "client sent header \"%V\" with "
                           "invalid value: \"%V\"",

From mdounin at mdounin.ru  Thu Feb 22 12:26:09 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 22 Feb 2018 15:26:09 +0300
Subject: Sub-requests poisoning r->variables cache
In-Reply-To: <CAOToG0RtQi9vUbR-qQbKz3E95ziuuX8DAQz-WyGbMctVsAo=PQ@mail.gmail.com>
References: <CAOToG0RtQi9vUbR-qQbKz3E95ziuuX8DAQz-WyGbMctVsAo=PQ@mail.gmail.com>
Message-ID: <20180222122609.GX24410@mdounin.ru>

Hello!

On Thu, Feb 22, 2018 at 11:53:14AM +1100, Lawrence Stewart via nginx-devel wrote:

> Greetings nginx developers,
> 
> The root cause analysis for a bug I observed during development of some
> custom nginx module code turns out to be an unfortunate interaction between
> sub-requests and indexed variable lookups using
> ngx_http_get_indexed_variable(). We're based on nginx 1.13.6.
> 
> The sequence of events is:
> 
> - Main request arrives and triggers an access-phase sub-request. Main
> request includes request variable "x=blah", but sub-request does not.
> 
> - Sub-request arrives at custom module's header filter
> 
> - Custom module looks up variable "x" with ngx_http_get_indexed_variable(),
> and it is not found in the sub-request. This causes the "not_found" flag to
> be set in the sr->variables var cache for variable "x", but
> because ngx_http_subrequest() shallow copies r->variables from parent
> request to sub-request, the not_found flag change affects the main
> request's r->variables too.
> 
> - Sub-request completes, main request is allowed to progress and arrives at
> custom module's header filter
> 
> - Custom module again looks up variable "x"
> with ngx_http_get_indexed_variable(), and even though "x" is present in
> r->args and I've confirmed can be found by
> ngx_http_arg(), ngx_http_get_indexed_variable() sees the not_found flag,
> does not refresh the variable cache and returns the poisoned
> ngx_http_variable_value_t.
> 
> There are ways to deal with this in our custom code, but it seems like a
> fundamental problem in the current design of the sub-request mechanism and
> I'm thinking that it may be appropriate to address this in the nginx core
> code e.g. by deep copying r->variables in ngx_http_subrequest(), or by
> somehow tracking when to invalidate the ngx_http_variable_value_t flags and
> refresh the r->variables cache.
> 
> Thoughts?

Variables are expected to be shared between the main request and 
subrequests.  If a variable can have different values in the main 
request and subrequests, consider marking it non-cacheable.  For 
example, the $arg_* variables are defined as follows:

    { ngx_string("arg_"), NULL, ngx_http_variable_argument,
      0, NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_PREFIX, 0 },

This is because request arguments can be changed, and, in 
particular, can be different in a subrequest.

While the approach with shared variables might not be convenient 
in some scenarious, it is certainly is in many of them - for 
example, you don't need to recalculate various map{} results, you 
can freely use variables in SSI includes (the actual use case 
subrequests were created for), and so on.

Given the above, I don't really think we need to change things 
here.  Suggested changes are likely to break more things they will 
fix.

-- 
Maxim Dounin
http://mdounin.ru/

From Johannes.Baiter at bsb-muenchen.de  Thu Feb 22 16:08:07 2018
From: Johannes.Baiter at bsb-muenchen.de (Johannes Baiter)
Date: Thu, 22 Feb 2018 17:08:07 +0100
Subject: [PATCH 1 of 2] Access log: Support for disabling escaping
Message-ID: <5A8EF8F7020000370008E19E@gwia.bsb-muenchen.de>

This patch adds a "none" option for the "escape" parameter on the
"log_format" directive.

Selecting this option will perform no escaping of variables in the
access log output and write the variable to the file as-is.
The use case for this is more freedom for custom logging setups.

In our case, we use structured logging in JSON. For this, we build a
JSON string in a Lua script that is then stored in a nginx variable.
The log_format in this case is simply the variable:

  # The script sets the $json_log variable
  log_by_lua_file /etc/nginx/lua/scripts/json_logging.lua;
  log_format json $json_log;

  access_log /local/nginx/log/access.log json;
  error_log /local/nginx/log/error.log;

Both "default" and "json" do not work in this case, since either will
result in invalid JSON output.
Updated test suites for both ngx_http and ngx_stream are provided in
the second patch.


# HG changeset patch
# User Johannes Baiter <johannes.baiter at bsb-muenchen.de>
# Date 1519312196 -3600
#      Thu Feb 22 16:09:56 2018 +0100
# Node ID 267e08661efcb58a9b4cd5fd941486b88a37f4f4
# Parent  aa60f5799a4cbfc6887ef099c588e84a75c69785
Access log: support for disabling escaping.

diff -r aa60f5799a4c -r 267e08661efc
src/http/modules/ngx_http_log_module.c
--- a/src/http/modules/ngx_http_log_module.c	Thu Feb 22 12:42:29 2018
+0300
+++ b/src/http/modules/ngx_http_log_module.c	Thu Feb 22 16:09:56 2018
+0100
@@ -90,6 +90,13 @@
 } ngx_http_log_var_t;
 
 
+typedef enum {
+    ngx_http_log_escape_default = 0,
+    ngx_http_log_escape_json,
+    ngx_http_log_escape_none
+} ngx_http_log_escape_t;
+
+
 static void ngx_http_log_write(ngx_http_request_t *r, ngx_http_log_t
*log,
     u_char *buf, size_t len);
 static ssize_t ngx_http_log_script_write(ngx_http_request_t *r,
@@ -126,12 +133,16 @@
     ngx_http_log_op_t *op);
 
 static ngx_int_t ngx_http_log_variable_compile(ngx_conf_t *cf,
-    ngx_http_log_op_t *op, ngx_str_t *value, ngx_uint_t json);
+    ngx_http_log_op_t *op, ngx_str_t *value, ngx_http_log_escape_t
escape);
 static size_t ngx_http_log_variable_getlen(ngx_http_request_t *r,
     uintptr_t data);
 static u_char *ngx_http_log_variable(ngx_http_request_t *r, u_char
*buf,
     ngx_http_log_op_t *op);
 static uintptr_t ngx_http_log_escape(u_char *dst, u_char *src, size_t
size);
+static size_t
ngx_http_log_unescaped_variable_getlen(ngx_http_request_t *r,
+    uintptr_t data);
+static u_char *ngx_http_log_unescaped_variable(ngx_http_request_t *r,
u_char *buf,
+    ngx_http_log_op_t *op);
 static size_t ngx_http_log_json_variable_getlen(ngx_http_request_t
*r,
     uintptr_t data);
 static u_char *ngx_http_log_json_variable(ngx_http_request_t *r,
u_char *buf,
@@ -905,7 +916,7 @@
 
 static ngx_int_t
 ngx_http_log_variable_compile(ngx_conf_t *cf, ngx_http_log_op_t *op,
-    ngx_str_t *value, ngx_uint_t json)
+    ngx_str_t *value, ngx_http_log_escape_t escape)
 {
     ngx_int_t  index;
 
@@ -916,10 +927,12 @@
 
     op->len = 0;
 
-    if (json) {
+    if (escape == ngx_http_log_escape_json) {
         op->getlen = ngx_http_log_json_variable_getlen;
         op->run = ngx_http_log_json_variable;
-
+    } else if (escape == ngx_http_log_escape_none) {
+        op->getlen = ngx_http_log_unescaped_variable_getlen;
+        op->run = ngx_http_log_unescaped_variable;
     } else {
         op->getlen = ngx_http_log_variable_getlen;
         op->run = ngx_http_log_variable;
@@ -1033,6 +1046,38 @@
 
 
 static size_t
+ngx_http_log_unescaped_variable_getlen(ngx_http_request_t *r,
uintptr_t data)
+{
+    ngx_http_variable_value_t  *value;
+
+    value = ngx_http_get_indexed_variable(r, data);
+
+    if (value == NULL || value->not_found) {
+        return 0;
+    }
+
+    value->escape = 0;
+    return value->len;
+}
+
+
+static u_char *
+ngx_http_log_unescaped_variable(ngx_http_request_t *r, u_char *buf,
+                                ngx_http_log_op_t *op)
+{
+    ngx_http_variable_value_t  *value;
+
+    value = ngx_http_get_indexed_variable(r, op->data);
+
+    if (value == NULL || value->not_found) {
+        return buf;
+    }
+
+    return ngx_cpymem(buf, value->data, value->len);
+}
+
+
+static size_t
 ngx_http_log_json_variable_getlen(ngx_http_request_t *r, uintptr_t
data)
 {
     uintptr_t                   len;
@@ -1536,19 +1581,21 @@
     size_t               i, len;
     ngx_str_t           *value, var;
     ngx_int_t           *flush;
-    ngx_uint_t           bracket, json;
+    ngx_uint_t           bracket;
     ngx_http_log_op_t   *op;
     ngx_http_log_var_t  *v;
+    ngx_http_log_escape_t escape;
 
-    json = 0;
+    escape = ngx_http_log_escape_default;
     value = args->elts;
 
     if (s < args->nelts && ngx_strncmp(value[s].data, "escape=", 7) ==
0) {
         data = value[s].data + 7;
 
         if (ngx_strcmp(data, "json") == 0) {
-            json = 1;
-
+            escape = ngx_http_log_escape_json;
+        } else if (ngx_strcmp(data, "none") == 0) {
+            escape = ngx_http_log_escape_none;
         } else if (ngx_strcmp(data, "default") != 0) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                                "unknown log format escaping \"%s\"",
data);
@@ -1636,7 +1683,7 @@
                     }
                 }
 
-                if (ngx_http_log_variable_compile(cf, op, &var, json)
+                if (ngx_http_log_variable_compile(cf, op, &var,
escape)
                     != NGX_OK)
                 {
                     return NGX_CONF_ERROR;
diff -r aa60f5799a4c -r 267e08661efc
src/stream/ngx_stream_log_module.c
--- a/src/stream/ngx_stream_log_module.c	Thu Feb 22 12:42:29 2018
+0300
+++ b/src/stream/ngx_stream_log_module.c	Thu Feb 22 16:09:56 2018
+0100
@@ -89,6 +89,13 @@
 } ngx_stream_log_var_t;
 
 
+typedef enum {
+    ngx_stream_log_escape_default = 0,
+    ngx_stream_log_escape_json,
+    ngx_stream_log_escape_none
+} ngx_stream_log_escape_t;
+
+
 static void ngx_stream_log_write(ngx_stream_session_t *s,
ngx_stream_log_t *log,
     u_char *buf, size_t len);
 static ssize_t ngx_stream_log_script_write(ngx_stream_session_t *s,
@@ -106,7 +113,7 @@
 static void ngx_stream_log_flush_handler(ngx_event_t *ev);
 
 static ngx_int_t ngx_stream_log_variable_compile(ngx_conf_t *cf,
-    ngx_stream_log_op_t *op, ngx_str_t *value, ngx_uint_t json);
+    ngx_stream_log_op_t *op, ngx_str_t *value, ngx_stream_log_escape_t
escape);
 static size_t ngx_stream_log_variable_getlen(ngx_stream_session_t *s,
     uintptr_t data);
 static u_char *ngx_stream_log_variable(ngx_stream_session_t *s, u_char
*buf,
@@ -682,7 +689,7 @@
 
 static ngx_int_t
 ngx_stream_log_variable_compile(ngx_conf_t *cf, ngx_stream_log_op_t
*op,
-    ngx_str_t *value, ngx_uint_t json)
+    ngx_str_t *value, ngx_stream_log_escape_t escape)
 {
     ngx_int_t  index;
 
@@ -693,10 +700,14 @@
 
     op->len = 0;
 
-    if (json) {
+    if (escape == ngx_stream_log_escape_json) {
         op->getlen = ngx_stream_log_json_variable_getlen;
         op->run = ngx_stream_log_json_variable;
 
+    } else if (escape == ngx_stream_log_escape_none) {
+        op->getlen = ngx_stream_log_unescaped_variable_getlen;
+        op->run = ngx_stream_log_unescaped_variable;
+
     } else {
         op->getlen = ngx_stream_log_variable_getlen;
         op->run = ngx_stream_log_variable;
@@ -811,6 +822,38 @@
 
 
 static size_t
+ngx_stream_log_unescaped_variable_getlen(ngx_stream_session_t *s,
uintptr_t data)
+{
+    ngx_stream_variable_value_t  *value;
+
+    value = ngx_stream_get_indexed_variable(s, data);
+
+    if (value == NULL || value->not_found) {
+        return 0;
+    }
+
+    value->escape = 0;
+    return value->len;
+}
+
+
+static u_char *
+ngx_http_log_unescaped_variable(ngx_stream_session_t *s, u_char *buf,
+                                ngx_stream_log_op_t *op)
+{
+    ngx_stream_variable_value_t  *value;
+
+    value = ngx_stream_get_indexed_variable(s, op->data);
+
+    if (value == NULL || value->not_found) {
+        return buf;
+    }
+
+    return ngx_cpymem(buf, value->data, value->len);
+}
+
+
+static size_t
 ngx_stream_log_json_variable_getlen(ngx_stream_session_t *s, uintptr_t
data)
 {
     uintptr_t                     len;
@@ -1265,17 +1308,21 @@
     size_t                 i, len;
     ngx_str_t             *value, var;
     ngx_int_t             *flush;
-    ngx_uint_t             bracket, json;
+    ngx_uint_t             bracket;
     ngx_stream_log_op_t   *op;
+    ngx_stream_log_escape_t escape;
 
-    json = 0;
+    escape = ngx_stream_log_escape_default;
     value = args->elts;
 
     if (s < args->nelts && ngx_strncmp(value[s].data, "escape=", 7) ==
0) {
         data = value[s].data + 7;
 
         if (ngx_strcmp(data, "json") == 0) {
-            json = 1;
+            escape = ngx_stream_log_escape_json;
+
+        } else if (ngx_strcmp(data, "none") == 0) {
+            escape = ngx_stream_log_escape_none;
 
         } else if (ngx_strcmp(data, "default") != 0) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
@@ -1350,7 +1397,7 @@
                     goto invalid;
                 }
 
-                if (ngx_stream_log_variable_compile(cf, op, &var,
json)
+                if (ngx_stream_log_variable_compile(cf, op, &var,
escape)
                     != NGX_OK)
                 {
                     return NGX_CONF_ERROR;


From Johannes.Baiter at bsb-muenchen.de  Thu Feb 22 16:08:12 2018
From: Johannes.Baiter at bsb-muenchen.de (Johannes Baiter)
Date: Thu, 22 Feb 2018 17:08:12 +0100
Subject: [PATCH 2 of 2] Access log: Support for disabling escaping [Tests]
Message-ID: <5A8EF8FC020000370008E1A1@gwia.bsb-muenchen.de>

These are the tests for the new "none" escaping option for the "escape"
parameter in the "log_format" directory.

# HG changeset patch
# User Johannes Baiter <johannes.baiter at bsb-muenchen.de>
# Date 1519314253 -3600
#      Thu Feb 22 16:44:13 2018 +0100
# Node ID 17a622c0d6becdfc60ee24ee80c92d324941208a
# Parent  5ac4aae1a740d165d01b44e8f61c12ee8de967f0
Tests: Update access log escaping tests for 'none' escaping

diff -r 5ac4aae1a740 -r 17a622c0d6be access_log_escape.t
--- a/access_log_escape.t	Wed Feb 21 20:32:07 2018 +0300
+++ b/access_log_escape.t	Thu Feb 22 16:44:13 2018 +0100
@@ -22,7 +22,7 @@
 select STDERR; $| = 1;
 select STDOUT; $| = 1;
 
-my $t = Test::Nginx->new()->has(qw/http/)->plan(2)
+my $t = Test::Nginx->new()->has(qw/http/)->plan(3)
 	->write_file_expand('nginx.conf', <<'EOF');
 
 %%TEST_GLOBALS%%
@@ -35,6 +35,7 @@
 http {
     %%TEST_GLOBALS_HTTP%%
 
+    log_format none     escape=none     $arg_a$arg_b$arg_c;
     log_format json     escape=json     $arg_a$arg_b$arg_c;
     log_format default  escape=default  $arg_a$arg_b$arg_c;
 
@@ -42,6 +43,7 @@
         listen       127.0.0.1:8080;
         server_name  localhost;
 
+        access_log %%TESTDIR%%/none.log none;
         access_log %%TESTDIR%%/json.log json;
         access_log %%TESTDIR%%/test.log default;
     }
@@ -57,6 +59,7 @@
 
 $t->stop();
 
+is($t->read_file('none.log'), '"1 \\ ' . pack("n", 0x1b1c) . ' "2' .
"\n", 'none');
 is($t->read_file('json.log'), '\"1 \\\\ \u001B\u001C \"2' . "\n",
'json');
 is($t->read_file('test.log'), '\x221 \x5C \x1B\x1C \x22-2' . "\n",
'default');
 
diff -r 5ac4aae1a740 -r 17a622c0d6be stream_access_log_escape.t
--- a/stream_access_log_escape.t	Wed Feb 21 20:32:07 2018 +0300
+++ b/stream_access_log_escape.t	Thu Feb 22 16:44:13 2018 +0100
@@ -40,6 +40,7 @@
         default "foo";
     }
 
+    log_format none     escape=none     $a$b$upstream_addr;
     log_format json     escape=json     $a$b$upstream_addr;
     log_format default  escape=default  $a$b$upstream_addr;
 
@@ -47,6 +48,7 @@
         listen       127.0.0.1:8080;
         return       ok;
 
+        access_log %%TESTDIR%%/none.log none;
         access_log %%TESTDIR%%/json.log json;
         access_log %%TESTDIR%%/test.log default;
     }
@@ -54,7 +56,7 @@
 
 EOF
 
-$t->run()->plan(2);
+$t->run()->plan(3);
 

###############################################################################
 
@@ -62,6 +64,7 @@
 
 $t->stop();
 
+is($t->read_file('none.log'), '" \\ "foo' . "\n", 'none');
 is($t->read_file('json.log'), '\" \\\\ \"foo' . "\n", 'json');
 is($t->read_file('test.log'), '\x22 \x5C \x22foo-' . "\n",
'default');
 


From arut at nginx.com  Thu Feb 22 17:06:41 2018
From: arut at nginx.com (Roman Arutyunyan)
Date: Thu, 22 Feb 2018 17:06:41 +0000
Subject: [nginx] Generate error for unsupported IPv6 transparent proxy.
Message-ID: <hg.8b70d4caa505.1519319201.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/8b70d4caa505
branches:  
changeset: 7217:8b70d4caa505
user:      Roman Arutyunyan <arut at nginx.com>
date:      Thu Feb 22 13:16:21 2018 +0300
description:
Generate error for unsupported IPv6 transparent proxy.

On some platforms (for example, Linux with glibc 2.12-2.25) IPv4 transparent
proxying is available, but IPv6 transparent proxying is not.  The entire feature
is enabled in this case and NGX_HAVE_TRANSPARENT_PROXY macro is set to 1.
Previously, an attempt to enable transparency for an IPv6 socket was silently
ignored in this case and was usually followed by a bind(2) EADDRNOTAVAIL error
(ticket #1487).  Now the error is generated for unavailable IPv6 transparent
proxy.

diffstat:

 src/event/ngx_event_connect.c |  9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diffs (20 lines):

diff -r aa60f5799a4c -r 8b70d4caa505 src/event/ngx_event_connect.c
--- a/src/event/ngx_event_connect.c	Thu Feb 22 12:42:29 2018 +0300
+++ b/src/event/ngx_event_connect.c	Thu Feb 22 13:16:21 2018 +0300
@@ -388,7 +388,16 @@ ngx_event_connect_set_transparent(ngx_pe
             return NGX_ERROR;
         }
 
+#else
+
+        ngx_log_error(NGX_LOG_ALERT, pc->log, 0,
+                      "could not enable transparent proxying for IPv6 "
+                      "on this platform");
+
+        return NGX_ERROR;
+
 #endif
+
         break;
 
 #endif /* NGX_HAVE_INET6 */

From Johannes.Baiter at bsb-muenchen.de  Thu Feb 22 17:12:52 2018
From: Johannes.Baiter at bsb-muenchen.de (Johannes Baiter)
Date: Thu, 22 Feb 2018 18:12:52 +0100
Subject: Antw: [PATCH 1 of 2] Access log: Support for disabling escaping
In-Reply-To: <5A8EF8F7020000370008E19E@gwia.bsb-muenchen.de>
References: <5A8EF8F7020000370008E19E@gwia.bsb-muenchen.de>
Message-ID: <5A8F0824020000370008E1B6@gwia.bsb-muenchen.de>

Sorry, I accidentally submitted an incomplete version of the patch.
Here is the corrected version.

# HG changeset patch
# User Johannes Baiter <johannes.baiter at bsb-muenchen.de>
# Date 1519312196 -3600
#      Thu Feb 22 16:09:56 2018 +0100
# Node ID 6c1e7ddade7dd75677cb2821f346e862866ba4e3
# Parent  aa60f5799a4cbfc6887ef099c588e84a75c69785
Access log: support for disabling escaping.

diff -r aa60f5799a4c -r 6c1e7ddade7d
src/http/modules/ngx_http_log_module.c
--- a/src/http/modules/ngx_http_log_module.c	Thu Feb 22 12:42:29 2018
+0300
+++ b/src/http/modules/ngx_http_log_module.c	Thu Feb 22 16:09:56 2018
+0100
@@ -90,6 +90,13 @@
 } ngx_http_log_var_t;
 
 
+typedef enum {
+    ngx_http_log_escape_default = 0,
+    ngx_http_log_escape_json,
+    ngx_http_log_escape_none
+} ngx_http_log_escape_t;
+
+
 static void ngx_http_log_write(ngx_http_request_t *r, ngx_http_log_t
*log,
     u_char *buf, size_t len);
 static ssize_t ngx_http_log_script_write(ngx_http_request_t *r,
@@ -126,12 +133,16 @@
     ngx_http_log_op_t *op);
 
 static ngx_int_t ngx_http_log_variable_compile(ngx_conf_t *cf,
-    ngx_http_log_op_t *op, ngx_str_t *value, ngx_uint_t json);
+    ngx_http_log_op_t *op, ngx_str_t *value, ngx_http_log_escape_t
escape);
 static size_t ngx_http_log_variable_getlen(ngx_http_request_t *r,
     uintptr_t data);
 static u_char *ngx_http_log_variable(ngx_http_request_t *r, u_char
*buf,
     ngx_http_log_op_t *op);
 static uintptr_t ngx_http_log_escape(u_char *dst, u_char *src, size_t
size);
+static size_t
ngx_http_log_unescaped_variable_getlen(ngx_http_request_t *r,
+    uintptr_t data);
+static u_char *ngx_http_log_unescaped_variable(ngx_http_request_t *r,
u_char *buf,
+    ngx_http_log_op_t *op);
 static size_t ngx_http_log_json_variable_getlen(ngx_http_request_t
*r,
     uintptr_t data);
 static u_char *ngx_http_log_json_variable(ngx_http_request_t *r,
u_char *buf,
@@ -905,7 +916,7 @@
 
 static ngx_int_t
 ngx_http_log_variable_compile(ngx_conf_t *cf, ngx_http_log_op_t *op,
-    ngx_str_t *value, ngx_uint_t json)
+    ngx_str_t *value, ngx_http_log_escape_t escape)
 {
     ngx_int_t  index;
 
@@ -916,10 +927,14 @@
 
     op->len = 0;
 
-    if (json) {
+    if (escape == ngx_http_log_escape_json) {
         op->getlen = ngx_http_log_json_variable_getlen;
         op->run = ngx_http_log_json_variable;
 
+    } else if (escape == ngx_http_log_escape_none) {
+        op->getlen = ngx_http_log_unescaped_variable_getlen;
+        op->run = ngx_http_log_unescaped_variable;
+
     } else {
         op->getlen = ngx_http_log_variable_getlen;
         op->run = ngx_http_log_variable;
@@ -1033,6 +1048,38 @@
 
 
 static size_t
+ngx_http_log_unescaped_variable_getlen(ngx_http_request_t *r,
uintptr_t data)
+{
+    ngx_http_variable_value_t  *value;
+
+    value = ngx_http_get_indexed_variable(r, data);
+
+    if (value == NULL || value->not_found) {
+        return 0;
+    }
+
+    value->escape = 0;
+    return value->len;
+}
+
+
+static u_char *
+ngx_http_log_unescaped_variable(ngx_http_request_t *r, u_char *buf,
+                                ngx_http_log_op_t *op)
+{
+    ngx_http_variable_value_t  *value;
+
+    value = ngx_http_get_indexed_variable(r, op->data);
+
+    if (value == NULL || value->not_found) {
+        return buf;
+    }
+
+    return ngx_cpymem(buf, value->data, value->len);
+}
+
+
+static size_t
 ngx_http_log_json_variable_getlen(ngx_http_request_t *r, uintptr_t
data)
 {
     uintptr_t                   len;
@@ -1536,19 +1583,21 @@
     size_t               i, len;
     ngx_str_t           *value, var;
     ngx_int_t           *flush;
-    ngx_uint_t           bracket, json;
+    ngx_uint_t           bracket;
     ngx_http_log_op_t   *op;
     ngx_http_log_var_t  *v;
+    ngx_http_log_escape_t escape;
 
-    json = 0;
+    escape = ngx_http_log_escape_default;
     value = args->elts;
 
     if (s < args->nelts && ngx_strncmp(value[s].data, "escape=", 7) ==
0) {
         data = value[s].data + 7;
 
         if (ngx_strcmp(data, "json") == 0) {
-            json = 1;
-
+            escape = ngx_http_log_escape_json;
+        } else if (ngx_strcmp(data, "none") == 0) {
+            escape = ngx_http_log_escape_none;
         } else if (ngx_strcmp(data, "default") != 0) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                                "unknown log format escaping \"%s\"",
data);
@@ -1636,7 +1685,7 @@
                     }
                 }
 
-                if (ngx_http_log_variable_compile(cf, op, &var, json)
+                if (ngx_http_log_variable_compile(cf, op, &var,
escape)
                     != NGX_OK)
                 {
                     return NGX_CONF_ERROR;
diff -r aa60f5799a4c -r 6c1e7ddade7d
src/stream/ngx_stream_log_module.c
--- a/src/stream/ngx_stream_log_module.c	Thu Feb 22 12:42:29 2018
+0300
+++ b/src/stream/ngx_stream_log_module.c	Thu Feb 22 16:09:56 2018
+0100
@@ -89,6 +89,13 @@
 } ngx_stream_log_var_t;
 
 
+typedef enum {
+    ngx_stream_log_escape_default = 0,
+    ngx_stream_log_escape_json,
+    ngx_stream_log_escape_none
+} ngx_stream_log_escape_t;
+
+
 static void ngx_stream_log_write(ngx_stream_session_t *s,
ngx_stream_log_t *log,
     u_char *buf, size_t len);
 static ssize_t ngx_stream_log_script_write(ngx_stream_session_t *s,
@@ -106,12 +113,16 @@
 static void ngx_stream_log_flush_handler(ngx_event_t *ev);
 
 static ngx_int_t ngx_stream_log_variable_compile(ngx_conf_t *cf,
-    ngx_stream_log_op_t *op, ngx_str_t *value, ngx_uint_t json);
+    ngx_stream_log_op_t *op, ngx_str_t *value, ngx_stream_log_escape_t
escape);
 static size_t ngx_stream_log_variable_getlen(ngx_stream_session_t *s,
     uintptr_t data);
 static u_char *ngx_stream_log_variable(ngx_stream_session_t *s, u_char
*buf,
     ngx_stream_log_op_t *op);
 static uintptr_t ngx_stream_log_escape(u_char *dst, u_char *src,
size_t size);
+static size_t
ngx_stream_log_unescaped_variable_getlen(ngx_stream_session_t *s,
+    uintptr_t data);
+static u_char *ngx_stream_log_unescaped_variable(ngx_stream_session_t
*s,
+    u_char *buf, ngx_stream_log_op_t *op);
 static size_t ngx_stream_log_json_variable_getlen(ngx_stream_session_t
*s,
     uintptr_t data);
 static u_char *ngx_stream_log_json_variable(ngx_stream_session_t *s,
@@ -682,7 +693,7 @@
 
 static ngx_int_t
 ngx_stream_log_variable_compile(ngx_conf_t *cf, ngx_stream_log_op_t
*op,
-    ngx_str_t *value, ngx_uint_t json)
+    ngx_str_t *value, ngx_stream_log_escape_t escape)
 {
     ngx_int_t  index;
 
@@ -693,10 +704,14 @@
 
     op->len = 0;
 
-    if (json) {
+    if (escape == ngx_stream_log_escape_json) {
         op->getlen = ngx_stream_log_json_variable_getlen;
         op->run = ngx_stream_log_json_variable;
 
+    } else if (escape == ngx_stream_log_escape_none) {
+        op->getlen = ngx_stream_log_unescaped_variable_getlen;
+        op->run = ngx_stream_log_unescaped_variable;
+
     } else {
         op->getlen = ngx_stream_log_variable_getlen;
         op->run = ngx_stream_log_variable;
@@ -811,6 +826,38 @@
 
 
 static size_t
+ngx_stream_log_unescaped_variable_getlen(ngx_stream_session_t *s,
uintptr_t data)
+{
+    ngx_stream_variable_value_t  *value;
+
+    value = ngx_stream_get_indexed_variable(s, data);
+
+    if (value == NULL || value->not_found) {
+        return 0;
+    }
+
+    value->escape = 0;
+    return value->len;
+}
+
+
+static u_char *
+ngx_stream_log_unescaped_variable(ngx_stream_session_t *s, u_char
*buf,
+                                  ngx_stream_log_op_t *op)
+{
+    ngx_stream_variable_value_t  *value;
+
+    value = ngx_stream_get_indexed_variable(s, op->data);
+
+    if (value == NULL || value->not_found) {
+        return buf;
+    }
+
+    return ngx_cpymem(buf, value->data, value->len);
+}
+
+
+static size_t
 ngx_stream_log_json_variable_getlen(ngx_stream_session_t *s, uintptr_t
data)
 {
     uintptr_t                     len;
@@ -1265,17 +1312,21 @@
     size_t                 i, len;
     ngx_str_t             *value, var;
     ngx_int_t             *flush;
-    ngx_uint_t             bracket, json;
+    ngx_uint_t             bracket;
     ngx_stream_log_op_t   *op;
+    ngx_stream_log_escape_t escape;
 
-    json = 0;
+    escape = ngx_stream_log_escape_default;
     value = args->elts;
 
     if (s < args->nelts && ngx_strncmp(value[s].data, "escape=", 7) ==
0) {
         data = value[s].data + 7;
 
         if (ngx_strcmp(data, "json") == 0) {
-            json = 1;
+            escape = ngx_stream_log_escape_json;
+
+        } else if (ngx_strcmp(data, "none") == 0) {
+            escape = ngx_stream_log_escape_none;
 
         } else if (ngx_strcmp(data, "default") != 0) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
@@ -1350,7 +1401,7 @@
                     goto invalid;
                 }
 
-                if (ngx_stream_log_variable_compile(cf, op, &var,
json)
+                if (ngx_stream_log_variable_compile(cf, op, &var,
escape)
                     != NGX_OK)
                 {
                     return NGX_CONF_ERROR; 
 
>>> "Johannes Baiter" <Johannes.Baiter at bsb-muenchen.de> 2/22/2018 5:08
PM >>> 
This patch adds a "none" option for the "escape" parameter on the
"log_format" directive.

Selecting this option will perform no escaping of variables in the
access log output and write the variable to the file as-is.
The use case for this is more freedom for custom logging setups.

In our case, we use structured logging in JSON. For this, we build a
JSON string in a Lua script that is then stored in a nginx variable.
The log_format in this case is simply the variable:

  # The script sets the $json_log variable
  log_by_lua_file /etc/nginx/lua/scripts/json_logging.lua;
  log_format json $json_log;

  access_log /local/nginx/log/access.log json;
  error_log /local/nginx/log/error.log;

Both "default" and "json" do not work in this case, since either will
result in invalid JSON output.
Updated test suites for both ngx_http and ngx_stream are provided in
the second patch.


# HG changeset patch
# User Johannes Baiter <johannes.baiter at bsb-muenchen.de>
# Date 1519312196 -3600
#      Thu Feb 22 16:09:56 2018 +0100
# Node ID 267e08661efcb58a9b4cd5fd941486b88a37f4f4
# Parent  aa60f5799a4cbfc6887ef099c588e84a75c69785
Access log: support for disabling escaping.

diff -r aa60f5799a4c -r 267e08661efc
src/http/modules/ngx_http_log_module.c
--- a/src/http/modules/ngx_http_log_module.c	Thu Feb 22 12:42:29
2018
+0300
+++ b/src/http/modules/ngx_http_log_module.c	Thu Feb 22 16:09:56
2018
+0100
@@ -90,6 +90,13 @@
 } ngx_http_log_var_t;
 
 
+typedef enum {
+    ngx_http_log_escape_default = 0,
+    ngx_http_log_escape_json,
+    ngx_http_log_escape_none
+} ngx_http_log_escape_t;
+
+
 static void ngx_http_log_write(ngx_http_request_t *r, ngx_http_log_t
*log,
     u_char *buf, size_t len);
 static ssize_t ngx_http_log_script_write(ngx_http_request_t *r,
@@ -126,12 +133,16 @@
     ngx_http_log_op_t *op);
 
 static ngx_int_t ngx_http_log_variable_compile(ngx_conf_t *cf,
-    ngx_http_log_op_t *op, ngx_str_t *value, ngx_uint_t json);
+    ngx_http_log_op_t *op, ngx_str_t *value, ngx_http_log_escape_t
escape);
 static size_t ngx_http_log_variable_getlen(ngx_http_request_t *r,
     uintptr_t data);
 static u_char *ngx_http_log_variable(ngx_http_request_t *r, u_char
*buf,
     ngx_http_log_op_t *op);
 static uintptr_t ngx_http_log_escape(u_char *dst, u_char *src, size_t
size);
+static size_t
ngx_http_log_unescaped_variable_getlen(ngx_http_request_t *r,
+    uintptr_t data);
+static u_char *ngx_http_log_unescaped_variable(ngx_http_request_t *r,
u_char *buf,
+    ngx_http_log_op_t *op);
 static size_t ngx_http_log_json_variable_getlen(ngx_http_request_t
*r,
     uintptr_t data);
 static u_char *ngx_http_log_json_variable(ngx_http_request_t *r,
u_char *buf,
@@ -905,7 +916,7 @@
 
 static ngx_int_t
 ngx_http_log_variable_compile(ngx_conf_t *cf, ngx_http_log_op_t *op,
-    ngx_str_t *value, ngx_uint_t json)
+    ngx_str_t *value, ngx_http_log_escape_t escape)
 {
     ngx_int_t  index;
 
@@ -916,10 +927,12 @@
 
     op->len = 0;
 
-    if (json) {
+    if (escape == ngx_http_log_escape_json) {
         op->getlen = ngx_http_log_json_variable_getlen;
         op->run = ngx_http_log_json_variable;
-
+    } else if (escape == ngx_http_log_escape_none) {
+        op->getlen = ngx_http_log_unescaped_variable_getlen;
+        op->run = ngx_http_log_unescaped_variable;
     } else {
         op->getlen = ngx_http_log_variable_getlen;
         op->run = ngx_http_log_variable;
@@ -1033,6 +1046,38 @@
 
 
 static size_t
+ngx_http_log_unescaped_variable_getlen(ngx_http_request_t *r,
uintptr_t data)
+{
+    ngx_http_variable_value_t  *value;
+
+    value = ngx_http_get_indexed_variable(r, data);
+
+    if (value == NULL || value->not_found) {
+        return 0;
+    }
+
+    value->escape = 0;
+    return value->len;
+}
+
+
+static u_char *
+ngx_http_log_unescaped_variable(ngx_http_request_t *r, u_char *buf,
+                                ngx_http_log_op_t *op)
+{
+    ngx_http_variable_value_t  *value;
+
+    value = ngx_http_get_indexed_variable(r, op->data);
+
+    if (value == NULL || value->not_found) {
+        return buf;
+    }
+
+    return ngx_cpymem(buf, value->data, value->len);
+}
+
+
+static size_t
 ngx_http_log_json_variable_getlen(ngx_http_request_t *r, uintptr_t
data)
 {
     uintptr_t                   len;
@@ -1536,19 +1581,21 @@
     size_t               i, len;
     ngx_str_t           *value, var;
     ngx_int_t           *flush;
-    ngx_uint_t           bracket, json;
+    ngx_uint_t           bracket;
     ngx_http_log_op_t   *op;
     ngx_http_log_var_t  *v;
+    ngx_http_log_escape_t escape;
 
-    json = 0;
+    escape = ngx_http_log_escape_default;
     value = args->elts;
 
     if (s < args->nelts && ngx_strncmp(value[s].data, "escape=", 7)
==
0) {
         data = value[s].data + 7;
 
         if (ngx_strcmp(data, "json") == 0) {
-            json = 1;
-
+            escape = ngx_http_log_escape_json;
+        } else if (ngx_strcmp(data, "none") == 0) {
+            escape = ngx_http_log_escape_none;
         } else if (ngx_strcmp(data, "default") != 0) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                                "unknown log format escaping \"%s\"",
data);
@@ -1636,7 +1683,7 @@
                     }
                 }
 
-                if (ngx_http_log_variable_compile(cf, op, &var, json)
+                if (ngx_http_log_variable_compile(cf, op, &var,
escape)
                     != NGX_OK)
                 {
                     return NGX_CONF_ERROR;
diff -r aa60f5799a4c -r 267e08661efc
src/stream/ngx_stream_log_module.c
--- a/src/stream/ngx_stream_log_module.c	Thu Feb 22 12:42:29
2018
+0300
+++ b/src/stream/ngx_stream_log_module.c	Thu Feb 22 16:09:56
2018
+0100
@@ -89,6 +89,13 @@
 } ngx_stream_log_var_t;
 
 
+typedef enum {
+    ngx_stream_log_escape_default = 0,
+    ngx_stream_log_escape_json,
+    ngx_stream_log_escape_none
+} ngx_stream_log_escape_t;
+
+
 static void ngx_stream_log_write(ngx_stream_session_t *s,
ngx_stream_log_t *log,
     u_char *buf, size_t len);
 static ssize_t ngx_stream_log_script_write(ngx_stream_session_t *s,
@@ -106,7 +113,7 @@
 static void ngx_stream_log_flush_handler(ngx_event_t *ev);
 
 static ngx_int_t ngx_stream_log_variable_compile(ngx_conf_t *cf,
-    ngx_stream_log_op_t *op, ngx_str_t *value, ngx_uint_t json);
+    ngx_stream_log_op_t *op, ngx_str_t *value,
ngx_stream_log_escape_t
escape);
 static size_t ngx_stream_log_variable_getlen(ngx_stream_session_t *s,
     uintptr_t data);
 static u_char *ngx_stream_log_variable(ngx_stream_session_t *s,
u_char
*buf,
@@ -682,7 +689,7 @@
 
 static ngx_int_t
 ngx_stream_log_variable_compile(ngx_conf_t *cf, ngx_stream_log_op_t
*op,
-    ngx_str_t *value, ngx_uint_t json)
+    ngx_str_t *value, ngx_stream_log_escape_t escape)
 {
     ngx_int_t  index;
 
@@ -693,10 +700,14 @@
 
     op->len = 0;
 
-    if (json) {
+    if (escape == ngx_stream_log_escape_json) {
         op->getlen = ngx_stream_log_json_variable_getlen;
         op->run = ngx_stream_log_json_variable;
 
+    } else if (escape == ngx_stream_log_escape_none) {
+        op->getlen = ngx_stream_log_unescaped_variable_getlen;
+        op->run = ngx_stream_log_unescaped_variable;
+
     } else {
         op->getlen = ngx_stream_log_variable_getlen;
         op->run = ngx_stream_log_variable;
@@ -811,6 +822,38 @@
 
 
 static size_t
+ngx_stream_log_unescaped_variable_getlen(ngx_stream_session_t *s,
uintptr_t data)
+{
+    ngx_stream_variable_value_t  *value;
+
+    value = ngx_stream_get_indexed_variable(s, data);
+
+    if (value == NULL || value->not_found) {
+        return 0;
+    }
+
+    value->escape = 0;
+    return value->len;
+}
+
+
+static u_char *
+ngx_http_log_unescaped_variable(ngx_stream_session_t *s, u_char *buf,
+                                ngx_stream_log_op_t *op)
+{
+    ngx_stream_variable_value_t  *value;
+
+    value = ngx_stream_get_indexed_variable(s, op->data);
+
+    if (value == NULL || value->not_found) {
+        return buf;
+    }
+
+    return ngx_cpymem(buf, value->data, value->len);
+}
+
+
+static size_t
 ngx_stream_log_json_variable_getlen(ngx_stream_session_t *s,
uintptr_t
data)
 {
     uintptr_t                     len;
@@ -1265,17 +1308,21 @@
     size_t                 i, len;
     ngx_str_t             *value, var;
     ngx_int_t             *flush;
-    ngx_uint_t             bracket, json;
+    ngx_uint_t             bracket;
     ngx_stream_log_op_t   *op;
+    ngx_stream_log_escape_t escape;
 
-    json = 0;
+    escape = ngx_stream_log_escape_default;
     value = args->elts;
 
     if (s < args->nelts && ngx_strncmp(value[s].data, "escape=", 7)
==
0) {
         data = value[s].data + 7;
 
         if (ngx_strcmp(data, "json") == 0) {
-            json = 1;
+            escape = ngx_stream_log_escape_json;
+
+        } else if (ngx_strcmp(data, "none") == 0) {
+            escape = ngx_stream_log_escape_none;
 
         } else if (ngx_strcmp(data, "default") != 0) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
@@ -1350,7 +1397,7 @@
                     goto invalid;
                 }
 
-                if (ngx_stream_log_variable_compile(cf, op, &var,
json)
+                if (ngx_stream_log_variable_compile(cf, op, &var,
escape)
                     != NGX_OK)
                 {
                     return NGX_CONF_ERROR;

_______________________________________________
nginx-devel mailing list
nginx-devel at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel


From fffilimonov at yandex.ru  Mon Feb 26 15:01:02 2018
From: fffilimonov at yandex.ru (Filimonov Vadim)
Date: Mon, 26 Feb 2018 18:01:02 +0300
Subject: prevent null character in error log
Message-ID: <182581519657262@web29o.yandex.ru>

# HG changeset patch
# User Vadim Filimonov <fffilimonov at yandex.ru>
# Date 1519656740 -7200
#      Mon Feb 26 16:52:20 2018 +0200
# Node ID 00de93c2804c0679adc33dee25cb948e61c04c05
# Parent  8b70d4caa505656bb5ee3f95246c4f39b62354ef
prevent null character in error log
https://trac.nginx.org/nginx/ticket/1494

diff -r 8b70d4caa505 -r 00de93c2804c src/http/modules/ngx_http_auth_basic_module.c
--- a/src/http/modules/ngx_http_auth_basic_module.c     Thu Feb 22 13:16:21 2018 +0300
+++ b/src/http/modules/ngx_http_auth_basic_module.c     Mon Feb 26 16:52:20 2018 +0200
@@ -266,8 +266,8 @@
     }

     ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
-                  "user \"%V\" was not found in \"%V\"",
-                  &r->headers_in.user, &user_file);
+                  "user \"%V\" was not found in \"%s\"",
+                  &r->headers_in.user, user_file.data);

     return ngx_http_auth_basic_set_realm(r, &realm);
 }

From mdounin at mdounin.ru  Mon Feb 26 17:52:13 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 26 Feb 2018 17:52:13 +0000
Subject: [nginx] Auth basic: prevent null character in error log (ticket
 #1494).
Message-ID: <hg.e48ac0136ee3.1519667533.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/e48ac0136ee3
branches:  
changeset: 7218:e48ac0136ee3
user:      Vadim Filimonov <fffilimonov at yandex.ru>
date:      Mon Feb 26 16:52:20 2018 +0200
description:
Auth basic: prevent null character in error log (ticket #1494).

diffstat:

 src/http/modules/ngx_http_auth_basic_module.c |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff --git a/src/http/modules/ngx_http_auth_basic_module.c b/src/http/modules/ngx_http_auth_basic_module.c
--- a/src/http/modules/ngx_http_auth_basic_module.c
+++ b/src/http/modules/ngx_http_auth_basic_module.c
@@ -266,8 +266,8 @@ ngx_http_auth_basic_handler(ngx_http_req
     }
 
     ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
-                  "user \"%V\" was not found in \"%V\"",
-                  &r->headers_in.user, &user_file);
+                  "user \"%V\" was not found in \"%s\"",
+                  &r->headers_in.user, user_file.data);
 
     return ngx_http_auth_basic_set_realm(r, &realm);
 }

From mdounin at mdounin.ru  Mon Feb 26 17:52:20 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 26 Feb 2018 20:52:20 +0300
Subject: prevent null character in error log
In-Reply-To: <182581519657262@web29o.yandex.ru>
References: <182581519657262@web29o.yandex.ru>
Message-ID: <20180226175220.GE89840@mdounin.ru>

Hello!

On Mon, Feb 26, 2018 at 06:01:02PM +0300, Filimonov Vadim wrote:

> # HG changeset patch
> # User Vadim Filimonov <fffilimonov at yandex.ru>
> # Date 1519656740 -7200
> #      Mon Feb 26 16:52:20 2018 +0200
> # Node ID 00de93c2804c0679adc33dee25cb948e61c04c05
> # Parent  8b70d4caa505656bb5ee3f95246c4f39b62354ef
> prevent null character in error log
> https://trac.nginx.org/nginx/ticket/1494

Should be something like:

Auth basic: prevent null character in error log (ticket #1494).

> 
> diff -r 8b70d4caa505 -r 00de93c2804c src/http/modules/ngx_http_auth_basic_module.c
> --- a/src/http/modules/ngx_http_auth_basic_module.c     Thu Feb 22 13:16:21 2018 +0300
> +++ b/src/http/modules/ngx_http_auth_basic_module.c     Mon Feb 26 16:52:20 2018 +0200
> @@ -266,8 +266,8 @@
>      }
> 
>      ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
> -                  "user \"%V\" was not found in \"%V\"",
> -                  &r->headers_in.user, &user_file);
> +                  "user \"%V\" was not found in \"%s\"",
> +                  &r->headers_in.user, user_file.data);
> 
>      return ngx_http_auth_basic_set_realm(r, &realm);
>  }

Thanks, committed with the above commit log changes.

The problem is that ccv.zero flag, as set during 
ngx_http_compile_complex_value() to produce null-terminated 
string, includes terminating null character into the string 
length.

Another solution would be to change/fix ccv.zero flag to produce 
silently zero-terminated string without including terminating null 
into the string length, though this would be a much more complex 
change.

-- 
Maxim Dounin
http://mdounin.ru/

From xeioex at nginx.com  Tue Feb 27 11:18:06 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 27 Feb 2018 11:18:06 +0000
Subject: [njs] Fixed memory leak in CLI.
Message-ID: <hg.ee00992356d9.1519730286.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/ee00992356d9
branches:  
changeset: 451:ee00992356d9
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Feb 27 14:10:36 2018 +0300
description:
Fixed memory leak in CLI.

diffstat:

 njs/njs.c |  8 ++++++--
 1 files changed, 6 insertions(+), 2 deletions(-)

diffs (25 lines):

diff -r 757271547b56 -r ee00992356d9 njs/njs.c
--- a/njs/njs.c	Tue Feb 20 19:12:55 2018 +0300
+++ b/njs/njs.c	Tue Feb 27 14:10:36 2018 +0300
@@ -253,6 +253,7 @@ njs_interactive_shell(njs_opts_t *opts, 
     }
 
     if (njs_externals_init(vm) != NXT_OK) {
+        fprintf(stderr, "failed to add external protos\n");
         return NXT_ERROR;
     }
 
@@ -389,8 +390,11 @@ njs_process_file(njs_opts_t *opts, njs_v
         goto done;
     }
 
-    if (njs_externals_init(vm) != NXT_OK) {
-        return NXT_ERROR;
+    ret = njs_externals_init(vm);
+    if (ret != NXT_OK) {
+        fprintf(stderr, "failed to add external protos\n");
+        ret = NXT_ERROR;
+        goto done;
     }
 
     ret = njs_process_script(vm, opts, &script, &out);

From xeioex at nginx.com  Tue Feb 27 11:18:06 2018
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 27 Feb 2018 11:18:06 +0000
Subject: [njs] Fixed the unit test's output format for a failed test.
Message-ID: <hg.0f1c3efcd894.1519730286.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/0f1c3efcd894
branches:  
changeset: 452:0f1c3efcd894
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Feb 27 14:11:00 2018 +0300
description:
Fixed the unit test's output format for a failed test.

diffstat:

 njs/test/njs_unit_test.c |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff -r ee00992356d9 -r 0f1c3efcd894 njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Tue Feb 27 14:10:36 2018 +0300
+++ b/njs/test/njs_unit_test.c	Tue Feb 27 14:11:00 2018 +0300
@@ -9466,8 +9466,8 @@ njs_unit_test(nxt_bool_t disassemble, nx
 
         printf("njs(\"%.*s\")\nexpected: \"%.*s\"\n     got: \"%.*s\"\n",
                (int) njs_test[i].script.length, njs_test[i].script.start,
-               (int) s.length, s.start, (int) njs_test[i].ret.length,
-               njs_test[i].ret.start);
+               (int) njs_test[i].ret.length, njs_test[i].ret.start,
+               (int) s.length, s.start);
 
         goto done;
     }

From vl at nginx.com  Tue Feb 27 14:16:53 2018
From: vl at nginx.com (Vladimir Homutov)
Date: Tue, 27 Feb 2018 14:16:53 +0000
Subject: [nginx] Modules compatibility: additional upstream metrics.
Message-ID: <hg.d0d32b33167d.1519741013.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/d0d32b33167d
branches:  
changeset: 7219:d0d32b33167d
user:      Vladimir Homutov <vl at nginx.com>
date:      Thu Feb 22 17:25:43 2018 +0300
description:
Modules compatibility: additional upstream metrics.

diffstat:

 src/http/ngx_http_upstream.h |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (11 lines):

diff -r e48ac0136ee3 -r d0d32b33167d src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h	Mon Feb 26 16:52:20 2018 +0200
+++ b/src/http/ngx_http_upstream.h	Thu Feb 22 17:25:43 2018 +0300
@@ -61,6 +61,7 @@ typedef struct {
     ngx_msec_t                       response_time;
     ngx_msec_t                       connect_time;
     ngx_msec_t                       header_time;
+    ngx_msec_t                       queue_time;
     off_t                            response_length;
     off_t                            bytes_received;
 

From cbranch at cloudflare.com  Tue Feb 27 22:32:40 2018
From: cbranch at cloudflare.com (Chris Branch)
Date: Tue, 27 Feb 2018 22:32:40 +0000
Subject: [PATCH] $request_scheme variable
In-Reply-To: <05f555d65a33ebf005fe.1488196710@cbranch-vm.localdomain>
References: <05f555d65a33ebf005fe.1488196710@cbranch-vm.localdomain>
Message-ID: <3ABC4237-9C2F-47E5-B075-CEE6FA874F9C@cloudflare.com>

Hi, just giving this patch some birthday bumps.

> On 27 Feb 2017, at 11:58, Chris Branch via nginx-devel <nginx-devel at nginx.org> wrote:
> 
> # HG changeset patch
> # User Chris Branch <cbranch at cloudflare.com>
> # Date 1488195909 0
> #      Mon Feb 27 11:45:09 2017 +0000
> # Node ID 05f555d65a33ebf005fedc569fb52eba3758e1d7
> # Parent  87cf6ddb41c216876d13cffa5e637a61b159362c
> $request_scheme variable.
> 
> Contains the URI scheme supplied by the client. If no scheme supplied,
> equivalent to $scheme.
> 
> Scheme can be supplied by the client in two ways:
> 
> * HTTP/2 :scheme pseudo-header.
> * HTTP/1 absolute URI in request line.
> 
> diff -r 87cf6ddb41c2 -r 05f555d65a33 src/http/ngx_http_variables.c
> --- a/src/http/ngx_http_variables.c	Fri Feb 17 17:01:27 2017 +0300
> +++ b/src/http/ngx_http_variables.c	Mon Feb 27 11:45:09 2017 +0000
> @@ -105,6 +105,8 @@
>     ngx_http_variable_value_t *v, uintptr_t data);
> static ngx_int_t ngx_http_variable_request_id(ngx_http_request_t *r,
>     ngx_http_variable_value_t *v, uintptr_t data);
> +static ngx_int_t ngx_http_variable_request_scheme(ngx_http_request_t *r,
> +    ngx_http_variable_value_t *v, uintptr_t data);
> static ngx_int_t ngx_http_variable_status(ngx_http_request_t *r,
>     ngx_http_variable_value_t *v, uintptr_t data);
> 
> @@ -288,6 +290,10 @@
>       ngx_http_variable_request_id,
>       0, 0, 0 },
> 
> +    { ngx_string("request_scheme"), NULL,
> +      ngx_http_variable_request_scheme,
> +      0, 0, 0 },
> +
>     { ngx_string("status"), NULL,
>       ngx_http_variable_status, 0,
>       NGX_HTTP_VAR_NOCACHEABLE, 0 },
> @@ -2158,6 +2164,24 @@
> 
> 
> static ngx_int_t
> +ngx_http_variable_request_scheme(ngx_http_request_t *r,
> +    ngx_http_variable_value_t *v, uintptr_t data)
> +{
> +    if (r->schema_start) {
> +        v->len = r->schema_end - r->schema_start;
> +        v->valid = 1;
> +        v->no_cacheable = 0;
> +        v->not_found = 0;
> +        v->data = r->schema_start;
> +
> +        return NGX_OK;
> +    }
> +
> +    return ngx_http_variable_scheme(r, v, data);
> +}
> +
> +
> +static ngx_int_t
> ngx_http_variable_connection(ngx_http_request_t *r,
>     ngx_http_variable_value_t *v, uintptr_t data)
> {
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel


From igor at sysoev.ru  Wed Feb 28 13:20:25 2018
From: igor at sysoev.ru (Igor Sysoev)
Date: Wed, 28 Feb 2018 13:20:25 +0000
Subject: [njs] Fixed String.prototype.toUTF8() function.
Message-ID: <hg.ab1f67b69707.1519824025.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/ab1f67b69707
branches:  
changeset: 453:ab1f67b69707
user:      Igor Sysoev <igor at sysoev.ru>
date:      Wed Feb 28 16:20:11 2018 +0300
description:
Fixed String.prototype.toUTF8() function.

A byte string returned by String.prototype.toUTF8() had length equal
to its size so the string can be processed later as an ASCII string.

diffstat:

 njs/njs_string.c         |  5 +++++
 njs/test/njs_unit_test.c |  9 +++++++++
 2 files changed, 14 insertions(+), 0 deletions(-)

diffs (34 lines):

diff -r 0f1c3efcd894 -r ab1f67b69707 njs/njs_string.c
--- a/njs/njs_string.c	Tue Feb 27 14:11:00 2018 +0300
+++ b/njs/njs_string.c	Wed Feb 28 16:20:11 2018 +0300
@@ -1051,6 +1051,11 @@ njs_string_slice(njs_vm_t *vm, njs_value
         start += slice->start;
         size = slice->length;
 
+        if (string->length == 0) {
+            /* Byte string. */
+            length = 0;
+        }
+
     } else {
         /* UTF-8 string. */
         end = start + string->size;
diff -r 0f1c3efcd894 -r ab1f67b69707 njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Tue Feb 27 14:11:00 2018 +0300
+++ b/njs/test/njs_unit_test.c	Wed Feb 28 16:20:11 2018 +0300
@@ -3529,6 +3529,15 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("'?'.toUTF8()[0]"),
       nxt_string("\xCE") },
 
+    { nxt_string("/^\\x80$/.test('\\x80'.toBytes())"),
+      nxt_string("true") },
+
+    { nxt_string("/^\\xC2\\x80$/.test('\\x80'.toUTF8())"),
+      nxt_string("true") },
+
+    { nxt_string("'?'.toUTF8().toBytes()"),
+      nxt_string("?") },
+
     { nxt_string("var a = 'a'.toBytes() + '?'; a + a.length"),
       nxt_string("a?3") },
 

From dpa-nginx at aegee.org  Wed Feb 28 10:52:18 2018
From: dpa-nginx at aegee.org (=?UTF-8?B?0JTQuNC70Y/QvSDQn9Cw0LvQsNGD0LfQvtCy?=)
Date: Tue, 28 Feb 2018 11:52:18 +0100
Subject: proxy_pass and concurrent connections
Message-ID: <b8a77f47-12f5-2ec9-8369-31771dead486@aegee.org>

Hello,

when I try to enter a bug at https://trac.nginx.org/nginx/newticket#ticket, choose as version 1.12.x and submit the system rejects the ticket with the message:

Warning: The ticket field 'nginx_version' is invalid: nginx_version is required

And here is the actual question:

proxy_pass can accept an URL or a server group. When a server group (upstream) is defined, then with max_conns= the maximum number of simultaneous connections can be specified.

The documentation of proxy_pass shall clarify, whether by default with URL only consequent connections are allowed/whether defining upstream is the only way to introduce parallelism towards the proxy.

Regards
   Dilian

From arut at nginx.com  Wed Feb 28 15:01:43 2018
From: arut at nginx.com (Roman Arutyunyan)
Date: Wed, 28 Feb 2018 15:01:43 +0000
Subject: [nginx] Generic subrequests in memory.
Message-ID: <hg.20f139e9ffa8.1519830103.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/20f139e9ffa8
branches:  
changeset: 7220:20f139e9ffa8
user:      Roman Arutyunyan <arut at nginx.com>
date:      Wed Feb 28 16:56:58 2018 +0300
description:
Generic subrequests in memory.

Previously, only the upstream response body could be accessed with the
NGX_HTTP_SUBREQUEST_IN_MEMORY feature.  Now any response body from a subrequest
can be saved in a memory buffer.  It is available as a single buffer in r->out
and the buffer size is configured by the subrequest_output_buffer_size
directive.

Upstream, proxy and fastcgi code used to handle the old-style feature is
removed.

diffstat:

 src/http/modules/ngx_http_fastcgi_module.c    |   30 ------
 src/http/modules/ngx_http_proxy_module.c      |   30 ------
 src/http/modules/ngx_http_ssi_filter_module.c |    8 +-
 src/http/ngx_http_core_module.c               |   21 ++++
 src/http/ngx_http_core_module.h               |    2 +
 src/http/ngx_http_postpone_filter_module.c    |   78 ++++++++++++++++
 src/http/ngx_http_upstream.c                  |  126 +-------------------------
 7 files changed, 107 insertions(+), 188 deletions(-)

diffs (428 lines):

diff -r d0d32b33167d -r 20f139e9ffa8 src/http/modules/ngx_http_fastcgi_module.c
--- a/src/http/modules/ngx_http_fastcgi_module.c	Thu Feb 22 17:25:43 2018 +0300
+++ b/src/http/modules/ngx_http_fastcgi_module.c	Wed Feb 28 16:56:58 2018 +0300
@@ -2512,36 +2512,6 @@ ngx_http_fastcgi_non_buffered_filter(voi
         break;
     }
 
-    /* provide continuous buffer for subrequests in memory */
-
-    if (r->subrequest_in_memory) {
-
-        cl = u->out_bufs;
-
-        if (cl) {
-            buf->pos = cl->buf->pos;
-        }
-
-        buf->last = buf->pos;
-
-        for (cl = u->out_bufs; cl; cl = cl->next) {
-            ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
-                           "http fastcgi in memory %p-%p %O",
-                           cl->buf->pos, cl->buf->last, ngx_buf_size(cl->buf));
-
-            if (buf->last == cl->buf->pos) {
-                buf->last = cl->buf->last;
-                continue;
-            }
-
-            buf->last = ngx_movemem(buf->last, cl->buf->pos,
-                                    cl->buf->last - cl->buf->pos);
-
-            cl->buf->pos = buf->last - (cl->buf->last - cl->buf->pos);
-            cl->buf->last = buf->last;
-        }
-    }
-
     return NGX_OK;
 }
 
diff -r d0d32b33167d -r 20f139e9ffa8 src/http/modules/ngx_http_proxy_module.c
--- a/src/http/modules/ngx_http_proxy_module.c	Thu Feb 22 17:25:43 2018 +0300
+++ b/src/http/modules/ngx_http_proxy_module.c	Wed Feb 28 16:56:58 2018 +0300
@@ -2321,36 +2321,6 @@ ngx_http_proxy_non_buffered_chunked_filt
         return NGX_ERROR;
     }
 
-    /* provide continuous buffer for subrequests in memory */
-
-    if (r->subrequest_in_memory) {
-
-        cl = u->out_bufs;
-
-        if (cl) {
-            buf->pos = cl->buf->pos;
-        }
-
-        buf->last = buf->pos;
-
-        for (cl = u->out_bufs; cl; cl = cl->next) {
-            ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
-                           "http proxy in memory %p-%p %O",
-                           cl->buf->pos, cl->buf->last, ngx_buf_size(cl->buf));
-
-            if (buf->last == cl->buf->pos) {
-                buf->last = cl->buf->last;
-                continue;
-            }
-
-            buf->last = ngx_movemem(buf->last, cl->buf->pos,
-                                    cl->buf->last - cl->buf->pos);
-
-            cl->buf->pos = buf->last - (cl->buf->last - cl->buf->pos);
-            cl->buf->last = buf->last;
-        }
-    }
-
     return NGX_OK;
 }
 
diff -r d0d32b33167d -r 20f139e9ffa8 src/http/modules/ngx_http_ssi_filter_module.c
--- a/src/http/modules/ngx_http_ssi_filter_module.c	Thu Feb 22 17:25:43 2018 +0300
+++ b/src/http/modules/ngx_http_ssi_filter_module.c	Wed Feb 28 16:56:58 2018 +0300
@@ -2231,9 +2231,11 @@ ngx_http_ssi_set_variable(ngx_http_reque
 {
     ngx_str_t  *value = data;
 
-    if (r->upstream) {
-        value->len = r->upstream->buffer.last - r->upstream->buffer.pos;
-        value->data = r->upstream->buffer.pos;
+    if (r->headers_out.status < NGX_HTTP_SPECIAL_RESPONSE
+        && r->out && r->out->buf)
+    {
+        value->len = r->out->buf->last - r->out->buf->pos;
+        value->data = r->out->buf->pos;
     }
 
     return rc;
diff -r d0d32b33167d -r 20f139e9ffa8 src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c	Thu Feb 22 17:25:43 2018 +0300
+++ b/src/http/ngx_http_core_module.c	Wed Feb 28 16:56:58 2018 +0300
@@ -399,6 +399,13 @@ static ngx_command_t  ngx_http_core_comm
       offsetof(ngx_http_core_loc_conf_t, sendfile_max_chunk),
       NULL },
 
+    { ngx_string("subrequest_output_buffer_size"),
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
+      ngx_conf_set_size_slot,
+      NGX_HTTP_LOC_CONF_OFFSET,
+      offsetof(ngx_http_core_loc_conf_t, subrequest_output_buffer_size),
+      NULL },
+
     { ngx_string("aio"),
       NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
       ngx_http_core_set_aio,
@@ -2237,6 +2244,12 @@ ngx_http_subrequest(ngx_http_request_t *
         return NGX_ERROR;
     }
 
+    if (r->subrequest_in_memory) {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "nested in-memory subrequest \"%V\"", uri);
+        return NGX_ERROR;
+    }
+
     sr = ngx_pcalloc(r->pool, sizeof(ngx_http_request_t));
     if (sr == NULL) {
         return NGX_ERROR;
@@ -2318,6 +2331,10 @@ ngx_http_subrequest(ngx_http_request_t *
 
     sr->log_handler = r->log_handler;
 
+    if (sr->subrequest_in_memory) {
+        sr->filter_need_in_memory = 1;
+    }
+
     if (!sr->background) {
         if (c->data == r && r->postponed == NULL) {
             c->data = sr;
@@ -3356,6 +3373,7 @@ ngx_http_core_create_loc_conf(ngx_conf_t
     clcf->internal = NGX_CONF_UNSET;
     clcf->sendfile = NGX_CONF_UNSET;
     clcf->sendfile_max_chunk = NGX_CONF_UNSET_SIZE;
+    clcf->subrequest_output_buffer_size = NGX_CONF_UNSET_SIZE;
     clcf->aio = NGX_CONF_UNSET;
     clcf->aio_write = NGX_CONF_UNSET;
 #if (NGX_THREADS)
@@ -3578,6 +3596,9 @@ ngx_http_core_merge_loc_conf(ngx_conf_t 
     ngx_conf_merge_value(conf->sendfile, prev->sendfile, 0);
     ngx_conf_merge_size_value(conf->sendfile_max_chunk,
                               prev->sendfile_max_chunk, 0);
+    ngx_conf_merge_size_value(conf->subrequest_output_buffer_size,
+                              prev->subrequest_output_buffer_size,
+                              (size_t) ngx_pagesize);
     ngx_conf_merge_value(conf->aio, prev->aio, NGX_HTTP_AIO_OFF);
     ngx_conf_merge_value(conf->aio_write, prev->aio_write, 0);
 #if (NGX_THREADS)
diff -r d0d32b33167d -r 20f139e9ffa8 src/http/ngx_http_core_module.h
--- a/src/http/ngx_http_core_module.h	Thu Feb 22 17:25:43 2018 +0300
+++ b/src/http/ngx_http_core_module.h	Wed Feb 28 16:56:58 2018 +0300
@@ -351,6 +351,8 @@ struct ngx_http_core_loc_conf_s {
     size_t        limit_rate_after;        /* limit_rate_after */
     size_t        sendfile_max_chunk;      /* sendfile_max_chunk */
     size_t        read_ahead;              /* read_ahead */
+    size_t        subrequest_output_buffer_size;
+                                           /* subrequest_output_buffer_size */
 
     ngx_msec_t    client_body_timeout;     /* client_body_timeout */
     ngx_msec_t    send_timeout;            /* send_timeout */
diff -r d0d32b33167d -r 20f139e9ffa8 src/http/ngx_http_postpone_filter_module.c
--- a/src/http/ngx_http_postpone_filter_module.c	Thu Feb 22 17:25:43 2018 +0300
+++ b/src/http/ngx_http_postpone_filter_module.c	Wed Feb 28 16:56:58 2018 +0300
@@ -12,6 +12,8 @@
 
 static ngx_int_t ngx_http_postpone_filter_add(ngx_http_request_t *r,
     ngx_chain_t *in);
+static ngx_int_t ngx_http_postpone_filter_in_memory(ngx_http_request_t *r,
+    ngx_chain_t *in);
 static ngx_int_t ngx_http_postpone_filter_init(ngx_conf_t *cf);
 
 
@@ -60,6 +62,10 @@ ngx_http_postpone_filter(ngx_http_reques
     ngx_log_debug3(NGX_LOG_DEBUG_HTTP, c->log, 0,
                    "http postpone filter \"%V?%V\" %p", &r->uri, &r->args, in);
 
+    if (r->subrequest_in_memory) {
+        return ngx_http_postpone_filter_in_memory(r, in);
+    }
+
     if (r != c->data) {
 
         if (in) {
@@ -172,6 +178,78 @@ found:
 
 
 static ngx_int_t
+ngx_http_postpone_filter_in_memory(ngx_http_request_t *r, ngx_chain_t *in)
+{
+    size_t                     len;
+    ngx_buf_t                 *b;
+    ngx_connection_t          *c;
+    ngx_http_core_loc_conf_t  *clcf;
+
+    c = r->connection;
+
+    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0,
+                   "http postpone filter in memory");
+
+    if (r->out == NULL) {
+        r->out = ngx_alloc_chain_link(r->pool);
+        if (r->out == NULL) {
+            return NGX_ERROR;
+        }
+
+        clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module);
+
+        if (r->headers_out.content_length_n != -1) {
+            len = r->headers_out.content_length_n;
+
+            if (len > clcf->subrequest_output_buffer_size) {
+                ngx_log_error(NGX_LOG_ERR, c->log, 0,
+                              "too big subrequest response: %uz", len);
+                return NGX_ERROR;
+            }
+
+        } else {
+            len = clcf->subrequest_output_buffer_size;
+        }
+
+        b = ngx_create_temp_buf(r->pool, len);
+        if (b == NULL) {
+            return NGX_ERROR;
+        }
+
+        b->last_buf = 1;
+
+        r->out->buf = b;
+        r->out->next = NULL;
+    }
+
+    b = r->out->buf;
+
+    for ( /* void */ ; in; in = in->next) {
+
+        if (ngx_buf_special(in->buf)) {
+            continue;
+        }
+
+        len = in->buf->last - in->buf->pos;
+
+        if (len > (size_t) (b->end - b->last)) {
+            ngx_log_error(NGX_LOG_ERR, c->log, 0,
+                          "too big subrequest response");
+            return NGX_ERROR;
+        }
+
+        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0,
+                       "http postpone filter in memory %uz bytes", len);
+
+        b->last = ngx_cpymem(b->last, in->buf->pos, len);
+        in->buf->pos = in->buf->last;
+    }
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
 ngx_http_postpone_filter_init(ngx_conf_t *cf)
 {
     ngx_http_next_body_filter = ngx_http_top_body_filter;
diff -r d0d32b33167d -r 20f139e9ffa8 src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Thu Feb 22 17:25:43 2018 +0300
+++ b/src/http/ngx_http_upstream.c	Wed Feb 28 16:56:58 2018 +0300
@@ -55,8 +55,6 @@ static ngx_int_t ngx_http_upstream_inter
 static ngx_int_t ngx_http_upstream_test_connect(ngx_connection_t *c);
 static ngx_int_t ngx_http_upstream_process_headers(ngx_http_request_t *r,
     ngx_http_upstream_t *u);
-static void ngx_http_upstream_process_body_in_memory(ngx_http_request_t *r,
-    ngx_http_upstream_t *u);
 static void ngx_http_upstream_send_response(ngx_http_request_t *r,
     ngx_http_upstream_t *u);
 static void ngx_http_upstream_upgrade(ngx_http_request_t *r,
@@ -2335,45 +2333,7 @@ ngx_http_upstream_process_header(ngx_htt
         return;
     }
 
-    if (!r->subrequest_in_memory) {
-        ngx_http_upstream_send_response(r, u);
-        return;
-    }
-
-    /* subrequest content in memory */
-
-    if (u->input_filter == NULL) {
-        u->input_filter_init = ngx_http_upstream_non_buffered_filter_init;
-        u->input_filter = ngx_http_upstream_non_buffered_filter;
-        u->input_filter_ctx = r;
-    }
-
-    if (u->input_filter_init(u->input_filter_ctx) == NGX_ERROR) {
-        ngx_http_upstream_finalize_request(r, u, NGX_ERROR);
-        return;
-    }
-
-    n = u->buffer.last - u->buffer.pos;
-
-    if (n) {
-        u->buffer.last = u->buffer.pos;
-
-        u->state->response_length += n;
-
-        if (u->input_filter(u->input_filter_ctx, n) == NGX_ERROR) {
-            ngx_http_upstream_finalize_request(r, u, NGX_ERROR);
-            return;
-        }
-    }
-
-    if (u->length == 0) {
-        ngx_http_upstream_finalize_request(r, u, 0);
-        return;
-    }
-
-    u->read_event_handler = ngx_http_upstream_process_body_in_memory;
-
-    ngx_http_upstream_process_body_in_memory(r, u);
+    ngx_http_upstream_send_response(r, u);
 }
 
 
@@ -2776,84 +2736,6 @@ ngx_http_upstream_process_headers(ngx_ht
 
 
 static void
-ngx_http_upstream_process_body_in_memory(ngx_http_request_t *r,
-    ngx_http_upstream_t *u)
-{
-    size_t             size;
-    ssize_t            n;
-    ngx_buf_t         *b;
-    ngx_event_t       *rev;
-    ngx_connection_t  *c;
-
-    c = u->peer.connection;
-    rev = c->read;
-
-    ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0,
-                   "http upstream process body in memory");
-
-    if (rev->timedout) {
-        ngx_connection_error(c, NGX_ETIMEDOUT, "upstream timed out");
-        ngx_http_upstream_finalize_request(r, u, NGX_HTTP_GATEWAY_TIME_OUT);
-        return;
-    }
-
-    b = &u->buffer;
-
-    for ( ;; ) {
-
-        size = b->end - b->last;
-
-        if (size == 0) {
-            ngx_log_error(NGX_LOG_ALERT, c->log, 0,
-                          "upstream buffer is too small to read response");
-            ngx_http_upstream_finalize_request(r, u, NGX_ERROR);
-            return;
-        }
-
-        n = c->recv(c, b->last, size);
-
-        if (n == NGX_AGAIN) {
-            break;
-        }
-
-        if (n == 0 || n == NGX_ERROR) {
-            ngx_http_upstream_finalize_request(r, u, n);
-            return;
-        }
-
-        u->state->bytes_received += n;
-        u->state->response_length += n;
-
-        if (u->input_filter(u->input_filter_ctx, n) == NGX_ERROR) {
-            ngx_http_upstream_finalize_request(r, u, NGX_ERROR);
-            return;
-        }
-
-        if (!rev->ready) {
-            break;
-        }
-    }
-
-    if (u->length == 0) {
-        ngx_http_upstream_finalize_request(r, u, 0);
-        return;
-    }
-
-    if (ngx_handle_read_event(rev, 0) != NGX_OK) {
-        ngx_http_upstream_finalize_request(r, u, NGX_ERROR);
-        return;
-    }
-
-    if (rev->active) {
-        ngx_add_timer(rev, u->conf->read_timeout);
-
-    } else if (rev->timer_set) {
-        ngx_del_timer(rev);
-    }
-}
-
-
-static void
 ngx_http_upstream_send_response(ngx_http_request_t *r, ngx_http_upstream_t *u)
 {
     ssize_t                    n;
@@ -4359,12 +4241,6 @@ ngx_http_upstream_finalize_request(ngx_h
 
 #endif
 
-    if (r->subrequest_in_memory
-        && u->headers_in.status_n >= NGX_HTTP_SPECIAL_RESPONSE)
-    {
-        u->buffer.last = u->buffer.pos;
-    }
-
     r->read_event_handler = ngx_http_block_reading;
 
     if (rc == NGX_DECLINED) {

From mdounin at mdounin.ru  Wed Feb 28 15:52:31 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 28 Feb 2018 18:52:31 +0300
Subject: proxy_pass and concurrent connections
In-Reply-To: <b8a77f47-12f5-2ec9-8369-31771dead486@aegee.org>
References: <b8a77f47-12f5-2ec9-8369-31771dead486@aegee.org>
Message-ID: <20180228155231.GQ89840@mdounin.ru>

Hello!

On Wed, Feb 28, 2018 at 11:52:18AM +0100, ????? ???????? wrote:

> when I try to enter a bug at 
> https://trac.nginx.org/nginx/newticket#ticket, choose as version 
> 1.12.x and submit the system rejects the ticket with the 
> message:
> 
> Warning: The ticket field 'nginx_version' is invalid: 
> nginx_version is required

That's about the "nginx -V" field you've left blank.

> And here is the actual question:
> 
> proxy_pass can accept an URL or a server group. When a server 
> group (upstream) is defined, then with max_conns= the maximum 
> number of simultaneous connections can be specified.
> 
> The documentation of proxy_pass shall clarify, whether by 
> default with URL only consequent connections are allowed/whether 
> defining upstream is the only way to introduce parallelism 
> towards the proxy.

Or not, bacause the answer is obvious unless you have very strange 
background.  Moreover, the documentation explicitly explains the 
default for "max_conns", so this should be obvious regardless of 
the background:

: max_conns=number
: limits the maximum number of simultaneous active connections to 
: the proxied server (1.11.5). Default value is zero, meaning there 
: is no limit.

If you for some reason think that only consequent connections are 
allowed, most likely you are facing limitations of your backend.

-- 
Maxim Dounin
http://mdounin.ru/

From arut at nginx.com  Wed Feb 28 16:16:53 2018
From: arut at nginx.com (Roman Arutyunyan)
Date: Wed, 28 Feb 2018 16:16:53 +0000
Subject: [njs] Skip empty buffers in HTTP response send().
Message-ID: <hg.c86a0cc40ce5.1519834613.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/c86a0cc40ce5
branches:  
changeset: 454:c86a0cc40ce5
user:      Roman Arutyunyan <arut at nginx.com>
date:      Wed Feb 28 19:16:25 2018 +0300
description:
Skip empty buffers in HTTP response send().

Such buffers lead to send errors and should never be sent.

diffstat:

 nginx/ngx_http_js_module.c |  4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diffs (14 lines):

diff -r ab1f67b69707 -r c86a0cc40ce5 nginx/ngx_http_js_module.c
--- a/nginx/ngx_http_js_module.c	Wed Feb 28 16:20:11 2018 +0300
+++ b/nginx/ngx_http_js_module.c	Wed Feb 28 19:16:25 2018 +0300
@@ -891,6 +891,10 @@ ngx_http_js_ext_send(njs_vm_t *vm, njs_v
                 return NJS_ERROR;
             }
 
+            if (s.length == 0) {
+                continue;
+            }
+
             /* TODO: njs_value_release(vm, value) in buf completion */
 
             ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,

From vl at nginx.com  Wed Feb 28 16:21:49 2018
From: vl at nginx.com (Vladimir Homutov)
Date: Wed, 28 Feb 2018 19:21:49 +0300
Subject: Antw: [PATCH 1 of 2] Access log: Support for disabling escaping
In-Reply-To: <5A8F0824020000370008E1B6@gwia.bsb-muenchen.de>
References: <5A8EF8F7020000370008E19E@gwia.bsb-muenchen.de>
 <5A8F0824020000370008E1B6@gwia.bsb-muenchen.de>
Message-ID: <20180228162149.GA5997@vlpc>

On Thu, Feb 22, 2018 at 06:12:52PM +0100, Johannes Baiter wrote:
> Sorry, I accidentally submitted an incomplete version of the patch.
> Here is the corrected version.
>

Hello,

I've slightly updated the patch (also note your mail client have broken
it - you may want to update settings to avoid this), please take a look.

See also ticket 1450: https://trac.nginx.org/nginx/ticket/1450

-------------- next part --------------
# HG changeset patch
# User Vladimir Homutov <vl at nginx.com>
# Date 1519834295 -10800
#      Wed Feb 28 19:11:35 2018 +0300
# Node ID d420ce6b46768ea7eb23bdec84f992212293af19
# Parent  20f139e9ffa84f1a1db6039bbbb547cd35fc4534
Access log: support for disabling escaping (ticket #1450).

Based on patches by Johannes Baiter <johannes.baiter at bsb-muenchen.de>
and Calin Don.

diff --git a/src/http/modules/ngx_http_log_module.c b/src/http/modules/ngx_http_log_module.c
--- a/src/http/modules/ngx_http_log_module.c
+++ b/src/http/modules/ngx_http_log_module.c
@@ -90,6 +90,11 @@ typedef struct {
 } ngx_http_log_var_t;
 
 
+#define NGX_HTTP_LOG_ESCAPE_DEFAULT  0
+#define NGX_HTTP_LOG_ESCAPE_JSON     1
+#define NGX_HTTP_LOG_ESCAPE_NONE     2
+
+
 static void ngx_http_log_write(ngx_http_request_t *r, ngx_http_log_t *log,
     u_char *buf, size_t len);
 static ssize_t ngx_http_log_script_write(ngx_http_request_t *r,
@@ -126,7 +131,7 @@ static u_char *ngx_http_log_request_leng
     ngx_http_log_op_t *op);
 
 static ngx_int_t ngx_http_log_variable_compile(ngx_conf_t *cf,
-    ngx_http_log_op_t *op, ngx_str_t *value, ngx_uint_t json);
+    ngx_http_log_op_t *op, ngx_str_t *value, ngx_uint_t escape);
 static size_t ngx_http_log_variable_getlen(ngx_http_request_t *r,
     uintptr_t data);
 static u_char *ngx_http_log_variable(ngx_http_request_t *r, u_char *buf,
@@ -136,6 +141,10 @@ static size_t ngx_http_log_json_variable
     uintptr_t data);
 static u_char *ngx_http_log_json_variable(ngx_http_request_t *r, u_char *buf,
     ngx_http_log_op_t *op);
+static size_t ngx_http_log_unescaped_variable_getlen(ngx_http_request_t *r,
+    uintptr_t data);
+static u_char *ngx_http_log_unescaped_variable(ngx_http_request_t *r,
+    u_char *buf, ngx_http_log_op_t *op);
 
 
 static void *ngx_http_log_create_main_conf(ngx_conf_t *cf);
@@ -905,7 +914,7 @@ ngx_http_log_request_length(ngx_http_req
 
 static ngx_int_t
 ngx_http_log_variable_compile(ngx_conf_t *cf, ngx_http_log_op_t *op,
-    ngx_str_t *value, ngx_uint_t json)
+    ngx_str_t *value, ngx_uint_t escape)
 {
     ngx_int_t  index;
 
@@ -916,11 +925,18 @@ ngx_http_log_variable_compile(ngx_conf_t
 
     op->len = 0;
 
-    if (json) {
+    switch (escape) {
+    case NGX_HTTP_LOG_ESCAPE_JSON:
         op->getlen = ngx_http_log_json_variable_getlen;
         op->run = ngx_http_log_json_variable;
+        break;
 
-    } else {
+    case NGX_HTTP_LOG_ESCAPE_NONE:
+        op->getlen = ngx_http_log_unescaped_variable_getlen;
+        op->run = ngx_http_log_unescaped_variable;
+        break;
+
+    default: /* NGX_HTTP_LOG_ESCAPE_DEFAULT */
         op->getlen = ngx_http_log_variable_getlen;
         op->run = ngx_http_log_variable;
     }
@@ -1073,6 +1089,39 @@ ngx_http_log_json_variable(ngx_http_requ
 }
 
 
+static size_t
+ngx_http_log_unescaped_variable_getlen(ngx_http_request_t *r, uintptr_t data)
+{
+    ngx_http_variable_value_t  *value;
+
+    value = ngx_http_get_indexed_variable(r, data);
+
+    if (value == NULL || value->not_found) {
+        return 0;
+    }
+
+    value->escape = 0;
+
+    return value->len;
+}
+
+
+static u_char *
+ngx_http_log_unescaped_variable(ngx_http_request_t *r, u_char *buf,
+    ngx_http_log_op_t *op)
+{
+    ngx_http_variable_value_t  *value;
+
+    value = ngx_http_get_indexed_variable(r, op->data);
+
+    if (value == NULL || value->not_found) {
+        return buf;
+    }
+
+    return ngx_cpymem(buf, value->data, value->len);
+}
+
+
 static void *
 ngx_http_log_create_main_conf(ngx_conf_t *cf)
 {
@@ -1536,18 +1585,21 @@ ngx_http_log_compile_format(ngx_conf_t *
     size_t               i, len;
     ngx_str_t           *value, var;
     ngx_int_t           *flush;
-    ngx_uint_t           bracket, json;
+    ngx_uint_t           bracket, escape;
     ngx_http_log_op_t   *op;
     ngx_http_log_var_t  *v;
 
-    json = 0;
+    escape = NGX_HTTP_LOG_ESCAPE_DEFAULT;
     value = args->elts;
 
     if (s < args->nelts && ngx_strncmp(value[s].data, "escape=", 7) == 0) {
         data = value[s].data + 7;
 
         if (ngx_strcmp(data, "json") == 0) {
-            json = 1;
+            escape = NGX_HTTP_LOG_ESCAPE_JSON;
+
+        } else if (ngx_strcmp(data, "none") == 0) {
+            escape = NGX_HTTP_LOG_ESCAPE_NONE;
 
         } else if (ngx_strcmp(data, "default") != 0) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
@@ -1636,7 +1688,7 @@ ngx_http_log_compile_format(ngx_conf_t *
                     }
                 }
 
-                if (ngx_http_log_variable_compile(cf, op, &var, json)
+                if (ngx_http_log_variable_compile(cf, op, &var, escape)
                     != NGX_OK)
                 {
                     return NGX_CONF_ERROR;
diff --git a/src/stream/ngx_stream_log_module.c b/src/stream/ngx_stream_log_module.c
--- a/src/stream/ngx_stream_log_module.c
+++ b/src/stream/ngx_stream_log_module.c
@@ -89,6 +89,11 @@ typedef struct {
 } ngx_stream_log_var_t;
 
 
+#define NGX_STREAM_LOG_ESCAPE_DEFAULT  0
+#define NGX_STREAM_LOG_ESCAPE_JSON     1
+#define NGX_STREAM_LOG_ESCAPE_NONE     2
+
+
 static void ngx_stream_log_write(ngx_stream_session_t *s, ngx_stream_log_t *log,
     u_char *buf, size_t len);
 static ssize_t ngx_stream_log_script_write(ngx_stream_session_t *s,
@@ -106,7 +111,7 @@ static void ngx_stream_log_flush(ngx_ope
 static void ngx_stream_log_flush_handler(ngx_event_t *ev);
 
 static ngx_int_t ngx_stream_log_variable_compile(ngx_conf_t *cf,
-    ngx_stream_log_op_t *op, ngx_str_t *value, ngx_uint_t json);
+    ngx_stream_log_op_t *op, ngx_str_t *value, ngx_uint_t escape);
 static size_t ngx_stream_log_variable_getlen(ngx_stream_session_t *s,
     uintptr_t data);
 static u_char *ngx_stream_log_variable(ngx_stream_session_t *s, u_char *buf,
@@ -116,7 +121,10 @@ static size_t ngx_stream_log_json_variab
     uintptr_t data);
 static u_char *ngx_stream_log_json_variable(ngx_stream_session_t *s,
     u_char *buf, ngx_stream_log_op_t *op);
-
+static size_t ngx_stream_log_unescaped_variable_getlen(ngx_stream_session_t *s,
+    uintptr_t data);
+static u_char *ngx_stream_log_unescaped_variable(ngx_stream_session_t *s,
+    u_char *buf, ngx_stream_log_op_t *op);
 
 static void *ngx_stream_log_create_main_conf(ngx_conf_t *cf);
 static void *ngx_stream_log_create_srv_conf(ngx_conf_t *cf);
@@ -682,7 +690,7 @@ ngx_stream_log_copy_long(ngx_stream_sess
 
 static ngx_int_t
 ngx_stream_log_variable_compile(ngx_conf_t *cf, ngx_stream_log_op_t *op,
-    ngx_str_t *value, ngx_uint_t json)
+    ngx_str_t *value, ngx_uint_t escape)
 {
     ngx_int_t  index;
 
@@ -693,11 +701,18 @@ ngx_stream_log_variable_compile(ngx_conf
 
     op->len = 0;
 
-    if (json) {
+    switch (escape) {
+    case NGX_STREAM_LOG_ESCAPE_JSON:
         op->getlen = ngx_stream_log_json_variable_getlen;
         op->run = ngx_stream_log_json_variable;
+        break;
 
-    } else {
+    case NGX_STREAM_LOG_ESCAPE_NONE:
+        op->getlen = ngx_stream_log_unescaped_variable_getlen;
+        op->run = ngx_stream_log_unescaped_variable;
+        break;
+
+    default: /* NGX_STREAM_LOG_ESCAPE_DEFAULT */
         op->getlen = ngx_stream_log_variable_getlen;
         op->run = ngx_stream_log_variable;
     }
@@ -851,6 +866,39 @@ ngx_stream_log_json_variable(ngx_stream_
 }
 
 
+static size_t
+ngx_stream_log_unescaped_variable_getlen(ngx_stream_session_t *s, uintptr_t data)
+{
+    ngx_stream_variable_value_t  *value;
+
+    value = ngx_stream_get_indexed_variable(s, data);
+
+    if (value == NULL || value->not_found) {
+        return 0;
+    }
+
+    value->escape = 0;
+
+    return value->len;
+}
+
+
+static u_char *
+ngx_stream_log_unescaped_variable(ngx_stream_session_t *s, u_char *buf,
+                                  ngx_stream_log_op_t *op)
+{
+    ngx_stream_variable_value_t  *value;
+
+    value = ngx_stream_get_indexed_variable(s, op->data);
+
+    if (value == NULL || value->not_found) {
+        return buf;
+    }
+
+    return ngx_cpymem(buf, value->data, value->len);
+}
+
+
 static void *
 ngx_stream_log_create_main_conf(ngx_conf_t *cf)
 {
@@ -1265,17 +1313,20 @@ ngx_stream_log_compile_format(ngx_conf_t
     size_t                 i, len;
     ngx_str_t             *value, var;
     ngx_int_t             *flush;
-    ngx_uint_t             bracket, json;
+    ngx_uint_t             bracket, escape;
     ngx_stream_log_op_t   *op;
 
-    json = 0;
+    escape = NGX_STREAM_LOG_ESCAPE_DEFAULT;
     value = args->elts;
 
     if (s < args->nelts && ngx_strncmp(value[s].data, "escape=", 7) == 0) {
         data = value[s].data + 7;
 
         if (ngx_strcmp(data, "json") == 0) {
-            json = 1;
+            escape = NGX_STREAM_LOG_ESCAPE_JSON;
+
+        } else if (ngx_strcmp(data, "none") == 0) {
+            escape = NGX_STREAM_LOG_ESCAPE_NONE;
 
         } else if (ngx_strcmp(data, "default") != 0) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
@@ -1350,7 +1401,7 @@ ngx_stream_log_compile_format(ngx_conf_t
                     goto invalid;
                 }
 
-                if (ngx_stream_log_variable_compile(cf, op, &var, json)
+                if (ngx_stream_log_variable_compile(cf, op, &var, escape)
                     != NGX_OK)
                 {
                     return NGX_CONF_ERROR;

From mdounin at mdounin.ru  Wed Feb 28 17:09:13 2018
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 28 Feb 2018 20:09:13 +0300
Subject: [PATCH] $request_scheme variable
In-Reply-To: <3ABC4237-9C2F-47E5-B075-CEE6FA874F9C@cloudflare.com>
References: <05f555d65a33ebf005fe.1488196710@cbranch-vm.localdomain>
 <3ABC4237-9C2F-47E5-B075-CEE6FA874F9C@cloudflare.com>
Message-ID: <20180228170913.GR89840@mdounin.ru>

Hello!

On Tue, Feb 27, 2018 at 10:32:40PM +0000, Chris Branch via nginx-devel wrote:

> Hi, just giving this patch some birthday bumps.
> 
> > On 27 Feb 2017, at 11:58, Chris Branch via nginx-devel <nginx-devel at nginx.org> wrote:
> > 
> > # HG changeset patch
> > # User Chris Branch <cbranch at cloudflare.com>
> > # Date 1488195909 0
> > #      Mon Feb 27 11:45:09 2017 +0000
> > # Node ID 05f555d65a33ebf005fedc569fb52eba3758e1d7
> > # Parent  87cf6ddb41c216876d13cffa5e637a61b159362c
> > $request_scheme variable.
> > 
> > Contains the URI scheme supplied by the client. If no scheme supplied,
> > equivalent to $scheme.
> > 
> > Scheme can be supplied by the client in two ways:
> > 
> > * HTTP/2 :scheme pseudo-header.
> > * HTTP/1 absolute URI in request line.

The $scheme variable is already documented as

: $scheme
: request scheme, ?http? or ?https?

and introducing additional variable with the $request_scheme might 
not be a good idea.

If we really need this for some reason, we should rather consider 
changing $scheme instead, like we do with $host.  This might be a 
bad idea for other reasons though, as an ability to supply 
incorrect $scheme might cause security problems.

-- 
Maxim Dounin
http://mdounin.ru/

From dilyan.palauzov at aegee.org  Wed Feb 28 16:30:09 2018
From: dilyan.palauzov at aegee.org (=?UTF-8?B?0JTQuNC70Y/QvSDQn9Cw0LvQsNGD0LfQvtCy?=)
Date: Wed, 28 Feb 2018 17:30:09 +0100
Subject: proxy_pass and concurrent connections
In-Reply-To: <20180228155231.GQ89840@mdounin.ru>
References: <c8a77f47-12f5-2ec9-8369-31771dead486@aegee.org>
 <20180228155231.GQ89840@mdounin.ru>
Message-ID: <4c790c48-25ba-ec2c-4f0b-f8f28e57de8a@aegee.org>

Hello,

thanks for your answer.

The documentation is explicit when an upstream server is used with proxy_pass, but it says nothing, when proxy_pass is used with an URL.  Being explicit somewhere and stating nothing on the same manner in an alternative scenario leaves space for interpretations, hence my question.

The text "Warning: The ticket field 'nginx_version' is invalid: nginx_version is required " is misleading.  It is not a warning but a permanent error.

Moreover the 'Version' field was set, the message shall be "The 'nginx -V' field at the bottom is required" (or alternatively remove the Version field and leave only 'nging -V', then it will be absolutely clear what is meant).

Regards
   Dilian

On 02/28/18 16:52, Maxim Dounin wrote:
> Hello!
> 
> On Wed, Feb 28, 2018 at 11:52:18AM +0100, ????? ???????? wrote:
> 
>> when I try to enter a bug at
>> https://trac.nginx.org/nginx/newticket#ticket, choose as version
>> 1.12.x and submit the system rejects the ticket with the
>> message:
>>
>> Warning: The ticket field 'nginx_version' is invalid:
>> nginx_version is required
> 
> That's about the "nginx -V" field you've left blank.
> 
>> And here is the actual question:
>>
>> proxy_pass can accept an URL or a server group. When a server
>> group (upstream) is defined, then with max_conns= the maximum
>> number of simultaneous connections can be specified.
>>
>> The documentation of proxy_pass shall clarify, whether by
>> default with URL only consequent connections are allowed/whether
>> defining upstream is the only way to introduce parallelism
>> towards the proxy.
> 
> Or not, bacause the answer is obvious unless you have very strange
> background.  Moreover, the documentation explicitly explains the
> default for "max_conns", so this should be obvious regardless of
> the background:
> 
> : max_conns=number
> : limits the maximum number of simultaneous active connections to
> : the proxied server (1.11.5). Default value is zero, meaning there
> : is no limit.
> 
> If you for some reason think that only consequent connections are
> allowed, most likely you are facing limitations of your backend.
>