From hongzhidao at gmail.com  Thu Sep  1 00:20:31 2016
From: hongzhidao at gmail.com (=?UTF-8?B?5rSq5b+X6YGT?=)
Date: Thu, 1 Sep 2016 08:20:31 +0800
Subject: nginx dso coredump
In-Reply-To: <20160831165039.GP1855@mdounin.ru>
References: <CAJuLd0hX9BMhQfBYVpsqbPSx8zmj-iWDfz49fogxrkB2ZB_okw@mail.gmail.com>
 <20160830112223.GI1855@mdounin.ru>
 <CAJuLd0i4Ea+CAABZk2=qaorP_1E0huBnyd31AE=0UDNqXmnM=Q@mail.gmail.com>
 <20160830130505.GM1855@mdounin.ru>
 <CAJuLd0ghP4An8L2Z3ujJsEVnAJLhM_X-Y5czWS0RL17S=t2u5w@mail.gmail.com>
 <20160830161412.GO1855@mdounin.ru>
 <CAJuLd0jhmocnOwgxXkBV76-P2YZCC_N++T+O8LA=V3tE9qBiDA@mail.gmail.com>
 <20160831165039.GP1855@mdounin.ru>
Message-ID: <CAJuLd0gGaOfbkWmWjO6xCvcEXPAaL8_qSOXsw_fPGad-Fxx6bA@mail.gmail.com>

Thanks

2016-09-01 0:50 GMT+08:00 Maxim Dounin <mdounin at mdounin.ru>:

> Hello!
>
> On Wed, Aug 31, 2016 at 01:19:14AM +0800, ??? wrote:
>
> > Hi.
> >
> > It?s do really a funny experience. Now I use in the following way.
> >
> > [ngx_http_hello_module]
> >
> >
> > 1. development:
> >
> > ngx_http_hello_module.c  => ngx_http_hello_module.so
> >
> >
> > 2. production
> >
> > mv modules/ngx_http_hello_module.so  modules/ngx_http_hello_module.so.2
> >
> > upload ngx_http_hello_module.so  to modules directory, and keep the same
> > name
> >
> > -- configuration --
> >
> > load_module  module/ngx_http_hello_module.so;  # It always use the same
> > name.
> >
> > I can use lsof to check the so file used by nginx.pid or
> nginx.pid.oldbin.
> >
> >
> > Is it the right way? Thanks again.
>
> This should be fine, yes.
>
> --
> Maxim Dounin
> http://nginx.org/
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160901/625eb103/attachment.html>

From piotrsikora at google.com  Thu Sep  1 01:31:30 2016
From: piotrsikora at google.com (Piotr Sikora)
Date: Wed, 31 Aug 2016 18:31:30 -0700
Subject: [PATCH] HTTP: add support for trailers in HTTP responses
In-Reply-To: <20160823142212.GS24741@mdounin.ru>
References: <20160721134430.GV57459@mdounin.ru>
 <fc15b8aa8f020bbc1cdb.1470027503@piotrsikora.sfo.corp.google.com>
 <CAF-CG+LUF0Ob4SFYPbxoj9QWGv0W7peeYR7meyNFWrerCK9AOA@mail.gmail.com>
 <20160801132154.GH57459@mdounin.ru>
 <CAF-CG++zTweYeoCxaO+43YYNyoEiJSguocFUmwFnuFpTnDrAOA@mail.gmail.com>
 <20160804022412.GW57459@mdounin.ru>
 <CAF-CG++TizSEwYyEUK+VE10W_Vmj+pk2kx_DrYaee+ez=uT6BA@mail.gmail.com>
 <20160823142212.GS24741@mdounin.ru>
Message-ID: <CAF-CG+KPw0LPieBpXpNfmcvUu_bGYBiphW8HpopbEE6b5Uc7EQ@mail.gmail.com>

Hey Maxim,

> Could you please describe one of the uses cases in details?
> That's what I'm asking all the time, and still didn't get even a
> single description of a real-world use case.

You were definitely given some real-world use cases in this thread already:

http://mailman.nginx.org/pipermail/nginx-devel/2016-June/008441.html
http://mailman.nginx.org/pipermail/nginx-devel/2016-July/008586.html
http://mailman.nginx.org/pipermail/nginx-devel/2016-August/008619.html

But fine, I can repeat a few of those:

1. Checksums: web server or proxy sends "Content-SHA256:
$response_body_sha256" back to the client. Since it doesn't know the
checksum ahead of time (either because it's a streaming proxy or
because it's not feasible to read the whole file twice), it must sent
it in the trailers. Thanks to the Fetch API, JavaScript clients in the
browser can verify that the checksum matches what was received.

2. Logging: in distributed systems, it's not always feasible or even
possible to write logs at each server that was involved in generating
response, so backend servers can pass information to log (processing
time, profiling data, etc.) in the trailers, so that everything can be
aggregated and/or logged in one place, usually by the front-end proxy.

3. Trailing status (Microservices): servers send "Status: OK" in the
trailers after successfully completing request. This is needed,
because a lot of things can go wrong between sending "200 OK" status
code in the headers and completing response.

Is that enough?

> The same applies to HTTP/1.1.  Yet trailers are (mostly?) unused
> for many years.  Moreover, it is clearly understood now that
> merging trailer headers to headers as supposed by HTTP/1.1 is a
> security disaster.

As stated previously, I have no intention of merging them together.

> Merging trailers with headers can happen somewhere else as long as
> you pass trailers through.  And it will be perfectly in line with
> HTTP/1.1 specs.

Yes, but then it's not NGINX's problem, is it?

> The only potentially bad thing that can happen without forcing
> chunked transfer encoding is that trailers configured won't be
> sent if Content-Length is know.  If this is critical for a use
> case, the Content-Length can be explicitly removed with additional
> configuration.

If you already configured NGINX to send and/or pass trailers, then why
would you also need to configure NGINX to send and/or pass trailer
even if Content-Length is present? This doesn't make any sense.

Furthermore, even with such simple configuration:

    location / {
        add_trailer "Content-SHA256" "$request_body_sha256";
        root /var/www;
        gzip on;
    }

clients that support gzip would receive Content-SHA256 trailer, while
those that didn't request gzip would not. Again, this doesn't make any
sense.

> Additionally, this is not something expected to happen when
> proxying, as Content-Length won't be known anyway if there are
> trailers in the upstream response.

That's only assuming HTTP/1.1. There might be other upstream protocols
that produce trailers or trailer-like metadata.

> On the other hand, forcing chunked transfer encoding based on "TE:
> trailers" looks all the way wrong:
>
> - it will change the behaviour of nginx for such clients, even if
>   there are no other reasons to do so;

Like I previously said, I'm happy to add r->expect_trailers, so that
modules that want to emit trailers could signal that, in which case
chunked encoding would be forced only when trailers are expected to
appear.

> - it won't change the behaviour for other HTTP/1.1 clients who
>   actually support trailers but doesn't advertize it via "TE:
>   trailers", and thus trailers will be lost in some cases anyway.

Again, I'm fine dropping this requirement...

> Gzip is something you explicitly enable in the configuration to
> change the content of some responses.  It changes the content, and
> this in turn results in chunked transfer encoding when using
> HTTP/1.1.

You need to explicitly add and/or enable passing of trailers as well.

> In contrast, trailers is something exists only in case of chunked
> transfer encoding.  Depending on the use case, it may make sense
> to either:
>
> 1. once trailers are configured, force chunked encoding and sent
>    trailers (if it is possible to do so); or

Isn't that what I'm proposing?

> 2. once trailers are configured, sent them as long as chunked
>    encoding is used.
>
> Obviously enough, second approach is more flexible.  And, as long
> as there is a way to force chunked encoding by other means, allows
> to do exactly the same thing as the first one, plus some additional
> things.

I honestly don't see what value would this add, other than confusing
people as to why trailers sometimes work and sometimes don't.

Best regards,
Piotr Sikora


From pluknet at nginx.com  Thu Sep  1 10:30:15 2016
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Thu, 01 Sep 2016 10:30:15 +0000
Subject: [njs] Attribute visibility detection has been disabled.
Message-ID: <hg.af6c17324584.1472725815.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/af6c17324584
branches:  
changeset: 167:af6c17324584
user:      Sergey Kandaurov <pluknet at nginx.com>
date:      Thu Sep 01 13:28:44 2016 +0300
description:
Attribute visibility detection has been disabled.

Old GNU ld linker hangs on Linux ppc64le platform.

diffstat:

 nxt/auto/clang |  13 -------------
 1 files changed, 0 insertions(+), 13 deletions(-)

diffs (23 lines):

diff -r 715b147cbfe0 -r af6c17324584 nxt/auto/clang
--- a/nxt/auto/clang	Wed Aug 31 19:28:08 2016 +0300
+++ b/nxt/auto/clang	Thu Sep 01 13:28:44 2016 +0300
@@ -202,19 +202,6 @@ nxt_feature_test="int main() {
 . ${NXT_AUTO}feature
 
 
-nxt_feature="GCC __attribute__ visibility"
-nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_VISIBILITY
-nxt_feature_run=
-nxt_feature_path=
-nxt_feature_libs=
-nxt_feature_test="int n __attribute__ ((visibility(\"default\")));
-
-                  int main() {
-                      return 1;
-                  }"
-. ${NXT_AUTO}feature
-
-
 nxt_feature="GCC __attribute__ aligned"
 nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_ALIGNED
 nxt_feature_run=


From arut at nginx.com  Thu Sep  1 11:37:49 2016
From: arut at nginx.com (Roman Arutyunyan)
Date: Thu, 01 Sep 2016 11:37:49 +0000
Subject: [nginx] Realip: fixed uninitialized memory access.
Message-ID: <hg.6b1b8c4b7a95.1472729869.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/6b1b8c4b7a95
branches:  
changeset: 6671:6b1b8c4b7a95
user:      Roman Arutyunyan <arut at nginx.com>
date:      Thu Sep 01 14:33:51 2016 +0300
description:
Realip: fixed uninitialized memory access.

Previously, the realip module could be left with uninitialized context after an
error in the ngx_http_realip_set_addr() function.  That context could be later
accessed by $realip_remote_addr and $realip_remote_port variable handlers.

diffstat:

 src/http/modules/ngx_http_realip_module.c |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (19 lines):

diff -r c6372a40c2a7 -r 6b1b8c4b7a95 src/http/modules/ngx_http_realip_module.c
--- a/src/http/modules/ngx_http_realip_module.c	Fri Aug 26 15:33:07 2016 +0300
+++ b/src/http/modules/ngx_http_realip_module.c	Thu Sep 01 14:33:51 2016 +0300
@@ -264,7 +264,6 @@ ngx_http_realip_set_addr(ngx_http_reques
     }
 
     ctx = cln->data;
-    ngx_http_set_ctx(r, ctx, ngx_http_realip_module);
 
     c = r->connection;
 
@@ -282,6 +281,7 @@ ngx_http_realip_set_addr(ngx_http_reques
     ngx_memcpy(p, text, len);
 
     cln->handler = ngx_http_realip_cleanup;
+    ngx_http_set_ctx(r, ctx, ngx_http_realip_module);
 
     ctx->connection = c;
     ctx->sockaddr = c->sockaddr;


From igor at sysoev.ru  Thu Sep  1 13:12:58 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Thu, 01 Sep 2016 13:12:58 +0000
Subject: [njs] Now Date.parse() and Date() accept shortcut date strings.
Message-ID: <hg.19758fdcd554.1472735578.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/19758fdcd554
branches:  
changeset: 168:19758fdcd554
user:      Igor Sysoev <igor at sysoev.ru>
date:      Thu Sep 01 16:12:31 2016 +0300
description:
Now Date.parse() and Date() accept shortcut date strings.

diffstat:

 njs/njs_date.c           |  305 ++++++++++++++++++++++++++++++++--------------
 njs/test/njs_unit_test.c |   39 ++++++
 2 files changed, 247 insertions(+), 97 deletions(-)

diffs (487 lines):

diff -r af6c17324584 -r 19758fdcd554 njs/njs_date.c
--- a/njs/njs_date.c	Thu Sep 01 13:28:44 2016 +0300
+++ b/njs/njs_date.c	Thu Sep 01 16:12:31 2016 +0300
@@ -44,6 +44,10 @@
 
 
 static nxt_noinline double njs_date_string_parse(njs_value_t *date);
+static double njs_date_rfc2822_string_parse(struct tm *tm, const u_char *p,
+    const u_char *end);
+static double njs_date_js_string_parse(struct tm *tm, const u_char *p,
+    const u_char *end);
 static const u_char *njs_date_skip_week_day(const u_char *p, const u_char *end);
 static const u_char *njs_date_skip_spaces(const u_char *p, const u_char *end);
 static nxt_noinline nxt_int_t njs_date_month_parse(const u_char *p,
@@ -297,9 +301,9 @@ njs_date_parse(njs_vm_t *vm, njs_value_t
 static nxt_noinline double
 njs_date_string_parse(njs_value_t *date)
 {
-    int                ext, ms, gmtoff;
+    int                ext, ms;
     struct tm          tm;
-    nxt_bool_t         sign;
+    nxt_bool_t         sign, week;
     const u_char       *p, *next, *end;
     njs_string_prop_t  string;
 
@@ -308,7 +312,7 @@ njs_date_string_parse(njs_value_t *date)
     p = string.start;
     end = p + string.size;
 
-    if (nxt_slow_path(p + 10 >= end)) {
+    if (nxt_slow_path(p >= end)) {
         return NJS_NAN;
     }
 
@@ -324,11 +328,21 @@ njs_date_string_parse(njs_value_t *date)
         sign = 0;
     }
 
+    tm.tm_mon = 0;
+    tm.tm_mday = 1;
+    tm.tm_hour = 0;
+    tm.tm_min = 0;
+    tm.tm_sec = 0;
+
     next = njs_date_number_parse(&tm.tm_year, p, end, 4);
 
     if (next != NULL) {
         /* ISO-8601 format: "1970-09-28T06:00:00.000Z" */
 
+        if (next == end) {
+            goto year;
+        }
+
         if (*next != '-') {
             /* Extended ISO-8601 format: "+001970-09-28T06:00:00.000Z" */
 
@@ -347,6 +361,10 @@ njs_date_string_parse(njs_value_t *date)
                 tm.tm_year = -tm.tm_year;
             }
 
+            if (next == end) {
+                goto year;
+            }
+
             if (*next != '-') {
                 return NJS_NAN;
             }
@@ -361,7 +379,11 @@ njs_date_string_parse(njs_value_t *date)
 
         tm.tm_mon--;
 
-        if (nxt_slow_path(p >= end || *p != '-')) {
+        if (p == end) {
+            goto done;
+        }
+
+        if (nxt_slow_path(*p != '-')) {
             return NJS_NAN;
         }
 
@@ -370,7 +392,11 @@ njs_date_string_parse(njs_value_t *date)
             return NJS_NAN;
         }
 
-        if (nxt_slow_path(p >= end || *p != 'T')) {
+        if (p == end) {
+            goto done;
+        }
+
+        if (nxt_slow_path(*p != 'T')) {
             return NJS_NAN;
         }
 
@@ -379,6 +405,10 @@ njs_date_string_parse(njs_value_t *date)
             return NJS_NAN;
         }
 
+        if (p == end) {
+            goto done;
+        }
+
         if (nxt_slow_path(p >= end || *p != '.')) {
             return NJS_NAN;
         }
@@ -399,86 +429,68 @@ njs_date_string_parse(njs_value_t *date)
         return NJS_NAN;
     }
 
-    p = njs_date_skip_week_day(p, end);
-    if (nxt_slow_path(p == NULL)) {
-        return NJS_NAN;
+    week = 1;
+
+    for ( ;; ) {
+        next = njs_date_number_parse(&tm.tm_mday, p, end, 2);
+
+        if (next != NULL) {
+            /*
+             * RFC 2822 format:
+             *   "Mon, 28 Sep 1970 06:00:00 GMT",
+             *   "Mon, 28 Sep 1970 06:00:00 UTC",
+             *   "Mon, 28 Sep 1970 12:00:00 +0600".
+             */
+            return njs_date_rfc2822_string_parse(&tm, next, end);
+        }
+
+        tm.tm_mon = njs_date_month_parse(p, end);
+
+        if (tm.tm_mon >= 0) {
+            /* Date.toString() format: "Mon Sep 28 1970 12:00:00 GMT+0600". */
+
+            return njs_date_js_string_parse(&tm, p + 3, end);
+        }
+
+        if (!week) {
+            return NJS_NAN;
+        }
+
+        p = njs_date_skip_week_day(p, end);
+        if (nxt_slow_path(p == NULL)) {
+            return NJS_NAN;
+        }
+
+        p = njs_date_skip_spaces(p, end);
+        if (nxt_slow_path(p == NULL)) {
+            return NJS_NAN;
+        }
+
+        week = 0;
     }
 
+year:
+
+    tm.tm_year -= 1900;
+
+done:
+
+    return njs_timegm(&tm) * 1000;
+}
+
+
+static double
+njs_date_rfc2822_string_parse(struct tm *tm, const u_char *p, const u_char *end)
+{
+    int  gmtoff;
+
     p = njs_date_skip_spaces(p, end);
     if (nxt_slow_path(p == NULL)) {
         return NJS_NAN;
     }
 
-    next = njs_date_number_parse(&tm.tm_mday, p, end, 2);
-
-    if (next != NULL) {
-        /*
-         * RFC 2822 format:
-         *   "Mon, 28 Sep 1970 06:00:00 GMT",
-         *   "Mon, 28 Sep 1970 06:00:00 UTC",
-         *   "Mon, 28 Sep 1970 12:00:00 +0600".
-         */
-        p = njs_date_skip_spaces(next, end);
-        if (nxt_slow_path(p == NULL)) {
-            return NJS_NAN;
-        }
-
-        tm.tm_mon = njs_date_month_parse(p, end);
-        if (nxt_slow_path(tm.tm_mon < 0)) {
-            return NJS_NAN;
-        }
-
-        p = njs_date_skip_spaces(p + 3, end);
-        if (nxt_slow_path(p == NULL)) {
-            return NJS_NAN;
-        }
-
-        p = njs_date_number_parse(&tm.tm_year, p, end, 4);
-        if (nxt_slow_path(p == NULL)) {
-            return NJS_NAN;
-        }
-
-        tm.tm_year -= 1900;
-
-        p = njs_date_skip_spaces(p, end);
-        if (nxt_slow_path(p == NULL)) {
-            return NJS_NAN;
-        }
-
-        p = njs_date_time_parse(&tm, p, end);
-        if (nxt_slow_path(p == NULL)) {
-            return NJS_NAN;
-        }
-
-        p = njs_date_skip_spaces(p, end);
-        if (nxt_slow_path(p == NULL)) {
-            return NJS_NAN;
-        }
-
-        if (p + 2 < end) {
-
-            if ((p[0] == 'G' && p[1] == 'M' && p[2] == 'T')
-                || (p[0] == 'U' && p[1] == 'T' && p[2] == 'C'))
-            {
-                gmtoff = 0;
-
-            } else {
-                gmtoff = njs_date_gmtoff_parse(p, end);
-                if (nxt_slow_path(gmtoff == -1)) {
-                    return NJS_NAN;
-                }
-            }
-
-            return (njs_timegm(&tm) - gmtoff * 60) * 1000;
-        }
-
-        return NJS_NAN;
-    }
-
-    /* Date.toString() format: "Mon Sep 28 1970 12:00:00 GMT+0600". */
-
-    tm.tm_mon = njs_date_month_parse(p, end);
-    if (nxt_slow_path(tm.tm_mon < 0)) {
+    tm->tm_mon = njs_date_month_parse(p, end);
+    if (nxt_slow_path(tm->tm_mon < 0)) {
         return NJS_NAN;
     }
 
@@ -487,48 +499,141 @@ njs_date_string_parse(njs_value_t *date)
         return NJS_NAN;
     }
 
-    p = njs_date_number_parse(&tm.tm_mday, p, end, 2);
+    p = njs_date_number_parse(&tm->tm_year, p, end, 4);
     if (nxt_slow_path(p == NULL)) {
         return NJS_NAN;
     }
 
-    p = njs_date_skip_spaces(p, end);
-    if (nxt_slow_path(p == NULL)) {
-        return NJS_NAN;
+    tm->tm_year -= 1900;
+
+    if (p == end) {
+        goto done;
     }
 
-    p = njs_date_number_parse(&tm.tm_year, p, end, 4);
-    if (nxt_slow_path(p == NULL)) {
-        return NJS_NAN;
-    }
-
-    tm.tm_year -= 1900;
-
     p = njs_date_skip_spaces(p, end);
     if (nxt_slow_path(p == NULL)) {
         return NJS_NAN;
     }
 
-    p = njs_date_time_parse(&tm, p, end);
+    if (p == end) {
+        goto done;
+    }
+
+    p = njs_date_time_parse(tm, p, end);
     if (nxt_slow_path(p == NULL)) {
         return NJS_NAN;
     }
 
+    if (p == end) {
+        goto done;
+    }
+
     p = njs_date_skip_spaces(p, end);
     if (nxt_slow_path(p == NULL)) {
         return NJS_NAN;
     }
 
-    if (p + 2 < end && p[0] == 'G' && p[1] == 'M' && p[2] == 'T') {
-
-        gmtoff = njs_date_gmtoff_parse(&p[3], end);
-
-        if (nxt_fast_path(gmtoff != -1)) {
-            return (njs_timegm(&tm) - gmtoff * 60) * 1000;
+    if (p == end) {
+        goto done;
+    }
+
+    if (nxt_slow_path(p + 2 >= end)) {
+        return NJS_NAN;
+    }
+
+    if ((p[0] == 'G' && p[1] == 'M' && p[2] == 'T')
+        || (p[0] == 'U' && p[1] == 'T' && p[2] == 'C'))
+    {
+        gmtoff = 0;
+
+    } else {
+        gmtoff = njs_date_gmtoff_parse(p, end);
+
+        if (nxt_slow_path(gmtoff == -1)) {
+            return NJS_NAN;
         }
     }
 
+    return (njs_timegm(tm) - gmtoff * 60) * 1000;
+
+done:
+
+    return njs_timegm(tm) * 1000;
+}
+
+
+static double
+njs_date_js_string_parse(struct tm *tm, const u_char *p, const u_char *end)
+{
+    int  gmtoff;
+
+    p = njs_date_skip_spaces(p, end);
+    if (nxt_slow_path(p == NULL)) {
+        return NJS_NAN;
+    }
+
+    p = njs_date_number_parse(&tm->tm_mday, p, end, 2);
+    if (nxt_slow_path(p == NULL)) {
+        return NJS_NAN;
+    }
+
+    p = njs_date_skip_spaces(p, end);
+    if (nxt_slow_path(p == NULL)) {
+        return NJS_NAN;
+    }
+
+    p = njs_date_number_parse(&tm->tm_year, p, end, 4);
+    if (nxt_slow_path(p == NULL)) {
+        return NJS_NAN;
+    }
+
+    tm->tm_year -= 1900;
+
+    if (p == end) {
+        goto done;
+    }
+
+    p = njs_date_skip_spaces(p, end);
+    if (nxt_slow_path(p == NULL)) {
+        return NJS_NAN;
+    }
+
+    if (p == end) {
+        goto done;
+    }
+
+    p = njs_date_time_parse(tm, p, end);
+    if (nxt_slow_path(p == NULL)) {
+        return NJS_NAN;
+    }
+
+    if (p == end) {
+        goto done;
+    }
+
+    p = njs_date_skip_spaces(p, end);
+    if (nxt_slow_path(p == NULL)) {
+        return NJS_NAN;
+    }
+
+    if (p == end) {
+        goto done;
+    }
+
+    if (p + 2 < end && p[0] == 'G' && p[1] == 'M' && p[2] == 'T') {
+
+        gmtoff = njs_date_gmtoff_parse(&p[3], end);
+
+        if (nxt_fast_path(gmtoff != -1)) {
+            return (njs_timegm(tm) - gmtoff * 60) * 1000;
+        }
+    }
+
     return NJS_NAN;
+
+done:
+
+    return njs_timegm(tm) * 1000;
 }
 
 
@@ -559,6 +664,8 @@ njs_date_skip_spaces(const u_char *p, co
 
             p++;
         }
+
+        return p;
     }
 
     return NULL;
@@ -670,7 +777,11 @@ njs_date_time_parse(struct tm *tm, const
         return p;
     }
 
-    if (nxt_slow_path(p >= end || *p != ':')) {
+    if (p == end) {
+        return p;
+    }
+
+    if (nxt_slow_path(*p != ':')) {
         return NULL;
     }
 
diff -r af6c17324584 -r 19758fdcd554 njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Thu Sep 01 13:28:44 2016 +0300
+++ b/njs/test/njs_unit_test.c	Thu Sep 01 16:12:31 2016 +0300
@@ -4515,15 +4515,54 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("Date.parse()"),
       nxt_string("NaN") },
 
+    { nxt_string("Date.parse('2011')"),
+      nxt_string("1293840000000") },
+
+    { nxt_string("Date.parse('+002011')"),
+      nxt_string("1293840000000") },
+
+    { nxt_string("Date.parse('2011-06')"),
+      nxt_string("1306886400000") },
+
+    { nxt_string("Date.parse('2011-06-24')"),
+      nxt_string("1308873600000") },
+
+    { nxt_string("Date.parse('2011-06-24T06')"),
+      nxt_string("NaN") },
+
+    { nxt_string("Date.parse('2011-06-24T06:01')"),
+      nxt_string("1308895260000") },
+
+    { nxt_string("Date.parse('2011-06-24T06:01:02')"),
+      nxt_string("1308895262000") },
+
     { nxt_string("Date.parse('2011-06-24T06:01:02.625Z')"),
       nxt_string("1308895262625") },
 
+    { nxt_string("Date.parse('24 Jun 2011')"),
+      nxt_string("1308873600000") },
+
+    { nxt_string("Date.parse('Fri, 24 Jun 2011 18:48')"),
+      nxt_string("1308941280000") },
+
+    { nxt_string("Date.parse('Fri, 24 Jun 2011 18:48:02')"),
+      nxt_string("1308941282000") },
+
     { nxt_string("Date.parse('Fri, 24 Jun 2011 18:48:02 GMT')"),
       nxt_string("1308941282000") },
 
     { nxt_string("Date.parse('Fri, 24 Jun 2011 18:48:02 +1245')"),
       nxt_string("1308895382000") },
 
+    { nxt_string("Date.parse('Jun 24 2011')"),
+      nxt_string("1308873600000") },
+
+    { nxt_string("Date.parse('Fri Jun 24 2011 18:48')"),
+      nxt_string("1308941280000") },
+
+    { nxt_string("Date.parse('Fri Jun 24 2011 18:48:02')"),
+      nxt_string("1308941282000") },
+
     { nxt_string("Date.parse('Fri Jun 24 2011 18:48:02 GMT+1245')"),
       nxt_string("1308895382000") },
 


From mdounin at mdounin.ru  Thu Sep  1 15:25:54 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 1 Sep 2016 18:25:54 +0300
Subject: [PATCH] SSL: fix order of checks during SSL certificate
 verification
In-Reply-To: <CAF-CG++APc6Ct4N2gCFxyJWobdwxYt53ZFjbqTA56FF=hY7Q3w@mail.gmail.com>
References: <cd72e0a1164abd70aafd.1470176695@piotrsikora.sfo.corp.google.com>
 <20160804035352.GY57459@mdounin.ru>
 <CAF-CG+KBkeG1tWetT0iK2QJBwywZL8cTRwqvrb7mUgKOzbjHww@mail.gmail.com>
 <20160818003702.GE24741@mdounin.ru>
 <CAF-CG+LHx+xb3x+S-N6C0Wek5XsQgXLcfLAC0bkBsg_ZXHvTmw@mail.gmail.com>
 <20160821140211.GH24741@mdounin.ru>
 <CAF-CG++APc6Ct4N2gCFxyJWobdwxYt53ZFjbqTA56FF=hY7Q3w@mail.gmail.com>
Message-ID: <20160901152553.GY1855@mdounin.ru>

Hello!

On Wed, Aug 31, 2016 at 03:24:19PM -0700, Piotr Sikora wrote:

> > At this point nginx just uses the interface provided by the
> > OpenSSL library, exactly as documented.  The fact that the
> > interface is flawed (and documented to be flawed to make sure
> > people will use it properly) has nothing to do with nginx use of
> > it.
> 
> And yet, NGINX doesn't use it properly.

Uhm, nginx uses the interface according to the documentation.  
That is, when SSL_get_verify_result() return X509_V_OK, it uses 
SSL_get_peer_certificate() to differentiate two of its possible 
meanings:

    if (SSL_get_verify_result(c->ssl->connection) != X509_V_OK) {
        ngx_str_set(s, "FAILED");
        return NGX_OK;
    }

    cert = SSL_get_peer_certificate(c->ssl->connection);

    if (cert) {
        ngx_str_set(s, "SUCCESS");

    } else {
        ngx_str_set(s, "NONE");
    }

This is exactly in line with the SSL_get_verify_result() 
documentation, as previously quoted here: 

       X509_V_OK
           The verification succeeded or no peer certificate was presented.

I don't understand why you think that nginx doesn't use it 
properly.  If you still think nginx "doesn't use it properly", 
please elaborate.

A different order of checks may make nginx more robust against API 
changes similar to one BoringSSL tried to implement.  But it has 
nothing to do with proper use of the existing API.

> > Your patches assume that there are no conflicts between nginx
> > error codes (NGX_OK, NGX_DECLINED) and SSL_get_verify_result()
> > error codes.  While this is currently true, this is not something
> > I would rely on, even assuming OpenSSL only.
> 
> You didn't like the previous approach, when logging was done in
> ngx_event_openssl.c (because of the use of "client" and "upstream"),
> and you don't like this approach, when logging is done outside of
> ngx_event_openssl.c (since the error code must be passed back to the
> caller), which is actually one of the solutions you suggested.

The "one of the solutions you suggested" claim isn't really true.  
I never suggested such a solution.  Quoting myself, 
http://mailman.nginx.org/pipermail/nginx-devel/2016-August/008680.html:

: I can't say I like either of the variants.  (1) will require
: memory allocations, (2) looks hardly portable (what if another
: library will have different rc values? or will have more than one
: error string to print?), and (3) looks strange.

The (2) here corresponds to the variant in question you suggested.

-- 
Maxim Dounin
http://nginx.org/


From mdounin at mdounin.ru  Thu Sep  1 15:31:01 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 01 Sep 2016 15:31:01 +0000
Subject: [nginx] Event pipe: process data after recv_chain() errors.
Message-ID: <hg.0fa883e92895.1472743861.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/0fa883e92895
branches:  
changeset: 6672:0fa883e92895
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Sep 01 18:29:55 2016 +0300
description:
Event pipe: process data after recv_chain() errors.

When c->recv_chain() returns an error, it is possible that we already
have some data previously read, e.g., in preread buffer.  And in some
cases it may be even a complete response.  Changed c->recv_chain() error
handling to process the data, much like it is already done if kevent
reports about an error.

This change, in particular, fixes processing of small responses
when an upstream fails to properly close a connection with lingering and
therefore the connection is reset, but the response is already fully
obtained by nginx (see ticket #1037).

diffstat:

 src/event/ngx_event_pipe.c |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff --git a/src/event/ngx_event_pipe.c b/src/event/ngx_event_pipe.c
--- a/src/event/ngx_event_pipe.c
+++ b/src/event/ngx_event_pipe.c
@@ -300,7 +300,7 @@ ngx_event_pipe_read_upstream(ngx_event_p
 
             if (n == NGX_ERROR) {
                 p->upstream_error = 1;
-                return NGX_ERROR;
+                break;
             }
 
             if (n == NGX_AGAIN) {


From mdounin at mdounin.ru  Thu Sep  1 17:06:45 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 01 Sep 2016 17:06:45 +0000
Subject: [nginx] Event pipe: do not set file's thread_handler if not needed.
Message-ID: <hg.e4c1f5b32868.1472749605.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/e4c1f5b32868
branches:  
changeset: 6673:e4c1f5b32868
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Sep 01 20:05:23 2016 +0300
description:
Event pipe: do not set file's thread_handler if not needed.

This fixes a problem with aio threads and sendfile with aio_write switched
off, as observed with range requests after fc72784b1f52 (1.9.13).  Potential
problems with sendfile in threads were previously described in 9fd738b85fad,
and this seems to be one of them.

The problem occurred as file's thread_handler was set to NULL by event pipe
code after a sendfile thread task was scheduled.  As a result, no sendfile
completion code was executed, and the same buffer was additionally sent
using non-threaded sendfile.  Fix is to avoid modifying file's thread_handler
if aio_write is switched off.

Note that with "aio_write on" it is still possible that sendfile will use
thread_handler as set by event pipe.  This is believed to be safe though,
as handlers used are compatible.

diffstat:

 src/event/ngx_event_pipe.c |  10 ++++++----
 1 files changed, 6 insertions(+), 4 deletions(-)

diffs (20 lines):

diff --git a/src/event/ngx_event_pipe.c b/src/event/ngx_event_pipe.c
--- a/src/event/ngx_event_pipe.c
+++ b/src/event/ngx_event_pipe.c
@@ -815,10 +815,12 @@ ngx_event_pipe_write_chain_to_temp_file(
     }
 
 #if (NGX_THREADS)
-    p->temp_file->thread_write = p->thread_handler ? 1 : 0;
-    p->temp_file->file.thread_task = p->thread_task;
-    p->temp_file->file.thread_handler = p->thread_handler;
-    p->temp_file->file.thread_ctx = p->thread_ctx;
+    if (p->thread_handler) {
+        p->temp_file->thread_write = 1;
+        p->temp_file->file.thread_task = p->thread_task;
+        p->temp_file->file.thread_handler = p->thread_handler;
+        p->temp_file->file.thread_ctx = p->thread_ctx;
+    }
 #endif
 
     n = ngx_write_chain_to_temp_file(p->temp_file, out);


From piotrsikora at google.com  Thu Sep  1 21:16:37 2016
From: piotrsikora at google.com (Piotr Sikora)
Date: Thu, 1 Sep 2016 14:16:37 -0700
Subject: [PATCH] SSL: fix order of checks during SSL certificate
 verification
In-Reply-To: <20160901152553.GY1855@mdounin.ru>
References: <cd72e0a1164abd70aafd.1470176695@piotrsikora.sfo.corp.google.com>
 <20160804035352.GY57459@mdounin.ru>
 <CAF-CG+KBkeG1tWetT0iK2QJBwywZL8cTRwqvrb7mUgKOzbjHww@mail.gmail.com>
 <20160818003702.GE24741@mdounin.ru>
 <CAF-CG+LHx+xb3x+S-N6C0Wek5XsQgXLcfLAC0bkBsg_ZXHvTmw@mail.gmail.com>
 <20160821140211.GH24741@mdounin.ru>
 <CAF-CG++APc6Ct4N2gCFxyJWobdwxYt53ZFjbqTA56FF=hY7Q3w@mail.gmail.com>
 <20160901152553.GY1855@mdounin.ru>
Message-ID: <CAF-CG+Ji_JeE6cpy5EMi4uz5Fg1Qa+bU-5fGzdGBPJtAmM6K_w@mail.gmail.com>

Hey Maxim,

> I don't understand why you think that nginx doesn't use it
> properly.  If you still think nginx "doesn't use it properly",
> please elaborate.

Per BUGS section:

    SSL_get_verify_result() is only useful in connection with
SSL_get_peer_certificate.

But the code you pasted, i.e.:

    if (SSL_get_verify_result(c->ssl->connection) != X509_V_OK) {
        ngx_str_set(s, "FAILED");
        return NGX_OK;
    }

    cert = SSL_get_peer_certificate(c->ssl->connection);

uses result of SSL_get_verify_result() without ever calling
SSL_get_peer_ceritficate(), which is what the BUGS section warns
against.

> The "one of the solutions you suggested" claim isn't really true.
> I never suggested such a solution.  Quoting myself,
> http://mailman.nginx.org/pipermail/nginx-devel/2016-August/008680.html:
>
> : I can't say I like either of the variants.  (1) will require
> : memory allocations, (2) looks hardly portable (what if another
> : library will have different rc values? or will have more than one
> : error string to print?), and (3) looks strange.
>
> The (2) here corresponds to the variant in question you suggested.

I was referring to:

    Alternatively,
    we can consider abstracting printing of verification results
    errors with something similar to ngx_ssl_error().

which is basically (2), unless I've misunderstood you.

But that's not really important... what's important is which approach
would be acceptable for your? Because the only reason for the change
in previous patch was the fact that you didn't like my original
version, which printed "client" and "upstream" in ngx_ssl_openssl.c.

Best regards,
Piotr Sikora


From mdounin at mdounin.ru  Fri Sep  2 12:48:34 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 2 Sep 2016 15:48:34 +0300
Subject: [PATCH] SSL: fix order of checks during SSL certificate
 verification
In-Reply-To: <CAF-CG+Ji_JeE6cpy5EMi4uz5Fg1Qa+bU-5fGzdGBPJtAmM6K_w@mail.gmail.com>
References: <cd72e0a1164abd70aafd.1470176695@piotrsikora.sfo.corp.google.com>
 <20160804035352.GY57459@mdounin.ru>
 <CAF-CG+KBkeG1tWetT0iK2QJBwywZL8cTRwqvrb7mUgKOzbjHww@mail.gmail.com>
 <20160818003702.GE24741@mdounin.ru>
 <CAF-CG+LHx+xb3x+S-N6C0Wek5XsQgXLcfLAC0bkBsg_ZXHvTmw@mail.gmail.com>
 <20160821140211.GH24741@mdounin.ru>
 <CAF-CG++APc6Ct4N2gCFxyJWobdwxYt53ZFjbqTA56FF=hY7Q3w@mail.gmail.com>
 <20160901152553.GY1855@mdounin.ru>
 <CAF-CG+Ji_JeE6cpy5EMi4uz5Fg1Qa+bU-5fGzdGBPJtAmM6K_w@mail.gmail.com>
Message-ID: <20160902124834.GB1855@mdounin.ru>

Hello!

On Thu, Sep 01, 2016 at 02:16:37PM -0700, Piotr Sikora wrote:

> Hey Maxim,
> 
> > I don't understand why you think that nginx doesn't use it
> > properly.  If you still think nginx "doesn't use it properly",
> > please elaborate.
> 
> Per BUGS section:
> 
>     SSL_get_verify_result() is only useful in connection with
> SSL_get_peer_certificate.
> 
> But the code you pasted, i.e.:
> 
>     if (SSL_get_verify_result(c->ssl->connection) != X509_V_OK) {
>         ngx_str_set(s, "FAILED");
>         return NGX_OK;
>     }
> 
>     cert = SSL_get_peer_certificate(c->ssl->connection);
> 
> uses result of SSL_get_verify_result() without ever calling
> SSL_get_peer_ceritficate(), which is what the BUGS section warns
> against.

You are misreading the BUGS section.  It doesn't say that 
SSL_get_peer_certificate() must be always called when 
SSL_get_verify_result() is called.  It says that SSL_get_verify_result() is 
only useful in connection with SSL_get_peer_certificate().  More 
specifically, this is needed to distinguish two of the different 
meanings of X509_V_OK.  And this is what nginx does.

> > The "one of the solutions you suggested" claim isn't really true.
> > I never suggested such a solution.  Quoting myself,
> > http://mailman.nginx.org/pipermail/nginx-devel/2016-August/008680.html:
> >
> > : I can't say I like either of the variants.  (1) will require
> > : memory allocations, (2) looks hardly portable (what if another
> > : library will have different rc values? or will have more than one
> > : error string to print?), and (3) looks strange.
> >
> > The (2) here corresponds to the variant in question you suggested.
> 
> I was referring to:
> 
>     Alternatively,
>     we can consider abstracting printing of verification results
>     errors with something similar to ngx_ssl_error().
> 
> which is basically (2), unless I've misunderstood you.

The difference between ngx_ssl_error() and what you've suggested 
is that ngx_ssl_error() doesn't try to cast errors to an nginx rc 
value.  Instead, it uses the error stack saved in the relevant 
connection object.

> But that's not really important... what's important is which approach
> would be acceptable for your? Because the only reason for the change
> in previous patch was the fact that you didn't like my original
> version, which printed "client" and "upstream" in ngx_ssl_openssl.c.

As previously suggested, it might be a good solution to use "peer", as 
already used in serveral error messages in ngx_event_openssl.c

-- 
Maxim Dounin
http://nginx.org/


From vl at nginx.com  Fri Sep  2 15:28:29 2016
From: vl at nginx.com (Vladimir Homutov)
Date: Fri, 02 Sep 2016 15:28:29 +0000
Subject: [nginx] Stream: the $status variable.
Message-ID: <hg.38143d1abdec.1472830109.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/38143d1abdec
branches:  
changeset: 6674:38143d1abdec
user:      Roman Arutyunyan <arut at nginx.com>
date:      Thu Aug 11 20:22:23 2016 +0300
description:
Stream: the $status variable.

The stream session status is one of the following:

200 - normal completion
403 - access forbidden
500 - internal server error
502 - bad gateway
503 - limit conn

diffstat:

 src/stream/ngx_stream.h               |  11 ++++-
 src/stream/ngx_stream_handler.c       |  46 +++++++++++++++-----
 src/stream/ngx_stream_proxy_module.c  |  75 +++++++++++++++++-----------------
 src/stream/ngx_stream_return_module.c |  14 +++---
 src/stream/ngx_stream_variables.c     |  23 ++++++++++
 5 files changed, 112 insertions(+), 57 deletions(-)

diffs (589 lines):

diff -r e4c1f5b32868 -r 38143d1abdec src/stream/ngx_stream.h
--- a/src/stream/ngx_stream.h	Thu Sep 01 20:05:23 2016 +0300
+++ b/src/stream/ngx_stream.h	Thu Aug 11 20:22:23 2016 +0300
@@ -26,6 +26,13 @@ typedef struct ngx_stream_session_s  ngx
 #include <ngx_stream_upstream_round_robin.h>
 
 
+#define NGX_STREAM_OK                        200
+#define NGX_STREAM_FORBIDDEN                 403
+#define NGX_STREAM_INTERNAL_SERVER_ERROR     500
+#define NGX_STREAM_BAD_GATEWAY               502
+#define NGX_STREAM_SERVICE_UNAVAILABLE       503
+
+
 typedef struct {
     void                         **main_conf;
     void                         **srv_conf;
@@ -173,6 +180,8 @@ struct ngx_stream_session_s {
     int                           *captures;
     u_char                        *captures_data;
 #endif
+
+    ngx_uint_t                     status;
 };
 
 
@@ -223,7 +232,7 @@ typedef struct {
 
 
 void ngx_stream_init_connection(ngx_connection_t *c);
-void ngx_stream_close_connection(ngx_connection_t *c);
+void ngx_stream_finalize_session(ngx_stream_session_t *s, ngx_uint_t rc);
 
 
 extern ngx_module_t  ngx_stream_module;
diff -r e4c1f5b32868 -r 38143d1abdec src/stream/ngx_stream_handler.c
--- a/src/stream/ngx_stream_handler.c	Thu Sep 01 20:05:23 2016 +0300
+++ b/src/stream/ngx_stream_handler.c	Thu Aug 11 20:22:23 2016 +0300
@@ -11,6 +11,7 @@
 #include <ngx_stream.h>
 
 
+static void ngx_stream_close_connection(ngx_connection_t *c);
 static u_char *ngx_stream_log_error(ngx_log_t *log, u_char *buf, size_t len);
 static void ngx_stream_init_session(ngx_connection_t *c);
 
@@ -166,8 +167,13 @@ ngx_stream_init_connection(ngx_connectio
     if (cmcf->limit_conn_handler) {
         rc = cmcf->limit_conn_handler(s);
 
-        if (rc != NGX_DECLINED) {
-            ngx_stream_close_connection(c);
+        if (rc == NGX_ERROR) {
+            ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+            return;
+        }
+
+        if (rc == NGX_ABORT) {
+            ngx_stream_finalize_session(s, NGX_STREAM_SERVICE_UNAVAILABLE);
             return;
         }
     }
@@ -175,8 +181,13 @@ ngx_stream_init_connection(ngx_connectio
     if (cmcf->access_handler) {
         rc = cmcf->access_handler(s);
 
-        if (rc != NGX_OK && rc != NGX_DECLINED) {
-            ngx_stream_close_connection(c);
+        if (rc == NGX_ERROR) {
+            ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+            return;
+        }
+
+        if (rc == NGX_ABORT) {
+            ngx_stream_finalize_session(s, NGX_STREAM_FORBIDDEN);
             return;
         }
     }
@@ -194,7 +205,7 @@ ngx_stream_init_connection(ngx_connectio
         {
             ngx_connection_error(c, ngx_socket_errno,
                                  "setsockopt(TCP_NODELAY) failed");
-            ngx_stream_close_connection(c);
+            ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
             return;
         }
 
@@ -215,7 +226,7 @@ ngx_stream_init_connection(ngx_connectio
             ngx_log_error(NGX_LOG_ERR, c->log, 0,
                           "no \"ssl_certificate\" is defined "
                           "in server listening on SSL port");
-            ngx_stream_close_connection(c);
+            ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
             return;
         }
 
@@ -242,7 +253,7 @@ ngx_stream_init_session(ngx_connection_t
 
     s->ctx = ngx_pcalloc(c->pool, sizeof(void *) * ngx_stream_max_module);
     if (s->ctx == NULL) {
-        ngx_stream_close_connection(c);
+        ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
@@ -258,15 +269,14 @@ ngx_stream_ssl_init_connection(ngx_ssl_t
     ngx_stream_session_t   *s;
     ngx_stream_ssl_conf_t  *sslcf;
 
+    s = c->data;
+
     if (ngx_ssl_create_connection(ssl, c, 0) == NGX_ERROR) {
-        ngx_stream_close_connection(c);
+        ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
     if (ngx_ssl_handshake(c) == NGX_AGAIN) {
-
-        s = c->data;
-
         sslcf = ngx_stream_get_module_srv_conf(s, ngx_stream_ssl_module);
 
         ngx_add_timer(c->read, sslcf->handshake_timeout);
@@ -284,7 +294,7 @@ static void
 ngx_stream_ssl_handshake_handler(ngx_connection_t *c)
 {
     if (!c->ssl->handshaked) {
-        ngx_stream_close_connection(c);
+        ngx_stream_finalize_session(c->data, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
@@ -299,6 +309,18 @@ ngx_stream_ssl_handshake_handler(ngx_con
 
 
 void
+ngx_stream_finalize_session(ngx_stream_session_t *s, ngx_uint_t rc)
+{
+    ngx_log_debug1(NGX_LOG_DEBUG_STREAM, s->connection->log, 0,
+                   "finalize stream session: %i", rc);
+
+    s->status = rc;
+
+    ngx_stream_close_connection(s->connection);
+}
+
+
+static void
 ngx_stream_close_connection(ngx_connection_t *c)
 {
     ngx_pool_t  *pool;
diff -r e4c1f5b32868 -r 38143d1abdec src/stream/ngx_stream_proxy_module.c
--- a/src/stream/ngx_stream_proxy_module.c	Thu Sep 01 20:05:23 2016 +0300
+++ b/src/stream/ngx_stream_proxy_module.c	Thu Aug 11 20:22:23 2016 +0300
@@ -73,7 +73,7 @@ static ngx_int_t ngx_stream_proxy_test_c
 static void ngx_stream_proxy_process(ngx_stream_session_t *s,
     ngx_uint_t from_upstream, ngx_uint_t do_write);
 static void ngx_stream_proxy_next_upstream(ngx_stream_session_t *s);
-static void ngx_stream_proxy_finalize(ngx_stream_session_t *s, ngx_int_t rc);
+static void ngx_stream_proxy_finalize(ngx_stream_session_t *s, ngx_uint_t rc);
 static u_char *ngx_stream_proxy_log_error(ngx_log_t *log, u_char *buf,
     size_t len);
 
@@ -368,7 +368,7 @@ ngx_stream_proxy_handler(ngx_stream_sess
 
     u = ngx_pcalloc(c->pool, sizeof(ngx_stream_upstream_t));
     if (u == NULL) {
-        ngx_stream_proxy_finalize(s, NGX_ERROR);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
@@ -380,7 +380,7 @@ ngx_stream_proxy_handler(ngx_stream_sess
     u->peer.log_error = NGX_ERROR_ERR;
 
     if (ngx_stream_proxy_set_local(s, u, pscf->local) != NGX_OK) {
-        ngx_stream_proxy_finalize(s, NGX_ERROR);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
@@ -395,7 +395,7 @@ ngx_stream_proxy_handler(ngx_stream_sess
     if (c->type == SOCK_STREAM) {
         p = ngx_pnalloc(c->pool, pscf->buffer_size);
         if (p == NULL) {
-            ngx_stream_proxy_finalize(s, NGX_ERROR);
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
             return;
         }
 
@@ -418,7 +418,7 @@ ngx_stream_proxy_handler(ngx_stream_sess
             p = ngx_proxy_protocol_write(c, u->downstream_buf.last,
                                          u->downstream_buf.end);
             if (p == NULL) {
-                ngx_stream_proxy_finalize(s, NGX_ERROR);
+                ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
                 return;
             }
 
@@ -433,7 +433,7 @@ ngx_stream_proxy_handler(ngx_stream_sess
 
     if (pscf->upstream_value) {
         if (ngx_stream_proxy_eval(s, pscf) != NGX_OK) {
-            ngx_stream_proxy_finalize(s, NGX_ERROR);
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
             return;
         }
     }
@@ -457,14 +457,14 @@ ngx_stream_proxy_handler(ngx_stream_sess
             {
                 ngx_log_error(NGX_LOG_ERR, c->log, 0,
                               "no port in upstream \"%V\"", host);
-                ngx_stream_proxy_finalize(s, NGX_ERROR);
+                ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
                 return;
             }
 
             if (ngx_stream_upstream_create_round_robin_peer(s, u->resolved)
                 != NGX_OK)
             {
-                ngx_stream_proxy_finalize(s, NGX_ERROR);
+                ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
                 return;
             }
 
@@ -493,7 +493,7 @@ ngx_stream_proxy_handler(ngx_stream_sess
         if (u->resolved->port == 0) {
             ngx_log_error(NGX_LOG_ERR, c->log, 0,
                           "no port in upstream \"%V\"", host);
-            ngx_stream_proxy_finalize(s, NGX_ERROR);
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
             return;
         }
 
@@ -503,14 +503,14 @@ ngx_stream_proxy_handler(ngx_stream_sess
 
         ctx = ngx_resolve_start(cscf->resolver, &temp);
         if (ctx == NULL) {
-            ngx_stream_proxy_finalize(s, NGX_ERROR);
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
             return;
         }
 
         if (ctx == NGX_NO_RESOLVER) {
             ngx_log_error(NGX_LOG_ERR, c->log, 0,
                           "no resolver defined to resolve %V", host);
-            ngx_stream_proxy_finalize(s, NGX_ERROR);
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
             return;
         }
 
@@ -523,7 +523,7 @@ ngx_stream_proxy_handler(ngx_stream_sess
 
         if (ngx_resolve_name(ctx) != NGX_OK) {
             u->resolved->ctx = NULL;
-            ngx_stream_proxy_finalize(s, NGX_ERROR);
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
             return;
         }
 
@@ -534,7 +534,7 @@ found:
 
     if (uscf == NULL) {
         ngx_log_error(NGX_LOG_ALERT, c->log, 0, "no upstream configuration");
-        ngx_stream_proxy_finalize(s, NGX_ERROR);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
@@ -543,7 +543,7 @@ found:
 #endif
 
     if (uscf->peer.init(s, uscf) != NGX_OK) {
-        ngx_stream_proxy_finalize(s, NGX_ERROR);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
@@ -682,13 +682,13 @@ ngx_stream_proxy_connect(ngx_stream_sess
     ngx_log_debug1(NGX_LOG_DEBUG_STREAM, c->log, 0, "proxy connect: %i", rc);
 
     if (rc == NGX_ERROR) {
-        ngx_stream_proxy_finalize(s, NGX_ERROR);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
     if (rc == NGX_BUSY) {
         ngx_log_error(NGX_LOG_ERR, c->log, 0, "no live upstreams");
-        ngx_stream_proxy_finalize(s, NGX_DECLINED);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_BAD_GATEWAY);
         return;
     }
 
@@ -801,7 +801,7 @@ ngx_stream_proxy_init_upstream(ngx_strea
     if (u->upstream_buf.start == NULL) {
         p = ngx_pnalloc(c->pool, pscf->buffer_size);
         if (p == NULL) {
-            ngx_stream_proxy_finalize(s, NGX_ERROR);
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
             return;
         }
 
@@ -851,7 +851,7 @@ ngx_stream_proxy_send_proxy_protocol(ngx
 
     p = ngx_proxy_protocol_write(c, buf, buf + NGX_PROXY_PROTOCOL_MAX_HEADER);
     if (p == NULL) {
-        ngx_stream_proxy_finalize(s, NGX_ERROR);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return NGX_ERROR;
     }
 
@@ -865,7 +865,7 @@ ngx_stream_proxy_send_proxy_protocol(ngx
 
     if (n == NGX_AGAIN) {
         if (ngx_handle_write_event(pc->write, 0) != NGX_OK) {
-            ngx_stream_proxy_finalize(s, NGX_ERROR);
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
             return NGX_ERROR;
         }
 
@@ -879,7 +879,7 @@ ngx_stream_proxy_send_proxy_protocol(ngx
     }
 
     if (n == NGX_ERROR) {
-        ngx_stream_proxy_finalize(s, NGX_DECLINED);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_OK);
         return NGX_ERROR;
     }
 
@@ -895,7 +895,7 @@ ngx_stream_proxy_send_proxy_protocol(ngx
         ngx_log_error(NGX_LOG_ERR, c->log, 0,
                       "could not send PROXY protocol header at once");
 
-        ngx_stream_proxy_finalize(s, NGX_DECLINED);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
 
         return NGX_ERROR;
     }
@@ -947,20 +947,20 @@ ngx_stream_proxy_ssl_init_connection(ngx
     if (ngx_ssl_create_connection(pscf->ssl, pc, NGX_SSL_BUFFER|NGX_SSL_CLIENT)
         != NGX_OK)
     {
-        ngx_stream_proxy_finalize(s, NGX_ERROR);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
     if (pscf->ssl_server_name || pscf->ssl_verify) {
         if (ngx_stream_proxy_ssl_name(s) != NGX_OK) {
-            ngx_stream_proxy_finalize(s, NGX_ERROR);
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
             return;
         }
     }
 
     if (pscf->ssl_session_reuse) {
         if (u->peer.set_session(&u->peer, u->peer.data) != NGX_OK) {
-            ngx_stream_proxy_finalize(s, NGX_ERROR);
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
             return;
         }
     }
@@ -1166,7 +1166,7 @@ ngx_stream_proxy_resolve_handler(ngx_res
                       &ctx->name, ctx->state,
                       ngx_resolver_strerror(ctx->state));
 
-        ngx_stream_proxy_finalize(s, NGX_ERROR);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
@@ -1192,7 +1192,7 @@ ngx_stream_proxy_resolve_handler(ngx_res
 #endif
 
     if (ngx_stream_upstream_create_round_robin_peer(s, ur) != NGX_OK) {
-        ngx_stream_proxy_finalize(s, NGX_ERROR);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
@@ -1245,7 +1245,8 @@ ngx_stream_proxy_process_connection(ngx_
 
             if (!ev->ready) {
                 if (ngx_handle_read_event(ev, 0) != NGX_OK) {
-                    ngx_stream_proxy_finalize(s, NGX_ERROR);
+                    ngx_stream_proxy_finalize(s,
+                                              NGX_STREAM_INTERNAL_SERVER_ERROR);
                     return;
                 }
 
@@ -1279,7 +1280,7 @@ ngx_stream_proxy_process_connection(ngx_
             }
 
             ngx_connection_error(c, NGX_ETIMEDOUT, "connection timed out");
-            ngx_stream_proxy_finalize(s, NGX_DECLINED);
+            ngx_stream_proxy_finalize(s, NGX_STREAM_OK);
             return;
         }
 
@@ -1289,7 +1290,7 @@ ngx_stream_proxy_process_connection(ngx_
                        "stream connection delayed");
 
         if (ngx_handle_read_event(ev, 0) != NGX_OK) {
-            ngx_stream_proxy_finalize(s, NGX_ERROR);
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         }
 
         return;
@@ -1407,7 +1408,7 @@ ngx_stream_proxy_process(ngx_stream_sess
 
         c->log->handler = handler;
 
-        ngx_stream_proxy_finalize(s, NGX_OK);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_OK);
         return;
     }
 
@@ -1449,7 +1450,7 @@ ngx_stream_proxy_process(ngx_stream_sess
                         return;
                     }
 
-                    ngx_stream_proxy_finalize(s, NGX_DECLINED);
+                    ngx_stream_proxy_finalize(s, NGX_STREAM_OK);
                     return;
                 }
 
@@ -1540,20 +1541,20 @@ ngx_stream_proxy_process(ngx_stream_sess
 
         c->log->handler = handler;
 
-        ngx_stream_proxy_finalize(s, NGX_OK);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_OK);
         return;
     }
 
     flags = src->read->eof ? NGX_CLOSE_EVENT : 0;
 
     if (!src->shared && ngx_handle_read_event(src->read, flags) != NGX_OK) {
-        ngx_stream_proxy_finalize(s, NGX_ERROR);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
     if (dst) {
         if (!dst->shared && ngx_handle_write_event(dst->write, 0) != NGX_OK) {
-            ngx_stream_proxy_finalize(s, NGX_ERROR);
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
             return;
         }
 
@@ -1593,7 +1594,7 @@ ngx_stream_proxy_next_upstream(ngx_strea
         || !pscf->next_upstream
         || (timeout && ngx_current_msec - u->peer.start_time >= timeout))
     {
-        ngx_stream_proxy_finalize(s, NGX_DECLINED);
+        ngx_stream_proxy_finalize(s, NGX_STREAM_BAD_GATEWAY);
         return;
     }
 
@@ -1621,7 +1622,7 @@ ngx_stream_proxy_next_upstream(ngx_strea
 
 
 static void
-ngx_stream_proxy_finalize(ngx_stream_session_t *s, ngx_int_t rc)
+ngx_stream_proxy_finalize(ngx_stream_session_t *s, ngx_uint_t rc)
 {
     ngx_connection_t       *pc;
     ngx_stream_upstream_t  *u;
@@ -1664,7 +1665,7 @@ ngx_stream_proxy_finalize(ngx_stream_ses
 
 noupstream:
 
-    ngx_stream_close_connection(s->connection);
+    ngx_stream_finalize_session(s, rc);
 }
 
 
diff -r e4c1f5b32868 -r 38143d1abdec src/stream/ngx_stream_return_module.c
--- a/src/stream/ngx_stream_return_module.c	Thu Sep 01 20:05:23 2016 +0300
+++ b/src/stream/ngx_stream_return_module.c	Thu Aug 11 20:22:23 2016 +0300
@@ -83,7 +83,7 @@ ngx_stream_return_handler(ngx_stream_ses
     rscf = ngx_stream_get_module_srv_conf(s, ngx_stream_return_module);
 
     if (ngx_stream_complex_value(s, &rscf->text, &text) != NGX_OK) {
-        ngx_stream_close_connection(c);
+        ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
@@ -91,13 +91,13 @@ ngx_stream_return_handler(ngx_stream_ses
                    "stream return text: \"%V\"", &text);
 
     if (text.len == 0) {
-        ngx_stream_close_connection(c);
+        ngx_stream_finalize_session(s, NGX_STREAM_OK);
         return;
     }
 
     ctx = ngx_pcalloc(c->pool, sizeof(ngx_stream_return_ctx_t));
     if (ctx == NULL) {
-        ngx_stream_close_connection(c);
+        ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
@@ -126,7 +126,7 @@ ngx_stream_return_write_handler(ngx_even
 
     if (ev->timedout) {
         ngx_connection_error(c, NGX_ETIMEDOUT, "connection timed out");
-        ngx_stream_close_connection(c);
+        ngx_stream_finalize_session(s, NGX_STREAM_OK);
         return;
     }
 
@@ -137,7 +137,7 @@ ngx_stream_return_write_handler(ngx_even
 
         n = c->send(c, b->pos, b->last - b->pos);
         if (n == NGX_ERROR) {
-            ngx_stream_close_connection(c);
+            ngx_stream_finalize_session(s, NGX_STREAM_OK);
             return;
         }
 
@@ -145,14 +145,14 @@ ngx_stream_return_write_handler(ngx_even
             b->pos += n;
 
             if (b->pos == b->last) {
-                ngx_stream_close_connection(c);
+                ngx_stream_finalize_session(s, NGX_STREAM_OK);
                 return;
             }
         }
     }
 
     if (ngx_handle_write_event(ev, 0) != NGX_OK) {
-        ngx_stream_close_connection(c);
+        ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
     }
 
diff -r e4c1f5b32868 -r 38143d1abdec src/stream/ngx_stream_variables.c
--- a/src/stream/ngx_stream_variables.c	Thu Sep 01 20:05:23 2016 +0300
+++ b/src/stream/ngx_stream_variables.c	Thu Aug 11 20:22:23 2016 +0300
@@ -25,6 +25,8 @@ static ngx_int_t ngx_stream_variable_byt
     ngx_stream_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_stream_variable_session_time(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_stream_variable_status(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_stream_variable_connection(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
 
@@ -70,6 +72,9 @@ static ngx_stream_variable_t  ngx_stream
     { ngx_string("session_time"), NULL, ngx_stream_variable_session_time,
       0, NGX_STREAM_VAR_NOCACHEABLE, 0 },
 
+    { ngx_string("status"), NULL, ngx_stream_variable_status,
+      0, NGX_STREAM_VAR_NOCACHEABLE, 0 },
+
     { ngx_string("connection"), NULL,
       ngx_stream_variable_connection, 0, 0, 0 },
 
@@ -530,6 +535,24 @@ ngx_stream_variable_session_time(ngx_str
 
 
 static ngx_int_t
+ngx_stream_variable_status(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data)
+{
+    v->data = ngx_pnalloc(s->connection->pool, NGX_INT_T_LEN);
+    if (v->data == NULL) {
+        return NGX_ERROR;
+    }
+
+    v->len = ngx_sprintf(v->data, "%03ui", s->status) - v->data;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
 ngx_stream_variable_connection(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data)
 {


From vl at nginx.com  Fri Sep  2 15:28:32 2016
From: vl at nginx.com (Vladimir Homutov)
Date: Fri, 02 Sep 2016 15:28:32 +0000
Subject: [nginx] Stream: the $upstream_addr variable.
Message-ID: <hg.ab9b4fd8c5b7.1472830112.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/ab9b4fd8c5b7
branches:  
changeset: 6675:ab9b4fd8c5b7
user:      Vladimir Homutov <vl at nginx.com>
date:      Fri Sep 02 18:27:05 2016 +0300
description:
Stream: the $upstream_addr variable.

Keeps the full address of the upstream server.  If several servers were
contacted during proxying, their addresses are separated by commas,
e.g. "192.168.1.1:80, 192.168.1.2:80".

diffstat:

 src/stream/ngx_stream.h              |   3 +-
 src/stream/ngx_stream_proxy_module.c |  17 ++++++
 src/stream/ngx_stream_upstream.c     |  92 +++++++++++++++++++++++++++++++++++-
 src/stream/ngx_stream_upstream.h     |   6 ++
 4 files changed, 116 insertions(+), 2 deletions(-)

diffs (193 lines):

diff -r 38143d1abdec -r ab9b4fd8c5b7 src/stream/ngx_stream.h
--- a/src/stream/ngx_stream.h	Thu Aug 11 20:22:23 2016 +0300
+++ b/src/stream/ngx_stream.h	Fri Sep 02 18:27:05 2016 +0300
@@ -172,7 +172,8 @@ struct ngx_stream_session_s {
     void                         **srv_conf;
 
     ngx_stream_upstream_t         *upstream;
-
+    ngx_array_t                   *upstream_states;
+                                           /* of ngx_stream_upstream_state_t */
     ngx_stream_variable_value_t   *variables;
 
 #if (NGX_PCRE)
diff -r 38143d1abdec -r ab9b4fd8c5b7 src/stream/ngx_stream_proxy_module.c
--- a/src/stream/ngx_stream_proxy_module.c	Thu Aug 11 20:22:23 2016 +0300
+++ b/src/stream/ngx_stream_proxy_module.c	Fri Sep 02 18:27:05 2016 +0300
@@ -392,6 +392,13 @@ ngx_stream_proxy_handler(ngx_stream_sess
     c->write->handler = ngx_stream_proxy_downstream_handler;
     c->read->handler = ngx_stream_proxy_downstream_handler;
 
+    s->upstream_states = ngx_array_create(c->pool, 1,
+                                          sizeof(ngx_stream_upstream_state_t));
+    if (s->upstream_states == NULL) {
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+        return;
+    }
+
     if (c->type == SOCK_STREAM) {
         p = ngx_pnalloc(c->pool, pscf->buffer_size);
         if (p == NULL) {
@@ -677,6 +684,14 @@ ngx_stream_proxy_connect(ngx_stream_sess
 
     u = s->upstream;
 
+    u->state = ngx_array_push(s->upstream_states);
+    if (u->state == NULL) {
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+        return;
+    }
+
+    ngx_memzero(u->state, sizeof(ngx_stream_upstream_state_t));
+
     rc = ngx_event_connect_peer(&u->peer);
 
     ngx_log_debug1(NGX_LOG_DEBUG_STREAM, c->log, 0, "proxy connect: %i", rc);
@@ -686,6 +701,8 @@ ngx_stream_proxy_connect(ngx_stream_sess
         return;
     }
 
+    u->state->peer = u->peer.name;
+
     if (rc == NGX_BUSY) {
         ngx_log_error(NGX_LOG_ERR, c->log, 0, "no live upstreams");
         ngx_stream_proxy_finalize(s, NGX_STREAM_BAD_GATEWAY);
diff -r 38143d1abdec -r ab9b4fd8c5b7 src/stream/ngx_stream_upstream.c
--- a/src/stream/ngx_stream_upstream.c	Thu Aug 11 20:22:23 2016 +0300
+++ b/src/stream/ngx_stream_upstream.c	Fri Sep 02 18:27:05 2016 +0300
@@ -10,6 +10,10 @@
 #include <ngx_stream.h>
 
 
+static ngx_int_t ngx_stream_upstream_add_variables(ngx_conf_t *cf);
+static ngx_int_t ngx_stream_upstream_addr_variable(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data);
+
 static char *ngx_stream_upstream(ngx_conf_t *cf, ngx_command_t *cmd,
     void *dummy);
 static char *ngx_stream_upstream_server(ngx_conf_t *cf, ngx_command_t *cmd,
@@ -39,7 +43,7 @@ static ngx_command_t  ngx_stream_upstrea
 
 
 static ngx_stream_module_t  ngx_stream_upstream_module_ctx = {
-    NULL,                                  /* preconfiguration */
+    ngx_stream_upstream_add_variables,     /* preconfiguration */
     NULL,                                  /* postconfiguration */
 
     ngx_stream_upstream_create_main_conf,  /* create main configuration */
@@ -66,6 +70,92 @@ ngx_module_t  ngx_stream_upstream_module
 };
 
 
+static ngx_stream_variable_t  ngx_stream_upstream_vars[] = {
+
+    { ngx_string("upstream_addr"), NULL,
+      ngx_stream_upstream_addr_variable, 0,
+      NGX_STREAM_VAR_NOCACHEABLE, 0 },
+
+    { ngx_null_string, NULL, NULL, 0, 0, 0 }
+};
+
+
+static ngx_int_t
+ngx_stream_upstream_add_variables(ngx_conf_t *cf)
+{
+    ngx_stream_variable_t  *var, *v;
+
+    for (v = ngx_stream_upstream_vars; v->name.len; v++) {
+        var = ngx_stream_add_variable(cf, &v->name, v->flags);
+        if (var == NULL) {
+            return NGX_ERROR;
+        }
+
+        var->get_handler = v->get_handler;
+        var->data = v->data;
+    }
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_stream_upstream_addr_variable(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data)
+{
+    u_char                       *p;
+    size_t                        len;
+    ngx_uint_t                    i;
+    ngx_stream_upstream_state_t  *state;
+
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+
+    if (s->upstream_states == NULL || s->upstream_states->nelts == 0) {
+        v->not_found = 1;
+        return NGX_OK;
+    }
+
+    len = 0;
+    state = s->upstream_states->elts;
+
+    for (i = 0; i < s->upstream_states->nelts; i++) {
+        if (state[i].peer) {
+            len += state[i].peer->len;
+        }
+
+        len += 2;
+    }
+
+    p = ngx_pnalloc(s->connection->pool, len);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
+
+    v->data = p;
+
+    i = 0;
+
+    for ( ;; ) {
+        if (state[i].peer) {
+            p = ngx_cpymem(p, state[i].peer->data, state[i].peer->len);
+        }
+
+        if (++i == s->upstream_states->nelts) {
+            break;
+        }
+
+        *p++ = ',';
+        *p++ = ' ';
+    }
+
+    v->len = p - v->data;
+
+    return NGX_OK;
+}
+
+
 static char *
 ngx_stream_upstream(ngx_conf_t *cf, ngx_command_t *cmd, void *dummy)
 {
diff -r 38143d1abdec -r ab9b4fd8c5b7 src/stream/ngx_stream_upstream.h
--- a/src/stream/ngx_stream_upstream.h	Thu Aug 11 20:22:23 2016 +0300
+++ b/src/stream/ngx_stream_upstream.h	Fri Sep 02 18:27:05 2016 +0300
@@ -79,6 +79,11 @@ struct ngx_stream_upstream_srv_conf_s {
 
 
 typedef struct {
+    ngx_str_t                         *peer;
+} ngx_stream_upstream_state_t;
+
+
+typedef struct {
     ngx_str_t                          host;
     in_port_t                          port;
     ngx_uint_t                         no_port; /* unsigned no_port:1 */
@@ -104,6 +109,7 @@ typedef struct {
     ngx_str_t                          ssl_name;
 #endif
     ngx_stream_upstream_resolved_t    *resolved;
+    ngx_stream_upstream_state_t       *state;
     unsigned                           connected:1;
     unsigned                           proxy_protocol:1;
 } ngx_stream_upstream_t;


From vl at nginx.com  Fri Sep  2 15:28:34 2016
From: vl at nginx.com (Vladimir Homutov)
Date: Fri, 02 Sep 2016 15:28:34 +0000
Subject: [nginx] Stream: $upstream_bytes_sent and $upstream_bytes_received.
Message-ID: <hg.df3a7c029dec.1472830114.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/df3a7c029dec
branches:  
changeset: 6676:df3a7c029dec
user:      Vladimir Homutov <vl at nginx.com>
date:      Fri Sep 02 18:27:08 2016 +0300
description:
Stream: $upstream_bytes_sent and $upstream_bytes_received.

diffstat:

 src/stream/ngx_stream_proxy_module.c |  14 ++++++-
 src/stream/ngx_stream_upstream.c     |  63 ++++++++++++++++++++++++++++++++++++
 src/stream/ngx_stream_upstream.h     |   3 +
 3 files changed, 78 insertions(+), 2 deletions(-)

diffs (136 lines):

diff -r ab9b4fd8c5b7 -r df3a7c029dec src/stream/ngx_stream_proxy_module.c
--- a/src/stream/ngx_stream_proxy_module.c	Fri Sep 02 18:27:05 2016 +0300
+++ b/src/stream/ngx_stream_proxy_module.c	Fri Sep 02 18:27:08 2016 +0300
@@ -1630,6 +1630,9 @@ ngx_stream_proxy_next_upstream(ngx_strea
         }
 #endif
 
+        u->state->bytes_received = u->received;
+        u->state->bytes_sent = pc->sent;
+
         ngx_close_connection(pc);
         u->peer.connection = NULL;
     }
@@ -1658,13 +1661,20 @@ ngx_stream_proxy_finalize(ngx_stream_ses
         u->resolved->ctx = NULL;
     }
 
+    pc = u->peer.connection;
+
+    if (u->state) {
+        if (pc) {
+            u->state->bytes_received = u->received;
+            u->state->bytes_sent = pc->sent;
+        }
+    }
+
     if (u->peer.free && u->peer.sockaddr) {
         u->peer.free(&u->peer, u->peer.data, 0);
         u->peer.sockaddr = NULL;
     }
 
-    pc = u->peer.connection;
-
     if (pc) {
         ngx_log_debug1(NGX_LOG_DEBUG_STREAM, s->connection->log, 0,
                        "close stream proxy upstream connection: %d", pc->fd);
diff -r ab9b4fd8c5b7 -r df3a7c029dec src/stream/ngx_stream_upstream.c
--- a/src/stream/ngx_stream_upstream.c	Fri Sep 02 18:27:05 2016 +0300
+++ b/src/stream/ngx_stream_upstream.c	Fri Sep 02 18:27:08 2016 +0300
@@ -13,6 +13,8 @@
 static ngx_int_t ngx_stream_upstream_add_variables(ngx_conf_t *cf);
 static ngx_int_t ngx_stream_upstream_addr_variable(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_stream_upstream_bytes_variable(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data);
 
 static char *ngx_stream_upstream(ngx_conf_t *cf, ngx_command_t *cmd,
     void *dummy);
@@ -76,6 +78,14 @@ static ngx_stream_variable_t  ngx_stream
       ngx_stream_upstream_addr_variable, 0,
       NGX_STREAM_VAR_NOCACHEABLE, 0 },
 
+    { ngx_string("upstream_bytes_sent"), NULL,
+      ngx_stream_upstream_bytes_variable, 0,
+      NGX_STREAM_VAR_NOCACHEABLE, 0 },
+
+    { ngx_string("upstream_bytes_received"), NULL,
+      ngx_stream_upstream_bytes_variable, 1,
+      NGX_STREAM_VAR_NOCACHEABLE, 0 },
+
     { ngx_null_string, NULL, NULL, 0, 0, 0 }
 };
 
@@ -156,6 +166,59 @@ ngx_stream_upstream_addr_variable(ngx_st
 }
 
 
+static ngx_int_t
+ngx_stream_upstream_bytes_variable(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data)
+{
+    u_char                       *p;
+    size_t                        len;
+    ngx_uint_t                    i;
+    ngx_stream_upstream_state_t  *state;
+
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+
+    if (s->upstream_states == NULL || s->upstream_states->nelts == 0) {
+        v->not_found = 1;
+        return NGX_OK;
+    }
+
+    len = s->upstream_states->nelts * (NGX_OFF_T_LEN + 2);
+
+    p = ngx_pnalloc(s->connection->pool, len);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
+
+    v->data = p;
+
+    i = 0;
+    state = s->upstream_states->elts;
+
+    for ( ;; ) {
+
+        if (data == 1) {
+            p = ngx_sprintf(p, "%O", state[i].bytes_received);
+
+        } else {
+            p = ngx_sprintf(p, "%O", state[i].bytes_sent);
+        }
+
+        if (++i == s->upstream_states->nelts) {
+            break;
+        }
+
+        *p++ = ',';
+        *p++ = ' ';
+    }
+
+    v->len = p - v->data;
+
+    return NGX_OK;
+}
+
+
 static char *
 ngx_stream_upstream(ngx_conf_t *cf, ngx_command_t *cmd, void *dummy)
 {
diff -r ab9b4fd8c5b7 -r df3a7c029dec src/stream/ngx_stream_upstream.h
--- a/src/stream/ngx_stream_upstream.h	Fri Sep 02 18:27:05 2016 +0300
+++ b/src/stream/ngx_stream_upstream.h	Fri Sep 02 18:27:08 2016 +0300
@@ -79,6 +79,9 @@ struct ngx_stream_upstream_srv_conf_s {
 
 
 typedef struct {
+    off_t                              bytes_sent;
+    off_t                              bytes_received;
+
     ngx_str_t                         *peer;
 } ngx_stream_upstream_state_t;
 


From vl at nginx.com  Fri Sep  2 15:28:37 2016
From: vl at nginx.com (Vladimir Homutov)
Date: Fri, 02 Sep 2016 15:28:37 +0000
Subject: [nginx] Stream: upstream response time variables.
Message-ID: <hg.c02290241cbe.1472830117.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/c02290241cbe
branches:  
changeset: 6677:c02290241cbe
user:      Vladimir Homutov <vl at nginx.com>
date:      Fri Sep 02 18:27:12 2016 +0300
description:
Stream: upstream response time variables.

The $upstream_connect_time, $upstream_first_byte_time and
$upstream_session_time variables keep corresponding times.

diffstat:

 src/stream/ngx_stream_proxy_module.c |  19 ++++++++
 src/stream/ngx_stream_upstream.c     |  81 ++++++++++++++++++++++++++++++++++++
 src/stream/ngx_stream_upstream.h     |   3 +
 3 files changed, 103 insertions(+), 0 deletions(-)

diffs (175 lines):

diff -r df3a7c029dec -r c02290241cbe src/stream/ngx_stream_proxy_module.c
--- a/src/stream/ngx_stream_proxy_module.c	Fri Sep 02 18:27:08 2016 +0300
+++ b/src/stream/ngx_stream_proxy_module.c	Fri Sep 02 18:27:12 2016 +0300
@@ -684,6 +684,10 @@ ngx_stream_proxy_connect(ngx_stream_sess
 
     u = s->upstream;
 
+    if (u->state) {
+        u->state->response_time = ngx_current_msec - u->state->response_time;
+    }
+
     u->state = ngx_array_push(s->upstream_states);
     if (u->state == NULL) {
         ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
@@ -692,6 +696,10 @@ ngx_stream_proxy_connect(ngx_stream_sess
 
     ngx_memzero(u->state, sizeof(ngx_stream_upstream_state_t));
 
+    u->state->connect_time = (ngx_msec_t) -1;
+    u->state->first_byte_time = (ngx_msec_t) -1;
+    u->state->response_time = ngx_current_msec;
+
     rc = ngx_event_connect_peer(&u->peer);
 
     ngx_log_debug1(NGX_LOG_DEBUG_STREAM, c->log, 0, "proxy connect: %i", rc);
@@ -813,6 +821,8 @@ ngx_stream_proxy_init_upstream(ngx_strea
         }
     }
 
+    u->state->connect_time = ngx_current_msec - u->state->response_time;
+
     c->log->action = "proxying connection";
 
     if (u->upstream_buf.start == NULL) {
@@ -1518,6 +1528,13 @@ ngx_stream_proxy_process(ngx_stream_sess
                     }
                 }
 
+                if (from_upstream) {
+                    if (u->state->first_byte_time == (ngx_msec_t) -1) {
+                        u->state->first_byte_time = ngx_current_msec
+                                                    - u->state->response_time;
+                    }
+                }
+
                 if (c->type == SOCK_DGRAM && ++u->responses == pscf->responses)
                 {
                     src->read->ready = 0;
@@ -1664,6 +1681,8 @@ ngx_stream_proxy_finalize(ngx_stream_ses
     pc = u->peer.connection;
 
     if (u->state) {
+        u->state->response_time = ngx_current_msec - u->state->response_time;
+
         if (pc) {
             u->state->bytes_received = u->received;
             u->state->bytes_sent = pc->sent;
diff -r df3a7c029dec -r c02290241cbe src/stream/ngx_stream_upstream.c
--- a/src/stream/ngx_stream_upstream.c	Fri Sep 02 18:27:08 2016 +0300
+++ b/src/stream/ngx_stream_upstream.c	Fri Sep 02 18:27:12 2016 +0300
@@ -13,6 +13,8 @@
 static ngx_int_t ngx_stream_upstream_add_variables(ngx_conf_t *cf);
 static ngx_int_t ngx_stream_upstream_addr_variable(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_stream_upstream_response_time_variable(
+    ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_stream_upstream_bytes_variable(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
 
@@ -82,6 +84,18 @@ static ngx_stream_variable_t  ngx_stream
       ngx_stream_upstream_bytes_variable, 0,
       NGX_STREAM_VAR_NOCACHEABLE, 0 },
 
+    { ngx_string("upstream_connect_time"), NULL,
+      ngx_stream_upstream_response_time_variable, 2,
+      NGX_STREAM_VAR_NOCACHEABLE, 0 },
+
+    { ngx_string("upstream_first_byte_time"), NULL,
+      ngx_stream_upstream_response_time_variable, 1,
+      NGX_STREAM_VAR_NOCACHEABLE, 0 },
+
+    { ngx_string("upstream_session_time"), NULL,
+      ngx_stream_upstream_response_time_variable, 0,
+      NGX_STREAM_VAR_NOCACHEABLE, 0 },
+
     { ngx_string("upstream_bytes_received"), NULL,
       ngx_stream_upstream_bytes_variable, 1,
       NGX_STREAM_VAR_NOCACHEABLE, 0 },
@@ -219,6 +233,73 @@ ngx_stream_upstream_bytes_variable(ngx_s
 }
 
 
+static ngx_int_t
+ngx_stream_upstream_response_time_variable(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data)
+{
+    u_char                       *p;
+    size_t                        len;
+    ngx_uint_t                    i;
+    ngx_msec_int_t                ms;
+    ngx_stream_upstream_state_t  *state;
+
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+
+    if (s->upstream_states == NULL || s->upstream_states->nelts == 0) {
+        v->not_found = 1;
+        return NGX_OK;
+    }
+
+    len = s->upstream_states->nelts * (NGX_TIME_T_LEN + 4 + 2);
+
+    p = ngx_pnalloc(s->connection->pool, len);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
+
+    v->data = p;
+
+    i = 0;
+    state = s->upstream_states->elts;
+
+    for ( ;; ) {
+
+        if (data == 1) {
+            if (state[i].first_byte_time == (ngx_msec_t) -1) {
+                *p++ = '-';
+                goto next;
+            }
+
+            ms = state[i].first_byte_time;
+
+        } else if (data == 2 && state[i].connect_time != (ngx_msec_t) -1) {
+            ms = state[i].connect_time;
+
+        } else {
+            ms = state[i].response_time;
+        }
+
+        ms = ngx_max(ms, 0);
+        p = ngx_sprintf(p, "%T.%03M", (time_t) ms / 1000, ms % 1000);
+
+    next:
+
+        if (++i == s->upstream_states->nelts) {
+            break;
+        }
+
+        *p++ = ',';
+        *p++ = ' ';
+    }
+
+    v->len = p - v->data;
+
+    return NGX_OK;
+}
+
+
 static char *
 ngx_stream_upstream(ngx_conf_t *cf, ngx_command_t *cmd, void *dummy)
 {
diff -r df3a7c029dec -r c02290241cbe src/stream/ngx_stream_upstream.h
--- a/src/stream/ngx_stream_upstream.h	Fri Sep 02 18:27:08 2016 +0300
+++ b/src/stream/ngx_stream_upstream.h	Fri Sep 02 18:27:12 2016 +0300
@@ -79,6 +79,9 @@ struct ngx_stream_upstream_srv_conf_s {
 
 
 typedef struct {
+    ngx_msec_t                         response_time;
+    ngx_msec_t                         connect_time;
+    ngx_msec_t                         first_byte_time;
     off_t                              bytes_sent;
     off_t                              bytes_received;
 


From piotrsikora at google.com  Fri Sep  2 23:18:53 2016
From: piotrsikora at google.com (Piotr Sikora)
Date: Fri, 2 Sep 2016 16:18:53 -0700
Subject: [PATCH] SSL: fix order of checks during SSL certificate
 verification
In-Reply-To: <20160902124834.GB1855@mdounin.ru>
References: <cd72e0a1164abd70aafd.1470176695@piotrsikora.sfo.corp.google.com>
 <20160804035352.GY57459@mdounin.ru>
 <CAF-CG+KBkeG1tWetT0iK2QJBwywZL8cTRwqvrb7mUgKOzbjHww@mail.gmail.com>
 <20160818003702.GE24741@mdounin.ru>
 <CAF-CG+LHx+xb3x+S-N6C0Wek5XsQgXLcfLAC0bkBsg_ZXHvTmw@mail.gmail.com>
 <20160821140211.GH24741@mdounin.ru>
 <CAF-CG++APc6Ct4N2gCFxyJWobdwxYt53ZFjbqTA56FF=hY7Q3w@mail.gmail.com>
 <20160901152553.GY1855@mdounin.ru>
 <CAF-CG+Ji_JeE6cpy5EMi4uz5Fg1Qa+bU-5fGzdGBPJtAmM6K_w@mail.gmail.com>
 <20160902124834.GB1855@mdounin.ru>
Message-ID: <CAF-CG+KPgDTFvmQ4d3rpFMijKv0POQLTfeWNsxCBs4ARmsS0OQ@mail.gmail.com>

Hey Maxim,

> You are misreading the BUGS section.  It doesn't say that
> SSL_get_peer_certificate() must be always called when
> SSL_get_verify_result() is called.  It says that SSL_get_verify_result() is
> only useful in connection with SSL_get_peer_certificate().

Those 2 sentences are mutually exclusive, if result of
SSL_get_verify_result() is useless without SSL_get_peer_certificate(),
then those two should be called together, or more precisely,
SSL_get_peer_certificate() should be called before
SSL_get_verify_result().

Also, for what it's worth, NGINX & netty-tcnative are the only two
consumers of this OpenSSL API (that we link against BoringSSL) that
use those functions in this order.

> The difference between ngx_ssl_error() and what you've suggested
> is that ngx_ssl_error() doesn't try to cast errors to an nginx rc
> value.  Instead, it uses the error stack saved in the relevant
> connection object.

Except that SSL_get_verify_result() doesn't save its result on the
error stack, so what I suggested is as close to ngx_ssl_error() as
possible.

> As previously suggested, it might be a good solution to use "peer", as
> already used in serveral error messages in ngx_event_openssl.c

Again, could you elaborate why the use of "client" in
ngx_ssl_verify_client() and "upstream" in ngx_ssl_verify_host() is
wrong?

IMHO, the only case in which "peer" would make sense is if we wanted
to introduce common ngx_ssl_verify_peer() function that handles both
cases, which I don't think is a good idea.

Best regards,
Piotr Sikora


From mdounin at mdounin.ru  Sat Sep  3 15:29:14 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sat, 3 Sep 2016 18:29:14 +0300
Subject: [PATCH] SSL: fix order of checks during SSL certificate
 verification
In-Reply-To: <CAF-CG+KPgDTFvmQ4d3rpFMijKv0POQLTfeWNsxCBs4ARmsS0OQ@mail.gmail.com>
References: <20160804035352.GY57459@mdounin.ru>
 <CAF-CG+KBkeG1tWetT0iK2QJBwywZL8cTRwqvrb7mUgKOzbjHww@mail.gmail.com>
 <20160818003702.GE24741@mdounin.ru>
 <CAF-CG+LHx+xb3x+S-N6C0Wek5XsQgXLcfLAC0bkBsg_ZXHvTmw@mail.gmail.com>
 <20160821140211.GH24741@mdounin.ru>
 <CAF-CG++APc6Ct4N2gCFxyJWobdwxYt53ZFjbqTA56FF=hY7Q3w@mail.gmail.com>
 <20160901152553.GY1855@mdounin.ru>
 <CAF-CG+Ji_JeE6cpy5EMi4uz5Fg1Qa+bU-5fGzdGBPJtAmM6K_w@mail.gmail.com>
 <20160902124834.GB1855@mdounin.ru>
 <CAF-CG+KPgDTFvmQ4d3rpFMijKv0POQLTfeWNsxCBs4ARmsS0OQ@mail.gmail.com>
Message-ID: <20160903152914.GB86582@mdounin.ru>

Hello!

On Fri, Sep 02, 2016 at 04:18:53PM -0700, Piotr Sikora wrote:

> Hey Maxim,
> 
> > You are misreading the BUGS section.  It doesn't say that
> > SSL_get_peer_certificate() must be always called when
> > SSL_get_verify_result() is called.  It says that SSL_get_verify_result() is
> > only useful in connection with SSL_get_peer_certificate().
> 
> Those 2 sentences are mutually exclusive, if result of
> SSL_get_verify_result() is useless without SSL_get_peer_certificate(),
> then those two should be called together,

No, your are incorrect here.  "In connection with" means that 
SSL_get_peer_certificate() should be used, but doesn't require it 
to be used always, in all cases.  In particular, 
SSL_get_peer_certificate() is useless when SSL_get_verify_result() 
returns anything but X509_V_OK.

> or more precisely,
> SSL_get_peer_certificate() should be called before
> SSL_get_verify_result().

This is simply not true, sorry.

[...]

> > The difference between ngx_ssl_error() and what you've suggested
> > is that ngx_ssl_error() doesn't try to cast errors to an nginx rc
> > value.  Instead, it uses the error stack saved in the relevant
> > connection object.
> 
> Except that SSL_get_verify_result() doesn't save its result on the
> error stack, so what I suggested is as close to ngx_ssl_error() as
> possible.

What your patch does is what you initially suggested in (2) 
and I objected against.

Obviously enough, SSL_get_verify_result() doesn't use error stack 
in OpenSSL, and implementing something like ngx_ssl_error() (or 
extending ngx_ssl_error() itself) would require additional work to 
save the verify result.

> > As previously suggested, it might be a good solution to use "peer", as
> > already used in serveral error messages in ngx_event_openssl.c
> 
> Again, could you elaborate why the use of "client" in
> ngx_ssl_verify_client() and "upstream" in ngx_ssl_verify_host() is
> wrong?

Because ngx_ssl_verify_host() is expected to be a generic 
function, and it can be used in situations different from talking 
to upstream servers.

-- 
Maxim Dounin
http://nginx.org/


From piotrsikora at google.com  Sat Sep  3 22:27:35 2016
From: piotrsikora at google.com (Piotr Sikora)
Date: Sat, 3 Sep 2016 15:27:35 -0700
Subject: [PATCH] SSL: fix order of checks during SSL certificate
 verification
In-Reply-To: <20160903152914.GB86582@mdounin.ru>
References: <20160804035352.GY57459@mdounin.ru>
 <CAF-CG+KBkeG1tWetT0iK2QJBwywZL8cTRwqvrb7mUgKOzbjHww@mail.gmail.com>
 <20160818003702.GE24741@mdounin.ru>
 <CAF-CG+LHx+xb3x+S-N6C0Wek5XsQgXLcfLAC0bkBsg_ZXHvTmw@mail.gmail.com>
 <20160821140211.GH24741@mdounin.ru>
 <CAF-CG++APc6Ct4N2gCFxyJWobdwxYt53ZFjbqTA56FF=hY7Q3w@mail.gmail.com>
 <20160901152553.GY1855@mdounin.ru>
 <CAF-CG+Ji_JeE6cpy5EMi4uz5Fg1Qa+bU-5fGzdGBPJtAmM6K_w@mail.gmail.com>
 <20160902124834.GB1855@mdounin.ru>
 <CAF-CG+KPgDTFvmQ4d3rpFMijKv0POQLTfeWNsxCBs4ARmsS0OQ@mail.gmail.com>
 <20160903152914.GB86582@mdounin.ru>
Message-ID: <CAF-CG+Kjj_dbC6OC=r-EDfa+ppn7rxExm0PB4FpQPSNNWDRW+A@mail.gmail.com>

Hey Maxim,

> No, your are incorrect here.  "In connection with" means that
> SSL_get_peer_certificate() should be used, but doesn't require it
> to be used always, in all cases.  In particular,
> SSL_get_peer_certificate() is useless when SSL_get_verify_result()
> returns anything but X509_V_OK.

Sigh, why do you insist on checking status of verification of client
certificate that wasn't sent in the first place?

> Because ngx_ssl_verify_host() is expected to be a generic
> function, and it can be used in situations different from talking
> to upstream servers.

Like what, exactly?

Also, for the record, are you fine with "client" in
ngx_ssl_verify_client() or is that also expected to be generic
function?

Best regards,
Piotr Sikora


From mdounin at mdounin.ru  Mon Sep  5 14:16:03 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 5 Sep 2016 17:16:03 +0300
Subject: [PATCH] SSL: fix order of checks during SSL certificate
 verification
In-Reply-To: <CAF-CG+Kjj_dbC6OC=r-EDfa+ppn7rxExm0PB4FpQPSNNWDRW+A@mail.gmail.com>
References: <20160818003702.GE24741@mdounin.ru>
 <CAF-CG+LHx+xb3x+S-N6C0Wek5XsQgXLcfLAC0bkBsg_ZXHvTmw@mail.gmail.com>
 <20160821140211.GH24741@mdounin.ru>
 <CAF-CG++APc6Ct4N2gCFxyJWobdwxYt53ZFjbqTA56FF=hY7Q3w@mail.gmail.com>
 <20160901152553.GY1855@mdounin.ru>
 <CAF-CG+Ji_JeE6cpy5EMi4uz5Fg1Qa+bU-5fGzdGBPJtAmM6K_w@mail.gmail.com>
 <20160902124834.GB1855@mdounin.ru>
 <CAF-CG+KPgDTFvmQ4d3rpFMijKv0POQLTfeWNsxCBs4ARmsS0OQ@mail.gmail.com>
 <20160903152914.GB86582@mdounin.ru>
 <CAF-CG+Kjj_dbC6OC=r-EDfa+ppn7rxExm0PB4FpQPSNNWDRW+A@mail.gmail.com>
Message-ID: <20160905141603.GD86582@mdounin.ru>

Hello!

On Sat, Sep 03, 2016 at 03:27:35PM -0700, Piotr Sikora wrote:

> Hey Maxim,
> 
> > No, your are incorrect here.  "In connection with" means that
> > SSL_get_peer_certificate() should be used, but doesn't require it
> > to be used always, in all cases.  In particular,
> > SSL_get_peer_certificate() is useless when SSL_get_verify_result()
> > returns anything but X509_V_OK.
> 
> Sigh, why do you insist on checking status of verification of client
> certificate that wasn't sent in the first place?

It's not me who insist on anything.  It's you who insist that the 
current code is wrong.  It's not.

> > Because ngx_ssl_verify_host() is expected to be a generic
> > function, and it can be used in situations different from talking
> > to upstream servers.
> 
> Like what, exactly?

For example, it can be used to verify a host of auth_http server 
in mail, or OCSP responder - if we'll implement SSL there.

> Also, for the record, are you fine with "client" in
> ngx_ssl_verify_client() or is that also expected to be generic
> function?

Yes, more or less.  I'm not fine with the ngx_ssl_verify_client() 
implementation as suggested in patches I've seen so far, as it 
seems too biased to the current use of client verification in http 
module, but it's a different question.

-- 
Maxim Dounin
http://nginx.org/


From vl at nginx.com  Mon Sep  5 14:53:18 2016
From: vl at nginx.com (Vladimir Homutov)
Date: Mon, 05 Sep 2016 14:53:18 +0000
Subject: [nginx] Stream: log module.
Message-ID: <hg.0125b151c9a5.1473087198.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/0125b151c9a5
branches:  
changeset: 6678:0125b151c9a5
user:      Vladimir Homutov <vl at nginx.com>
date:      Mon Sep 05 17:50:16 2016 +0300
description:
Stream: log module.

diffstat:

 auto/modules                       |     2 +
 src/stream/ngx_stream.h            |     1 +
 src/stream/ngx_stream_handler.c    |     8 +
 src/stream/ngx_stream_log_module.c |  1474 ++++++++++++++++++++++++++++++++++++
 src/stream/ngx_stream_script.c     |    67 +
 src/stream/ngx_stream_script.h     |     4 +
 6 files changed, 1556 insertions(+), 0 deletions(-)

diffs (truncated from 1622 to 1000 lines):

diff -r c02290241cbe -r 0125b151c9a5 auto/modules
--- a/auto/modules	Fri Sep 02 18:27:12 2016 +0300
+++ b/auto/modules	Mon Sep 05 17:50:16 2016 +0300
@@ -971,6 +971,7 @@ if [ $STREAM != NO ]; then
 
     ngx_module_name="ngx_stream_module \
                      ngx_stream_core_module \
+                     ngx_stream_log_module \
                      ngx_stream_proxy_module \
                      ngx_stream_upstream_module"
     ngx_module_incs="src/stream"
@@ -984,6 +985,7 @@ if [ $STREAM != NO ]; then
                      src/stream/ngx_stream_script.c \
                      src/stream/ngx_stream_handler.c \
                      src/stream/ngx_stream_core_module.c \
+                     src/stream/ngx_stream_log_module.c \
                      src/stream/ngx_stream_proxy_module.c \
                      src/stream/ngx_stream_upstream.c \
                      src/stream/ngx_stream_upstream_round_robin.c"
diff -r c02290241cbe -r 0125b151c9a5 src/stream/ngx_stream.h
--- a/src/stream/ngx_stream.h	Fri Sep 02 18:27:12 2016 +0300
+++ b/src/stream/ngx_stream.h	Mon Sep 05 17:50:16 2016 +0300
@@ -121,6 +121,7 @@ typedef struct {
 
     ngx_stream_access_pt           limit_conn_handler;
     ngx_stream_access_pt           access_handler;
+    ngx_stream_access_pt           access_log_handler;
 
     ngx_hash_t                     variables_hash;
 
diff -r c02290241cbe -r 0125b151c9a5 src/stream/ngx_stream_handler.c
--- a/src/stream/ngx_stream_handler.c	Fri Sep 02 18:27:12 2016 +0300
+++ b/src/stream/ngx_stream_handler.c	Mon Sep 05 17:50:16 2016 +0300
@@ -311,11 +311,19 @@ ngx_stream_ssl_handshake_handler(ngx_con
 void
 ngx_stream_finalize_session(ngx_stream_session_t *s, ngx_uint_t rc)
 {
+    ngx_stream_core_main_conf_t  *cmcf;
+
     ngx_log_debug1(NGX_LOG_DEBUG_STREAM, s->connection->log, 0,
                    "finalize stream session: %i", rc);
 
     s->status = rc;
 
+    cmcf = ngx_stream_get_module_main_conf(s, ngx_stream_core_module);
+
+    if (cmcf->access_log_handler) {
+        (void) cmcf->access_log_handler(s);
+    }
+
     ngx_stream_close_connection(s->connection);
 }
 
diff -r c02290241cbe -r 0125b151c9a5 src/stream/ngx_stream_log_module.c
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/src/stream/ngx_stream_log_module.c	Mon Sep 05 17:50:16 2016 +0300
@@ -0,0 +1,1474 @@
+
+/*
+ * Copyright (C) Igor Sysoev
+ * Copyright (C) Nginx, Inc.
+ */
+
+
+#include <ngx_config.h>
+#include <ngx_core.h>
+#include <ngx_stream.h>
+
+#if (NGX_ZLIB)
+#include <zlib.h>
+#endif
+
+
+typedef struct ngx_stream_log_op_s  ngx_stream_log_op_t;
+
+typedef u_char *(*ngx_stream_log_op_run_pt) (ngx_stream_session_t *s,
+    u_char *buf, ngx_stream_log_op_t *op);
+
+typedef size_t (*ngx_stream_log_op_getlen_pt) (ngx_stream_session_t *s,
+    uintptr_t data);
+
+
+struct ngx_stream_log_op_s {
+    size_t                       len;
+    ngx_stream_log_op_getlen_pt  getlen;
+    ngx_stream_log_op_run_pt     run;
+    uintptr_t                    data;
+};
+
+
+typedef struct {
+    ngx_str_t                    name;
+    ngx_array_t                 *flushes;
+    ngx_array_t                 *ops;        /* array of ngx_stream_log_op_t */
+} ngx_stream_log_fmt_t;
+
+
+typedef struct {
+    ngx_array_t                  formats;    /* array of ngx_stream_log_fmt_t */
+} ngx_stream_log_main_conf_t;
+
+
+typedef struct {
+    u_char                      *start;
+    u_char                      *pos;
+    u_char                      *last;
+
+    ngx_event_t                 *event;
+    ngx_msec_t                   flush;
+    ngx_int_t                    gzip;
+} ngx_stream_log_buf_t;
+
+
+typedef struct {
+    ngx_array_t                 *lengths;
+    ngx_array_t                 *values;
+} ngx_stream_log_script_t;
+
+
+typedef struct {
+    ngx_open_file_t             *file;
+    ngx_stream_log_script_t     *script;
+    time_t                       disk_full_time;
+    time_t                       error_log_time;
+    ngx_syslog_peer_t           *syslog_peer;
+    ngx_stream_log_fmt_t        *format;
+    ngx_stream_complex_value_t  *filter;
+} ngx_stream_log_t;
+
+
+typedef struct {
+    ngx_array_t                 *logs;       /* array of ngx_stream_log_t */
+
+    ngx_open_file_cache_t       *open_file_cache;
+    time_t                       open_file_cache_valid;
+    ngx_uint_t                   open_file_cache_min_uses;
+
+    ngx_uint_t                   off;        /* unsigned  off:1 */
+} ngx_stream_log_srv_conf_t;
+
+
+typedef struct {
+    ngx_str_t                    name;
+    size_t                       len;
+    ngx_stream_log_op_run_pt     run;
+} ngx_stream_log_var_t;
+
+
+static void ngx_stream_log_write(ngx_stream_session_t *s, ngx_stream_log_t *log,
+    u_char *buf, size_t len);
+static ssize_t ngx_stream_log_script_write(ngx_stream_session_t *s,
+    ngx_stream_log_script_t *script, u_char **name, u_char *buf, size_t len);
+
+#if (NGX_ZLIB)
+static ssize_t ngx_stream_log_gzip(ngx_fd_t fd, u_char *buf, size_t len,
+    ngx_int_t level, ngx_log_t *log);
+
+static void *ngx_stream_log_gzip_alloc(void *opaque, u_int items, u_int size);
+static void ngx_stream_log_gzip_free(void *opaque, void *address);
+#endif
+
+static void ngx_stream_log_flush(ngx_open_file_t *file, ngx_log_t *log);
+static void ngx_stream_log_flush_handler(ngx_event_t *ev);
+
+static ngx_int_t ngx_stream_log_variable_compile(ngx_conf_t *cf,
+    ngx_stream_log_op_t *op, ngx_str_t *value);
+static size_t ngx_stream_log_variable_getlen(ngx_stream_session_t *s,
+    uintptr_t data);
+static u_char *ngx_stream_log_variable(ngx_stream_session_t *s, u_char *buf,
+    ngx_stream_log_op_t *op);
+static uintptr_t ngx_stream_log_escape(u_char *dst, u_char *src, size_t size);
+
+
+static void *ngx_stream_log_create_main_conf(ngx_conf_t *cf);
+static void *ngx_stream_log_create_srv_conf(ngx_conf_t *cf);
+static char *ngx_stream_log_merge_srv_conf(ngx_conf_t *cf, void *parent,
+    void *child);
+static char *ngx_stream_log_set_log(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf);
+static char *ngx_stream_log_set_format(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf);
+static char *ngx_stream_log_compile_format(ngx_conf_t *cf,
+    ngx_array_t *flushes, ngx_array_t *ops, ngx_array_t *args, ngx_uint_t s);
+static char *ngx_stream_log_open_file_cache(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf);
+static ngx_int_t ngx_stream_log_init(ngx_conf_t *cf);
+
+
+static ngx_command_t  ngx_stream_log_commands[] = {
+
+    { ngx_string("log_format"),
+      NGX_STREAM_MAIN_CONF|NGX_CONF_2MORE,
+      ngx_stream_log_set_format,
+      NGX_STREAM_MAIN_CONF_OFFSET,
+      0,
+      NULL },
+
+    { ngx_string("access_log"),
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_1MORE,
+      ngx_stream_log_set_log,
+      NGX_STREAM_SRV_CONF_OFFSET,
+      0,
+      NULL },
+
+    { ngx_string("open_log_file_cache"),
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_TAKE1234,
+      ngx_stream_log_open_file_cache,
+      NGX_STREAM_SRV_CONF_OFFSET,
+      0,
+      NULL },
+
+      ngx_null_command
+};
+
+
+static ngx_stream_module_t  ngx_stream_log_module_ctx = {
+    NULL,                                  /* preconfiguration */
+    ngx_stream_log_init,                   /* postconfiguration */
+
+    ngx_stream_log_create_main_conf,       /* create main configuration */
+    NULL,                                  /* init main configuration */
+
+    ngx_stream_log_create_srv_conf,        /* create server configuration */
+    ngx_stream_log_merge_srv_conf          /* merge server configuration */
+};
+
+
+ngx_module_t  ngx_stream_log_module = {
+    NGX_MODULE_V1,
+    &ngx_stream_log_module_ctx,            /* module context */
+    ngx_stream_log_commands,               /* module directives */
+    NGX_STREAM_MODULE,                     /* module type */
+    NULL,                                  /* init master */
+    NULL,                                  /* init module */
+    NULL,                                  /* init process */
+    NULL,                                  /* init thread */
+    NULL,                                  /* exit thread */
+    NULL,                                  /* exit process */
+    NULL,                                  /* exit master */
+    NGX_MODULE_V1_PADDING
+};
+
+
+static ngx_int_t
+ngx_stream_log_handler(ngx_stream_session_t *s)
+{
+    u_char                     *line, *p;
+    size_t                      len, size;
+    ssize_t                     n;
+    ngx_str_t                   val;
+    ngx_uint_t                  i, l;
+    ngx_stream_log_t           *log;
+    ngx_stream_log_op_t        *op;
+    ngx_stream_log_buf_t       *buffer;
+    ngx_stream_log_srv_conf_t  *lscf;
+
+    ngx_log_debug0(NGX_LOG_DEBUG_STREAM, s->connection->log, 0,
+                   "stream log handler");
+
+    lscf = ngx_stream_get_module_srv_conf(s, ngx_stream_log_module);
+
+    if (lscf->off || lscf->logs == NULL) {
+        return NGX_OK;
+    }
+
+    log = lscf->logs->elts;
+    for (l = 0; l < lscf->logs->nelts; l++) {
+
+        if (log[l].filter) {
+            if (ngx_stream_complex_value(s, log[l].filter, &val) != NGX_OK) {
+                return NGX_ERROR;
+            }
+
+            if (val.len == 0 || (val.len == 1 && val.data[0] == '0')) {
+                continue;
+            }
+        }
+
+        if (ngx_time() == log[l].disk_full_time) {
+
+            /*
+             * on FreeBSD writing to a full filesystem with enabled softupdates
+             * may block process for much longer time than writing to non-full
+             * filesystem, so we skip writing to a log for one second
+             */
+
+            continue;
+        }
+
+        ngx_stream_script_flush_no_cacheable_variables(s,
+                                                       log[l].format->flushes);
+
+        len = 0;
+        op = log[l].format->ops->elts;
+        for (i = 0; i < log[l].format->ops->nelts; i++) {
+            if (op[i].len == 0) {
+                len += op[i].getlen(s, op[i].data);
+
+            } else {
+                len += op[i].len;
+            }
+        }
+
+        if (log[l].syslog_peer) {
+
+            /* length of syslog's PRI and HEADER message parts */
+            len += sizeof("<255>Jan 01 00:00:00 ") - 1
+                   + ngx_cycle->hostname.len + 1
+                   + log[l].syslog_peer->tag.len + 2;
+
+            goto alloc_line;
+        }
+
+        len += NGX_LINEFEED_SIZE;
+
+        buffer = log[l].file ? log[l].file->data : NULL;
+
+        if (buffer) {
+
+            if (len > (size_t) (buffer->last - buffer->pos)) {
+
+                ngx_stream_log_write(s, &log[l], buffer->start,
+                                     buffer->pos - buffer->start);
+
+                buffer->pos = buffer->start;
+            }
+
+            if (len <= (size_t) (buffer->last - buffer->pos)) {
+
+                p = buffer->pos;
+
+                if (buffer->event && p == buffer->start) {
+                    ngx_add_timer(buffer->event, buffer->flush);
+                }
+
+                for (i = 0; i < log[l].format->ops->nelts; i++) {
+                    p = op[i].run(s, p, &op[i]);
+                }
+
+                ngx_linefeed(p);
+
+                buffer->pos = p;
+
+                continue;
+            }
+
+            if (buffer->event && buffer->event->timer_set) {
+                ngx_del_timer(buffer->event);
+            }
+        }
+
+    alloc_line:
+
+        line = ngx_pnalloc(s->connection->pool, len);
+        if (line == NULL) {
+            return NGX_ERROR;
+        }
+
+        p = line;
+
+        if (log[l].syslog_peer) {
+            p = ngx_syslog_add_header(log[l].syslog_peer, line);
+        }
+
+        for (i = 0; i < log[l].format->ops->nelts; i++) {
+            p = op[i].run(s, p, &op[i]);
+        }
+
+        if (log[l].syslog_peer) {
+
+            size = p - line;
+
+            n = ngx_syslog_send(log[l].syslog_peer, line, size);
+
+            if (n < 0) {
+                ngx_log_error(NGX_LOG_WARN, s->connection->log, 0,
+                              "send() to syslog failed");
+
+            } else if ((size_t) n != size) {
+                ngx_log_error(NGX_LOG_WARN, s->connection->log, 0,
+                              "send() to syslog has written only %z of %uz",
+                              n, size);
+            }
+
+            continue;
+        }
+
+        ngx_linefeed(p);
+
+        ngx_stream_log_write(s, &log[l], line, p - line);
+    }
+
+    return NGX_OK;
+}
+
+
+static void
+ngx_stream_log_write(ngx_stream_session_t *s, ngx_stream_log_t *log,
+    u_char *buf, size_t len)
+{
+    u_char                *name;
+    time_t                 now;
+    ssize_t                n;
+    ngx_err_t              err;
+#if (NGX_ZLIB)
+    ngx_stream_log_buf_t  *buffer;
+#endif
+
+    if (log->script == NULL) {
+        name = log->file->name.data;
+
+#if (NGX_ZLIB)
+        buffer = log->file->data;
+
+        if (buffer && buffer->gzip) {
+            n = ngx_stream_log_gzip(log->file->fd, buf, len, buffer->gzip,
+                                    s->connection->log);
+        } else {
+            n = ngx_write_fd(log->file->fd, buf, len);
+        }
+#else
+        n = ngx_write_fd(log->file->fd, buf, len);
+#endif
+
+    } else {
+        name = NULL;
+        n = ngx_stream_log_script_write(s, log->script, &name, buf, len);
+    }
+
+    if (n == (ssize_t) len) {
+        return;
+    }
+
+    now = ngx_time();
+
+    if (n == -1) {
+        err = ngx_errno;
+
+        if (err == NGX_ENOSPC) {
+            log->disk_full_time = now;
+        }
+
+        if (now - log->error_log_time > 59) {
+            ngx_log_error(NGX_LOG_ALERT, s->connection->log, err,
+                          ngx_write_fd_n " to \"%s\" failed", name);
+
+            log->error_log_time = now;
+        }
+
+        return;
+    }
+
+    if (now - log->error_log_time > 59) {
+        ngx_log_error(NGX_LOG_ALERT, s->connection->log, 0,
+                      ngx_write_fd_n " to \"%s\" was incomplete: %z of %uz",
+                      name, n, len);
+
+        log->error_log_time = now;
+    }
+}
+
+
+static ssize_t
+ngx_stream_log_script_write(ngx_stream_session_t *s,
+    ngx_stream_log_script_t *script, u_char **name, u_char *buf, size_t len)
+{
+    ssize_t                     n;
+    ngx_str_t                   log;
+    ngx_open_file_info_t        of;
+    ngx_stream_log_srv_conf_t  *lscf;
+
+    if (ngx_stream_script_run(s, &log, script->lengths->elts, 1,
+                              script->values->elts)
+        == NULL)
+    {
+        /* simulate successful logging */
+        return len;
+    }
+
+    log.data[log.len - 1] = '\0';
+    *name = log.data;
+
+    ngx_log_debug1(NGX_LOG_DEBUG_STREAM, s->connection->log, 0,
+                   "stream log \"%s\"", log.data);
+
+    lscf = ngx_stream_get_module_srv_conf(s, ngx_stream_log_module);
+
+    ngx_memzero(&of, sizeof(ngx_open_file_info_t));
+
+    of.log = 1;
+    of.valid = lscf->open_file_cache_valid;
+    of.min_uses = lscf->open_file_cache_min_uses;
+    of.directio = NGX_OPEN_FILE_DIRECTIO_OFF;
+
+    if (ngx_open_cached_file(lscf->open_file_cache, &log, &of,
+                             s->connection->pool)
+        != NGX_OK)
+    {
+        ngx_log_error(NGX_LOG_CRIT, s->connection->log, ngx_errno,
+                      "%s \"%s\" failed", of.failed, log.data);
+        /* simulate successful logging */
+        return len;
+    }
+
+    ngx_log_debug1(NGX_LOG_DEBUG_STREAM, s->connection->log, 0,
+                   "stream log #%d", of.fd);
+
+    n = ngx_write_fd(of.fd, buf, len);
+
+    return n;
+}
+
+
+#if (NGX_ZLIB)
+
+static ssize_t
+ngx_stream_log_gzip(ngx_fd_t fd, u_char *buf, size_t len, ngx_int_t level,
+    ngx_log_t *log)
+{
+    int          rc, wbits, memlevel;
+    u_char      *out;
+    size_t       size;
+    ssize_t      n;
+    z_stream     zstream;
+    ngx_err_t    err;
+    ngx_pool_t  *pool;
+
+    wbits = MAX_WBITS;
+    memlevel = MAX_MEM_LEVEL - 1;
+
+    while ((ssize_t) len < ((1 << (wbits - 1)) - 262)) {
+        wbits--;
+        memlevel--;
+    }
+
+    /*
+     * This is a formula from deflateBound() for conservative upper bound of
+     * compressed data plus 18 bytes of gzip wrapper.
+     */
+
+    size = len + ((len + 7) >> 3) + ((len + 63) >> 6) + 5 + 18;
+
+    ngx_memzero(&zstream, sizeof(z_stream));
+
+    pool = ngx_create_pool(256, log);
+    if (pool == NULL) {
+        /* simulate successful logging */
+        return len;
+    }
+
+    pool->log = log;
+
+    zstream.zalloc = ngx_stream_log_gzip_alloc;
+    zstream.zfree = ngx_stream_log_gzip_free;
+    zstream.opaque = pool;
+
+    out = ngx_pnalloc(pool, size);
+    if (out == NULL) {
+        goto done;
+    }
+
+    zstream.next_in = buf;
+    zstream.avail_in = len;
+    zstream.next_out = out;
+    zstream.avail_out = size;
+
+    rc = deflateInit2(&zstream, (int) level, Z_DEFLATED, wbits + 16, memlevel,
+                      Z_DEFAULT_STRATEGY);
+
+    if (rc != Z_OK) {
+        ngx_log_error(NGX_LOG_ALERT, log, 0, "deflateInit2() failed: %d", rc);
+        goto done;
+    }
+
+    ngx_log_debug4(NGX_LOG_DEBUG_STREAM, log, 0,
+                   "deflate in: ni:%p no:%p ai:%ud ao:%ud",
+                   zstream.next_in, zstream.next_out,
+                   zstream.avail_in, zstream.avail_out);
+
+    rc = deflate(&zstream, Z_FINISH);
+
+    if (rc != Z_STREAM_END) {
+        ngx_log_error(NGX_LOG_ALERT, log, 0,
+                      "deflate(Z_FINISH) failed: %d", rc);
+        goto done;
+    }
+
+    ngx_log_debug5(NGX_LOG_DEBUG_STREAM, log, 0,
+                   "deflate out: ni:%p no:%p ai:%ud ao:%ud rc:%d",
+                   zstream.next_in, zstream.next_out,
+                   zstream.avail_in, zstream.avail_out,
+                   rc);
+
+    size -= zstream.avail_out;
+
+    rc = deflateEnd(&zstream);
+
+    if (rc != Z_OK) {
+        ngx_log_error(NGX_LOG_ALERT, log, 0, "deflateEnd() failed: %d", rc);
+        goto done;
+    }
+
+    n = ngx_write_fd(fd, out, size);
+
+    if (n != (ssize_t) size) {
+        err = (n == -1) ? ngx_errno : 0;
+
+        ngx_destroy_pool(pool);
+
+        ngx_set_errno(err);
+        return -1;
+    }
+
+done:
+
+    ngx_destroy_pool(pool);
+
+    /* simulate successful logging */
+    return len;
+}
+
+
+static void *
+ngx_stream_log_gzip_alloc(void *opaque, u_int items, u_int size)
+{
+    ngx_pool_t *pool = opaque;
+
+    ngx_log_debug2(NGX_LOG_DEBUG_STREAM, pool->log, 0,
+                   "gzip alloc: n:%ud s:%ud", items, size);
+
+    return ngx_palloc(pool, items * size);
+}
+
+
+static void
+ngx_stream_log_gzip_free(void *opaque, void *address)
+{
+#if 0
+    ngx_pool_t *pool = opaque;
+
+    ngx_log_debug1(NGX_LOG_DEBUG_STREAM, pool->log, 0,
+                   "gzip free: %p", address);
+#endif
+}
+
+#endif
+
+
+static void
+ngx_stream_log_flush(ngx_open_file_t *file, ngx_log_t *log)
+{
+    size_t                 len;
+    ssize_t                n;
+    ngx_stream_log_buf_t  *buffer;
+
+    buffer = file->data;
+
+    len = buffer->pos - buffer->start;
+
+    if (len == 0) {
+        return;
+    }
+
+#if (NGX_ZLIB)
+    if (buffer->gzip) {
+        n = ngx_stream_log_gzip(file->fd, buffer->start, len, buffer->gzip,
+                                log);
+    } else {
+        n = ngx_write_fd(file->fd, buffer->start, len);
+    }
+#else
+    n = ngx_write_fd(file->fd, buffer->start, len);
+#endif
+
+    if (n == -1) {
+        ngx_log_error(NGX_LOG_ALERT, log, ngx_errno,
+                      ngx_write_fd_n " to \"%s\" failed",
+                      file->name.data);
+
+    } else if ((size_t) n != len) {
+        ngx_log_error(NGX_LOG_ALERT, log, 0,
+                      ngx_write_fd_n " to \"%s\" was incomplete: %z of %uz",
+                      file->name.data, n, len);
+    }
+
+    buffer->pos = buffer->start;
+
+    if (buffer->event && buffer->event->timer_set) {
+        ngx_del_timer(buffer->event);
+    }
+}
+
+
+static void
+ngx_stream_log_flush_handler(ngx_event_t *ev)
+{
+    ngx_open_file_t       *file;
+    ngx_stream_log_buf_t  *buffer;
+
+    ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ev->log, 0,
+                   "stream log buffer flush handler");
+
+    if (ev->timedout) {
+        ngx_stream_log_flush(ev->data, ev->log);
+        return;
+    }
+
+    /* cancel the flush timer for graceful shutdown */
+
+    file = ev->data;
+    buffer = file->data;
+
+    buffer->event = NULL;
+}
+
+
+static u_char *
+ngx_stream_log_copy_short(ngx_stream_session_t *s, u_char *buf,
+    ngx_stream_log_op_t *op)
+{
+    size_t     len;
+    uintptr_t  data;
+
+    len = op->len;
+    data = op->data;
+
+    while (len--) {
+        *buf++ = (u_char) (data & 0xff);
+        data >>= 8;
+    }
+
+    return buf;
+}
+
+
+static u_char *
+ngx_stream_log_copy_long(ngx_stream_session_t *s, u_char *buf,
+    ngx_stream_log_op_t *op)
+{
+    return ngx_cpymem(buf, (u_char *) op->data, op->len);
+}
+
+
+static ngx_int_t
+ngx_stream_log_variable_compile(ngx_conf_t *cf, ngx_stream_log_op_t *op,
+    ngx_str_t *value)
+{
+    ngx_int_t  index;
+
+    index = ngx_stream_get_variable_index(cf, value);
+    if (index == NGX_ERROR) {
+        return NGX_ERROR;
+    }
+
+    op->len = 0;
+    op->getlen = ngx_stream_log_variable_getlen;
+    op->run = ngx_stream_log_variable;
+    op->data = index;
+
+    return NGX_OK;
+}
+
+
+static size_t
+ngx_stream_log_variable_getlen(ngx_stream_session_t *s, uintptr_t data)
+{
+    uintptr_t                     len;
+    ngx_stream_variable_value_t  *value;
+
+    value = ngx_stream_get_indexed_variable(s, data);
+
+    if (value == NULL || value->not_found) {
+        return 1;
+    }
+
+    len = ngx_stream_log_escape(NULL, value->data, value->len);
+
+    value->escape = len ? 1 : 0;
+
+    return value->len + len * 3;
+}
+
+
+static u_char *
+ngx_stream_log_variable(ngx_stream_session_t *s, u_char *buf,
+    ngx_stream_log_op_t *op)
+{
+    ngx_stream_variable_value_t  *value;
+
+    value = ngx_stream_get_indexed_variable(s, op->data);
+
+    if (value == NULL || value->not_found) {
+        *buf = '-';
+        return buf + 1;
+    }
+
+    if (value->escape == 0) {
+        return ngx_cpymem(buf, value->data, value->len);
+
+    } else {
+        return (u_char *) ngx_stream_log_escape(buf, value->data, value->len);
+    }
+}
+
+
+static uintptr_t
+ngx_stream_log_escape(u_char *dst, u_char *src, size_t size)
+{
+    ngx_uint_t      n;
+    static u_char   hex[] = "0123456789ABCDEF";
+
+    static uint32_t   escape[] = {
+        0xffffffff, /* 1111 1111 1111 1111  1111 1111 1111 1111 */
+
+                    /* ?>=< ;:98 7654 3210  /.-, +*)( '&%$ #"!  */
+        0x00000004, /* 0000 0000 0000 0000  0000 0000 0000 0100 */
+
+                    /* _^]\ [ZYX WVUT SRQP  ONML KJIH GFED CBA@ */
+        0x10000000, /* 0001 0000 0000 0000  0000 0000 0000 0000 */
+
+                    /*  ~}| {zyx wvut srqp  onml kjih gfed cba` */
+        0x80000000, /* 1000 0000 0000 0000  0000 0000 0000 0000 */
+
+        0xffffffff, /* 1111 1111 1111 1111  1111 1111 1111 1111 */
+        0xffffffff, /* 1111 1111 1111 1111  1111 1111 1111 1111 */
+        0xffffffff, /* 1111 1111 1111 1111  1111 1111 1111 1111 */
+        0xffffffff, /* 1111 1111 1111 1111  1111 1111 1111 1111 */
+    };
+
+
+    if (dst == NULL) {
+
+        /* find the number of the characters to be escaped */
+
+        n = 0;
+
+        while (size) {
+            if (escape[*src >> 5] & (1U << (*src & 0x1f))) {
+                n++;
+            }
+            src++;
+            size--;
+        }
+
+        return (uintptr_t) n;
+    }
+
+    while (size) {
+        if (escape[*src >> 5] & (1U << (*src & 0x1f))) {
+            *dst++ = '\\';
+            *dst++ = 'x';
+            *dst++ = hex[*src >> 4];
+            *dst++ = hex[*src & 0xf];
+            src++;
+
+        } else {
+            *dst++ = *src++;
+        }
+        size--;
+    }
+
+    return (uintptr_t) dst;
+}
+
+
+static void *
+ngx_stream_log_create_main_conf(ngx_conf_t *cf)
+{
+    ngx_stream_log_main_conf_t  *conf;
+
+    conf = ngx_pcalloc(cf->pool, sizeof(ngx_stream_log_main_conf_t));
+    if (conf == NULL) {
+        return NULL;
+    }
+
+    if (ngx_array_init(&conf->formats, cf->pool, 4,
+                       sizeof(ngx_stream_log_fmt_t))
+        != NGX_OK)
+    {
+        return NULL;
+    }
+
+    return conf;
+}
+
+
+static void *
+ngx_stream_log_create_srv_conf(ngx_conf_t *cf)
+{
+    ngx_stream_log_srv_conf_t  *conf;
+
+    conf = ngx_pcalloc(cf->pool, sizeof(ngx_stream_log_srv_conf_t));
+    if (conf == NULL) {
+        return NULL;
+    }
+
+    conf->open_file_cache = NGX_CONF_UNSET_PTR;
+
+    return conf;
+}
+
+
+static char *
+ngx_stream_log_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child)
+{
+    ngx_stream_log_srv_conf_t *prev = parent;
+    ngx_stream_log_srv_conf_t *conf = child;
+
+    if (conf->open_file_cache == NGX_CONF_UNSET_PTR) {
+
+        conf->open_file_cache = prev->open_file_cache;
+        conf->open_file_cache_valid = prev->open_file_cache_valid;
+        conf->open_file_cache_min_uses = prev->open_file_cache_min_uses;
+
+        if (conf->open_file_cache == NGX_CONF_UNSET_PTR) {
+            conf->open_file_cache = NULL;
+        }
+    }
+
+    if (conf->logs || conf->off) {
+        return NGX_CONF_OK;
+    }
+
+    conf->logs = prev->logs;
+    conf->off = prev->off;
+
+    return NGX_CONF_OK;
+}
+
+
+static char *
+ngx_stream_log_set_log(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
+{
+    ngx_stream_log_srv_conf_t *lscf = conf;
+
+    ssize_t                              size;
+    ngx_int_t                            gzip;
+    ngx_uint_t                           i, n;
+    ngx_msec_t                           flush;
+    ngx_str_t                           *value, name, s;
+    ngx_stream_log_t                    *log;
+    ngx_syslog_peer_t                   *peer;
+    ngx_stream_log_buf_t                *buffer;
+    ngx_stream_log_fmt_t                *fmt;
+    ngx_stream_script_compile_t          sc;
+    ngx_stream_log_main_conf_t          *lmcf;
+    ngx_stream_compile_complex_value_t   ccv;
+
+    value = cf->args->elts;
+
+    if (ngx_strcmp(value[1].data, "off") == 0) {
+        lscf->off = 1;
+        if (cf->args->nelts == 2) {
+            return NGX_CONF_OK;
+        }
+
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                           "invalid parameter \"%V\"", &value[2]);
+        return NGX_CONF_ERROR;
+    }
+
+    if (lscf->logs == NULL) {
+        lscf->logs = ngx_array_create(cf->pool, 2, sizeof(ngx_stream_log_t));
+        if (lscf->logs == NULL) {
+            return NGX_CONF_ERROR;
+        }
+    }
+
+    lmcf = ngx_stream_conf_get_module_main_conf(cf, ngx_stream_log_module);
+
+    log = ngx_array_push(lscf->logs);
+    if (log == NULL) {
+        return NGX_CONF_ERROR;
+    }
+
+    ngx_memzero(log, sizeof(ngx_stream_log_t));
+
+
+    if (ngx_strncmp(value[1].data, "syslog:", 7) == 0) {
+
+        peer = ngx_pcalloc(cf->pool, sizeof(ngx_syslog_peer_t));
+        if (peer == NULL) {
+            return NGX_CONF_ERROR;
+        }
+
+        if (ngx_syslog_process_conf(cf, peer) != NGX_CONF_OK) {
+            return NGX_CONF_ERROR;
+        }
+
+        log->syslog_peer = peer;
+
+        goto process_formats;
+    }
+
+    n = ngx_stream_script_variables_count(&value[1]);
+
+    if (n == 0) {
+        log->file = ngx_conf_open_file(cf->cycle, &value[1]);
+        if (log->file == NULL) {
+            return NGX_CONF_ERROR;


From xeioex at nginx.com  Wed Sep  7 15:28:41 2016
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 07 Sep 2016 15:28:41 +0000
Subject: [nginx] Stream: postpone session initialization under accept mutex.
Message-ID: <hg.40e8ce405859.1473262121.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/40e8ce405859
branches:  
changeset: 6679:40e8ce405859
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Sep 06 21:28:13 2016 +0300
description:
Stream: postpone session initialization under accept mutex.

Previously, it was possible that some system calls could be
invoked while holding the accept mutex.  This is clearly
wrong as it prevents incoming connections from being accepted
as quickly as possible.

diffstat:

 src/stream/ngx_stream.h         |   4 ++++
 src/stream/ngx_stream_handler.c |  41 ++++++++++++++++++++++++++++++++++++++---
 2 files changed, 42 insertions(+), 3 deletions(-)

diffs (104 lines):

diff -r 0125b151c9a5 -r 40e8ce405859 src/stream/ngx_stream.h
--- a/src/stream/ngx_stream.h	Mon Sep 05 17:50:16 2016 +0300
+++ b/src/stream/ngx_stream.h	Tue Sep 06 21:28:13 2016 +0300
@@ -184,6 +184,10 @@ struct ngx_stream_session_s {
 #endif
 
     ngx_uint_t                     status;
+
+#if (NGX_STREAM_SSL)
+    ngx_uint_t                     ssl;  /* unsigned  ssl:1; */
+#endif
 };
 
 
diff -r 0125b151c9a5 -r 40e8ce405859 src/stream/ngx_stream_handler.c
--- a/src/stream/ngx_stream_handler.c	Mon Sep 05 17:50:16 2016 +0300
+++ b/src/stream/ngx_stream_handler.c	Tue Sep 06 21:28:13 2016 +0300
@@ -13,6 +13,7 @@
 
 static void ngx_stream_close_connection(ngx_connection_t *c);
 static u_char *ngx_stream_log_error(ngx_log_t *log, u_char *buf, size_t len);
+static void ngx_stream_init_session_handler(ngx_event_t *rev);
 static void ngx_stream_init_session(ngx_connection_t *c);
 
 #if (NGX_STREAM_SSL)
@@ -24,12 +25,11 @@ static void ngx_stream_ssl_handshake_han
 void
 ngx_stream_init_connection(ngx_connection_t *c)
 {
-    int                           tcp_nodelay;
     u_char                        text[NGX_SOCKADDR_STRLEN];
     size_t                        len;
-    ngx_int_t                     rc;
     ngx_uint_t                    i;
     ngx_time_t                   *tp;
+    ngx_event_t                  *rev;
     struct sockaddr              *sa;
     ngx_stream_port_t            *port;
     struct sockaddr_in           *sin;
@@ -130,6 +130,10 @@ ngx_stream_init_connection(ngx_connectio
     s->main_conf = addr_conf->ctx->main_conf;
     s->srv_conf = addr_conf->ctx->srv_conf;
 
+#if (NGX_STREAM_SSL)
+    s->ssl = addr_conf->ssl;
+#endif
+
     s->connection = c;
     c->data = s;
 
@@ -164,6 +168,35 @@ ngx_stream_init_connection(ngx_connectio
     s->start_sec = tp->sec;
     s->start_msec = tp->msec;
 
+    rev = c->read;
+    rev->handler = ngx_stream_init_session_handler;
+
+    if (ngx_use_accept_mutex) {
+        ngx_post_event(rev, &ngx_posted_events);
+        return;
+    }
+
+    rev->handler(rev);
+}
+
+
+static void
+ngx_stream_init_session_handler(ngx_event_t *rev)
+{
+    int                           tcp_nodelay;
+    ngx_int_t                     rc;
+    ngx_connection_t             *c;
+    ngx_stream_session_t         *s;
+    ngx_stream_core_srv_conf_t   *cscf;
+    ngx_stream_core_main_conf_t  *cmcf;
+
+    c = rev->data;
+    s = c->data;
+
+    c->log->action = "initializing session";
+
+    cmcf = ngx_stream_get_module_main_conf(s, ngx_stream_core_module);
+
     if (cmcf->limit_conn_handler) {
         rc = cmcf->limit_conn_handler(s);
 
@@ -192,6 +225,8 @@ ngx_stream_init_connection(ngx_connectio
         }
     }
 
+    cscf = ngx_stream_get_module_srv_conf(s, ngx_stream_core_module);
+
     if (c->type == SOCK_STREAM
         && cscf->tcp_nodelay
         && c->tcp_nodelay == NGX_TCP_NODELAY_UNSET)
@@ -219,7 +254,7 @@ ngx_stream_init_connection(ngx_connectio
 
     sslcf = ngx_stream_get_module_srv_conf(s, ngx_stream_ssl_module);
 
-    if (addr_conf->ssl) {
+    if (s->ssl) {
         c->log->action = "SSL handshaking";
 
         if (sslcf->ssl.ctx == NULL) {


From xeioex at nginx.com  Wed Sep  7 15:28:44 2016
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 07 Sep 2016 15:28:44 +0000
Subject: [nginx] Stream: the "proxy_protocol" parameter of the "listen"
 directive.
Message-ID: <hg.7357abd1fa8c.1473262124.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/7357abd1fa8c
branches:  
changeset: 6680:7357abd1fa8c
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Sep 06 21:28:16 2016 +0300
description:
Stream: the "proxy_protocol" parameter of the "listen" directive.

diffstat:

 src/stream/ngx_stream.c             |   2 +
 src/stream/ngx_stream.h             |   7 ++-
 src/stream/ngx_stream_core_module.c |  20 +++++++
 src/stream/ngx_stream_handler.c     |  94 +++++++++++++++++++++++++++++++++++++
 4 files changed, 122 insertions(+), 1 deletions(-)

diffs (234 lines):

diff -r 40e8ce405859 -r 7357abd1fa8c src/stream/ngx_stream.c
--- a/src/stream/ngx_stream.c	Tue Sep 06 21:28:13 2016 +0300
+++ b/src/stream/ngx_stream.c	Tue Sep 06 21:28:16 2016 +0300
@@ -455,6 +455,7 @@ ngx_stream_add_addrs(ngx_conf_t *cf, ngx
 #if (NGX_STREAM_SSL)
         addrs[i].conf.ssl = addr[i].opt.ssl;
 #endif
+        addrs[i].conf.proxy_protocol = addr[i].opt.proxy_protocol;
 
         len = ngx_sock_ntop(&addr[i].opt.sockaddr.sockaddr, addr[i].opt.socklen,
                             buf, NGX_SOCKADDR_STRLEN, 1);
@@ -504,6 +505,7 @@ ngx_stream_add_addrs6(ngx_conf_t *cf, ng
 #if (NGX_STREAM_SSL)
         addrs6[i].conf.ssl = addr[i].opt.ssl;
 #endif
+        addrs6[i].conf.proxy_protocol = addr[i].opt.proxy_protocol;
 
         len = ngx_sock_ntop(&addr[i].opt.sockaddr.sockaddr, addr[i].opt.socklen,
                             buf, NGX_SOCKADDR_STRLEN, 1);
diff -r 40e8ce405859 -r 7357abd1fa8c src/stream/ngx_stream.h
--- a/src/stream/ngx_stream.h	Tue Sep 06 21:28:13 2016 +0300
+++ b/src/stream/ngx_stream.h	Tue Sep 06 21:28:16 2016 +0300
@@ -27,6 +27,7 @@ typedef struct ngx_stream_session_s  ngx
 
 
 #define NGX_STREAM_OK                        200
+#define NGX_STREAM_BAD_REQUEST               400
 #define NGX_STREAM_FORBIDDEN                 403
 #define NGX_STREAM_INTERNAL_SERVER_ERROR     500
 #define NGX_STREAM_BAD_GATEWAY               502
@@ -58,6 +59,7 @@ typedef struct {
     unsigned                       reuseport:1;
 #endif
     unsigned                       so_keepalive:2;
+    unsigned                       proxy_protocol:1;
 #if (NGX_HAVE_KEEPALIVE_TUNABLE)
     int                            tcp_keepidle;
     int                            tcp_keepintvl;
@@ -72,8 +74,9 @@ typedef struct {
     ngx_stream_conf_ctx_t         *ctx;
     ngx_str_t                      addr_text;
 #if (NGX_STREAM_SSL)
-    ngx_uint_t                     ssl;    /* unsigned   ssl:1; */
+    unsigned                       ssl:1;
 #endif
+    unsigned                       proxy_protocol:1;
 } ngx_stream_addr_conf_t;
 
 typedef struct {
@@ -153,6 +156,8 @@ typedef struct {
     ngx_msec_t                     resolver_timeout;
     ngx_resolver_t                *resolver;
 
+    ngx_msec_t                     proxy_protocol_timeout;
+
     ngx_uint_t                     listen;  /* unsigned  listen:1; */
 } ngx_stream_core_srv_conf_t;
 
diff -r 40e8ce405859 -r 7357abd1fa8c src/stream/ngx_stream_core_module.c
--- a/src/stream/ngx_stream_core_module.c	Tue Sep 06 21:28:13 2016 +0300
+++ b/src/stream/ngx_stream_core_module.c	Tue Sep 06 21:28:16 2016 +0300
@@ -77,6 +77,13 @@ static ngx_command_t  ngx_stream_core_co
       offsetof(ngx_stream_core_srv_conf_t, resolver_timeout),
       NULL },
 
+    { ngx_string("proxy_protocol_timeout"),
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_TAKE1,
+      ngx_conf_set_msec_slot,
+      NGX_STREAM_SRV_CONF_OFFSET,
+      offsetof(ngx_stream_core_srv_conf_t, proxy_protocol_timeout),
+      NULL },
+
     { ngx_string("tcp_nodelay"),
       NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_FLAG,
       ngx_conf_set_flag_slot,
@@ -192,6 +199,7 @@ ngx_stream_core_create_srv_conf(ngx_conf
     cscf->file_name = cf->conf_file->file.name.data;
     cscf->line = cf->conf_file->line;
     cscf->resolver_timeout = NGX_CONF_UNSET_MSEC;
+    cscf->proxy_protocol_timeout = NGX_CONF_UNSET_MSEC;
     cscf->tcp_nodelay = NGX_CONF_UNSET;
 
     return cscf;
@@ -240,6 +248,9 @@ ngx_stream_core_merge_srv_conf(ngx_conf_
         }
     }
 
+    ngx_conf_merge_msec_value(conf->proxy_protocol_timeout,
+                              prev->proxy_protocol_timeout, 5000);
+
     ngx_conf_merge_value(conf->tcp_nodelay, prev->tcp_nodelay, 1);
 
     return NGX_CONF_OK;
@@ -572,6 +583,11 @@ ngx_stream_core_listen(ngx_conf_t *cf, n
 #endif
         }
 
+        if (ngx_strcmp(value[i].data, "proxy_protocol") == 0) {
+            ls->proxy_protocol = 1;
+            continue;
+        }
+
         ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "the invalid \"%V\" parameter", &value[i]);
         return NGX_CONF_ERROR;
@@ -591,6 +607,10 @@ ngx_stream_core_listen(ngx_conf_t *cf, n
         if (ls->so_keepalive) {
             return "\"so_keepalive\" parameter is incompatible with \"udp\"";
         }
+
+        if (ls->proxy_protocol) {
+            return "\"proxy_protocol\" parameter is incompatible with \"udp\"";
+        }
     }
 
     als = cmcf->listen.elts;
diff -r 40e8ce405859 -r 7357abd1fa8c src/stream/ngx_stream_handler.c
--- a/src/stream/ngx_stream_handler.c	Tue Sep 06 21:28:13 2016 +0300
+++ b/src/stream/ngx_stream_handler.c	Tue Sep 06 21:28:16 2016 +0300
@@ -13,6 +13,7 @@
 
 static void ngx_stream_close_connection(ngx_connection_t *c);
 static u_char *ngx_stream_log_error(ngx_log_t *log, u_char *buf, size_t len);
+static void ngx_stream_proxy_protocol_handler(ngx_event_t *rev);
 static void ngx_stream_init_session_handler(ngx_event_t *rev);
 static void ngx_stream_init_session(ngx_connection_t *c);
 
@@ -171,6 +172,23 @@ ngx_stream_init_connection(ngx_connectio
     rev = c->read;
     rev->handler = ngx_stream_init_session_handler;
 
+    if (addr_conf->proxy_protocol) {
+        c->log->action = "reading PROXY protocol";
+
+        rev->handler = ngx_stream_proxy_protocol_handler;
+
+        if (!rev->ready) {
+            ngx_add_timer(rev, cscf->proxy_protocol_timeout);
+
+            if (ngx_handle_read_event(rev, 0) != NGX_OK) {
+                ngx_stream_finalize_session(s,
+                                            NGX_STREAM_INTERNAL_SERVER_ERROR);
+            }
+
+            return;
+        }
+    }
+
     if (ngx_use_accept_mutex) {
         ngx_post_event(rev, &ngx_posted_events);
         return;
@@ -181,6 +199,82 @@ ngx_stream_init_connection(ngx_connectio
 
 
 static void
+ngx_stream_proxy_protocol_handler(ngx_event_t *rev)
+{
+    u_char                      *p, buf[NGX_PROXY_PROTOCOL_MAX_HEADER];
+    size_t                       size;
+    ssize_t                      n;
+    ngx_err_t                    err;
+    ngx_connection_t            *c;
+    ngx_stream_session_t        *s;
+    ngx_stream_core_srv_conf_t  *cscf;
+
+    c = rev->data;
+    s = c->data;
+
+    ngx_log_debug0(NGX_LOG_DEBUG_STREAM, c->log, 0,
+                   "stream PROXY protocol handler");
+
+    if (rev->timedout) {
+        ngx_log_error(NGX_LOG_INFO, c->log, NGX_ETIMEDOUT, "client timed out");
+        ngx_stream_finalize_session(s, NGX_STREAM_OK);
+        return;
+    }
+
+    n = recv(c->fd, (char *) buf, sizeof(buf), MSG_PEEK);
+
+    err = ngx_socket_errno;
+
+    ngx_log_debug1(NGX_LOG_DEBUG_STREAM, c->log, 0, "recv(): %z", n);
+
+    if (n == -1) {
+        if (err == NGX_EAGAIN) {
+            rev->ready = 0;
+
+            if (!rev->timer_set) {
+                cscf = ngx_stream_get_module_srv_conf(s,
+                                                      ngx_stream_core_module);
+
+                ngx_add_timer(rev, cscf->proxy_protocol_timeout);
+            }
+
+            if (ngx_handle_read_event(rev, 0) != NGX_OK) {
+                ngx_stream_finalize_session(s,
+                                            NGX_STREAM_INTERNAL_SERVER_ERROR);
+            }
+
+            return;
+        }
+
+        ngx_connection_error(c, err, "recv() failed");
+
+        ngx_stream_finalize_session(s, NGX_STREAM_OK);
+        return;
+    }
+
+    if (rev->timer_set) {
+        ngx_del_timer(rev);
+    }
+
+    p = ngx_proxy_protocol_read(c, buf, buf + n);
+
+    if (p == NULL) {
+        ngx_stream_finalize_session(s, NGX_STREAM_BAD_REQUEST);
+        return;
+    }
+
+    size = p - buf;
+
+    if (c->recv(c, buf, size) != (ssize_t) size) {
+        ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+        return;
+    }
+
+    ngx_stream_init_session_handler(rev);
+}
+
+
+static void
 ngx_stream_init_session_handler(ngx_event_t *rev)
 {
     int                           tcp_nodelay;


From xeioex at nginx.com  Wed Sep  7 15:28:46 2016
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 07 Sep 2016 15:28:46 +0000
Subject: [nginx] Stream: $proxy_protocol_addr and $proxy_protocol_port.
Message-ID: <hg.b9f78a4e3597.1473262126.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/b9f78a4e3597
branches:  
changeset: 6681:b9f78a4e3597
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Sep 06 21:28:16 2016 +0300
description:
Stream: $proxy_protocol_addr and $proxy_protocol_port.

diffstat:

 src/stream/ngx_stream_variables.c |  50 +++++++++++++++++++++++++++++++++++++++
 1 files changed, 50 insertions(+), 0 deletions(-)

diffs (74 lines):

diff -r 7357abd1fa8c -r b9f78a4e3597 src/stream/ngx_stream_variables.c
--- a/src/stream/ngx_stream_variables.c	Tue Sep 06 21:28:16 2016 +0300
+++ b/src/stream/ngx_stream_variables.c	Tue Sep 06 21:28:16 2016 +0300
@@ -17,6 +17,10 @@ static ngx_int_t ngx_stream_variable_rem
     ngx_stream_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_stream_variable_remote_port(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_stream_variable_proxy_protocol_addr(
+    ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_stream_variable_proxy_protocol_port(
+    ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_stream_variable_server_addr(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_stream_variable_server_port(ngx_stream_session_t *s,
@@ -57,6 +61,12 @@ static ngx_stream_variable_t  ngx_stream
     { ngx_string("remote_port"), NULL,
       ngx_stream_variable_remote_port, 0, 0, 0 },
 
+    { ngx_string("proxy_protocol_addr"), NULL,
+      ngx_stream_variable_proxy_protocol_addr, 0, 0, 0 },
+
+    { ngx_string("proxy_protocol_port"), NULL,
+      ngx_stream_variable_proxy_protocol_port, 0, 0, 0 },
+
     { ngx_string("server_addr"), NULL,
       ngx_stream_variable_server_addr, 0, 0, 0 },
 
@@ -418,6 +428,46 @@ ngx_stream_variable_remote_port(ngx_stre
 
 
 static ngx_int_t
+ngx_stream_variable_proxy_protocol_addr(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data)
+{
+    v->len = s->connection->proxy_protocol_addr.len;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+    v->data = s->connection->proxy_protocol_addr.data;
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_stream_variable_proxy_protocol_port(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data)
+{
+    ngx_uint_t  port;
+
+    v->len = 0;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+
+    v->data = ngx_pnalloc(s->connection->pool, sizeof("65535") - 1);
+    if (v->data == NULL) {
+        return NGX_ERROR;
+    }
+
+    port = s->connection->proxy_protocol_port;
+
+    if (port > 0 && port < 65536) {
+        v->len = ngx_sprintf(v->data, "%ui", port) - v->data;
+    }
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
 ngx_stream_variable_server_addr(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data)
 {


From xeioex at nginx.com  Wed Sep  7 15:28:49 2016
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 07 Sep 2016 15:28:49 +0000
Subject: [nginx] Stream: allow using the session context inside handlers.
Message-ID: <hg.db422604ceb0.1473262129.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/db422604ceb0
branches:  
changeset: 6682:db422604ceb0
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Tue Sep 06 21:28:17 2016 +0300
description:
Stream: allow using the session context inside handlers.

Previously, it was not possible to use the stream context
inside ngx_stream_init_connection() handlers.  Now, limit_conn,
access handlers, as well as those added later, can create
their own contexts.

diffstat:

 src/stream/ngx_stream_handler.c |  37 ++++++++++++++++---------------------
 1 files changed, 16 insertions(+), 21 deletions(-)

diffs (77 lines):

diff -r b9f78a4e3597 -r db422604ceb0 src/stream/ngx_stream_handler.c
--- a/src/stream/ngx_stream_handler.c	Tue Sep 06 21:28:16 2016 +0300
+++ b/src/stream/ngx_stream_handler.c	Tue Sep 06 21:28:17 2016 +0300
@@ -15,7 +15,6 @@ static void ngx_stream_close_connection(
 static u_char *ngx_stream_log_error(ngx_log_t *log, u_char *buf, size_t len);
 static void ngx_stream_proxy_protocol_handler(ngx_event_t *rev);
 static void ngx_stream_init_session_handler(ngx_event_t *rev);
-static void ngx_stream_init_session(ngx_connection_t *c);
 
 #if (NGX_STREAM_SSL)
 static void ngx_stream_ssl_init_connection(ngx_ssl_t *ssl, ngx_connection_t *c);
@@ -154,6 +153,12 @@ ngx_stream_init_connection(ngx_connectio
     c->log->action = "initializing connection";
     c->log_error = NGX_ERROR_INFO;
 
+    s->ctx = ngx_pcalloc(c->pool, sizeof(void *) * ngx_stream_max_module);
+    if (s->ctx == NULL) {
+        ngx_stream_close_connection(c);
+        return;
+    }
+
     cmcf = ngx_stream_get_module_main_conf(s, ngx_stream_core_module);
 
     s->variables = ngx_pcalloc(s->connection->pool,
@@ -365,27 +370,8 @@ ngx_stream_init_session_handler(ngx_even
     }
 #endif
 
-    ngx_stream_init_session(c);
-}
-
-
-static void
-ngx_stream_init_session(ngx_connection_t *c)
-{
-    ngx_stream_session_t        *s;
-    ngx_stream_core_srv_conf_t  *cscf;
-
-    s = c->data;
     c->log->action = "handling client connection";
 
-    cscf = ngx_stream_get_module_srv_conf(s, ngx_stream_core_module);
-
-    s->ctx = ngx_pcalloc(c->pool, sizeof(void *) * ngx_stream_max_module);
-    if (s->ctx == NULL) {
-        ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
-        return;
-    }
-
     cscf->handler(s);
 }
 
@@ -422,6 +408,9 @@ ngx_stream_ssl_init_connection(ngx_ssl_t
 static void
 ngx_stream_ssl_handshake_handler(ngx_connection_t *c)
 {
+    ngx_stream_session_t        *s;
+    ngx_stream_core_srv_conf_t  *cscf;
+
     if (!c->ssl->handshaked) {
         ngx_stream_finalize_session(c->data, NGX_STREAM_INTERNAL_SERVER_ERROR);
         return;
@@ -431,7 +420,13 @@ ngx_stream_ssl_handshake_handler(ngx_con
         ngx_del_timer(c->read);
     }
 
-    ngx_stream_init_session(c);
+    c->log->action = "handling client connection";
+
+    s = c->data;
+
+    cscf = ngx_stream_get_module_srv_conf(s, ngx_stream_core_module);
+
+    cscf->handler(s);
 }
 
 #endif


From xeioex at nginx.com  Wed Sep  7 15:28:52 2016
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 07 Sep 2016 15:28:52 +0000
Subject: [nginx] Core: introduced ngx_cidr_match() function.
Message-ID: <hg.b802b7e1d9bc.1473262132.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/b802b7e1d9bc
branches:  
changeset: 6683:b802b7e1d9bc
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Wed Sep 07 13:56:53 2016 +0300
description:
Core: introduced ngx_cidr_match() function.

diffstat:

 src/core/ngx_inet.c             |   87 ++++++++++++++++++++++++
 src/core/ngx_inet.h             |    1 +
 src/http/ngx_http_core_module.c |  144 +++++++++------------------------------
 3 files changed, 123 insertions(+), 109 deletions(-)

diffs (267 lines):

diff -r db422604ceb0 -r b802b7e1d9bc src/core/ngx_inet.c
--- a/src/core/ngx_inet.c	Tue Sep 06 21:28:17 2016 +0300
+++ b/src/core/ngx_inet.c	Wed Sep 07 13:56:53 2016 +0300
@@ -466,6 +466,93 @@ ngx_ptocidr(ngx_str_t *text, ngx_cidr_t 
 
 
 ngx_int_t
+ngx_cidr_match(struct sockaddr *sa, ngx_array_t *cidrs)
+{
+#if (NGX_HAVE_INET6)
+    u_char           *p;
+#endif
+    in_addr_t         inaddr;
+    ngx_cidr_t       *cidr;
+    ngx_uint_t        family, i;
+#if (NGX_HAVE_INET6)
+    ngx_uint_t        n;
+    struct in6_addr  *inaddr6;
+#endif
+
+#if (NGX_SUPPRESS_WARN)
+    inaddr = 0;
+#if (NGX_HAVE_INET6)
+    inaddr6 = NULL;
+#endif
+#endif
+
+    family = sa->sa_family;
+
+    if (family == AF_INET) {
+        inaddr = ((struct sockaddr_in *) sa)->sin_addr.s_addr;
+    }
+
+#if (NGX_HAVE_INET6)
+    else if (family == AF_INET6) {
+        inaddr6 = &((struct sockaddr_in6 *) sa)->sin6_addr;
+
+        if (IN6_IS_ADDR_V4MAPPED(inaddr6)) {
+            family = AF_INET;
+
+            p = inaddr6->s6_addr;
+
+            inaddr = p[12] << 24;
+            inaddr += p[13] << 16;
+            inaddr += p[14] << 8;
+            inaddr += p[15];
+
+            inaddr = htonl(inaddr);
+        }
+    }
+#endif
+
+    for (cidr = cidrs->elts, i = 0; i < cidrs->nelts; i++) {
+        if (cidr[i].family != family) {
+            goto next;
+        }
+
+        switch (family) {
+
+#if (NGX_HAVE_INET6)
+        case AF_INET6:
+            for (n = 0; n < 16; n++) {
+                if ((inaddr6->s6_addr[n] & cidr[i].u.in6.mask.s6_addr[n])
+                    != cidr[i].u.in6.addr.s6_addr[n])
+                {
+                    goto next;
+                }
+            }
+            break;
+#endif
+
+#if (NGX_HAVE_UNIX_DOMAIN)
+        case AF_UNIX:
+            break;
+#endif
+
+        default: /* AF_INET */
+            if ((inaddr & cidr[i].u.in.mask) != cidr[i].u.in.addr) {
+                goto next;
+            }
+            break;
+        }
+
+        return NGX_OK;
+
+    next:
+        continue;
+    }
+
+    return NGX_DECLINED;
+}
+
+
+ngx_int_t
 ngx_parse_addr(ngx_pool_t *pool, ngx_addr_t *addr, u_char *text, size_t len)
 {
     in_addr_t             inaddr;
diff -r db422604ceb0 -r b802b7e1d9bc src/core/ngx_inet.h
--- a/src/core/ngx_inet.h	Tue Sep 06 21:28:17 2016 +0300
+++ b/src/core/ngx_inet.h	Wed Sep 07 13:56:53 2016 +0300
@@ -113,6 +113,7 @@ size_t ngx_sock_ntop(struct sockaddr *sa
     size_t len, ngx_uint_t port);
 size_t ngx_inet_ntop(int family, void *addr, u_char *text, size_t len);
 ngx_int_t ngx_ptocidr(ngx_str_t *text, ngx_cidr_t *cidr);
+ngx_int_t ngx_cidr_match(struct sockaddr *sa, ngx_array_t *cidrs);
 ngx_int_t ngx_parse_addr(ngx_pool_t *pool, ngx_addr_t *addr, u_char *text,
     size_t len);
 ngx_int_t ngx_parse_addr_port(ngx_pool_t *pool, ngx_addr_t *addr,
diff -r db422604ceb0 -r b802b7e1d9bc src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c	Tue Sep 06 21:28:17 2016 +0300
+++ b/src/http/ngx_http_core_module.c	Wed Sep 07 13:56:53 2016 +0300
@@ -2823,120 +2823,46 @@ static ngx_int_t
 ngx_http_get_forwarded_addr_internal(ngx_http_request_t *r, ngx_addr_t *addr,
     u_char *xff, size_t xfflen, ngx_array_t *proxies, int recursive)
 {
-    u_char           *p;
-    in_addr_t         inaddr;
-    ngx_int_t         rc;
-    ngx_addr_t        paddr;
-    ngx_cidr_t       *cidr;
-    ngx_uint_t        family, i;
-#if (NGX_HAVE_INET6)
-    ngx_uint_t        n;
-    struct in6_addr  *inaddr6;
-#endif
-
-#if (NGX_SUPPRESS_WARN)
-    inaddr = 0;
-#if (NGX_HAVE_INET6)
-    inaddr6 = NULL;
-#endif
-#endif
-
-    family = addr->sockaddr->sa_family;
-
-    if (family == AF_INET) {
-        inaddr = ((struct sockaddr_in *) addr->sockaddr)->sin_addr.s_addr;
-    }
-
-#if (NGX_HAVE_INET6)
-    else if (family == AF_INET6) {
-        inaddr6 = &((struct sockaddr_in6 *) addr->sockaddr)->sin6_addr;
-
-        if (IN6_IS_ADDR_V4MAPPED(inaddr6)) {
-            family = AF_INET;
-
-            p = inaddr6->s6_addr;
-
-            inaddr = p[12] << 24;
-            inaddr += p[13] << 16;
-            inaddr += p[14] << 8;
-            inaddr += p[15];
-
-            inaddr = htonl(inaddr);
-        }
-    }
-#endif
-
-    for (cidr = proxies->elts, i = 0; i < proxies->nelts; i++) {
-        if (cidr[i].family != family) {
-            goto next;
-        }
-
-        switch (family) {
-
-#if (NGX_HAVE_INET6)
-        case AF_INET6:
-            for (n = 0; n < 16; n++) {
-                if ((inaddr6->s6_addr[n] & cidr[i].u.in6.mask.s6_addr[n])
-                    != cidr[i].u.in6.addr.s6_addr[n])
-                {
-                    goto next;
-                }
-            }
-            break;
-#endif
-
-#if (NGX_HAVE_UNIX_DOMAIN)
-        case AF_UNIX:
-            break;
-#endif
-
-        default: /* AF_INET */
-            if ((inaddr & cidr[i].u.in.mask) != cidr[i].u.in.addr) {
-                goto next;
-            }
+    u_char      *p;
+    ngx_int_t    rc;
+    ngx_addr_t   paddr;
+
+    if (ngx_cidr_match(addr->sockaddr, proxies) != NGX_OK) {
+        return NGX_DECLINED;
+    }
+
+    for (p = xff + xfflen - 1; p > xff; p--, xfflen--) {
+        if (*p != ' ' && *p != ',') {
             break;
         }
-
-        for (p = xff + xfflen - 1; p > xff; p--, xfflen--) {
-            if (*p != ' ' && *p != ',') {
-                break;
-            }
-        }
-
-        for ( /* void */ ; p > xff; p--) {
-            if (*p == ' ' || *p == ',') {
-                p++;
-                break;
-            }
-        }
-
-        if (ngx_parse_addr_port(r->pool, &paddr, p, xfflen - (p - xff))
-            != NGX_OK)
-        {
-            return NGX_DECLINED;
+    }
+
+    for ( /* void */ ; p > xff; p--) {
+        if (*p == ' ' || *p == ',') {
+            p++;
+            break;
         }
-
-        *addr = paddr;
-
-        if (recursive && p > xff) {
-            rc = ngx_http_get_forwarded_addr_internal(r, addr, xff, p - 1 - xff,
-                                                      proxies, 1);
-
-            if (rc == NGX_DECLINED) {
-                return NGX_DONE;
-            }
-
-            /* rc == NGX_OK || rc == NGX_DONE  */
-            return rc;
+    }
+
+    if (ngx_parse_addr_port(r->pool, &paddr, p, xfflen - (p - xff)) != NGX_OK) {
+        return NGX_DECLINED;
+    }
+
+    *addr = paddr;
+
+    if (recursive && p > xff) {
+        rc = ngx_http_get_forwarded_addr_internal(r, addr, xff, p - 1 - xff,
+                                                  proxies, 1);
+
+        if (rc == NGX_DECLINED) {
+            return NGX_DONE;
         }
 
-        return NGX_OK;
-
-    next:
-        continue;
-    }
-
-    return NGX_DECLINED;
+        /* rc == NGX_OK || rc == NGX_DONE  */
+        return rc;
+    }
+
+    return NGX_OK;
 }
 
 


From xeioex at nginx.com  Wed Sep  7 15:28:54 2016
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Wed, 07 Sep 2016 15:28:54 +0000
Subject: [nginx] Stream: realip module.
Message-ID: <hg.9cac11efb205.1473262134.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/9cac11efb205
branches:  
changeset: 6684:9cac11efb205
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Thu Sep 01 14:45:33 2016 +0300
description:
Stream: realip module.

diffstat:

 auto/modules                          |   10 +
 auto/options                          |    3 +
 src/stream/ngx_stream.h               |    1 +
 src/stream/ngx_stream_handler.c       |    9 +
 src/stream/ngx_stream_realip_module.c |  342 ++++++++++++++++++++++++++++++++++
 5 files changed, 365 insertions(+), 0 deletions(-)

diffs (423 lines):

diff -r b802b7e1d9bc -r 9cac11efb205 auto/modules
--- a/auto/modules	Wed Sep 07 13:56:53 2016 +0300
+++ b/auto/modules	Thu Sep 01 14:45:33 2016 +0300
@@ -1007,6 +1007,16 @@ if [ $STREAM != NO ]; then
         . auto/module
     fi
 
+    if [ $STREAM_REALIP = YES ]; then
+        ngx_module_name=ngx_stream_realip_module
+        ngx_module_deps=
+        ngx_module_srcs=src/stream/ngx_stream_realip_module.c
+        ngx_module_libs=
+        ngx_module_link=$STREAM_REALIP
+
+        . auto/module
+    fi
+
     if [ $STREAM_LIMIT_CONN = YES ]; then
         ngx_module_name=ngx_stream_limit_conn_module
         ngx_module_deps=
diff -r b802b7e1d9bc -r 9cac11efb205 auto/options
--- a/auto/options	Wed Sep 07 13:56:53 2016 +0300
+++ b/auto/options	Thu Sep 01 14:45:33 2016 +0300
@@ -115,6 +115,7 @@ MAIL_SMTP=YES
 
 STREAM=NO
 STREAM_SSL=NO
+STREAM_REALIP=NO
 STREAM_LIMIT_CONN=YES
 STREAM_ACCESS=YES
 STREAM_GEO=YES
@@ -296,6 +297,7 @@ use the \"--with-mail_ssl_module\" optio
         --with-stream)                   STREAM=YES                 ;;
         --with-stream=dynamic)           STREAM=DYNAMIC             ;;
         --with-stream_ssl_module)        STREAM_SSL=YES             ;;
+        --with-stream_realip_module)     STREAM_REALIP=YES          ;;
         --with-stream_geoip_module)      STREAM_GEOIP=YES           ;;
         --with-stream_geoip_module=dynamic)
                                          STREAM_GEOIP=DYNAMIC       ;;
@@ -503,6 +505,7 @@ cat << END
   --with-stream                      enable TCP/UDP proxy module
   --with-stream=dynamic              enable dynamic TCP/UDP proxy module
   --with-stream_ssl_module           enable ngx_stream_ssl_module
+  --with-stream_realip_module        enable ngx_stream_realip_module
   --with-stream_geoip_module         enable ngx_stream_geoip_module
   --with-stream_geoip_module=dynamic enable dynamic ngx_stream_geoip_module
   --without-stream_limit_conn_module disable ngx_stream_limit_conn_module
diff -r b802b7e1d9bc -r 9cac11efb205 src/stream/ngx_stream.h
--- a/src/stream/ngx_stream.h	Wed Sep 07 13:56:53 2016 +0300
+++ b/src/stream/ngx_stream.h	Thu Sep 01 14:45:33 2016 +0300
@@ -122,6 +122,7 @@ typedef struct {
     ngx_array_t                    servers;     /* ngx_stream_core_srv_conf_t */
     ngx_array_t                    listen;      /* ngx_stream_listen_t */
 
+    ngx_stream_access_pt           realip_handler;
     ngx_stream_access_pt           limit_conn_handler;
     ngx_stream_access_pt           access_handler;
     ngx_stream_access_pt           access_log_handler;
diff -r b802b7e1d9bc -r 9cac11efb205 src/stream/ngx_stream_handler.c
--- a/src/stream/ngx_stream_handler.c	Wed Sep 07 13:56:53 2016 +0300
+++ b/src/stream/ngx_stream_handler.c	Thu Sep 01 14:45:33 2016 +0300
@@ -296,6 +296,15 @@ ngx_stream_init_session_handler(ngx_even
 
     cmcf = ngx_stream_get_module_main_conf(s, ngx_stream_core_module);
 
+    if (cmcf->realip_handler) {
+        rc = cmcf->realip_handler(s);
+
+        if (rc == NGX_ERROR) {
+            ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+            return;
+        }
+    }
+
     if (cmcf->limit_conn_handler) {
         rc = cmcf->limit_conn_handler(s);
 
diff -r b802b7e1d9bc -r 9cac11efb205 src/stream/ngx_stream_realip_module.c
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/src/stream/ngx_stream_realip_module.c	Thu Sep 01 14:45:33 2016 +0300
@@ -0,0 +1,342 @@
+
+/*
+ * Copyright (C) Igor Sysoev
+ * Copyright (C) Nginx, Inc.
+ */
+
+
+#include <ngx_config.h>
+#include <ngx_core.h>
+#include <ngx_stream.h>
+
+
+typedef struct {
+    ngx_array_t       *from;     /* array of ngx_cidr_t */
+} ngx_stream_realip_srv_conf_t;
+
+
+typedef struct {
+    struct sockaddr   *sockaddr;
+    socklen_t          socklen;
+    ngx_str_t          addr_text;
+} ngx_stream_realip_ctx_t;
+
+
+static ngx_int_t ngx_stream_realip_handler(ngx_stream_session_t *s);
+static ngx_int_t ngx_stream_realip_set_addr(ngx_stream_session_t *s,
+    ngx_addr_t *addr);
+static char *ngx_stream_realip_from(ngx_conf_t *cf, ngx_command_t *cmd,
+    void *conf);
+static void *ngx_stream_realip_create_srv_conf(ngx_conf_t *cf);
+static char *ngx_stream_realip_merge_srv_conf(ngx_conf_t *cf, void *parent,
+    void *child);
+static ngx_int_t ngx_stream_realip_add_variables(ngx_conf_t *cf);
+static ngx_int_t ngx_stream_realip_init(ngx_conf_t *cf);
+
+
+static ngx_int_t ngx_stream_realip_remote_addr_variable(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_stream_realip_remote_port_variable(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data);
+
+
+static ngx_command_t  ngx_stream_realip_commands[] = {
+
+    { ngx_string("set_real_ip_from"),
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_TAKE1,
+      ngx_stream_realip_from,
+      NGX_STREAM_SRV_CONF_OFFSET,
+      0,
+      NULL },
+
+      ngx_null_command
+};
+
+
+static ngx_stream_module_t  ngx_stream_realip_module_ctx = {
+    ngx_stream_realip_add_variables,       /* preconfiguration */
+    ngx_stream_realip_init,                /* postconfiguration */
+
+    NULL,                                  /* create main configuration */
+    NULL,                                  /* init main configuration */
+
+    ngx_stream_realip_create_srv_conf,     /* create server configuration */
+    ngx_stream_realip_merge_srv_conf       /* merge server configuration */
+};
+
+
+ngx_module_t  ngx_stream_realip_module = {
+    NGX_MODULE_V1,
+    &ngx_stream_realip_module_ctx,         /* module context */
+    ngx_stream_realip_commands,            /* module directives */
+    NGX_STREAM_MODULE,                     /* module type */
+    NULL,                                  /* init master */
+    NULL,                                  /* init module */
+    NULL,                                  /* init process */
+    NULL,                                  /* init thread */
+    NULL,                                  /* exit thread */
+    NULL,                                  /* exit process */
+    NULL,                                  /* exit master */
+    NGX_MODULE_V1_PADDING
+};
+
+
+static ngx_stream_variable_t  ngx_stream_realip_vars[] = {
+
+    { ngx_string("realip_remote_addr"), NULL,
+      ngx_stream_realip_remote_addr_variable, 0, 0, 0 },
+
+    { ngx_string("realip_remote_port"), NULL,
+      ngx_stream_realip_remote_port_variable, 0, 0, 0 },
+
+    { ngx_null_string, NULL, NULL, 0, 0, 0 }
+};
+
+
+static ngx_int_t
+ngx_stream_realip_handler(ngx_stream_session_t *s)
+{
+    ngx_addr_t                     addr;
+    ngx_connection_t              *c;
+    ngx_stream_realip_srv_conf_t  *rscf;
+
+    rscf = ngx_stream_get_module_srv_conf(s, ngx_stream_realip_module);
+
+    if (rscf->from == NULL) {
+        return NGX_DECLINED;
+    }
+
+    c = s->connection;
+
+    if (c->proxy_protocol_addr.len == 0) {
+        return NGX_DECLINED;
+    }
+
+    if (ngx_cidr_match(c->sockaddr, rscf->from) != NGX_OK) {
+        return NGX_DECLINED;
+    }
+
+    if (ngx_parse_addr(c->pool, &addr, c->proxy_protocol_addr.data,
+                       c->proxy_protocol_addr.len)
+        != NGX_OK)
+    {
+        return NGX_DECLINED;
+    }
+
+    ngx_inet_set_port(addr.sockaddr, c->proxy_protocol_port);
+
+    return ngx_stream_realip_set_addr(s, &addr);
+}
+
+
+static ngx_int_t
+ngx_stream_realip_set_addr(ngx_stream_session_t *s, ngx_addr_t *addr)
+{
+    size_t                    len;
+    u_char                   *p;
+    u_char                    text[NGX_SOCKADDR_STRLEN];
+    ngx_connection_t         *c;
+    ngx_stream_realip_ctx_t  *ctx;
+
+    c = s->connection;
+
+    ctx = ngx_palloc(c->pool, sizeof(ngx_stream_realip_ctx_t));
+    if (ctx == NULL) {
+        return NGX_ERROR;
+    }
+
+    len = ngx_sock_ntop(addr->sockaddr, addr->socklen, text,
+                        NGX_SOCKADDR_STRLEN, 0);
+    if (len == 0) {
+        return NGX_ERROR;
+    }
+
+    p = ngx_pnalloc(c->pool, len);
+    if (p == NULL) {
+        return NGX_ERROR;
+    }
+
+    ngx_memcpy(p, text, len);
+
+    ngx_stream_set_ctx(s, ctx, ngx_stream_realip_module);
+
+    ctx->sockaddr = c->sockaddr;
+    ctx->socklen = c->socklen;
+    ctx->addr_text = c->addr_text;
+
+    c->sockaddr = addr->sockaddr;
+    c->socklen = addr->socklen;
+    c->addr_text.len = len;
+    c->addr_text.data = p;
+
+    return NGX_DECLINED;
+}
+
+
+static char *
+ngx_stream_realip_from(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
+{
+    ngx_stream_realip_srv_conf_t *rscf = conf;
+
+    ngx_int_t                rc;
+    ngx_str_t               *value;
+    ngx_cidr_t              *cidr;
+
+    value = cf->args->elts;
+
+    if (rscf->from == NULL) {
+        rscf->from = ngx_array_create(cf->pool, 2,
+                                      sizeof(ngx_cidr_t));
+        if (rscf->from == NULL) {
+            return NGX_CONF_ERROR;
+        }
+    }
+
+    cidr = ngx_array_push(rscf->from);
+    if (cidr == NULL) {
+        return NGX_CONF_ERROR;
+    }
+
+#if (NGX_HAVE_UNIX_DOMAIN)
+
+    if (ngx_strcmp(value[1].data, "unix:") == 0) {
+        cidr->family = AF_UNIX;
+        return NGX_CONF_OK;
+    }
+
+#endif
+
+    rc = ngx_ptocidr(&value[1], cidr);
+
+    if (rc == NGX_ERROR) {
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid parameter \"%V\"",
+                           &value[1]);
+        return NGX_CONF_ERROR;
+    }
+
+    if (rc == NGX_DONE) {
+        ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
+                           "low address bits of %V are meaningless", &value[1]);
+    }
+
+    return NGX_CONF_OK;
+}
+
+
+static void *
+ngx_stream_realip_create_srv_conf(ngx_conf_t *cf)
+{
+    ngx_stream_realip_srv_conf_t  *conf;
+
+    conf = ngx_pcalloc(cf->pool, sizeof(ngx_stream_realip_srv_conf_t));
+    if (conf == NULL) {
+        return NULL;
+    }
+
+    /*
+     * set by ngx_pcalloc():
+     *
+     *     conf->from = NULL;
+     */
+
+    return conf;
+}
+
+
+static char *
+ngx_stream_realip_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child)
+{
+    ngx_stream_realip_srv_conf_t *prev = parent;
+    ngx_stream_realip_srv_conf_t *conf = child;
+
+    if (conf->from == NULL) {
+        conf->from = prev->from;
+    }
+
+    return NGX_CONF_OK;
+}
+
+
+static ngx_int_t
+ngx_stream_realip_add_variables(ngx_conf_t *cf)
+{
+    ngx_stream_variable_t  *var, *v;
+
+    for (v = ngx_stream_realip_vars; v->name.len; v++) {
+        var = ngx_stream_add_variable(cf, &v->name, v->flags);
+        if (var == NULL) {
+            return NGX_ERROR;
+        }
+
+        var->get_handler = v->get_handler;
+        var->data = v->data;
+    }
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_stream_realip_init(ngx_conf_t *cf)
+{
+    ngx_stream_core_main_conf_t  *cmcf;
+
+    cmcf = ngx_stream_conf_get_module_main_conf(cf, ngx_stream_core_module);
+
+    cmcf->realip_handler = ngx_stream_realip_handler;
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_stream_realip_remote_addr_variable(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data)
+{
+    ngx_str_t                *addr_text;
+    ngx_stream_realip_ctx_t  *ctx;
+
+    ctx = ngx_stream_get_module_ctx(s, ngx_stream_realip_module);
+
+    addr_text = ctx ? &ctx->addr_text : &s->connection->addr_text;
+
+    v->len = addr_text->len;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+    v->data = addr_text->data;
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_stream_realip_remote_port_variable(ngx_stream_session_t *s,
+    ngx_stream_variable_value_t *v, uintptr_t data)
+{
+    ngx_uint_t                port;
+    struct sockaddr          *sa;
+    ngx_stream_realip_ctx_t  *ctx;
+
+    ctx = ngx_stream_get_module_ctx(s, ngx_stream_realip_module);
+
+    sa = ctx ? ctx->sockaddr : s->connection->sockaddr;
+
+    v->len = 0;
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+
+    v->data = ngx_pnalloc(s->connection->pool, sizeof("65535") - 1);
+    if (v->data == NULL) {
+        return NGX_ERROR;
+    }
+
+    port = ngx_inet_get_port(sa);
+
+    if (port > 0 && port < 65536) {
+        v->len = ngx_sprintf(v->data, "%ui", port) - v->data;
+    }
+
+    return NGX_OK;
+}


From xeioex at nginx.com  Thu Sep  8 12:51:41 2016
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Thu, 08 Sep 2016 12:51:41 +0000
Subject: [nginx] Stream: increase default value for proxy_protocol_timeout to
 30s.
Message-ID: <hg.4a16fceea03b.1473339101.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/4a16fceea03b
branches:  
changeset: 6685:4a16fceea03b
user:      Dmitry Volyntsev <xeioex at nginx.com>
date:      Thu Sep 08 15:51:36 2016 +0300
description:
Stream: increase default value for proxy_protocol_timeout to 30s.

diffstat:

 src/stream/ngx_stream_core_module.c |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff -r 9cac11efb205 -r 4a16fceea03b src/stream/ngx_stream_core_module.c
--- a/src/stream/ngx_stream_core_module.c	Thu Sep 01 14:45:33 2016 +0300
+++ b/src/stream/ngx_stream_core_module.c	Thu Sep 08 15:51:36 2016 +0300
@@ -249,7 +249,7 @@ ngx_stream_core_merge_srv_conf(ngx_conf_
     }
 
     ngx_conf_merge_msec_value(conf->proxy_protocol_timeout,
-                              prev->proxy_protocol_timeout, 5000);
+                              prev->proxy_protocol_timeout, 30000);
 
     ngx_conf_merge_value(conf->tcp_nodelay, prev->tcp_nodelay, 1);
 


From wangyun at us.ibm.com  Thu Sep  8 18:44:04 2016
From: wangyun at us.ibm.com (Yun Wang)
Date: Thu, 8 Sep 2016 18:44:04 +0000
Subject: proxy_ignore_client_abort on; not working as expected
Message-ID: <OF3F719FDE.C34206EC-ON00258028.00648D60-00258028.0066E975@notes.na.collabserv.com>

An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160908/7ea28972/attachment.html>

From mdounin at mdounin.ru  Thu Sep  8 21:25:38 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 9 Sep 2016 00:25:38 +0300
Subject: proxy_ignore_client_abort on; not working as expected
In-Reply-To: <OF3F719FDE.C34206EC-ON00258028.00648D60-00258028.0066E975@notes.na.collabserv.com>
References: <OF3F719FDE.C34206EC-ON00258028.00648D60-00258028.0066E975@notes.na.collabserv.com>
Message-ID: <20160908212538.GL86582@mdounin.ru>

Hello!

On Thu, Sep 08, 2016 at 06:44:04PM +0000, Yun Wang wrote:

>    Hi,
> 
>    We have a client that uses a half-close connection to get response from
>    the server, but nginx closes the connection to the client without
>    waiting for the response from server, even
>    the proxy_ignore_client_abort is set to on. Without nginx in between,
>    things work fine.
> 
>    Here are the events:
>    1. Client establishes the connection (SYN, SYN ACK, ACK).
>    2. Client closes _half_ of the connection by sending a FIN/ACK packet.
>    3. Nginx receives the FIN/ACK from the client, forwards it to the
>    server, and responds to the client with a FIN/ACK right afterwards,
>    which closes the connection on its own.
>    4. When server starts sending data to the client, nginx responds with
>    a RST because the client side connection has been closed.
> 
>    Here is the config file I'm using:
> 
>    events {
>      worker_connections  1024;
>    }
>    http {
>      upstream proxy {
>        keepalive 1;
>        server 172.17.0.1:8089;
>      }
>      map $http_upgrade $connection_upgrade {
>        default upgrade;
>        ''      "";
>      }
>      server {
>          listen    7001;
>          proxy_ignore_client_abort on;
>          location / {
>                  proxy_pass [1]http://proxy;
>                  proxy_set_header Connection $connection_upgrade;
>                  proxy_set_header Upgrade $http_upgrade;
>                  proxy_http_version 1.1;
>          }
>      }
>    }
> 
>    From the documentation, proxy_ignore_client_abort seems to be the
>    perfect match for this scenario, but I'm not sure why it's not working
>    for our setup.
> 
>    Any suggestion is appreciated!

Your config suggests that you are probably see the problem with 
the upgraded connections.  Upgraded connections are closed 
once any party (either client or server) closes the connection and 
there are no pending data.  The proxy_ignore_client_abort 
directive doesn't apply to upgraded connections.

-- 
Maxim Dounin
http://nginx.org/


From savetherbtz at gmail.com  Mon Sep 12 09:52:34 2016
From: savetherbtz at gmail.com (Alexey Ivanov)
Date: Mon, 12 Sep 2016 02:52:34 -0700
Subject: [PATCH] Added the $upstream_connection variable
In-Reply-To: <CAG43V7MPf4P5EnatGVut7Hy2Ta9CUUGw6z_jPhrrYqN93VWA_w@mail.gmail.com>
References: <CAG43V7MPf4P5EnatGVut7Hy2Ta9CUUGw6z_jPhrrYqN93VWA_w@mail.gmail.com>
Message-ID: <EBE4EE5A-9A0D-4842-A0CA-8C9D21A95E44@gmail.com>

+1 to that.

Connection reuse to an upstream is a very important metric for Edge->DC communication.
In our production since we have nginx on both sides we are are gathering that metric from the other side of the other side of a connection. I assume not everybody have that luxury, therefore that stat would be useful.

> On Aug 31, 2016, at 6:38 AM, Jason Stangroome <jason at codeassassin.com> wrote:
> 
> Hello,
> 
> I am using nginx primarily as a proxy and I am looking to improve the
> visibility and control over keepalive connections to upstreams.
> 
> I have several patches I would like to submit for your consideration
> but I am relatively unfamiliar with the code base so I've started
> simple and I appreciate your feedback. I must say the code base has
> been very approachable to work with.
> 
> To begin I am adding support for an $upstream_connection variable for
> use with the log_format directive. This is essentially the same as the
> $connection variable but applies to upstream connections instead of
> downstream.
> 
> The intent of this variable is to help understand which requests are
> being serviced by the same upstream keepalive connection and which are
> using different connections.
> 
> I think I have followed the Contributing Changes page at nginx.org.
> I've honoured the existing code formatting and my `hg export` output
> follows my signature. I have also executed the tests from the
> nginx-tests repository in a Ubuntu Trusty environment but I did not
> have many nginx modules included in my build.
> 
> Regards,
> 
> Jason
> --
> # HG changeset patch
> # User Jason Stangroome <jason at codeassassin.com>
> # Date 1472649436 0
> #      Wed Aug 31 13:17:16 2016 +0000
> # Node ID f06c8a934e3f3ceac2ff393a391234e225cbfcf1
> # Parent  c6372a40c2a731d8816160bf8f55a7a50050c2ac
> Added the $upstream_connection variable
> 
> Allows the connection identifier of the upstream connection used to service a
> proxied request to be logged in the access.log to understand which requests
> are using which upstream keepalive connections.
> 
> diff -r c6372a40c2a7 -r f06c8a934e3f src/http/ngx_http_upstream.c
> --- a/src/http/ngx_http_upstream.c Fri Aug 26 15:33:07 2016 +0300
> +++ b/src/http/ngx_http_upstream.c Wed Aug 31 13:17:16 2016 +0000
> @@ -161,6 +161,9 @@
> static ngx_int_t ngx_http_upstream_response_length_variable(
>     ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data);
> 
> +static ngx_int_t ngx_http_upstream_connection_variable(
> +    ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data);
> +
> static char *ngx_http_upstream(ngx_conf_t *cf, ngx_command_t *cmd,
> void *dummy);
> static char *ngx_http_upstream_server(ngx_conf_t *cf, ngx_command_t *cmd,
>     void *conf);
> @@ -395,6 +398,10 @@
>       ngx_http_upstream_response_length_variable, 1,
>       NGX_HTTP_VAR_NOCACHEABLE, 0 },
> 
> +    { ngx_string("upstream_connection"), NULL,
> +      ngx_http_upstream_connection_variable, 0,
> +      NGX_HTTP_VAR_NOCACHEABLE, 0 },
> +
> #if (NGX_HTTP_CACHE)
> 
>     { ngx_string("upstream_cache_status"), NULL,
> @@ -1804,6 +1811,7 @@
> 
>     if (u->state->connect_time == (ngx_msec_t) -1) {
>         u->state->connect_time = ngx_current_msec - u->state->response_time;
> +        u->state->connection_number = c->number;
>     }
> 
>     if (!u->request_sent && ngx_http_upstream_test_connect(c) != NGX_OK) {
> @@ -5291,6 +5299,67 @@
> }
> 
> 
> +static ngx_int_t
> +ngx_http_upstream_connection_variable(ngx_http_request_t *r,
> +    ngx_http_variable_value_t *v, uintptr_t data)
> +{
> +    u_char                     *p;
> +    size_t                      len;
> +    ngx_uint_t                  i;
> +    ngx_http_upstream_state_t  *state;
> +
> +    v->valid = 1;
> +    v->no_cacheable = 0;
> +    v->not_found = 0;
> +
> +    if (r->upstream_states == NULL || r->upstream_states->nelts == 0) {
> +        v->not_found = 1;
> +        return NGX_OK;
> +    }
> +
> +    len = r->upstream_states->nelts * (NGX_ATOMIC_T_LEN + 2);
> +
> +    p = ngx_pnalloc(r->pool, len);
> +    if (p == NULL) {
> +        return NGX_ERROR;
> +    }
> +
> +    v->data = p;
> +
> +    i = 0;
> +    state = r->upstream_states->elts;
> +
> +    for ( ;; ) {
> +
> +        p = ngx_sprintf(p, "%uA", state[i].connection_number);
> +
> +        if (++i == r->upstream_states->nelts) {
> +            break;
> +        }
> +
> +        if (state[i].peer) {
> +            *p++ = ',';
> +            *p++ = ' ';
> +
> +        } else {
> +            *p++ = ' ';
> +            *p++ = ':';
> +            *p++ = ' ';
> +
> +            if (++i == r->upstream_states->nelts) {
> +                break;
> +            }
> +
> +            continue;
> +        }
> +    }
> +
> +    v->len = p - v->data;
> +
> +    return NGX_OK;
> +}
> +
> +
> ngx_int_t
> ngx_http_upstream_header_variable(ngx_http_request_t *r,
>     ngx_http_variable_value_t *v, uintptr_t data)
> diff -r c6372a40c2a7 -r f06c8a934e3f src/http/ngx_http_upstream.h
> --- a/src/http/ngx_http_upstream.h Fri Aug 26 15:33:07 2016 +0300
> +++ b/src/http/ngx_http_upstream.h Wed Aug 31 13:17:16 2016 +0000
> @@ -66,6 +66,8 @@
>     off_t                            bytes_received;
> 
>     ngx_str_t                       *peer;
> +
> +    ngx_atomic_uint_t                connection_number;
> } ngx_http_upstream_state_t;
> 
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160912/6f6fb219/attachment.bin>

From mdounin at mdounin.ru  Mon Sep 12 15:50:03 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 12 Sep 2016 18:50:03 +0300
Subject: [PATCH 2 of 2] Core: add ngx_atomic_store() and ngx_atomic_load()
In-Reply-To: <40765d8ee4dd29089b0e.1471480172@piotrsikora.sfo.corp.google.com>
References: <2f2ec92c3af93c11e195.1471480171@piotrsikora.sfo.corp.google.com>
 <40765d8ee4dd29089b0e.1471480172@piotrsikora.sfo.corp.google.com>
Message-ID: <20160912155003.GB1527@mdounin.ru>

Hello!

On Wed, Aug 17, 2016 at 05:29:32PM -0700, Piotr Sikora wrote:

> # HG changeset patch
> # User Piotr Sikora <piotrsikora at google.com>
> # Date 1471265532 25200
> #      Mon Aug 15 05:52:12 2016 -0700
> # Node ID 40765d8ee4dd29089b0e60ed5b6099ac624e804e
> # Parent  2f2ec92c3af93c11e195fb6d805df57518fede7c
> Core: add ngx_atomic_store() and ngx_atomic_load().
> 
> Those functions must be used to prevent data races between
> threads operating concurrently on the same variables.
> 
> No performance loss measured in microbenchmarks on x86_64.
> 
> No binary changes when compiled without __atomic intrinsics.
> 
> Found with ThreadSanitizer.
> 
> Signed-off-by: Piotr Sikora <piotrsikora at google.com>

[...]

>  #define ngx_trylock(lock, value)                                              \
> -    (*(lock) == 0 && ngx_atomic_cmp_set(lock, 0, value))
> +    (ngx_atomic_load(lock) == 0 && ngx_atomic_cmp_set(lock, 0, value))

The "*(lock) == 0" check here is just an optimization, it only 
ensures that the lock is likely to succed.  Atomicity is provided 
by the ngx_atomic_cmp_set() operation following the check.  If the 
check returns a wrong result due to non-atomic load - this won't 
do any harm.  The idea is that a quick-and-dirty non-atomic 
reading can be used to optimize things when the lock is already 
obtained by another process.  This is especially important in 
spinlocks like in ngx_shmtx_lock().

The same is believed to apply to the other places changed as well.  
If you think there are places where atomic reading is critical - 
please highlight these particular places.

-- 
Maxim Dounin
http://nginx.org/


From pluknet at nginx.com  Mon Sep 12 16:05:18 2016
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 12 Sep 2016 16:05:18 +0000
Subject: [nginx] SSL: factored out digest and cipher in session ticket
 callback.
Message-ID: <hg.f28e74f02c88.1473696318.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/f28e74f02c88
branches:  
changeset: 6686:f28e74f02c88
user:      Sergey Kandaurov <pluknet at nginx.com>
date:      Mon Sep 12 18:57:42 2016 +0300
description:
SSL: factored out digest and cipher in session ticket callback.

No functional changes.

diffstat:

 src/event/ngx_event_openssl.c |  28 ++++++++++++++--------------
 1 files changed, 14 insertions(+), 14 deletions(-)

diffs (66 lines):

diff -r 4a16fceea03b -r f28e74f02c88 src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c	Thu Sep 08 15:51:36 2016 +0300
+++ b/src/event/ngx_event_openssl.c	Mon Sep 12 18:57:42 2016 +0300
@@ -2941,13 +2941,6 @@ failed:
 }
 
 
-#ifdef OPENSSL_NO_SHA256
-#define ngx_ssl_session_ticket_md  EVP_sha1
-#else
-#define ngx_ssl_session_ticket_md  EVP_sha256
-#endif
-
-
 static int
 ngx_ssl_session_ticket_key_callback(ngx_ssl_conn_t *ssl_conn,
     unsigned char *name, unsigned char *iv, EVP_CIPHER_CTX *ectx,
@@ -2958,6 +2951,8 @@ ngx_ssl_session_ticket_key_callback(ngx_
     ngx_array_t                   *keys;
     ngx_connection_t              *c;
     ngx_ssl_session_ticket_key_t  *key;
+    const EVP_MD                  *digest;
+    const EVP_CIPHER              *cipher;
 #if (NGX_DEBUG)
     u_char                         buf[32];
 #endif
@@ -2965,6 +2960,13 @@ ngx_ssl_session_ticket_key_callback(ngx_
     c = ngx_ssl_get_connection(ssl_conn);
     ssl_ctx = c->ssl->session_ctx;
 
+    cipher = EVP_aes_128_cbc();
+#ifdef OPENSSL_NO_SHA256
+    digest = EVP_sha1();
+#else
+    digest = EVP_sha256();
+#endif
+
     keys = SSL_CTX_get_ex_data(ssl_ctx, ngx_ssl_session_ticket_keys_index);
     if (keys == NULL) {
         return -1;
@@ -2980,10 +2982,9 @@ ngx_ssl_session_ticket_key_callback(ngx_
                        ngx_hex_dump(buf, key[0].name, 16) - buf, buf,
                        SSL_session_reused(ssl_conn) ? "reused" : "new");
 
-        RAND_bytes(iv, 16);
-        EVP_EncryptInit_ex(ectx, EVP_aes_128_cbc(), NULL, key[0].aes_key, iv);
-        HMAC_Init_ex(hctx, key[0].hmac_key, 16,
-                     ngx_ssl_session_ticket_md(), NULL);
+        RAND_bytes(iv, EVP_CIPHER_iv_length(cipher));
+        EVP_EncryptInit_ex(ectx, cipher, NULL, key[0].aes_key, iv);
+        HMAC_Init_ex(hctx, key[0].hmac_key, 16, digest, NULL);
         ngx_memcpy(name, key[0].name, 16);
 
         return 1;
@@ -3010,9 +3011,8 @@ ngx_ssl_session_ticket_key_callback(ngx_
                        ngx_hex_dump(buf, key[i].name, 16) - buf, buf,
                        (i == 0) ? " (default)" : "");
 
-        HMAC_Init_ex(hctx, key[i].hmac_key, 16,
-                     ngx_ssl_session_ticket_md(), NULL);
-        EVP_DecryptInit_ex(ectx, EVP_aes_128_cbc(), NULL, key[i].aes_key, iv);
+        HMAC_Init_ex(hctx, key[i].hmac_key, 16, digest, NULL);
+        EVP_DecryptInit_ex(ectx, cipher, NULL, key[i].aes_key, iv);
 
         return (i == 0) ? 1 : 2 /* renew */;
     }


From pluknet at nginx.com  Mon Sep 12 16:05:21 2016
From: pluknet at nginx.com (Sergey Kandaurov)
Date: Mon, 12 Sep 2016 16:05:21 +0000
Subject: [nginx] SSL: improved session ticket callback error handling.
Message-ID: <hg.dfa626cdde6b.1473696321.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/dfa626cdde6b
branches:  
changeset: 6687:dfa626cdde6b
user:      Sergey Kandaurov <pluknet at nginx.com>
date:      Mon Sep 12 18:57:42 2016 +0300
description:
SSL: improved session ticket callback error handling.

Prodded by Guido Vranken.

diffstat:

 src/event/ngx_event_openssl.c |  35 ++++++++++++++++++++++++++++++++---
 1 files changed, 32 insertions(+), 3 deletions(-)

diffs (54 lines):

diff -r f28e74f02c88 -r dfa626cdde6b src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c	Mon Sep 12 18:57:42 2016 +0300
+++ b/src/event/ngx_event_openssl.c	Mon Sep 12 18:57:42 2016 +0300
@@ -2982,9 +2982,26 @@ ngx_ssl_session_ticket_key_callback(ngx_
                        ngx_hex_dump(buf, key[0].name, 16) - buf, buf,
                        SSL_session_reused(ssl_conn) ? "reused" : "new");
 
-        RAND_bytes(iv, EVP_CIPHER_iv_length(cipher));
-        EVP_EncryptInit_ex(ectx, cipher, NULL, key[0].aes_key, iv);
+        if (RAND_bytes(iv, EVP_CIPHER_iv_length(cipher)) != 1) {
+            ngx_ssl_error(NGX_LOG_ALERT, c->log, 0, "RAND_bytes() failed");
+            return -1;
+        }
+
+        if (EVP_EncryptInit_ex(ectx, cipher, NULL, key[0].aes_key, iv) != 1) {
+            ngx_ssl_error(NGX_LOG_ALERT, c->log, 0,
+                          "EVP_EncryptInit_ex() failed");
+            return -1;
+        }
+
+#if OPENSSL_VERSION_NUMBER >= 0x10000000L
+        if (HMAC_Init_ex(hctx, key[0].hmac_key, 16, digest, NULL) != 1) {
+            ngx_ssl_error(NGX_LOG_ALERT, c->log, 0, "HMAC_Init_ex() failed");
+            return -1;
+        }
+#else
         HMAC_Init_ex(hctx, key[0].hmac_key, 16, digest, NULL);
+#endif
+
         ngx_memcpy(name, key[0].name, 16);
 
         return 1;
@@ -3011,8 +3028,20 @@ ngx_ssl_session_ticket_key_callback(ngx_
                        ngx_hex_dump(buf, key[i].name, 16) - buf, buf,
                        (i == 0) ? " (default)" : "");
 
+#if OPENSSL_VERSION_NUMBER >= 0x10000000L
+        if (HMAC_Init_ex(hctx, key[i].hmac_key, 16, digest, NULL) != 1) {
+            ngx_ssl_error(NGX_LOG_ALERT, c->log, 0, "HMAC_Init_ex() failed");
+            return -1;
+        }
+#else
         HMAC_Init_ex(hctx, key[i].hmac_key, 16, digest, NULL);
-        EVP_DecryptInit_ex(ectx, cipher, NULL, key[i].aes_key, iv);
+#endif
+
+        if (EVP_DecryptInit_ex(ectx, cipher, NULL, key[i].aes_key, iv) != 1) {
+            ngx_ssl_error(NGX_LOG_ALERT, c->log, 0,
+                          "EVP_DecryptInit_ex() failed");
+            return -1;
+        }
 
         return (i == 0) ? 1 : 2 /* renew */;
     }


From mdounin at mdounin.ru  Mon Sep 12 17:52:56 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 12 Sep 2016 17:52:56 +0000
Subject: [nginx] OCSP stapling: fixed using wrong responder with multiple
 certs.
Message-ID: <hg.6acbe9964ceb.1473702776.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/6acbe9964ceb
branches:  
changeset: 6688:6acbe9964ceb
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Sep 12 20:11:06 2016 +0300
description:
OCSP stapling: fixed using wrong responder with multiple certs.

diffstat:

 src/event/ngx_event_openssl_stapling.c |  3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diffs (20 lines):

diff --git a/src/event/ngx_event_openssl_stapling.c b/src/event/ngx_event_openssl_stapling.c
--- a/src/event/ngx_event_openssl_stapling.c
+++ b/src/event/ngx_event_openssl_stapling.c
@@ -376,6 +376,7 @@ ngx_ssl_stapling_responder(ngx_conf_t *c
 {
     ngx_url_t                  u;
     char                      *s;
+    ngx_str_t                  rsp;
     STACK_OF(OPENSSL_STRING)  *aia;
 
     if (responder->len == 0) {
@@ -403,6 +404,8 @@ ngx_ssl_stapling_responder(ngx_conf_t *c
             return NGX_DECLINED;
         }
 
+        responder = &rsp;
+
         responder->len = ngx_strlen(s);
         responder->data = ngx_palloc(cf->pool, responder->len);
         if (responder->data == NULL) {


From riramar at gmail.com  Mon Sep 12 18:56:24 2016
From: riramar at gmail.com (Ricardo Iramar dos Santos)
Date: Mon, 12 Sep 2016 15:56:24 -0300
Subject: Name resolution and opening connections
Message-ID: <CAE5Wca2g-zKJjfskr0Cq2fuD61-PnNT-HaJi0KhKhs3QoPNgaQ@mail.gmail.com>

Hi All,

This is my second post here so take easy on me.
I'm doing a security research about web servers and ddos
amplification. My guess is that web servers probably can reflect
HTTP(S) requests and be a "good" ddos amplification.
So maybe the best way to find this "reflected request" is looking
inside the code for functions like gethostbyname (since maybe the
server need to get the IP before open a connection).
Searching on github I was able to find only one .c file (ngx_inet.c)
in the line 1271 below:

h = gethostbyname((char *) host);

Some lines below that it seems the code is opening a socket connecting
(sorry... I'm not familiar with nginx code).

sin = ngx_pcalloc(pool, sizeof(struct sockaddr_in));
if (sin == NULL) {
return NGX_ERROR;
}
sin->sin_family = AF_INET;
sin->sin_port = port;
sin->sin_addr.s_addr = *(in_addr_t *) (h->h_addr_list[i]);
u->addrs[i].sockaddr = (struct sockaddr *) sin;
u->addrs[i].socklen = sizeof(struct sockaddr_in);
len = NGX_INET_ADDRSTRLEN + sizeof(":65535") - 1;

My question is this the best approach to reach my goal? In which case
the piece of code above would be invoked?
Sorry for the inconvenient for so many questions and I really
appreciate any help.

Thanks!
Ricardo Iramar


From ondrej.novy at gmail.com  Mon Sep 12 19:55:32 2016
From: ondrej.novy at gmail.com (=?UTF-8?B?T25kxZllaiBOb3bDvQ==?=)
Date: Mon, 12 Sep 2016 21:55:32 +0200
Subject: --with-openssl and OPENSSL_OPT
Message-ID: <CAOO6c=xcMUCMQNULw0F4ADNPQ+1_nnCXxRp6BaSP0VU+W8n8AQ@mail.gmail.com>

Hi,

I want to use OpenSSL 1.0.2 static linked with nginx. So I'm using
--with-openssl option. But I want to set OpenSSL configure options. Option
OPENSSL_OPT looks like correct way.

If I set this variable:
export OPENSSL_OPT=no-idea

After OpenSSL configure I got message:
*** Because of configuration changes, you MUST do the following before
*** building:

make depend

And building fails:
make[5]: *** No rule to make target '../../include/openssl/idea.h', needed
by 'e_idea.o'.  Stop.

I think you are not calling "make depend" after configuration of OpenSSL
(auto/lib/openssl/make*).

Thanks for help.

-- 
Best regards
 Ond?ej Nov?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160912/8a437682/attachment.html>

From mdounin at mdounin.ru  Tue Sep 13 01:40:54 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 13 Sep 2016 04:40:54 +0300
Subject: --with-openssl and OPENSSL_OPT
In-Reply-To: <CAOO6c=xcMUCMQNULw0F4ADNPQ+1_nnCXxRp6BaSP0VU+W8n8AQ@mail.gmail.com>
References: <CAOO6c=xcMUCMQNULw0F4ADNPQ+1_nnCXxRp6BaSP0VU+W8n8AQ@mail.gmail.com>
Message-ID: <20160913014054.GE1527@mdounin.ru>

Hello!

On Mon, Sep 12, 2016 at 09:55:32PM +0200, Ond?ej Nov? wrote:

> I want to use OpenSSL 1.0.2 static linked with nginx. So I'm using
> --with-openssl option. But I want to set OpenSSL configure options. Option
> OPENSSL_OPT looks like correct way.
> 
> If I set this variable:
> export OPENSSL_OPT=no-idea
> 
> After OpenSSL configure I got message:
> *** Because of configuration changes, you MUST do the following before
> *** building:
> 
> make depend
> 
> And building fails:
> make[5]: *** No rule to make target '../../include/openssl/idea.h', needed
> by 'e_idea.o'.  Stop.
> 
> I think you are not calling "make depend" after configuration of OpenSSL
> (auto/lib/openssl/make*).

If you need to configure openssl configure options which require 
"make depend", you can run "make depend" yourself before making 
nginx (and thus OpenSSL).

Alternatively, you can make OpenSSL yourself instead of asking 
nginx to do it for you.  Note that using --with-openssl option of 
nginx configure is not something required for static building with 
OpenSSL.  Rather, it's a convenient shortcut to make things easier 
in common cases.

-- 
Maxim Dounin
http://nginx.org/


From igor at sysoev.ru  Tue Sep 13 13:20:19 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Tue, 13 Sep 2016 13:20:19 +0000
Subject: [njs] Constructor function prototypes now have correct types and...
Message-ID: <hg.cb414901cccc.1473772819.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/cb414901cccc
branches:  
changeset: 169:cb414901cccc
user:      Igor Sysoev <igor at sysoev.ru>
date:      Tue Sep 13 16:19:26 2016 +0300
description:
Constructor function prototypes now have correct types and values.

diffstat:

 njs/njs_array.c          |    3 +-
 njs/njs_array.h          |    9 --
 njs/njs_builtin.c        |   49 ++++++++++++--
 njs/njs_date.c           |    3 +-
 njs/njs_date.h           |    6 -
 njs/njs_function.c       |    6 +-
 njs/njs_object.c         |   52 +++++++++------
 njs/njs_object.h         |    6 -
 njs/njs_regexp.c         |    3 +-
 njs/njs_regexp.h         |   16 -----
 njs/njs_vm.c             |    8 +-
 njs/njs_vm.h             |  150 +++++++++++++++++++++++++++++++---------------
 njs/test/njs_unit_test.c |   48 +++++++++++++++
 13 files changed, 234 insertions(+), 125 deletions(-)

diffs (762 lines):

diff -r 19758fdcd554 -r cb414901cccc njs/njs_array.c
--- a/njs/njs_array.c	Thu Sep 01 16:12:31 2016 +0300
+++ b/njs/njs_array.c	Tue Sep 13 16:19:26 2016 +0300
@@ -133,7 +133,8 @@ njs_array_alloc(njs_vm_t *vm, uint32_t l
     array->start = array->data;
     nxt_lvlhsh_init(&array->object.hash);
     nxt_lvlhsh_init(&array->object.shared_hash);
-    array->object.__proto__ = &vm->prototypes[NJS_PROTOTYPE_ARRAY];
+    array->object.__proto__ = &vm->prototypes[NJS_PROTOTYPE_ARRAY].object;
+    array->object.type = NJS_ARRAY;
     array->object.shared = 0;
     array->size = size;
     array->length = length;
diff -r 19758fdcd554 -r cb414901cccc njs/njs_array.h
--- a/njs/njs_array.h	Thu Sep 01 16:12:31 2016 +0300
+++ b/njs/njs_array.h	Tue Sep 13 16:19:26 2016 +0300
@@ -10,15 +10,6 @@
 
 #define NJS_ARRAY_SPARE  8
 
-struct njs_array_s {
-    /* Must be aligned to njs_value_t. */
-    njs_object_t         object;
-    uint32_t             size;
-    uint32_t             length;
-    njs_value_t          *start;
-    njs_value_t          *data;
-};
-
 
 njs_array_t *njs_array_alloc(njs_vm_t *vm, uint32_t length, uint32_t spare);
 njs_ret_t njs_array_string_add(njs_vm_t *vm, njs_array_t *array, u_char *start,
diff -r 19758fdcd554 -r cb414901cccc njs/njs_builtin.c
--- a/njs/njs_builtin.c	Thu Sep 01 16:12:31 2016 +0300
+++ b/njs/njs_builtin.c	Tue Sep 13 16:19:26 2016 +0300
@@ -37,10 +37,11 @@ typedef struct {
 nxt_int_t
 njs_builtin_objects_create(njs_vm_t *vm)
 {
-    nxt_int_t       ret;
-    nxt_uint_t      i;
-    njs_object_t    *objects, *prototypes;
-    njs_function_t  *functions, *constructors;
+    nxt_int_t               ret;
+    nxt_uint_t              i;
+    njs_object_t            *objects;
+    njs_function_t          *functions, *constructors;
+    njs_object_prototype_t  *prototypes;
 
     static const njs_object_init_t    *prototype_init[] = {
         &njs_object_prototype_init,
@@ -53,6 +54,29 @@ njs_builtin_objects_create(njs_vm_t *vm)
         &njs_date_prototype_init,
     };
 
+    static const njs_object_prototype_t  prototype_values[] = {
+        { .object.type = NJS_OBJECT },
+        { .object.type = NJS_ARRAY },
+
+        /*
+         * The .object.type field must be initialzed after the .value field,
+         * otherwise SunC 5.9 treats the .value as .object.value or so.
+         */
+        { .object_value = { .value = njs_value(NJS_BOOLEAN, 0, 0.0),
+                            .object.type = NJS_OBJECT_BOOLEAN } },
+
+        { .object_value = { .value = njs_value(NJS_NUMBER, 0, 0.0),
+                            .object.type = NJS_OBJECT_NUMBER } },
+
+        { .object_value = { .value = njs_string(""),
+                            .object.type = NJS_OBJECT_STRING } },
+
+        { .object.type = NJS_FUNCTION },
+        { .object.type = NJS_REGEXP },
+
+        { .date =         { .time = NJS_NAN, .object.type = NJS_DATE } },
+    };
+
     static const njs_object_init_t    *constructor_init[] = {
         &njs_object_constructor_init,
         &njs_array_constructor_init,
@@ -172,7 +196,9 @@ njs_builtin_objects_create(njs_vm_t *vm)
     prototypes = vm->shared->prototypes;
 
     for (i = NJS_PROTOTYPE_OBJECT; i < NJS_PROTOTYPE_MAX; i++) {
-        ret = njs_object_hash_create(vm, &prototypes[i].shared_hash,
+        prototypes[i] = prototype_values[i];
+
+        ret = njs_object_hash_create(vm, &prototypes[i].object.shared_hash,
                                      prototype_init[i]->properties,
                                      prototype_init[i]->items);
         if (nxt_slow_path(ret != NXT_OK)) {
@@ -180,6 +206,9 @@ njs_builtin_objects_create(njs_vm_t *vm)
         }
     }
 
+    prototypes[NJS_PROTOTYPE_REGEXP].regexp.pattern =
+                                     vm->empty_regexp.data.u.regexp->pattern;
+
     constructors = vm->shared->constructors;
 
     for (i = NJS_CONSTRUCTOR_OBJECT; i < NJS_CONSTRUCTOR_MAX; i++) {
@@ -249,22 +278,24 @@ njs_builtin_objects_clone(njs_vm_t *vm)
     size_t        size;
     nxt_uint_t    i;
     njs_value_t   *values;
-    njs_object_t  *function_prototype;
+    njs_object_t  *object_prototype, *function_prototype;
 
     /*
      * Copy both prototypes and constructors arrays by one memcpy()
      * because they are stored together.
      */
-    size = NJS_PROTOTYPE_MAX * sizeof(njs_object_t)
+    size = NJS_PROTOTYPE_MAX * sizeof(njs_object_prototype_t)
            + NJS_CONSTRUCTOR_MAX * sizeof(njs_function_t);
 
     memcpy(vm->prototypes, vm->shared->prototypes, size);
 
+    object_prototype = &vm->prototypes[NJS_PROTOTYPE_OBJECT].object;
+
     for (i = NJS_PROTOTYPE_ARRAY; i < NJS_PROTOTYPE_MAX; i++) {
-        vm->prototypes[i].__proto__ = &vm->prototypes[NJS_PROTOTYPE_OBJECT];
+        vm->prototypes[i].object.__proto__ = object_prototype;
     }
 
-    function_prototype = &vm->prototypes[NJS_CONSTRUCTOR_FUNCTION];
+    function_prototype = &vm->prototypes[NJS_CONSTRUCTOR_FUNCTION].object;
     values = vm->scopes[NJS_SCOPE_GLOBAL];
 
     for (i = NJS_CONSTRUCTOR_OBJECT; i < NJS_CONSTRUCTOR_MAX; i++) {
diff -r 19758fdcd554 -r cb414901cccc njs/njs_date.c
--- a/njs/njs_date.c	Thu Sep 01 16:12:31 2016 +0300
+++ b/njs/njs_date.c	Tue Sep 13 16:19:26 2016 +0300
@@ -151,8 +151,9 @@ njs_date_constructor(njs_vm_t *vm, njs_v
 
         nxt_lvlhsh_init(&date->object.hash);
         nxt_lvlhsh_init(&date->object.shared_hash);
+        date->object.type = NJS_DATE;
         date->object.shared = 0;
-        date->object.__proto__ = &vm->prototypes[NJS_PROTOTYPE_DATE];
+        date->object.__proto__ = &vm->prototypes[NJS_PROTOTYPE_DATE].object;
 
         date->time = time;
 
diff -r 19758fdcd554 -r cb414901cccc njs/njs_date.h
--- a/njs/njs_date.h	Thu Sep 01 16:12:31 2016 +0300
+++ b/njs/njs_date.h	Tue Sep 13 16:19:26 2016 +0300
@@ -8,12 +8,6 @@
 #define _NJS_DATE_H_INCLUDED_
 
 
-struct njs_date_s {
-    njs_object_t  object;
-    double        time;
-};
-
-
 njs_ret_t njs_date_constructor(njs_vm_t *vm, njs_value_t *args,
     nxt_uint_t nargs, njs_index_t unused);
 
diff -r 19758fdcd554 -r cb414901cccc njs/njs_function.c
--- a/njs/njs_function.c	Thu Sep 01 16:12:31 2016 +0300
+++ b/njs/njs_function.c	Tue Sep 13 16:19:26 2016 +0300
@@ -40,6 +40,7 @@ njs_function_alloc(njs_vm_t *vm)
          */
 
         function->object.shared_hash = vm->shared->function_prototype_hash;
+        function->object.type = NJS_FUNCTION;
         function->object.shared = 1;
         function->args_offset = 1;
 
@@ -69,7 +70,8 @@ njs_function_value_copy(njs_vm_t *vm, nj
 
     if (nxt_fast_path(function != NULL)) {
         *function = *value->data.u.function;
-        function->object.__proto__ = &vm->prototypes[NJS_PROTOTYPE_FUNCTION];
+        function->object.__proto__ =
+                                &vm->prototypes[NJS_PROTOTYPE_FUNCTION].object;
         function->object.shared = 0;
         value->data.u.function = function;
     }
@@ -527,7 +529,7 @@ njs_function_prototype_bind(njs_vm_t *vm
 
     *function = *args[0].data.u.function;
 
-    function->object.__proto__ = &vm->prototypes[NJS_PROTOTYPE_FUNCTION];
+    function->object.__proto__ = &vm->prototypes[NJS_PROTOTYPE_FUNCTION].object;
     function->object.shared = 0;
 
     if (nargs == 1) {
diff -r 19758fdcd554 -r cb414901cccc njs/njs_object.c
--- a/njs/njs_object.c	Thu Sep 01 16:12:31 2016 +0300
+++ b/njs/njs_object.c	Tue Sep 13 16:19:26 2016 +0300
@@ -35,7 +35,8 @@ njs_object_alloc(njs_vm_t *vm)
     if (nxt_fast_path(object != NULL)) {
         nxt_lvlhsh_init(&object->hash);
         nxt_lvlhsh_init(&object->shared_hash);
-        object->__proto__ = &vm->prototypes[NJS_PROTOTYPE_OBJECT];
+        object->__proto__ = &vm->prototypes[NJS_PROTOTYPE_OBJECT].object;
+        object->type = NJS_OBJECT;
         object->shared = 0;
     }
 
@@ -58,7 +59,7 @@ njs_object_value_copy(njs_vm_t *vm, njs_
 
     if (nxt_fast_path(object != NULL)) {
         *object = *value->data.u.object;
-        object->__proto__ = &vm->prototypes[NJS_PROTOTYPE_OBJECT];
+        object->__proto__ = &vm->prototypes[NJS_PROTOTYPE_OBJECT].object;
         object->shared = 0;
         value->data.u.object = object;
     }
@@ -78,10 +79,11 @@ njs_object_value_alloc(njs_vm_t *vm, con
     if (nxt_fast_path(ov != NULL)) {
         nxt_lvlhsh_init(&ov->object.hash);
         nxt_lvlhsh_init(&ov->object.shared_hash);
+        ov->object.type = njs_object_value_type(type);
         ov->object.shared = 0;
 
         index = njs_primitive_prototype_index(type);
-        ov->object.__proto__ = &vm->prototypes[index];
+        ov->object.__proto__ = &vm->prototypes[index].object;
 
         ov->value = *value;
     }
@@ -258,7 +260,7 @@ njs_object_constructor(njs_vm_t *vm, njs
                 return NXT_ERROR;
             }
 
-            type = NJS_OBJECT + value->type;
+            type = njs_object_value_type(value->type);
 
         } else {
             vm->exception = &njs_exception_type_error;
@@ -324,6 +326,7 @@ njs_object_create(njs_vm_t *vm, njs_valu
 njs_ret_t
 njs_primitive_prototype_get_proto(njs_vm_t *vm, njs_value_t *value)
 {
+    nxt_uint_t    index;
     njs_object_t  *proto;
 
     /*
@@ -331,14 +334,15 @@ njs_primitive_prototype_get_proto(njs_vm
      * and have to return different results for primitive type and for objects.
      */
     if (njs_is_object(value)) {
-         proto = value->data.u.object->__proto__;
+        proto = value->data.u.object->__proto__;
 
     } else {
-         proto = &vm->prototypes[njs_primitive_prototype_index(value->type)];
+        index = njs_primitive_prototype_index(value->type);
+        proto = &vm->prototypes[index].object;
     }
 
     vm->retval.data.u.object = proto;
-    vm->retval.type = NJS_OBJECT;
+    vm->retval.type = proto->type;
     vm->retval.data.truth = 1;
 
     return NXT_OK;
@@ -364,7 +368,7 @@ njs_object_prototype_create(njs_vm_t *vm
 
     if (index >= 0 && index < NJS_PROTOTYPE_MAX) {
         proto = njs_property_prototype_create(vm, &function->object.hash,
-                                              &vm->prototypes[index]);
+                                              &vm->prototypes[index].object);
     }
 
     if (proto == NULL) {
@@ -395,7 +399,7 @@ njs_property_prototype_create(njs_vm_t *
     /* GC */
 
     prop->value.data.u.object = prototype;
-    prop->value.type = NJS_OBJECT;
+    prop->value.type = prototype->type;
     prop->value.data.truth = 1;
 
     prop->enumerable = 0;
@@ -469,7 +473,7 @@ njs_object_prototype_get_proto(njs_vm_t 
 
     if (nxt_fast_path(proto != NULL)) {
         vm->retval.data.u.object = proto;
-        vm->retval.type = NJS_OBJECT;
+        vm->retval.type = proto->type;
         vm->retval.data.truth = 1;
 
     } else {
@@ -489,23 +493,25 @@ njs_object_prototype_get_proto(njs_vm_t 
 static njs_ret_t
 njs_object_prototype_create_constructor(njs_vm_t *vm, njs_value_t *value)
 {
-    int32_t       index;
-    njs_value_t   *cons;
-    njs_object_t  *prototype;
+    int32_t                 index;
+    njs_value_t             *cons;
+    njs_object_t            *object;
+    njs_object_prototype_t  *prototype;
 
     if (njs_is_object(value)) {
-        prototype = value->data.u.object;
+        object = value->data.u.object;
 
         do {
+            prototype = (njs_object_prototype_t *) object;
             index = prototype - vm->prototypes;
 
             if (index >= 0 && index < NJS_PROTOTYPE_MAX) {
                 goto found;
             }
 
-            prototype = prototype->__proto__;
+            object = object->__proto__;
 
-        } while (prototype != NULL);
+        } while (object != NULL);
 
         nxt_thread_log_alert("prototype not found");
 
@@ -518,7 +524,7 @@ njs_object_prototype_create_constructor(
 
 found:
 
-    cons = njs_property_constructor_create(vm, &prototype->hash,
+    cons = njs_property_constructor_create(vm, &prototype->object.hash,
                                           &vm->scopes[NJS_SCOPE_GLOBAL][index]);
     if (nxt_fast_path(cons != NULL)) {
         vm->retval = *cons;
@@ -604,8 +610,9 @@ njs_ret_t
 njs_object_prototype_to_string(njs_vm_t *vm, njs_value_t *args,
     nxt_uint_t nargs, njs_index_t unused)
 {
-    int32_t       index;
-    njs_object_t  *prototype;
+    int32_t                 index;
+    njs_object_t            *object;
+    njs_object_prototype_t  *prototype;
 
     static const njs_value_t  *class_name[] = {
         /* Primitives. */
@@ -633,9 +640,10 @@ njs_object_prototype_to_string(njs_vm_t 
     index = args[0].type;
 
     if (njs_is_object(&args[0])) {
-        prototype = args[0].data.u.object;
+        object = args[0].data.u.object;
 
         do {
+            prototype = (njs_object_prototype_t *) object;
             index = prototype - vm->prototypes;
 
             if (index >= 0 && index < NJS_PROTOTYPE_MAX) {
@@ -643,9 +651,9 @@ njs_object_prototype_to_string(njs_vm_t 
                 goto found;
             }
 
-            prototype = prototype->__proto__;
+            object = object->__proto__;
 
-        } while (prototype != NULL);
+        } while (object != NULL);
 
         nxt_thread_log_alert("prototype not found");
 
diff -r 19758fdcd554 -r cb414901cccc njs/njs_object.h
--- a/njs/njs_object.h	Thu Sep 01 16:12:31 2016 +0300
+++ b/njs/njs_object.h	Tue Sep 13 16:19:26 2016 +0300
@@ -8,12 +8,6 @@
 #define _NJS_OBJECT_H_INCLUDED_
 
 
-struct njs_object_value_s {
-    njs_object_t                object;
-    njs_value_t                 value;
-};
-
-
 typedef enum {
     NJS_PROPERTY = 0,
     NJS_GETTER,
diff -r 19758fdcd554 -r cb414901cccc njs/njs_regexp.c
--- a/njs/njs_regexp.c	Thu Sep 01 16:12:31 2016 +0300
+++ b/njs/njs_regexp.c	Tue Sep 13 16:19:26 2016 +0300
@@ -466,7 +466,8 @@ njs_regexp_alloc(njs_vm_t *vm, njs_regex
     if (nxt_fast_path(regexp != NULL)) {
         nxt_lvlhsh_init(&regexp->object.hash);
         nxt_lvlhsh_init(&regexp->object.shared_hash);
-        regexp->object.__proto__ = &vm->prototypes[NJS_PROTOTYPE_REGEXP];
+        regexp->object.__proto__ = &vm->prototypes[NJS_PROTOTYPE_REGEXP].object;
+        regexp->object.type = NJS_REGEXP;
         regexp->object.shared = 0;
         regexp->last_index = 0;
         regexp->pattern = pattern;
diff -r 19758fdcd554 -r cb414901cccc njs/njs_regexp.h
--- a/njs/njs_regexp.h	Thu Sep 01 16:12:31 2016 +0300
+++ b/njs/njs_regexp.h	Tue Sep 13 16:19:26 2016 +0300
@@ -16,22 +16,6 @@ typedef enum {
 } njs_regexp_flags_t;
 
 
-struct njs_regexp_s {
-    /* Must be aligned to njs_value_t. */
-    njs_object_t          object;
-
-    uint32_t              last_index;
-
-    njs_regexp_pattern_t  *pattern;
-
-    /*
-     * This string value can be unaligned since
-     * it never used in nJSVM operations.
-     */
-    njs_value_t           string;
-};
-
-
 njs_ret_t njs_regexp_init(njs_vm_t *vm);
 njs_ret_t njs_regexp_constructor(njs_vm_t *vm, njs_value_t *args,
     nxt_uint_t nargs, njs_index_t unused);
diff -r 19758fdcd554 -r cb414901cccc njs/njs_vm.c
--- a/njs/njs_vm.c	Thu Sep 01 16:12:31 2016 +0300
+++ b/njs/njs_vm.c	Tue Sep 13 16:19:26 2016 +0300
@@ -403,7 +403,8 @@ njs_vmcode_function(njs_vm_t *vm, njs_va
     function = nxt_mem_cache_zalloc(vm->mem_cache_pool, sizeof(njs_function_t));
 
     if (nxt_fast_path(function != NULL)) {
-        function->object.__proto__ = &vm->prototypes[NJS_PROTOTYPE_FUNCTION];
+        function->object.__proto__ =
+                                &vm->prototypes[NJS_PROTOTYPE_FUNCTION].object;
         function->args_offset = 1;
 
         code = (njs_vmcode_function_t *) vm->current;
@@ -929,7 +930,8 @@ njs_property_query(njs_vm_t *vm, njs_pro
             return NJS_PRIMITIVE_VALUE;
         }
 
-        obj = &vm->prototypes[njs_primitive_prototype_index(object->type)];
+        index = njs_primitive_prototype_index(object->type);
+        obj = &vm->prototypes[index].object;
         break;
 
     case NJS_STRING:
@@ -937,7 +939,7 @@ njs_property_query(njs_vm_t *vm, njs_pro
             return NXT_DECLINED;
         }
 
-        obj = &vm->prototypes[NJS_PROTOTYPE_STRING];
+        obj = &vm->prototypes[NJS_PROTOTYPE_STRING].object;
         break;
 
     case NJS_ARRAY:
diff -r 19758fdcd554 -r cb414901cccc njs/njs_vm.h
--- a/njs/njs_vm.h	Thu Sep 01 16:12:31 2016 +0300
+++ b/njs/njs_vm.h	Tue Sep 13 16:19:26 2016 +0300
@@ -111,6 +111,7 @@ typedef njs_ret_t (*njs_function_native_
 
 
 typedef struct njs_string_s           njs_string_t;
+typedef struct njs_object_s           njs_object_t;
 typedef struct njs_object_init_s      njs_object_init_t;
 typedef struct njs_object_value_s     njs_object_value_t;
 typedef struct njs_array_s            njs_array_t;
@@ -123,53 +124,6 @@ typedef struct njs_native_frame_s     nj
 typedef struct njs_property_next_s    njs_property_next_t;
 
 
-typedef struct njs_object_s           njs_object_t;
-
-struct njs_object_s {
-    /* A private hash of njs_object_prop_t. */
-    nxt_lvlhsh_t                      hash;
-
-    /* A shared hash of njs_object_prop_t. */
-    nxt_lvlhsh_t                      shared_hash;
-
-    /* An object __proto__. */
-    njs_object_t                      *__proto__;
-
-    uint32_t                          shared;  /* 1 bit */
-};
-
-
-#define NJS_ARGS_TYPES_MAX            5
-
-struct njs_function_s {
-    njs_object_t                      object;
-
-    uint8_t                           args_types[NJS_ARGS_TYPES_MAX];
-    uint8_t                           args_offset;
-
-    /*
-     * TODO Shared
-     * When function object is used as value: in assignments,
-     * as function argument, as property and as object to get properties.
-     */
-
-#if (NXT_64BIT)
-    uint8_t                           native;
-    uint8_t                           continuation_size;
-#else
-    uint8_t                           native;
-    uint8_t                           continuation_size;
-#endif
-
-    union {
-        njs_function_lambda_t         *lambda;
-        njs_function_native_t         native;
-    } u;
-
-    njs_value_t                       *bound;
-};
-
-
 typedef struct njs_continuation_s     njs_continuation_t;
 
 struct njs_continuation_s {
@@ -246,6 +200,97 @@ union njs_value_s {
 };
 
 
+struct njs_object_s {
+    /* A private hash of njs_object_prop_t. */
+    nxt_lvlhsh_t                      hash;
+
+    /* A shared hash of njs_object_prop_t. */
+    nxt_lvlhsh_t                      shared_hash;
+
+    /* An object __proto__. */
+    njs_object_t                      *__proto__;
+
+    /* The type is used in constructor prototypes. */
+    njs_value_type_t                  type:8;
+    uint8_t                           shared;  /* 1 bit */
+};
+
+
+struct njs_object_value_s {
+    njs_object_t                      object;
+    /* The value can be unaligned since it never used in nJSVM operations. */
+    njs_value_t                       value;
+};
+
+
+struct njs_array_s {
+    njs_object_t                      object;
+    uint32_t                          size;
+    uint32_t                          length;
+    njs_value_t                       *start;
+    njs_value_t                       *data;
+};
+
+
+#define NJS_ARGS_TYPES_MAX            5
+
+struct njs_function_s {
+    njs_object_t                      object;
+
+    uint8_t                           args_types[NJS_ARGS_TYPES_MAX];
+    uint8_t                           args_offset;
+
+    /*
+     * TODO Shared
+     * When function object is used as value: in assignments,
+     * as function argument, as property and as object to get properties.
+     */
+
+#if (NXT_64BIT)
+    uint8_t                           native;
+    uint8_t                           continuation_size;
+#else
+    uint8_t                           native;
+    uint8_t                           continuation_size;
+#endif
+
+    union {
+        njs_function_lambda_t         *lambda;
+        njs_function_native_t         native;
+    } u;
+
+    njs_value_t                       *bound;
+};
+
+
+struct njs_regexp_s {
+    njs_object_t                      object;
+    uint32_t                          last_index;
+    njs_regexp_pattern_t              *pattern;
+    /*
+     * This string value can be unaligned since
+     * it never used in nJSVM operations.
+     */
+    njs_value_t                       string;
+};
+
+
+struct njs_date_s {
+    njs_object_t                      object;
+    double                            time;
+};
+
+
+typedef union {
+    njs_object_t                      object;
+    njs_object_value_t                object_value;
+    njs_array_t                       array;
+    njs_function_t                    function;
+    njs_regexp_t                      regexp;
+    njs_date_t                        date;
+} njs_object_prototype_t;
+
+
 #define njs_value(_type, _truth, _number) {                                   \
     .data = {                                                                 \
         .type = _type,                                                        \
@@ -367,6 +412,10 @@ typedef njs_ret_t (*njs_vmcode_operation
     (((value)->type & NJS_OBJECT) != 0)
 
 
+#define njs_object_value_type(type)                                           \
+    (type + NJS_OBJECT)
+
+
 #define njs_is_array(value)                                                   \
     ((value)->type == NJS_ARRAY)
 
@@ -683,6 +732,9 @@ enum njs_prototypes_e {
 #define njs_primitive_prototype_index(type)                                   \
     (NJS_PROTOTYPE_BOOLEAN + ((type) - NJS_BOOLEAN))
 
+#define njs_prototype_type(index)                                             \
+    (index + NJS_OBJECT)
+
 
 enum njs_constructor_e {
     NJS_CONSTRUCTOR_OBJECT =   NJS_PROTOTYPE_OBJECT,
@@ -788,7 +840,7 @@ struct njs_vm_s {
      * they are copied from njs_vm_shared_t by single memcpy()
      * in njs_builtin_objects_clone().
      */
-    njs_object_t             prototypes[NJS_PROTOTYPE_MAX];
+    njs_object_prototype_t   prototypes[NJS_PROTOTYPE_MAX];
     njs_function_t           constructors[NJS_CONSTRUCTOR_MAX];
 
     nxt_mem_cache_pool_t     *mem_cache_pool;
@@ -829,7 +881,7 @@ struct njs_vm_shared_s {
      * The prototypes and constructors arrays must be togther because they are
      * copied to njs_vm_t by single memcpy() in njs_builtin_objects_clone().
      */
-    njs_object_t             prototypes[NJS_PROTOTYPE_MAX];
+    njs_object_prototype_t   prototypes[NJS_PROTOTYPE_MAX];
     njs_function_t           constructors[NJS_CONSTRUCTOR_MAX];
 };
 
diff -r 19758fdcd554 -r cb414901cccc njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Thu Sep 01 16:12:31 2016 +0300
+++ b/njs/test/njs_unit_test.c	Tue Sep 13 16:19:26 2016 +0300
@@ -4097,6 +4097,12 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("Object.prototype.__proto__ === null"),
       nxt_string("true") },
 
+    { nxt_string("Object.prototype.toString.call(Object.prototype)"),
+      nxt_string("[object Object]") },
+
+    { nxt_string("Object.prototype"),
+      nxt_string("[object Object]") },
+
     { nxt_string("Object.constructor === Function"),
       nxt_string("true") },
 
@@ -4154,6 +4160,12 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("Array.prototype.__proto__ === Object.prototype"),
       nxt_string("true") },
 
+    { nxt_string("Object.prototype.toString.call(Array.prototype)"),
+      nxt_string("[object Array]") },
+
+    { nxt_string("Array.prototype"),
+      nxt_string("") },
+
     { nxt_string("Array.constructor === Function"),
       nxt_string("true") },
 
@@ -4211,6 +4223,12 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("Boolean.prototype.__proto__ === Object.prototype"),
       nxt_string("true") },
 
+    { nxt_string("Object.prototype.toString.call(Boolean.prototype)"),
+      nxt_string("[object Boolean]") },
+
+    { nxt_string("Boolean.prototype"),
+      nxt_string("false") },
+
     { nxt_string("Boolean.constructor === Function"),
       nxt_string("true") },
 
@@ -4264,6 +4282,12 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("Number.prototype.__proto__ === Object.prototype"),
       nxt_string("true") },
 
+    { nxt_string("Object.prototype.toString.call(Number.prototype)"),
+      nxt_string("[object Number]") },
+
+    { nxt_string("Number.prototype"),
+      nxt_string("0") },
+
     { nxt_string("Number.constructor === Function"),
       nxt_string("true") },
 
@@ -4314,6 +4338,12 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("String.__proto__ === Function.prototype"),
       nxt_string("true") },
 
+    { nxt_string("Object.prototype.toString.call(String.prototype)"),
+      nxt_string("[object String]") },
+
+    { nxt_string("String.prototype"),
+      nxt_string("") },
+
     { nxt_string("String.prototype.length"),
       nxt_string("0") },
 
@@ -4359,6 +4389,12 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("Function.prototype.__proto__ === Object.prototype"),
       nxt_string("true") },
 
+    { nxt_string("Object.prototype.toString.call(Function.prototype)"),
+      nxt_string("[object Function]") },
+
+    { nxt_string("Function.prototype"),
+      nxt_string("[object Function]") },
+
     { nxt_string("Function.constructor === Function"),
       nxt_string("true") },
 
@@ -4389,6 +4425,12 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("RegExp.prototype.__proto__ === Object.prototype"),
       nxt_string("true") },
 
+    { nxt_string("Object.prototype.toString.call(RegExp.prototype)"),
+      nxt_string("[object RegExp]") },
+
+    { nxt_string("RegExp.prototype"),
+      nxt_string("/(?:)/") },
+
     { nxt_string("RegExp.constructor === Function"),
       nxt_string("true") },
 
@@ -4801,6 +4843,12 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("Date.prototype.__proto__ === Object.prototype"),
       nxt_string("true") },
 
+    { nxt_string("Date.prototype"),
+      nxt_string("Invalid Date") },
+
+    { nxt_string("Date.prototype.valueOf()"),
+      nxt_string("NaN") },
+
     { nxt_string("Date.constructor === Function"),
       nxt_string("true") },
 


From igor at sysoev.ru  Tue Sep 13 13:53:38 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Tue, 13 Sep 2016 13:53:38 +0000
Subject: [njs] Fixed building by GCC 4.
Message-ID: <hg.79e04e43f569.1473774818.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/79e04e43f569
branches:  
changeset: 170:79e04e43f569
user:      Igor Sysoev <igor at sysoev.ru>
date:      Tue Sep 13 16:53:24 2016 +0300
description:
Fixed building by GCC 4.

diffstat:

 njs/njs_builtin.c |  21 +++++++++++++--------
 1 files changed, 13 insertions(+), 8 deletions(-)

diffs (43 lines):

diff -r cb414901cccc -r 79e04e43f569 njs/njs_builtin.c
--- a/njs/njs_builtin.c	Tue Sep 13 16:19:26 2016 +0300
+++ b/njs/njs_builtin.c	Tue Sep 13 16:53:24 2016 +0300
@@ -55,26 +55,31 @@ njs_builtin_objects_create(njs_vm_t *vm)
     };
 
     static const njs_object_prototype_t  prototype_values[] = {
-        { .object.type = NJS_OBJECT },
-        { .object.type = NJS_ARRAY },
+        /*
+         * GCC 4 complains about unitialized .shared field,
+         * if the .type field is initialized as .object.type.
+         */
+        { .object =       { .type = NJS_OBJECT } },
+        { .object =       { .type = NJS_ARRAY } },
 
         /*
          * The .object.type field must be initialzed after the .value field,
          * otherwise SunC 5.9 treats the .value as .object.value or so.
          */
         { .object_value = { .value = njs_value(NJS_BOOLEAN, 0, 0.0),
-                            .object.type = NJS_OBJECT_BOOLEAN } },
+                            .object = { .type = NJS_OBJECT_BOOLEAN } } },
 
         { .object_value = { .value = njs_value(NJS_NUMBER, 0, 0.0),
-                            .object.type = NJS_OBJECT_NUMBER } },
+                            .object = { .type = NJS_OBJECT_NUMBER } } },
 
         { .object_value = { .value = njs_string(""),
-                            .object.type = NJS_OBJECT_STRING } },
+                            .object = { .type = NJS_OBJECT_STRING } } },
 
-        { .object.type = NJS_FUNCTION },
-        { .object.type = NJS_REGEXP },
+        { .object =       { .type = NJS_FUNCTION } },
+        { .object =       { .type = NJS_REGEXP } },
 
-        { .date =         { .time = NJS_NAN, .object.type = NJS_DATE } },
+        { .date =         { .time = NJS_NAN,
+                            .object = { .type = NJS_DATE } } },
     };
 
     static const njs_object_init_t    *constructor_init[] = {


From igor at sysoev.ru  Tue Sep 13 13:59:38 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Tue, 13 Sep 2016 13:59:38 +0000
Subject: [njs] Version 0.1.2.
Message-ID: <hg.5b066b4db54c.1473775178.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/5b066b4db54c
branches:  
changeset: 171:5b066b4db54c
user:      Igor Sysoev <igor at sysoev.ru>
date:      Tue Sep 13 16:59:09 2016 +0300
description:
Version 0.1.2.

diffstat:

 Makefile |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (10 lines):

diff -r 79e04e43f569 -r 5b066b4db54c Makefile
--- a/Makefile	Tue Sep 13 16:53:24 2016 +0300
+++ b/Makefile	Tue Sep 13 16:59:09 2016 +0300
@@ -1,5 +1,5 @@
 
-NJS_VER =	0.1.1
+NJS_VER =	0.1.2
 
 NXT_LIB =	nxt
 


From igor at sysoev.ru  Tue Sep 13 13:59:39 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Tue, 13 Sep 2016 13:59:39 +0000
Subject: [njs] Added tag 0.1.2 for changeset 5b066b4db54c
Message-ID: <hg.efd2f95aaeb4.1473775179.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/efd2f95aaeb4
branches:  
changeset: 172:efd2f95aaeb4
user:      Igor Sysoev <igor at sysoev.ru>
date:      Tue Sep 13 16:59:27 2016 +0300
description:
Added tag 0.1.2 for changeset 5b066b4db54c

diffstat:

 .hgtags |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (7 lines):

diff -r 5b066b4db54c -r efd2f95aaeb4 .hgtags
--- a/.hgtags	Tue Sep 13 16:59:09 2016 +0300
+++ b/.hgtags	Tue Sep 13 16:59:27 2016 +0300
@@ -1,2 +1,3 @@
 cdb8d20935ee96f1fa0c99d994d4b11cefcbc119 0.1.0
 0039a747d25a3e08792c23c43b75768896724031 0.1.1
+5b066b4db54c17dc0a9a72948474f36957462e87 0.1.2


From mdounin at mdounin.ru  Tue Sep 13 15:41:37 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 13 Sep 2016 15:41:37 +0000
Subject: [nginx] nginx-1.11.4-RELEASE
Message-ID: <hg.953512ca02c6.1473781297.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/953512ca02c6
branches:  
changeset: 6689:953512ca02c6
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Sep 13 18:39:23 2016 +0300
description:
nginx-1.11.4-RELEASE

diffstat:

 docs/xml/nginx/changes.xml |  105 +++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 105 insertions(+), 0 deletions(-)

diffs (115 lines):

diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml
--- a/docs/xml/nginx/changes.xml
+++ b/docs/xml/nginx/changes.xml
@@ -5,6 +5,111 @@
 <change_log title="nginx">
 
 
+<changes ver="1.11.4" date="13.09.2016">
+
+<change type="feature">
+<para lang="ru">
+?????????? $upstream_bytes_received.
+</para>
+<para lang="en">
+the $upstream_bytes_received variable.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+?????????? $bytes_received, $session_time, $protocol, $status,
+$upstream_addr, $upstream_bytes_sent, $upstream_bytes_received,
+$upstream_connect_time, $upstream_first_byte_time
+? $upstream_session_time ? ?????? stream.
+</para>
+<para lang="en">
+the $bytes_received, $session_time, $protocol, $status,
+$upstream_addr, $upstream_bytes_sent, $upstream_bytes_received,
+$upstream_connect_time, $upstream_first_byte_time,
+and $upstream_session_time variables in the stream module.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+?????? ngx_stream_log_module.
+</para>
+<para lang="en">
+the ngx_stream_log_module.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+???????? proxy_protocol ? ????????? listen,
+?????????? $proxy_protocol_addr ? $proxy_protocol_port
+? ?????? stream.
+</para>
+<para lang="en">
+the "proxy_protocol" parameter of the "listen" directive,
+the $proxy_protocol_addr and $proxy_protocol_port variables
+in the stream module.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+?????? ngx_stream_realip_module.
+</para>
+<para lang="en">
+the ngx_stream_realip_module.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+nginx ?? ????????? ? ??????? stream ? ??????? ngx_http_ssl_module,
+?? ??? ?????? ngx_stream_ssl_module;
+?????? ????????? ? 1.11.3.
+</para>
+<para lang="en">
+nginx could not be built with the stream module and the ngx_http_ssl_module,
+but without ngx_stream_ssl_module;
+the bug had appeared in 1.11.3.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+????? ?????? IP_BIND_ADDRESS_NO_PORT ?? ??????????????;
+?????? ????????? ? 1.11.2.
+</para>
+<para lang="en">
+the IP_BIND_ADDRESS_NO_PORT socket option was not used;
+the bug had appeared in 1.11.2.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+? ????????? ranges ????????? geo.
+</para>
+<para lang="en">
+in the "ranges" parameter of the "geo" directive.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ????????????? ???????? "aio threads" ? sendfile
+??? ???????????? ???????????? ?????; ?????? ????????? ? 1.9.13.
+</para>
+<para lang="en">
+an incorrect response might be returned
+when using the "aio threads" and "sendfile" directives;
+the bug had appeared in 1.9.13.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.11.3" date="26.07.2016">
 
 <change type="change">

From mdounin at mdounin.ru  Tue Sep 13 15:41:39 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 13 Sep 2016 15:41:39 +0000
Subject: [nginx] release-1.11.4 tag
Message-ID: <hg.9a4934f07bb4.1473781299.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/9a4934f07bb4
branches:  
changeset: 6690:9a4934f07bb4
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Sep 13 18:39:24 2016 +0300
description:
release-1.11.4 tag

diffstat:

 .hgtags |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (8 lines):

diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -402,3 +402,4 @@ 271ee30c6791847980cd139d31807541f5e569bf
 cb783d9cc19761e14e1285d91c38f4b84d0b8756 release-1.11.1
 4d3b3a13a8cf5fc3351a7f167d1c13325e00f21c release-1.11.2
 b83a067949a3384a49fd3d943eb8d0997b31f87b release-1.11.3
+953512ca02c6f63b4fcbbc3e10d0d9835896bf99 release-1.11.4


From novy at ondrej.org  Tue Sep 13 15:56:32 2016
From: novy at ondrej.org (Ondrej Novy)
Date: Tue, 13 Sep 2016 17:56:32 +0200
Subject: --with-openssl and OPENSSL_OPT
In-Reply-To: <20160913014054.GE1527@mdounin.ru>
References: <CAOO6c=xcMUCMQNULw0F4ADNPQ+1_nnCXxRp6BaSP0VU+W8n8AQ@mail.gmail.com>
 <20160913014054.GE1527@mdounin.ru>
Message-ID: <CAOO6c=yjHxmzOi9Pf_11uPyGt0Sx6-64cjNkT7GDiw=FjrAmqw@mail.gmail.com>

Hi,

2016-09-13 3:40 GMT+02:00 Maxim Dounin <mdounin at mdounin.ru>:

> If you need to configure openssl configure options which require
> "make depend", you can run "make depend" yourself before making
> nginx (and thus OpenSSL).
>

so I should run config+make depend+make myself? That doesn't work, nginx
calls config nevertheless and thus overwritting my config.

Alternatively, you can make OpenSSL yourself instead of asking
> nginx to do it for you.  Note that using --with-openssl option of
> nginx configure is not something required for static building with
> OpenSSL.  Rather, it's a convenient shortcut to make things easier
> in common cases.
>

I have built OpenSSL libs, so i have .s + .h files :). How to point nginx
to this files to static link them please?

Thank you.

-- 
Best regards
 Ond?ej Nov?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160913/248f2a64/attachment.html>

From mdounin at mdounin.ru  Tue Sep 13 16:55:58 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 13 Sep 2016 19:55:58 +0300
Subject: --with-openssl and OPENSSL_OPT
In-Reply-To: <CAOO6c=yjHxmzOi9Pf_11uPyGt0Sx6-64cjNkT7GDiw=FjrAmqw@mail.gmail.com>
References: <CAOO6c=xcMUCMQNULw0F4ADNPQ+1_nnCXxRp6BaSP0VU+W8n8AQ@mail.gmail.com>
 <20160913014054.GE1527@mdounin.ru>
 <CAOO6c=yjHxmzOi9Pf_11uPyGt0Sx6-64cjNkT7GDiw=FjrAmqw@mail.gmail.com>
Message-ID: <20160913165558.GS1527@mdounin.ru>

Hello!

On Tue, Sep 13, 2016 at 05:56:32PM +0200, Ondrej Novy wrote:

> 2016-09-13 3:40 GMT+02:00 Maxim Dounin <mdounin at mdounin.ru>:
> 
> > If you need to configure openssl configure options which require
> > "make depend", you can run "make depend" yourself before making
> > nginx (and thus OpenSSL).
> >
> 
> so I should run config+make depend+make myself? That doesn't work, nginx
> calls config nevertheless and thus overwritting my config.

Try something like this:

./configure --with-http_ssl_module --with-openssl=/path/to/openssl-1.0.2h --with-openssl-opt=no-idea
make
(cd /path/to/openssl-1.0.2h; make depend)
make

On the first "make", nginx will run OpenSSL config and then try to 
build it.  The build will fail due to no "make depend" (actually, 
you can stop it with Ctrl-C right after OpenSSL's configure).  
On the second "make", the build will succeed.

> Alternatively, you can make OpenSSL yourself instead of asking
> > nginx to do it for you.  Note that using --with-openssl option of
> > nginx configure is not something required for static building with
> > OpenSSL.  Rather, it's a convenient shortcut to make things easier
> > in common cases.
> >
> 
> I have built OpenSSL libs, so i have .s + .h files :). How to point nginx
> to this files to static link them please?

You need .h (include) and .a (static library) files.  Then use 
compiler and linker options to specify paths to include and 
library files, -I and -L respectively.  Use "--with-cc-opt" and 
"--with-ld-opt" to pass the options through nginx configure:

./configure --with-cc-opt="-I /path/to/openssl/include" --with-ld-opt="-L /path/to/openssl/lib"

Note that there should be no dynamic libraries (*.so) for OpenSSL 
available in the lib path specified, or they will be used instead 
of static libraries.  To prevent dynamic libraries from appearing 
you can configure OpenSSL with the "no-shared" flag, much like 
nginx does itself.

Full procedure to build both OpenSSL and nginx with it will look 
like this:

cd /path/to/openssl-1.0.2h
./config --prefix=`pwd`/.openssl no-shared no-idea
make depend
make
make install_sw
cd /path/to/nginx
./configure --with-http_ssl_module \
    --with-cc-opt="-I /path/to/openssl-1.0.2h/.openssl/include" \
    --with-ld-opt="-L /path/to/openssl-1.0.2h/.openssl/lib"
make

-- 
Maxim Dounin
http://nginx.org/


From ondrej.novy at gmail.com  Tue Sep 13 19:13:24 2016
From: ondrej.novy at gmail.com (=?UTF-8?B?T25kxZllaiBOb3bDvQ==?=)
Date: Tue, 13 Sep 2016 21:13:24 +0200
Subject: --with-openssl and OPENSSL_OPT
Message-ID: <CAOO6c=zvC9e-0R+NGwau7Oi7b-QnHdtsCfvecovVg=0r51wu3Q@mail.gmail.com>

Hi,

2016-09-13 18:55 GMT+02:00 Maxim Dounin <mdounin at mdounin.ru>:
>
> Try something like this:
>
> ./configure --with-http_ssl_module --with-openssl=/path/to/openssl-1.0.2h
> --with-openssl-opt=no-idea
> make
> (cd /path/to/openssl-1.0.2h; make depend)
> make
>

this is crazy :). Why you don't add make depend to nginx build phase of
OpenSSL? It should not harm anyone and build will succeed for first time.


> > I have built OpenSSL libs, so i have .s + .h files :). How to point nginx
> > to this files to static link them please?
>
> You need .h (include) and .a (static library) files.  Then use
>

sry, no .s but .a files.


> ./configure --with-cc-opt="-I /path/to/openssl/include" --with-ld-opt="-L
> /path/to/openssl/lib"
>

that looks cool, I will try it.

Thanks.

-- 
Best regards
 Ond?ej Nov?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160913/9cb74657/attachment.html>

From mdounin at mdounin.ru  Tue Sep 13 19:37:13 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 13 Sep 2016 22:37:13 +0300
Subject: --with-openssl and OPENSSL_OPT
In-Reply-To: <CAOO6c=zvC9e-0R+NGwau7Oi7b-QnHdtsCfvecovVg=0r51wu3Q@mail.gmail.com>
References: <CAOO6c=zvC9e-0R+NGwau7Oi7b-QnHdtsCfvecovVg=0r51wu3Q@mail.gmail.com>
Message-ID: <20160913193713.GT1527@mdounin.ru>

Hello!

On Tue, Sep 13, 2016 at 09:13:24PM +0200, Ond?ej Nov? wrote:

> Hi,
> 
> 2016-09-13 18:55 GMT+02:00 Maxim Dounin <mdounin at mdounin.ru>:
> >
> > Try something like this:
> >
> > ./configure --with-http_ssl_module --with-openssl=/path/to/openssl-1.0.2h
> > --with-openssl-opt=no-idea
> > make
> > (cd /path/to/openssl-1.0.2h; make depend)
> > make
> >
> 
> this is crazy :). Why you don't add make depend to nginx build phase of
> OpenSSL? It should not harm anyone and build will succeed for first time.

There is no way to find out if "make depend" is needed or not.  
On the other hand, it has noticeable cost, and not needed in most 
cases.

-- 
Maxim Dounin
http://nginx.org/


From ru at nginx.com  Thu Sep 15 07:52:15 2016
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 15 Sep 2016 07:52:15 +0000
Subject: [njs] Style: express requirement to use PCRE via $ngx_module_libs.
Message-ID: <hg.1745fcf4a5cc.1473925935.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/1745fcf4a5cc
branches:  
changeset: 173:1745fcf4a5cc
user:      Ruslan Ermilov <ru at nginx.com>
date:      Thu Sep 15 10:52:01 2016 +0300
description:
Style: express requirement to use PCRE via $ngx_module_libs.

diffstat:

 nginx/config |  8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)

diffs (32 lines):

diff -r efd2f95aaeb4 -r 1745fcf4a5cc nginx/config
--- a/nginx/config	Tue Sep 13 16:59:27 2016 +0300
+++ b/nginx/config	Thu Sep 15 10:52:01 2016 +0300
@@ -1,13 +1,11 @@
 ngx_addon_name="ngx_js_module"
 
-USE_PCRE=YES
-
 if test -n "$ngx_module_link"; then
     ngx_module_type=HTTP
     ngx_module_name=ngx_http_js_module
     ngx_module_incs="$ngx_addon_dir/../nxt $ngx_addon_dir/../njs"
     ngx_module_srcs="$ngx_addon_dir/ngx_http_js_module.c"
-    ngx_module_libs="$ngx_addon_dir/../build/libnjs.a -lm"
+    ngx_module_libs="PCRE $ngx_addon_dir/../build/libnjs.a -lm"
 
     . auto/module
 
@@ -15,10 +13,12 @@ if test -n "$ngx_module_link"; then
     ngx_module_name=ngx_stream_js_module
     ngx_module_incs="$ngx_addon_dir/../nxt $ngx_addon_dir/../njs"
     ngx_module_srcs="$ngx_addon_dir/ngx_stream_js_module.c"
-    ngx_module_libs="$ngx_addon_dir/../build/libnjs.a -lm"
+    ngx_module_libs="PCRE $ngx_addon_dir/../build/libnjs.a -lm"
 
     . auto/module
 else
+    USE_PCRE=YES
+
     HTTP_MODULES="$HTTP_MODULES ngx_http_js_module"
     STREAM_MODULES="$STREAM_MODULES ngx_stream_js_module"
     CORE_INCS="$CORE_INCS $ngx_addon_dir/../nxt $ngx_addon_dir/../njs"


From vl at nginx.com  Thu Sep 15 11:57:43 2016
From: vl at nginx.com (Vladimir Homutov)
Date: Thu, 15 Sep 2016 11:57:43 +0000
Subject: [nginx] Version bump.
Message-ID: <hg.4bce3edfac2c.1473940663.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/4bce3edfac2c
branches:  
changeset: 6691:4bce3edfac2c
user:      Vladimir Homutov <vl at nginx.com>
date:      Thu Sep 15 14:56:26 2016 +0300
description:
Version bump.

diffstat:

 src/core/nginx.h |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff -r 9a4934f07bb4 -r 4bce3edfac2c src/core/nginx.h
--- a/src/core/nginx.h	Tue Sep 13 18:39:24 2016 +0300
+++ b/src/core/nginx.h	Thu Sep 15 14:56:26 2016 +0300
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1011004
-#define NGINX_VERSION      "1.11.4"
+#define nginx_version      1011005
+#define NGINX_VERSION      "1.11.5"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #ifdef NGX_BUILD


From vl at nginx.com  Thu Sep 15 11:57:46 2016
From: vl at nginx.com (Vladimir Homutov)
Date: Thu, 15 Sep 2016 11:57:46 +0000
Subject: [nginx] Stream: filters.
Message-ID: <hg.56fc55e32f23.1473940666.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/56fc55e32f23
branches:  
changeset: 6692:56fc55e32f23
user:      Roman Arutyunyan <arut at nginx.com>
date:      Thu Sep 15 14:55:46 2016 +0300
description:
Stream: filters.

diffstat:

 auto/modules                                |    6 +-
 auto/sources                                |    1 +
 src/event/modules/ngx_iocp_module.c         |    2 +
 src/event/ngx_event.h                       |    1 +
 src/event/ngx_event_accept.c                |    1 +
 src/event/ngx_event_connect.c               |    1 +
 src/os/unix/ngx_darwin_init.c               |    1 +
 src/os/unix/ngx_freebsd_init.c              |    1 +
 src/os/unix/ngx_linux_init.c                |    1 +
 src/os/unix/ngx_os.h                        |    3 +
 src/os/unix/ngx_posix_init.c                |    1 +
 src/os/unix/ngx_solaris_init.c              |    1 +
 src/os/unix/ngx_udp_sendmsg_chain.c         |  245 +++++++++++++++++++++++++
 src/os/win32/ngx_os.h                       |    2 +
 src/os/win32/ngx_win32_init.c               |    2 +
 src/stream/ngx_stream.c                     |    3 +
 src/stream/ngx_stream.h                     |   10 +
 src/stream/ngx_stream_handler.c             |    4 +
 src/stream/ngx_stream_proxy_module.c        |  228 ++++++++++++++--------
 src/stream/ngx_stream_return_module.c       |   55 +++--
 src/stream/ngx_stream_upstream.h            |   10 +
 src/stream/ngx_stream_write_filter_module.c |  273 ++++++++++++++++++++++++++++
 22 files changed, 744 insertions(+), 108 deletions(-)

diffs (truncated from 1269 to 1000 lines):

diff -r 4bce3edfac2c -r 56fc55e32f23 auto/modules
--- a/auto/modules	Thu Sep 15 14:56:26 2016 +0300
+++ b/auto/modules	Thu Sep 15 14:55:46 2016 +0300
@@ -973,7 +973,8 @@ if [ $STREAM != NO ]; then
                      ngx_stream_core_module \
                      ngx_stream_log_module \
                      ngx_stream_proxy_module \
-                     ngx_stream_upstream_module"
+                     ngx_stream_upstream_module \
+                     ngx_stream_write_filter_module"
     ngx_module_incs="src/stream"
     ngx_module_deps="src/stream/ngx_stream.h \
                      src/stream/ngx_stream_variables.h \
@@ -988,7 +989,8 @@ if [ $STREAM != NO ]; then
                      src/stream/ngx_stream_log_module.c \
                      src/stream/ngx_stream_proxy_module.c \
                      src/stream/ngx_stream_upstream.c \
-                     src/stream/ngx_stream_upstream_round_robin.c"
+                     src/stream/ngx_stream_upstream_round_robin.c \
+                     src/stream/ngx_stream_write_filter_module.c"
 
     . auto/module
 
diff -r 4bce3edfac2c -r 56fc55e32f23 auto/sources
--- a/auto/sources	Thu Sep 15 14:56:26 2016 +0300
+++ b/auto/sources	Thu Sep 15 14:55:46 2016 +0300
@@ -167,6 +167,7 @@ UNIX_SRCS="$CORE_SRCS $EVENT_SRCS \
             src/os/unix/ngx_send.c \
             src/os/unix/ngx_writev_chain.c \
             src/os/unix/ngx_udp_send.c \
+            src/os/unix/ngx_udp_sendmsg_chain.c \
             src/os/unix/ngx_channel.c \
             src/os/unix/ngx_shmem.c \
             src/os/unix/ngx_process.c \
diff -r 4bce3edfac2c -r 56fc55e32f23 src/event/modules/ngx_iocp_module.c
--- a/src/event/modules/ngx_iocp_module.c	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/event/modules/ngx_iocp_module.c	Thu Sep 15 14:55:46 2016 +0300
@@ -93,6 +93,8 @@ ngx_os_io_t ngx_iocp_io = {
     NULL,
     ngx_udp_overlapped_wsarecv,
     NULL,
+    NULL,
+    NULL,
     ngx_overlapped_wsasend_chain,
     0
 };
diff -r 4bce3edfac2c -r 56fc55e32f23 src/event/ngx_event.h
--- a/src/event/ngx_event.h	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/event/ngx_event.h	Thu Sep 15 14:55:46 2016 +0300
@@ -430,6 +430,7 @@ extern ngx_os_io_t  ngx_io;
 #define ngx_send             ngx_io.send
 #define ngx_send_chain       ngx_io.send_chain
 #define ngx_udp_send         ngx_io.udp_send
+#define ngx_udp_send_chain   ngx_io.udp_send_chain
 
 
 #define NGX_EVENT_MODULE      0x544E5645  /* "EVNT" */
diff -r 4bce3edfac2c -r 56fc55e32f23 src/event/ngx_event_accept.c
--- a/src/event/ngx_event_accept.c	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/event/ngx_event_accept.c	Thu Sep 15 14:55:46 2016 +0300
@@ -467,6 +467,7 @@ ngx_event_recvmsg(ngx_event_t *ev)
         *log = ls->log;
 
         c->send = ngx_udp_send;
+        c->send_chain = ngx_udp_send_chain;
 
         c->log = log;
         c->pool->log = log;
diff -r 4bce3edfac2c -r 56fc55e32f23 src/event/ngx_event_connect.c
--- a/src/event/ngx_event_connect.c	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/event/ngx_event_connect.c	Thu Sep 15 14:55:46 2016 +0300
@@ -166,6 +166,7 @@ ngx_event_connect_peer(ngx_peer_connecti
     } else { /* type == SOCK_DGRAM */
         c->recv = ngx_udp_recv;
         c->send = ngx_send;
+        c->send_chain = ngx_udp_send_chain;
     }
 
     c->log_error = pc->log_error;
diff -r 4bce3edfac2c -r 56fc55e32f23 src/os/unix/ngx_darwin_init.c
--- a/src/os/unix/ngx_darwin_init.c	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/os/unix/ngx_darwin_init.c	Thu Sep 15 14:55:46 2016 +0300
@@ -24,6 +24,7 @@ static ngx_os_io_t ngx_darwin_io = {
     ngx_udp_unix_recv,
     ngx_unix_send,
     ngx_udp_unix_send,
+    ngx_udp_unix_sendmsg_chain,
 #if (NGX_HAVE_SENDFILE)
     ngx_darwin_sendfile_chain,
     NGX_IO_SENDFILE
diff -r 4bce3edfac2c -r 56fc55e32f23 src/os/unix/ngx_freebsd_init.c
--- a/src/os/unix/ngx_freebsd_init.c	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/os/unix/ngx_freebsd_init.c	Thu Sep 15 14:55:46 2016 +0300
@@ -33,6 +33,7 @@ static ngx_os_io_t ngx_freebsd_io = {
     ngx_udp_unix_recv,
     ngx_unix_send,
     ngx_udp_unix_send,
+    ngx_udp_unix_sendmsg_chain,
 #if (NGX_HAVE_SENDFILE)
     ngx_freebsd_sendfile_chain,
     NGX_IO_SENDFILE
diff -r 4bce3edfac2c -r 56fc55e32f23 src/os/unix/ngx_linux_init.c
--- a/src/os/unix/ngx_linux_init.c	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/os/unix/ngx_linux_init.c	Thu Sep 15 14:55:46 2016 +0300
@@ -19,6 +19,7 @@ static ngx_os_io_t ngx_linux_io = {
     ngx_udp_unix_recv,
     ngx_unix_send,
     ngx_udp_unix_send,
+    ngx_udp_unix_sendmsg_chain,
 #if (NGX_HAVE_SENDFILE)
     ngx_linux_sendfile_chain,
     NGX_IO_SENDFILE
diff -r 4bce3edfac2c -r 56fc55e32f23 src/os/unix/ngx_os.h
--- a/src/os/unix/ngx_os.h	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/os/unix/ngx_os.h	Thu Sep 15 14:55:46 2016 +0300
@@ -29,6 +29,7 @@ typedef struct {
     ngx_recv_pt        udp_recv;
     ngx_send_pt        send;
     ngx_send_pt        udp_send;
+    ngx_send_chain_pt  udp_send_chain;
     ngx_send_chain_pt  send_chain;
     ngx_uint_t         flags;
 } ngx_os_io_t;
@@ -49,6 +50,8 @@ ssize_t ngx_unix_send(ngx_connection_t *
 ngx_chain_t *ngx_writev_chain(ngx_connection_t *c, ngx_chain_t *in,
     off_t limit);
 ssize_t ngx_udp_unix_send(ngx_connection_t *c, u_char *buf, size_t size);
+ngx_chain_t *ngx_udp_unix_sendmsg_chain(ngx_connection_t *c, ngx_chain_t *in,
+    off_t limit);
 
 
 #if (IOV_MAX > 64)
diff -r 4bce3edfac2c -r 56fc55e32f23 src/os/unix/ngx_posix_init.c
--- a/src/os/unix/ngx_posix_init.c	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/os/unix/ngx_posix_init.c	Thu Sep 15 14:55:46 2016 +0300
@@ -25,6 +25,7 @@ ngx_os_io_t ngx_os_io = {
     ngx_udp_unix_recv,
     ngx_unix_send,
     ngx_udp_unix_send,
+    ngx_udp_unix_sendmsg_chain,
     ngx_writev_chain,
     0
 };
diff -r 4bce3edfac2c -r 56fc55e32f23 src/os/unix/ngx_solaris_init.c
--- a/src/os/unix/ngx_solaris_init.c	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/os/unix/ngx_solaris_init.c	Thu Sep 15 14:55:46 2016 +0300
@@ -20,6 +20,7 @@ static ngx_os_io_t ngx_solaris_io = {
     ngx_udp_unix_recv,
     ngx_unix_send,
     ngx_udp_unix_send,
+    ngx_udp_unix_sendmsg_chain,
 #if (NGX_HAVE_SENDFILE)
     ngx_solaris_sendfilev_chain,
     NGX_IO_SENDFILE
diff -r 4bce3edfac2c -r 56fc55e32f23 src/os/unix/ngx_udp_sendmsg_chain.c
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/src/os/unix/ngx_udp_sendmsg_chain.c	Thu Sep 15 14:55:46 2016 +0300
@@ -0,0 +1,245 @@
+
+/*
+ * Copyright (C) Igor Sysoev
+ * Copyright (C) Nginx, Inc.
+ */
+
+
+#include <ngx_config.h>
+#include <ngx_core.h>
+#include <ngx_event.h>
+
+
+static ngx_chain_t *ngx_udp_output_chain_to_iovec(ngx_iovec_t *vec,
+    ngx_chain_t *in, ngx_log_t *log);
+static ssize_t ngx_sendmsg(ngx_connection_t *c, ngx_iovec_t *vec);
+
+
+ngx_chain_t *
+ngx_udp_unix_sendmsg_chain(ngx_connection_t *c, ngx_chain_t *in, off_t limit)
+{
+    ssize_t        n;
+    off_t          send;
+    ngx_chain_t   *cl;
+    ngx_event_t   *wev;
+    ngx_iovec_t    vec;
+    struct iovec   iovs[NGX_IOVS_PREALLOCATE];
+
+    wev = c->write;
+
+    if (!wev->ready) {
+        return in;
+    }
+
+#if (NGX_HAVE_KQUEUE)
+
+    if ((ngx_event_flags & NGX_USE_KQUEUE_EVENT) && wev->pending_eof) {
+        (void) ngx_connection_error(c, wev->kq_errno,
+                               "kevent() reported about an closed connection");
+        wev->error = 1;
+        return NGX_CHAIN_ERROR;
+    }
+
+#endif
+
+    /* the maximum limit size is the maximum size_t value - the page size */
+
+    if (limit == 0 || limit > (off_t) (NGX_MAX_SIZE_T_VALUE - ngx_pagesize)) {
+        limit = NGX_MAX_SIZE_T_VALUE - ngx_pagesize;
+    }
+
+    send = 0;
+
+    vec.iovs = iovs;
+    vec.nalloc = NGX_IOVS_PREALLOCATE;
+
+    for ( ;; ) {
+
+        /* create the iovec and coalesce the neighbouring bufs */
+
+        cl = ngx_udp_output_chain_to_iovec(&vec, in, c->log);
+
+        if (cl == NGX_CHAIN_ERROR) {
+            return NGX_CHAIN_ERROR;
+        }
+
+        if (cl && cl->buf->in_file) {
+            ngx_log_error(NGX_LOG_ALERT, c->log, 0,
+                          "file buf in sendmsg "
+                          "t:%d r:%d f:%d %p %p-%p %p %O-%O",
+                          cl->buf->temporary,
+                          cl->buf->recycled,
+                          cl->buf->in_file,
+                          cl->buf->start,
+                          cl->buf->pos,
+                          cl->buf->last,
+                          cl->buf->file,
+                          cl->buf->file_pos,
+                          cl->buf->file_last);
+
+            ngx_debug_point();
+
+            return NGX_CHAIN_ERROR;
+        }
+
+        if (cl == in) {
+            return in;
+        }
+
+        send += vec.size;
+
+        n = ngx_sendmsg(c, &vec);
+
+        if (n == NGX_ERROR) {
+            return NGX_CHAIN_ERROR;
+        }
+
+        if (n == NGX_AGAIN) {
+            wev->ready = 0;
+            return in;
+        }
+
+        c->sent += n;
+
+        in = ngx_chain_update_sent(in, n);
+
+        if (send >= limit || in == NULL) {
+            return in;
+        }
+    }
+}
+
+
+static ngx_chain_t *
+ngx_udp_output_chain_to_iovec(ngx_iovec_t *vec, ngx_chain_t *in, ngx_log_t *log)
+{
+    size_t         total, size;
+    u_char        *prev;
+    ngx_uint_t     n, flush;
+    ngx_chain_t   *cl;
+    struct iovec  *iov;
+
+    cl = in;
+    iov = NULL;
+    prev = NULL;
+    total = 0;
+    n = 0;
+    flush = 0;
+
+    for ( /* void */ ; in && !flush; in = in->next) {
+
+        if (in->buf->flush || in->buf->last_buf) {
+            flush = 1;
+        }
+
+        if (ngx_buf_special(in->buf)) {
+            continue;
+        }
+
+        if (in->buf->in_file) {
+            break;
+        }
+
+        if (!ngx_buf_in_memory(in->buf)) {
+            ngx_log_error(NGX_LOG_ALERT, log, 0,
+                          "bad buf in output chain "
+                          "t:%d r:%d f:%d %p %p-%p %p %O-%O",
+                          in->buf->temporary,
+                          in->buf->recycled,
+                          in->buf->in_file,
+                          in->buf->start,
+                          in->buf->pos,
+                          in->buf->last,
+                          in->buf->file,
+                          in->buf->file_pos,
+                          in->buf->file_last);
+
+            ngx_debug_point();
+
+            return NGX_CHAIN_ERROR;
+        }
+
+        size = in->buf->last - in->buf->pos;
+
+        if (prev == in->buf->pos) {
+            iov->iov_len += size;
+
+        } else {
+            if (n == vec->nalloc) {
+                ngx_log_error(NGX_LOG_ALERT, log, 0,
+                              "too many parts in a datagram");
+                return NGX_CHAIN_ERROR;
+            }
+
+            iov = &vec->iovs[n++];
+
+            iov->iov_base = (void *) in->buf->pos;
+            iov->iov_len = size;
+        }
+
+        prev = in->buf->pos + size;
+        total += size;
+    }
+
+    if (!flush) {
+#if (NGX_SUPPRESS_WARN)
+        vec->size = 0;
+        vec->count = 0;
+#endif
+        return cl;
+    }
+
+    vec->count = n;
+    vec->size = total;
+
+    return in;
+}
+
+
+static ssize_t
+ngx_sendmsg(ngx_connection_t *c, ngx_iovec_t *vec)
+{
+    ssize_t        n;
+    ngx_err_t      err;
+    struct msghdr  msg;
+
+    ngx_memzero(&msg, sizeof(struct msghdr));
+
+    if (c->socklen) {
+        msg.msg_name = c->sockaddr;
+        msg.msg_namelen = c->socklen;
+    }
+
+    msg.msg_iov = vec->iovs;
+    msg.msg_iovlen = vec->count;
+
+eintr:
+
+    n = sendmsg(c->fd, &msg, 0);
+
+    ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0,
+                   "sendmsg: %z of %uz", n, vec->size);
+
+    if (n == -1) {
+        err = ngx_errno;
+
+        switch (err) {
+        case NGX_EAGAIN:
+            ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, err,
+                           "sendmsg() not ready");
+            return NGX_AGAIN;
+
+        case NGX_EINTR:
+            ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, err,
+                           "sendmsg() was interrupted");
+            goto eintr;
+
+        default:
+            c->write->error = 1;
+            ngx_connection_error(c, err, "sendmsg() failed");
+            return NGX_ERROR;
+        }
+    }
+
+    return n;
+}
diff -r 4bce3edfac2c -r 56fc55e32f23 src/os/win32/ngx_os.h
--- a/src/os/win32/ngx_os.h	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/os/win32/ngx_os.h	Thu Sep 15 14:55:46 2016 +0300
@@ -28,6 +28,8 @@ typedef struct {
     ngx_recv_chain_pt  recv_chain;
     ngx_recv_pt        udp_recv;
     ngx_send_pt        send;
+    ngx_send_pt        udp_send;
+    ngx_send_chain_pt  udp_send_chain;
     ngx_send_chain_pt  send_chain;
     ngx_uint_t         flags;
 } ngx_os_io_t;
diff -r 4bce3edfac2c -r 56fc55e32f23 src/os/win32/ngx_win32_init.c
--- a/src/os/win32/ngx_win32_init.c	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/os/win32/ngx_win32_init.c	Thu Sep 15 14:55:46 2016 +0300
@@ -25,6 +25,8 @@ ngx_os_io_t ngx_os_io = {
     ngx_wsarecv_chain,
     ngx_udp_wsarecv,
     ngx_wsasend,
+    NULL,
+    NULL,
     ngx_wsasend_chain,
     0
 };
diff -r 4bce3edfac2c -r 56fc55e32f23 src/stream/ngx_stream.c
--- a/src/stream/ngx_stream.c	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/stream/ngx_stream.c	Thu Sep 15 14:55:46 2016 +0300
@@ -27,6 +27,9 @@ static ngx_int_t ngx_stream_cmp_conf_add
 ngx_uint_t  ngx_stream_max_module;
 
 
+ngx_stream_filter_pt  ngx_stream_top_filter;
+
+
 static ngx_command_t  ngx_stream_commands[] = {
 
     { ngx_string("stream"),
diff -r 4bce3edfac2c -r 56fc55e32f23 src/stream/ngx_stream.h
--- a/src/stream/ngx_stream.h	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/stream/ngx_stream.h	Thu Sep 15 14:55:46 2016 +0300
@@ -243,6 +243,9 @@ typedef struct {
         NULL)
 
 
+#define NGX_STREAM_WRITE_BUFFERED  0x10
+
+
 void ngx_stream_init_connection(ngx_connection_t *c);
 void ngx_stream_finalize_session(ngx_stream_session_t *s, ngx_uint_t rc);
 
@@ -252,4 +255,11 @@ extern ngx_uint_t    ngx_stream_max_modu
 extern ngx_module_t  ngx_stream_core_module;
 
 
+typedef ngx_int_t (*ngx_stream_filter_pt)(ngx_stream_session_t *s,
+    ngx_chain_t *chain, ngx_uint_t from_upstream);
+
+
+extern ngx_stream_filter_pt  ngx_stream_top_filter;
+
+
 #endif /* _NGX_STREAM_H_INCLUDED_ */
diff -r 4bce3edfac2c -r 56fc55e32f23 src/stream/ngx_stream_handler.c
--- a/src/stream/ngx_stream_handler.c	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/stream/ngx_stream_handler.c	Thu Sep 15 14:55:46 2016 +0300
@@ -134,6 +134,10 @@ ngx_stream_init_connection(ngx_connectio
     s->ssl = addr_conf->ssl;
 #endif
 
+    if (c->buffer) {
+        s->received += c->buffer->last - c->buffer->pos;
+    }
+
     s->connection = c;
     c->data = s;
 
diff -r 4bce3edfac2c -r 56fc55e32f23 src/stream/ngx_stream_proxy_module.c
--- a/src/stream/ngx_stream_proxy_module.c	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/stream/ngx_stream_proxy_module.c	Thu Sep 15 14:55:46 2016 +0300
@@ -84,10 +84,10 @@ static char *ngx_stream_proxy_pass(ngx_c
     void *conf);
 static char *ngx_stream_proxy_bind(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf);
+
+#if (NGX_STREAM_SSL)
+
 static ngx_int_t ngx_stream_proxy_send_proxy_protocol(ngx_stream_session_t *s);
-
-#if (NGX_STREAM_SSL)
-
 static char *ngx_stream_proxy_ssl_password_file(ngx_conf_t *cf,
     ngx_command_t *cmd, void *conf);
 static void ngx_stream_proxy_ssl_init_connection(ngx_stream_session_t *s);
@@ -385,8 +385,6 @@ ngx_stream_proxy_handler(ngx_stream_sess
     }
 
     u->peer.type = c->type;
-
-    u->proxy_protocol = pscf->proxy_protocol;
     u->start_sec = ngx_time();
 
     c->write->handler = ngx_stream_proxy_downstream_handler;
@@ -411,28 +409,6 @@ ngx_stream_proxy_handler(ngx_stream_sess
         u->downstream_buf.pos = p;
         u->downstream_buf.last = p;
 
-        if (u->proxy_protocol
-#if (NGX_STREAM_SSL)
-            && pscf->ssl == NULL
-#endif
-            && pscf->buffer_size >= NGX_PROXY_PROTOCOL_MAX_HEADER)
-        {
-            /* optimization for a typical case */
-
-            ngx_log_debug0(NGX_LOG_DEBUG_STREAM, c->log, 0,
-                           "stream proxy send PROXY protocol header");
-
-            p = ngx_proxy_protocol_write(c, u->downstream_buf.last,
-                                         u->downstream_buf.end);
-            if (p == NULL) {
-                ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
-                return;
-            }
-
-            u->downstream_buf.last = p;
-            u->proxy_protocol = 0;
-        }
-
         if (c->read->ready) {
             ngx_post_event(c->read, &ngx_posted_events);
         }
@@ -682,8 +658,13 @@ ngx_stream_proxy_connect(ngx_stream_sess
 
     c->log->action = "connecting to upstream";
 
+    pscf = ngx_stream_get_module_srv_conf(s, ngx_stream_proxy_module);
+
     u = s->upstream;
 
+    u->connected = 0;
+    u->proxy_protocol = pscf->proxy_protocol;
+
     if (u->state) {
         u->state->response_time = ngx_current_msec - u->state->response_time;
     }
@@ -740,8 +721,6 @@ ngx_stream_proxy_connect(ngx_stream_sess
     pc->read->handler = ngx_stream_proxy_connect_handler;
     pc->write->handler = ngx_stream_proxy_connect_handler;
 
-    pscf = ngx_stream_get_module_srv_conf(s, ngx_stream_proxy_module);
-
     ngx_add_timer(pc->write, pscf->connect_timeout);
 }
 
@@ -751,6 +730,7 @@ ngx_stream_proxy_init_upstream(ngx_strea
 {
     int                           tcp_nodelay;
     u_char                       *p;
+    ngx_chain_t                  *cl;
     ngx_connection_t             *c, *pc;
     ngx_log_handler_pt            handler;
     ngx_stream_upstream_t        *u;
@@ -782,21 +762,26 @@ ngx_stream_proxy_init_upstream(ngx_strea
         pc->tcp_nodelay = NGX_TCP_NODELAY_SET;
     }
 
-    if (u->proxy_protocol) {
-        if (ngx_stream_proxy_send_proxy_protocol(s) != NGX_OK) {
+    pscf = ngx_stream_get_module_srv_conf(s, ngx_stream_proxy_module);
+
+#if (NGX_STREAM_SSL)
+
+    if (pc->type == SOCK_STREAM && pscf->ssl) {
+
+        if (u->proxy_protocol) {
+            if (ngx_stream_proxy_send_proxy_protocol(s) != NGX_OK) {
+                return;
+            }
+
+            u->proxy_protocol = 0;
+        }
+
+        if (pc->ssl == NULL) {
+            ngx_stream_proxy_ssl_init_connection(s);
             return;
         }
-
-        u->proxy_protocol = 0;
     }
 
-    pscf = ngx_stream_get_module_srv_conf(s, ngx_stream_proxy_module);
-
-#if (NGX_STREAM_SSL)
-    if (pc->type == SOCK_STREAM && pscf->ssl && pc->ssl == NULL) {
-        ngx_stream_proxy_ssl_init_connection(s);
-        return;
-    }
 #endif
 
     c = s->connection;
@@ -838,14 +823,66 @@ ngx_stream_proxy_init_upstream(ngx_strea
         u->upstream_buf.last = p;
     }
 
-    if (c->type == SOCK_DGRAM) {
-        s->received = c->buffer->last - c->buffer->pos;
-        u->downstream_buf = *c->buffer;
-
-        if (pscf->responses == 0) {
-            pc->read->ready = 0;
-            pc->read->eof = 1;
+    if (c->buffer && c->buffer->pos < c->buffer->last) {
+        ngx_log_debug1(NGX_LOG_DEBUG_STREAM, c->log, 0,
+                       "stream proxy add preread buffer: %uz",
+                       c->buffer->last - c->buffer->pos);
+
+        cl = ngx_chain_get_free_buf(c->pool, &u->free);
+        if (cl == NULL) {
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+            return;
         }
+
+        *cl->buf = *c->buffer;
+
+        cl->buf->tag = (ngx_buf_tag_t) &ngx_stream_proxy_module;
+        cl->buf->flush = 1;
+        cl->buf->last_buf = (c->type == SOCK_DGRAM);
+
+        cl->next = u->upstream_out;
+        u->upstream_out = cl;
+    }
+
+    if (u->proxy_protocol) {
+        ngx_log_debug0(NGX_LOG_DEBUG_STREAM, c->log, 0,
+                       "stream proxy add PROXY protocol header");
+
+        cl = ngx_chain_get_free_buf(c->pool, &u->free);
+        if (cl == NULL) {
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+            return;
+        }
+
+        p = ngx_pnalloc(c->pool, NGX_PROXY_PROTOCOL_MAX_HEADER);
+        if (p == NULL) {
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+            return;
+        }
+
+        cl->buf->pos = p;
+
+        p = ngx_proxy_protocol_write(c, p, p + NGX_PROXY_PROTOCOL_MAX_HEADER);
+        if (p == NULL) {
+            ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+            return;
+        }
+
+        cl->buf->last = p;
+        cl->buf->temporary = 1;
+        cl->buf->flush = 0;
+        cl->buf->last_buf = 0;
+        cl->buf->tag = (ngx_buf_tag_t) &ngx_stream_proxy_module;
+
+        cl->next = u->upstream_out;
+        u->upstream_out = cl;
+
+        u->proxy_protocol = 0;
+    }
+
+    if (c->type == SOCK_DGRAM && pscf->responses == 0) {
+        pc->read->ready = 0;
+        pc->read->eof = 1;
     }
 
     u->connected = 1;
@@ -861,6 +898,8 @@ ngx_stream_proxy_init_upstream(ngx_strea
 }
 
 
+#if (NGX_STREAM_SSL)
+
 static ngx_int_t
 ngx_stream_proxy_send_proxy_protocol(ngx_stream_session_t *s)
 {
@@ -931,8 +970,6 @@ ngx_stream_proxy_send_proxy_protocol(ngx
 }
 
 
-#if (NGX_STREAM_SSL)
-
 static char *
 ngx_stream_proxy_ssl_password_file(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf)
@@ -1412,8 +1449,10 @@ ngx_stream_proxy_process(ngx_stream_sess
     size_t                        size, limit_rate;
     ssize_t                       n;
     ngx_buf_t                    *b;
+    ngx_int_t                     rc;
     ngx_uint_t                    flags;
     ngx_msec_t                    delay;
+    ngx_chain_t                  *cl, **ll, **out, **busy;
     ngx_connection_t             *c, *pc, *src, *dst;
     ngx_log_handler_pt            handler;
     ngx_stream_upstream_t        *u;
@@ -1447,6 +1486,8 @@ ngx_stream_proxy_process(ngx_stream_sess
         b = &u->upstream_buf;
         limit_rate = pscf->download_rate;
         received = &u->received;
+        out = &u->downstream_out;
+        busy = &u->downstream_busy;
 
     } else {
         src = c;
@@ -1454,24 +1495,18 @@ ngx_stream_proxy_process(ngx_stream_sess
         b = &u->downstream_buf;
         limit_rate = pscf->upload_rate;
         received = &s->received;
+        out = &u->upstream_out;
+        busy = &u->upstream_busy;
     }
 
     for ( ;; ) {
 
-        if (do_write) {
-
-            size = b->last - b->pos;
-
-            if (size && dst && dst->write->ready) {
-
-                n = dst->send(dst, b->pos, size);
-
-                if (n == NGX_AGAIN && dst->shared) {
-                    /* cannot wait on a shared socket */
-                    n = NGX_ERROR;
-                }
-
-                if (n == NGX_ERROR) {
+        if (do_write && dst) {
+
+            if (*out || *busy || dst->buffered) {
+                rc = ngx_stream_top_filter(s, *out, from_upstream);
+
+                if (rc == NGX_ERROR) {
                     if (c->type == SOCK_DGRAM && !from_upstream) {
                         ngx_stream_proxy_next_upstream(s);
                         return;
@@ -1481,13 +1516,12 @@ ngx_stream_proxy_process(ngx_stream_sess
                     return;
                 }
 
-                if (n > 0) {
-                    b->pos += n;
-
-                    if (b->pos == b->last) {
-                        b->pos = b->start;
-                        b->last = b->start;
-                    }
+                ngx_chain_update_chains(c->pool, &u->free, busy, out,
+                                      (ngx_buf_tag_t) &ngx_stream_proxy_module);
+
+                if (*busy == NULL) {
+                    b->pos = b->start;
+                    b->last = b->start;
                 }
             }
         }
@@ -1514,11 +1548,21 @@ ngx_stream_proxy_process(ngx_stream_sess
 
             n = src->recv(src, b->last, size);
 
-            if (n == NGX_AGAIN || n == 0) {
+            if (n == NGX_AGAIN) {
                 break;
             }
 
-            if (n > 0) {
+            if (n == NGX_ERROR) {
+                if (c->type == SOCK_DGRAM && u->received == 0) {
+                    ngx_stream_proxy_next_upstream(s);
+                    return;
+                }
+
+                src->read->eof = 1;
+                n = 0;
+            }
+
+            if (n >= 0) {
                 if (limit_rate) {
                     delay = (ngx_msec_t) (n * 1000 / limit_rate);
 
@@ -1541,27 +1585,37 @@ ngx_stream_proxy_process(ngx_stream_sess
                     src->read->eof = 1;
                 }
 
+                for (ll = out; *ll; ll = &(*ll)->next) { /* void */ }
+
+                cl = ngx_chain_get_free_buf(c->pool, &u->free);
+                if (cl == NULL) {
+                    ngx_stream_proxy_finalize(s,
+                                              NGX_STREAM_INTERNAL_SERVER_ERROR);
+                    return;
+                }
+
+                *ll = cl;
+
+                cl->buf->pos = b->last;
+                cl->buf->last = b->last + n;
+                cl->buf->tag = (ngx_buf_tag_t) &ngx_stream_proxy_module;
+
+                cl->buf->temporary = (n ? 1 : 0);
+                cl->buf->last_buf = src->read->eof;
+                cl->buf->flush = 1;
+
                 *received += n;
                 b->last += n;
                 do_write = 1;
 
                 continue;
             }
-
-            if (n == NGX_ERROR) {
-                if (c->type == SOCK_DGRAM && u->received == 0) {
-                    ngx_stream_proxy_next_upstream(s);
-                    return;
-                }
-
-                src->read->eof = 1;
-            }
         }
 
         break;
     }
 
-    if (src->read->eof && (b->pos == b->last || (dst && dst->read->eof))) {
+    if (src->read->eof && dst && (dst->read->eof || !dst->buffered)) {
         handler = c->log->handler;
         c->log->handler = NULL;
 
@@ -1614,6 +1668,14 @@ ngx_stream_proxy_next_upstream(ngx_strea
                    "stream proxy next upstream");
 
     u = s->upstream;
+    pc = u->peer.connection;
+
+    if (u->upstream_out || u->upstream_busy || (pc && pc->buffered)) {
+        ngx_log_error(NGX_LOG_ERR, s->connection->log, 0,
+                      "pending buffers on next upstream");
+        ngx_stream_proxy_finalize(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+        return;
+    }
 
     if (u->peer.sockaddr) {
         u->peer.free(&u->peer, u->peer.data, NGX_PEER_FAILED);
@@ -1632,8 +1694,6 @@ ngx_stream_proxy_next_upstream(ngx_strea
         return;
     }
 
-    pc = u->peer.connection;
-
     if (pc) {
         ngx_log_debug1(NGX_LOG_DEBUG_STREAM, s->connection->log, 0,
                        "close proxy upstream connection: %d", pc->fd);
diff -r 4bce3edfac2c -r 56fc55e32f23 src/stream/ngx_stream_return_module.c
--- a/src/stream/ngx_stream_return_module.c	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/stream/ngx_stream_return_module.c	Thu Sep 15 14:55:46 2016 +0300
@@ -11,12 +11,12 @@
 
 
 typedef struct {
-    ngx_stream_complex_value_t  text;
+    ngx_stream_complex_value_t   text;
 } ngx_stream_return_srv_conf_t;
 
 
 typedef struct {
-    ngx_buf_t                   buf;
+    ngx_chain_t                 *out;
 } ngx_stream_return_ctx_t;
 
 
@@ -72,6 +72,7 @@ static void
 ngx_stream_return_handler(ngx_stream_session_t *s)
 {
     ngx_str_t                      text;
+    ngx_buf_t                     *b;
     ngx_connection_t              *c;
     ngx_stream_return_ctx_t       *ctx;
     ngx_stream_return_srv_conf_t  *rscf;
@@ -103,8 +104,25 @@ ngx_stream_return_handler(ngx_stream_ses
 
     ngx_stream_set_ctx(s, ctx, ngx_stream_return_module);
 
-    ctx->buf.pos = text.data;
-    ctx->buf.last = text.data + text.len;
+    b = ngx_calloc_buf(c->pool);
+    if (b == NULL) {
+        ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+        return;
+    }
+
+    b->memory = 1;
+    b->pos = text.data;
+    b->last = text.data + text.len;
+    b->last_buf = 1;
+
+    ctx->out = ngx_alloc_chain_link(c->pool);
+    if (ctx->out == NULL) {
+        ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+        return;
+    }
+
+    ctx->out->buf = b;
+    ctx->out->next = NULL;
 
     c->write->handler = ngx_stream_return_write_handler;
 
@@ -115,8 +133,6 @@ ngx_stream_return_handler(ngx_stream_ses
 static void
 ngx_stream_return_write_handler(ngx_event_t *ev)
 {
-    ssize_t                   n;
-    ngx_buf_t                *b;
     ngx_connection_t         *c;
     ngx_stream_session_t     *s;
     ngx_stream_return_ctx_t  *ctx;
@@ -130,25 +146,20 @@ ngx_stream_return_write_handler(ngx_even
         return;
     }
 
-    if (ev->ready) {
-        ctx = ngx_stream_get_module_ctx(s, ngx_stream_return_module);
-
-        b = &ctx->buf;
+    ctx = ngx_stream_get_module_ctx(s, ngx_stream_return_module);
 
-        n = c->send(c, b->pos, b->last - b->pos);
-        if (n == NGX_ERROR) {
-            ngx_stream_finalize_session(s, NGX_STREAM_OK);
-            return;
-        }
+    if (ngx_stream_top_filter(s, ctx->out, 1) == NGX_ERROR) {
+        ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+        return;
+    }
 
-        if (n > 0) {
-            b->pos += n;
+    ctx->out = NULL;
 
-            if (b->pos == b->last) {
-                ngx_stream_finalize_session(s, NGX_STREAM_OK);
-                return;
-            }
-        }
+    if (!c->buffered) {
+        ngx_log_debug0(NGX_LOG_DEBUG_STREAM, c->log, 0,
+                       "stream return done sending");
+        ngx_stream_finalize_session(s, NGX_STREAM_OK);
+        return;
     }
 
     if (ngx_handle_write_event(ev, 0) != NGX_OK) {
diff -r 4bce3edfac2c -r 56fc55e32f23 src/stream/ngx_stream_upstream.h
--- a/src/stream/ngx_stream_upstream.h	Thu Sep 15 14:56:26 2016 +0300
+++ b/src/stream/ngx_stream_upstream.h	Thu Sep 15 14:55:46 2016 +0300
@@ -106,14 +106,24 @@ typedef struct {
 
 typedef struct {
     ngx_peer_connection_t              peer;
+
     ngx_buf_t                          downstream_buf;
     ngx_buf_t                          upstream_buf;
+
+    ngx_chain_t                       *free;
+    ngx_chain_t                       *upstream_out;
+    ngx_chain_t                       *upstream_busy;
+    ngx_chain_t                       *downstream_out;
+    ngx_chain_t                       *downstream_busy;
+
     off_t                              received;
     time_t                             start_sec;
     ngx_uint_t                         responses;
+
 #if (NGX_STREAM_SSL)
     ngx_str_t                          ssl_name;
 #endif
+
     ngx_stream_upstream_resolved_t    *resolved;
     ngx_stream_upstream_state_t       *state;
     unsigned                           connected:1;
diff -r 4bce3edfac2c -r 56fc55e32f23 src/stream/ngx_stream_write_filter_module.c
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/src/stream/ngx_stream_write_filter_module.c	Thu Sep 15 14:55:46 2016 +0300
@@ -0,0 +1,273 @@
+
+/*
+ * Copyright (C) Igor Sysoev
+ * Copyright (C) Nginx, Inc.


From vl at nginx.com  Thu Sep 15 11:57:49 2016
From: vl at nginx.com (Vladimir Homutov)
Date: Thu, 15 Sep 2016 11:57:49 +0000
Subject: [nginx] Stream: phases.
Message-ID: <hg.3908156a51fa.1473940669.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/3908156a51fa
branches:  
changeset: 6693:3908156a51fa
user:      Roman Arutyunyan <arut at nginx.com>
date:      Thu Sep 15 14:55:54 2016 +0300
description:
Stream: phases.

diffstat:

 src/stream/ngx_stream.c                   |  110 ++++++++++++++++
 src/stream/ngx_stream.h                   |   57 +++++++-
 src/stream/ngx_stream_access_module.c     |   11 +-
 src/stream/ngx_stream_core_module.c       |  102 +++++++++++++++
 src/stream/ngx_stream_handler.c           |  198 ++++-------------------------
 src/stream/ngx_stream_limit_conn_module.c |   12 +-
 src/stream/ngx_stream_log_module.c        |    8 +-
 src/stream/ngx_stream_realip_module.c     |    8 +-
 src/stream/ngx_stream_ssl_module.c        |  108 ++++++++++++++++-
 9 files changed, 428 insertions(+), 186 deletions(-)

diffs (864 lines):

diff -r 56fc55e32f23 -r 3908156a51fa src/stream/ngx_stream.c
--- a/src/stream/ngx_stream.c	Thu Sep 15 14:55:46 2016 +0300
+++ b/src/stream/ngx_stream.c	Thu Sep 15 14:55:54 2016 +0300
@@ -12,6 +12,10 @@
 
 
 static char *ngx_stream_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf);
+static ngx_int_t ngx_stream_init_phases(ngx_conf_t *cf,
+    ngx_stream_core_main_conf_t *cmcf);
+static ngx_int_t ngx_stream_init_phase_handlers(ngx_conf_t *cf,
+    ngx_stream_core_main_conf_t *cmcf);
 static ngx_int_t ngx_stream_add_ports(ngx_conf_t *cf, ngx_array_t *ports,
     ngx_stream_listen_t *listen);
 static char *ngx_stream_optimize_servers(ngx_conf_t *cf, ngx_array_t *ports);
@@ -219,6 +223,10 @@ ngx_stream_block(ngx_conf_t *cf, ngx_com
         }
     }
 
+    if (ngx_stream_init_phases(cf, cmcf) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
+
     for (m = 0; cf->cycle->modules[m]; m++) {
         if (cf->cycle->modules[m]->type != NGX_STREAM_MODULE) {
             continue;
@@ -239,6 +247,9 @@ ngx_stream_block(ngx_conf_t *cf, ngx_com
 
     *cf = pcf;
 
+    if (ngx_stream_init_phase_handlers(cf, cmcf) != NGX_OK) {
+        return NGX_CONF_ERROR;
+    }
 
     if (ngx_array_init(&ports, cf->temp_pool, 4, sizeof(ngx_stream_conf_port_t))
         != NGX_OK)
@@ -259,6 +270,105 @@ ngx_stream_block(ngx_conf_t *cf, ngx_com
 
 
 static ngx_int_t
+ngx_stream_init_phases(ngx_conf_t *cf, ngx_stream_core_main_conf_t *cmcf)
+{
+    if (ngx_array_init(&cmcf->phases[NGX_STREAM_POST_ACCEPT_PHASE].handlers,
+                       cf->pool, 1, sizeof(ngx_stream_handler_pt))
+        != NGX_OK)
+    {
+        return NGX_ERROR;
+    }
+
+    if (ngx_array_init(&cmcf->phases[NGX_STREAM_PREACCESS_PHASE].handlers,
+                       cf->pool, 1, sizeof(ngx_stream_handler_pt))
+        != NGX_OK)
+    {
+        return NGX_ERROR;
+    }
+
+    if (ngx_array_init(&cmcf->phases[NGX_STREAM_ACCESS_PHASE].handlers,
+                       cf->pool, 1, sizeof(ngx_stream_handler_pt))
+        != NGX_OK)
+    {
+        return NGX_ERROR;
+    }
+
+#if (NGX_STREAM_SSL)
+    if (ngx_array_init(&cmcf->phases[NGX_STREAM_SSL_PHASE].handlers,
+                       cf->pool, 1, sizeof(ngx_stream_handler_pt))
+        != NGX_OK)
+    {
+        return NGX_ERROR;
+    }
+#endif
+
+    if (ngx_array_init(&cmcf->phases[NGX_STREAM_LOG_PHASE].handlers,
+                       cf->pool, 1, sizeof(ngx_stream_handler_pt))
+        != NGX_OK)
+    {
+        return NGX_ERROR;
+    }
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_stream_init_phase_handlers(ngx_conf_t *cf,
+    ngx_stream_core_main_conf_t *cmcf)
+{
+    ngx_int_t                     j;
+    ngx_uint_t                    i, n;
+    ngx_stream_handler_pt        *h;
+    ngx_stream_phase_handler_t   *ph;
+    ngx_stream_phase_handler_pt   checker;
+
+    n = 1 /* content phase */;
+
+    for (i = 0; i < NGX_STREAM_LOG_PHASE; i++) {
+        n += cmcf->phases[i].handlers.nelts;
+    }
+
+    ph = ngx_pcalloc(cf->pool,
+                     n * sizeof(ngx_stream_phase_handler_t) + sizeof(void *));
+    if (ph == NULL) {
+        return NGX_ERROR;
+    }
+
+    cmcf->phase_engine.handlers = ph;
+    n = 0;
+
+    for (i = 0; i < NGX_STREAM_LOG_PHASE; i++) {
+        h = cmcf->phases[i].handlers.elts;
+
+        switch (i) {
+
+        case NGX_STREAM_CONTENT_PHASE:
+            ph->checker = ngx_stream_core_content_phase;
+            n++;
+            ph++;
+
+            continue;
+
+        default:
+            checker = ngx_stream_core_generic_phase;
+        }
+
+        n += cmcf->phases[i].handlers.nelts;
+
+        for (j = cmcf->phases[i].handlers.nelts - 1; j >= 0; j--) {
+            ph->checker = checker;
+            ph->handler = h[j];
+            ph->next = n;
+            ph++;
+        }
+    }
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
 ngx_stream_add_ports(ngx_conf_t *cf, ngx_array_t *ports,
     ngx_stream_listen_t *listen)
 {
diff -r 56fc55e32f23 -r 3908156a51fa src/stream/ngx_stream.h
--- a/src/stream/ngx_stream.h	Thu Sep 15 14:55:46 2016 +0300
+++ b/src/stream/ngx_stream.h	Thu Sep 15 14:55:54 2016 +0300
@@ -115,17 +115,48 @@ typedef struct {
 } ngx_stream_conf_addr_t;
 
 
-typedef ngx_int_t (*ngx_stream_access_pt)(ngx_stream_session_t *s);
+typedef enum {
+    NGX_STREAM_POST_ACCEPT_PHASE = 0,
+    NGX_STREAM_PREACCESS_PHASE,
+    NGX_STREAM_ACCESS_PHASE,
+#if (NGX_STREAM_SSL)
+    NGX_STREAM_SSL_PHASE,
+#endif
+    NGX_STREAM_CONTENT_PHASE,
+    NGX_STREAM_LOG_PHASE
+} ngx_stream_phases;
+
+
+typedef struct ngx_stream_phase_handler_s  ngx_stream_phase_handler_t;
+
+typedef ngx_int_t (*ngx_stream_phase_handler_pt)(ngx_stream_session_t *s,
+    ngx_stream_phase_handler_t *ph);
+typedef ngx_int_t (*ngx_stream_handler_pt)(ngx_stream_session_t *s);
+typedef void (*ngx_stream_content_handler_pt)(ngx_stream_session_t *s);
+
+
+struct ngx_stream_phase_handler_s {
+    ngx_stream_phase_handler_pt    checker;
+    ngx_stream_handler_pt          handler;
+    ngx_uint_t                     next;
+};
+
+
+typedef struct {
+    ngx_stream_phase_handler_t    *handlers;
+} ngx_stream_phase_engine_t;
+
+
+typedef struct {
+    ngx_array_t                    handlers;
+} ngx_stream_phase_t;
 
 
 typedef struct {
     ngx_array_t                    servers;     /* ngx_stream_core_srv_conf_t */
     ngx_array_t                    listen;      /* ngx_stream_listen_t */
 
-    ngx_stream_access_pt           realip_handler;
-    ngx_stream_access_pt           limit_conn_handler;
-    ngx_stream_access_pt           access_handler;
-    ngx_stream_access_pt           access_log_handler;
+    ngx_stream_phase_engine_t      phase_engine;
 
     ngx_hash_t                     variables_hash;
 
@@ -136,14 +167,13 @@ typedef struct {
     ngx_uint_t                     variables_hash_bucket_size;
 
     ngx_hash_keys_arrays_t        *variables_keys;
+
+    ngx_stream_phase_t             phases[NGX_STREAM_LOG_PHASE + 1];
 } ngx_stream_core_main_conf_t;
 
 
-typedef void (*ngx_stream_handler_pt)(ngx_stream_session_t *s);
-
-
 typedef struct {
-    ngx_stream_handler_pt          handler;
+    ngx_stream_content_handler_pt  handler;
 
     ngx_stream_conf_ctx_t         *ctx;
 
@@ -189,6 +219,7 @@ struct ngx_stream_session_s {
     u_char                        *captures_data;
 #endif
 
+    ngx_int_t                      phase_handler;
     ngx_uint_t                     status;
 
 #if (NGX_STREAM_SSL)
@@ -246,7 +277,15 @@ typedef struct {
 #define NGX_STREAM_WRITE_BUFFERED  0x10
 
 
+void ngx_stream_core_run_phases(ngx_stream_session_t *s);
+ngx_int_t ngx_stream_core_generic_phase(ngx_stream_session_t *s,
+    ngx_stream_phase_handler_t *ph);
+ngx_int_t ngx_stream_core_content_phase(ngx_stream_session_t *s,
+    ngx_stream_phase_handler_t *ph);
+
+
 void ngx_stream_init_connection(ngx_connection_t *c);
+void ngx_stream_session_handler(ngx_event_t *rev);
 void ngx_stream_finalize_session(ngx_stream_session_t *s, ngx_uint_t rc);
 
 
diff -r 56fc55e32f23 -r 3908156a51fa src/stream/ngx_stream_access_module.c
--- a/src/stream/ngx_stream_access_module.c	Thu Sep 15 14:55:46 2016 +0300
+++ b/src/stream/ngx_stream_access_module.c	Thu Sep 15 14:55:54 2016 +0300
@@ -275,7 +275,7 @@ ngx_stream_access_found(ngx_stream_sessi
     if (deny) {
         ngx_log_error(NGX_LOG_ERR, s->connection->log, 0,
                       "access forbidden by rule");
-        return NGX_ABORT;
+        return NGX_STREAM_FORBIDDEN;
     }
 
     return NGX_OK;
@@ -443,10 +443,17 @@ ngx_stream_access_merge_srv_conf(ngx_con
 static ngx_int_t
 ngx_stream_access_init(ngx_conf_t *cf)
 {
+    ngx_stream_handler_pt        *h;
     ngx_stream_core_main_conf_t  *cmcf;
 
     cmcf = ngx_stream_conf_get_module_main_conf(cf, ngx_stream_core_module);
-    cmcf->access_handler = ngx_stream_access_handler;
+
+    h = ngx_array_push(&cmcf->phases[NGX_STREAM_ACCESS_PHASE].handlers);
+    if (h == NULL) {
+        return NGX_ERROR;
+    }
+
+    *h = ngx_stream_access_handler;
 
     return NGX_OK;
 }
diff -r 56fc55e32f23 -r 3908156a51fa src/stream/ngx_stream_core_module.c
--- a/src/stream/ngx_stream_core_module.c	Thu Sep 15 14:55:46 2016 +0300
+++ b/src/stream/ngx_stream_core_module.c	Thu Sep 15 14:55:54 2016 +0300
@@ -123,6 +123,108 @@ ngx_module_t  ngx_stream_core_module = {
 };
 
 
+void
+ngx_stream_core_run_phases(ngx_stream_session_t *s)
+{
+    ngx_int_t                     rc;
+    ngx_stream_phase_handler_t   *ph;
+    ngx_stream_core_main_conf_t  *cmcf;
+
+    cmcf = ngx_stream_get_module_main_conf(s, ngx_stream_core_module);
+
+    ph = cmcf->phase_engine.handlers;
+
+    while (ph[s->phase_handler].checker) {
+
+        rc = ph[s->phase_handler].checker(s, &ph[s->phase_handler]);
+
+        if (rc == NGX_OK) {
+            return;
+        }
+    }
+}
+
+
+ngx_int_t
+ngx_stream_core_generic_phase(ngx_stream_session_t *s,
+    ngx_stream_phase_handler_t *ph)
+{
+    ngx_int_t  rc;
+
+    /*
+     * generic phase checker,
+     * used by all phases, except for content
+     */
+
+    ngx_log_debug1(NGX_LOG_DEBUG_STREAM, s->connection->log, 0,
+                   "generic phase: %ui", s->phase_handler);
+
+    rc = ph->handler(s);
+
+    if (rc == NGX_OK) {
+        s->phase_handler = ph->next;
+        return NGX_AGAIN;
+    }
+
+    if (rc == NGX_DECLINED) {
+        s->phase_handler++;
+        return NGX_AGAIN;
+    }
+
+    if (rc == NGX_AGAIN || rc == NGX_DONE) {
+        return NGX_OK;
+    }
+
+    if (rc == NGX_ERROR) {
+        rc = NGX_STREAM_INTERNAL_SERVER_ERROR;
+    }
+
+    ngx_stream_finalize_session(s, rc);
+
+    return NGX_OK;
+}
+
+
+ngx_int_t
+ngx_stream_core_content_phase(ngx_stream_session_t *s,
+    ngx_stream_phase_handler_t *ph)
+{
+    int                          tcp_nodelay;
+    ngx_connection_t            *c;
+    ngx_stream_core_srv_conf_t  *cscf;
+
+    c = s->connection;
+
+    c->log->action = NULL;
+
+    cscf = ngx_stream_get_module_srv_conf(s, ngx_stream_core_module);
+
+    if (c->type == SOCK_STREAM
+        && cscf->tcp_nodelay
+        && c->tcp_nodelay == NGX_TCP_NODELAY_UNSET)
+    {
+        ngx_log_debug0(NGX_LOG_DEBUG_STREAM, c->log, 0, "tcp_nodelay");
+
+        tcp_nodelay = 1;
+
+        if (setsockopt(c->fd, IPPROTO_TCP, TCP_NODELAY,
+                       (const void *) &tcp_nodelay, sizeof(int)) == -1)
+        {
+            ngx_connection_error(c, ngx_socket_errno,
+                                 "setsockopt(TCP_NODELAY) failed");
+            ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+            return NGX_OK;
+        }
+
+        c->tcp_nodelay = NGX_TCP_NODELAY_SET;
+    }
+
+    cscf->handler(s);
+
+    return NGX_OK;
+}
+
+
 static ngx_int_t
 ngx_stream_core_preconfiguration(ngx_conf_t *cf)
 {
diff -r 56fc55e32f23 -r 3908156a51fa src/stream/ngx_stream_handler.c
--- a/src/stream/ngx_stream_handler.c	Thu Sep 15 14:55:46 2016 +0300
+++ b/src/stream/ngx_stream_handler.c	Thu Sep 15 14:55:54 2016 +0300
@@ -11,15 +11,10 @@
 #include <ngx_stream.h>
 
 
+static void ngx_stream_log_session(ngx_stream_session_t *s);
 static void ngx_stream_close_connection(ngx_connection_t *c);
 static u_char *ngx_stream_log_error(ngx_log_t *log, u_char *buf, size_t len);
 static void ngx_stream_proxy_protocol_handler(ngx_event_t *rev);
-static void ngx_stream_init_session_handler(ngx_event_t *rev);
-
-#if (NGX_STREAM_SSL)
-static void ngx_stream_ssl_init_connection(ngx_ssl_t *ssl, ngx_connection_t *c);
-static void ngx_stream_ssl_handshake_handler(ngx_connection_t *c);
-#endif
 
 
 void
@@ -154,7 +149,7 @@ ngx_stream_init_connection(ngx_connectio
     c->log->connection = c->number;
     c->log->handler = ngx_stream_log_error;
     c->log->data = s;
-    c->log->action = "initializing connection";
+    c->log->action = "initializing session";
     c->log_error = NGX_ERROR_INFO;
 
     s->ctx = ngx_pcalloc(c->pool, sizeof(void *) * ngx_stream_max_module);
@@ -179,7 +174,7 @@ ngx_stream_init_connection(ngx_connectio
     s->start_msec = tp->msec;
 
     rev = c->read;
-    rev->handler = ngx_stream_init_session_handler;
+    rev->handler = ngx_stream_session_handler;
 
     if (addr_conf->proxy_protocol) {
         c->log->action = "reading PROXY protocol";
@@ -279,189 +274,54 @@ ngx_stream_proxy_protocol_handler(ngx_ev
         return;
     }
 
-    ngx_stream_init_session_handler(rev);
+    c->log->action = "initializing session";
+
+    ngx_stream_session_handler(rev);
 }
 
 
-static void
-ngx_stream_init_session_handler(ngx_event_t *rev)
+void
+ngx_stream_session_handler(ngx_event_t *rev)
 {
-    int                           tcp_nodelay;
-    ngx_int_t                     rc;
-    ngx_connection_t             *c;
-    ngx_stream_session_t         *s;
-    ngx_stream_core_srv_conf_t   *cscf;
-    ngx_stream_core_main_conf_t  *cmcf;
+    ngx_connection_t      *c;
+    ngx_stream_session_t  *s;
 
     c = rev->data;
     s = c->data;
 
-    c->log->action = "initializing session";
-
-    cmcf = ngx_stream_get_module_main_conf(s, ngx_stream_core_module);
-
-    if (cmcf->realip_handler) {
-        rc = cmcf->realip_handler(s);
-
-        if (rc == NGX_ERROR) {
-            ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
-            return;
-        }
-    }
-
-    if (cmcf->limit_conn_handler) {
-        rc = cmcf->limit_conn_handler(s);
-
-        if (rc == NGX_ERROR) {
-            ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
-            return;
-        }
-
-        if (rc == NGX_ABORT) {
-            ngx_stream_finalize_session(s, NGX_STREAM_SERVICE_UNAVAILABLE);
-            return;
-        }
-    }
-
-    if (cmcf->access_handler) {
-        rc = cmcf->access_handler(s);
-
-        if (rc == NGX_ERROR) {
-            ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
-            return;
-        }
-
-        if (rc == NGX_ABORT) {
-            ngx_stream_finalize_session(s, NGX_STREAM_FORBIDDEN);
-            return;
-        }
-    }
-
-    cscf = ngx_stream_get_module_srv_conf(s, ngx_stream_core_module);
-
-    if (c->type == SOCK_STREAM
-        && cscf->tcp_nodelay
-        && c->tcp_nodelay == NGX_TCP_NODELAY_UNSET)
-    {
-        ngx_log_debug0(NGX_LOG_DEBUG_STREAM, c->log, 0, "tcp_nodelay");
-
-        tcp_nodelay = 1;
-
-        if (setsockopt(c->fd, IPPROTO_TCP, TCP_NODELAY,
-                       (const void *) &tcp_nodelay, sizeof(int)) == -1)
-        {
-            ngx_connection_error(c, ngx_socket_errno,
-                                 "setsockopt(TCP_NODELAY) failed");
-            ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
-            return;
-        }
-
-        c->tcp_nodelay = NGX_TCP_NODELAY_SET;
-    }
-
-
-#if (NGX_STREAM_SSL)
-    {
-    ngx_stream_ssl_conf_t  *sslcf;
-
-    sslcf = ngx_stream_get_module_srv_conf(s, ngx_stream_ssl_module);
-
-    if (s->ssl) {
-        c->log->action = "SSL handshaking";
-
-        if (sslcf->ssl.ctx == NULL) {
-            ngx_log_error(NGX_LOG_ERR, c->log, 0,
-                          "no \"ssl_certificate\" is defined "
-                          "in server listening on SSL port");
-            ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
-            return;
-        }
-
-        ngx_stream_ssl_init_connection(&sslcf->ssl, c);
-        return;
-    }
-    }
-#endif
-
-    c->log->action = "handling client connection";
-
-    cscf->handler(s);
+    ngx_stream_core_run_phases(s);
 }
 
 
-#if (NGX_STREAM_SSL)
-
-static void
-ngx_stream_ssl_init_connection(ngx_ssl_t *ssl, ngx_connection_t *c)
-{
-    ngx_stream_session_t   *s;
-    ngx_stream_ssl_conf_t  *sslcf;
-
-    s = c->data;
-
-    if (ngx_ssl_create_connection(ssl, c, 0) == NGX_ERROR) {
-        ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
-        return;
-    }
-
-    if (ngx_ssl_handshake(c) == NGX_AGAIN) {
-        sslcf = ngx_stream_get_module_srv_conf(s, ngx_stream_ssl_module);
-
-        ngx_add_timer(c->read, sslcf->handshake_timeout);
-
-        c->ssl->handler = ngx_stream_ssl_handshake_handler;
-
-        return;
-    }
-
-    ngx_stream_ssl_handshake_handler(c);
-}
-
-
-static void
-ngx_stream_ssl_handshake_handler(ngx_connection_t *c)
-{
-    ngx_stream_session_t        *s;
-    ngx_stream_core_srv_conf_t  *cscf;
-
-    if (!c->ssl->handshaked) {
-        ngx_stream_finalize_session(c->data, NGX_STREAM_INTERNAL_SERVER_ERROR);
-        return;
-    }
-
-    if (c->read->timer_set) {
-        ngx_del_timer(c->read);
-    }
-
-    c->log->action = "handling client connection";
-
-    s = c->data;
-
-    cscf = ngx_stream_get_module_srv_conf(s, ngx_stream_core_module);
-
-    cscf->handler(s);
-}
-
-#endif
-
-
 void
 ngx_stream_finalize_session(ngx_stream_session_t *s, ngx_uint_t rc)
 {
-    ngx_stream_core_main_conf_t  *cmcf;
-
     ngx_log_debug1(NGX_LOG_DEBUG_STREAM, s->connection->log, 0,
                    "finalize stream session: %i", rc);
 
     s->status = rc;
 
+    ngx_stream_log_session(s);
+
+    ngx_stream_close_connection(s->connection);
+}
+
+
+static void
+ngx_stream_log_session(ngx_stream_session_t *s)
+{
+    ngx_uint_t                    i, n;
+    ngx_stream_handler_pt        *log_handler;
+    ngx_stream_core_main_conf_t  *cmcf;
+
     cmcf = ngx_stream_get_module_main_conf(s, ngx_stream_core_module);
 
-    if (cmcf->access_log_handler) {
-        (void) cmcf->access_log_handler(s);
+    log_handler = cmcf->phases[NGX_STREAM_LOG_PHASE].handlers.elts;
+    n = cmcf->phases[NGX_STREAM_LOG_PHASE].handlers.nelts;
+
+    for (i = 0; i < n; i++) {
+        log_handler[i](s);
     }
-
-    ngx_stream_close_connection(s->connection);
 }
 
 
diff -r 56fc55e32f23 -r 3908156a51fa src/stream/ngx_stream_limit_conn_module.c
--- a/src/stream/ngx_stream_limit_conn_module.c	Thu Sep 15 14:55:46 2016 +0300
+++ b/src/stream/ngx_stream_limit_conn_module.c	Thu Sep 15 14:55:54 2016 +0300
@@ -178,7 +178,7 @@ ngx_stream_limit_conn_handler(ngx_stream
             if (node == NULL) {
                 ngx_shmtx_unlock(&shpool->mutex);
                 ngx_stream_limit_conn_cleanup_all(s->connection->pool);
-                return NGX_ABORT;
+                return NGX_STREAM_SERVICE_UNAVAILABLE;
             }
 
             lc = (ngx_stream_limit_conn_node_t *) &node->color;
@@ -203,7 +203,7 @@ ngx_stream_limit_conn_handler(ngx_stream
                               &limits[i].shm_zone->shm.name);
 
                 ngx_stream_limit_conn_cleanup_all(s->connection->pool);
-                return NGX_ABORT;
+                return NGX_STREAM_SERVICE_UNAVAILABLE;
             }
 
             lc->conn++;
@@ -630,11 +630,17 @@ ngx_stream_limit_conn(ngx_conf_t *cf, ng
 static ngx_int_t
 ngx_stream_limit_conn_init(ngx_conf_t *cf)
 {
+    ngx_stream_handler_pt        *h;
     ngx_stream_core_main_conf_t  *cmcf;
 
     cmcf = ngx_stream_conf_get_module_main_conf(cf, ngx_stream_core_module);
 
-    cmcf->limit_conn_handler = ngx_stream_limit_conn_handler;
+    h = ngx_array_push(&cmcf->phases[NGX_STREAM_PREACCESS_PHASE].handlers);
+    if (h == NULL) {
+        return NGX_ERROR;
+    }
+
+    *h = ngx_stream_limit_conn_handler;
 
     return NGX_OK;
 }
diff -r 56fc55e32f23 -r 3908156a51fa src/stream/ngx_stream_log_module.c
--- a/src/stream/ngx_stream_log_module.c	Thu Sep 15 14:55:46 2016 +0300
+++ b/src/stream/ngx_stream_log_module.c	Thu Sep 15 14:55:54 2016 +0300
@@ -1464,11 +1464,17 @@ ngx_stream_log_open_file_cache(ngx_conf_
 static ngx_int_t
 ngx_stream_log_init(ngx_conf_t *cf)
 {
+    ngx_stream_handler_pt        *h;
     ngx_stream_core_main_conf_t  *cmcf;
 
     cmcf = ngx_stream_conf_get_module_main_conf(cf, ngx_stream_core_module);
 
-    cmcf->access_log_handler = ngx_stream_log_handler;
+    h = ngx_array_push(&cmcf->phases[NGX_STREAM_LOG_PHASE].handlers);
+    if (h == NULL) {
+        return NGX_ERROR;
+    }
+
+    *h = ngx_stream_log_handler;
 
     return NGX_OK;
 }
diff -r 56fc55e32f23 -r 3908156a51fa src/stream/ngx_stream_realip_module.c
--- a/src/stream/ngx_stream_realip_module.c	Thu Sep 15 14:55:46 2016 +0300
+++ b/src/stream/ngx_stream_realip_module.c	Thu Sep 15 14:55:54 2016 +0300
@@ -279,11 +279,17 @@ ngx_stream_realip_add_variables(ngx_conf
 static ngx_int_t
 ngx_stream_realip_init(ngx_conf_t *cf)
 {
+    ngx_stream_handler_pt        *h;
     ngx_stream_core_main_conf_t  *cmcf;
 
     cmcf = ngx_stream_conf_get_module_main_conf(cf, ngx_stream_core_module);
 
-    cmcf->realip_handler = ngx_stream_realip_handler;
+    h = ngx_array_push(&cmcf->phases[NGX_STREAM_POST_ACCEPT_PHASE].handlers);
+    if (h == NULL) {
+        return NGX_ERROR;
+    }
+
+    *h = ngx_stream_realip_handler;
 
     return NGX_OK;
 }
diff -r 56fc55e32f23 -r 3908156a51fa src/stream/ngx_stream_ssl_module.c
--- a/src/stream/ngx_stream_ssl_module.c	Thu Sep 15 14:55:46 2016 +0300
+++ b/src/stream/ngx_stream_ssl_module.c	Thu Sep 15 14:55:54 2016 +0300
@@ -18,6 +18,10 @@ typedef ngx_int_t (*ngx_ssl_variable_han
 #define NGX_DEFAULT_ECDH_CURVE  "auto"
 
 
+static ngx_int_t ngx_stream_ssl_handler(ngx_stream_session_t *s);
+static ngx_int_t ngx_stream_ssl_init_connection(ngx_ssl_t *ssl,
+    ngx_connection_t *c);
+static void ngx_stream_ssl_handshake_handler(ngx_connection_t *c);
 static ngx_int_t ngx_stream_ssl_static_variable(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_stream_ssl_variable(ngx_stream_session_t *s,
@@ -32,6 +36,7 @@ static char *ngx_stream_ssl_password_fil
     void *conf);
 static char *ngx_stream_ssl_session_cache(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf);
+static ngx_int_t ngx_stream_ssl_init(ngx_conf_t *cf);
 
 
 static ngx_conf_bitmask_t  ngx_stream_ssl_protocols[] = {
@@ -143,7 +148,7 @@ static ngx_command_t  ngx_stream_ssl_com
 
 static ngx_stream_module_t  ngx_stream_ssl_module_ctx = {
     ngx_stream_ssl_add_variables,          /* preconfiguration */
-    NULL,                                  /* postconfiguration */
+    ngx_stream_ssl_init,                   /* postconfiguration */
 
     NULL,                                  /* create main configuration */
     NULL,                                  /* init main configuration */
@@ -194,6 +199,88 @@ static ngx_str_t ngx_stream_ssl_sess_id_
 
 
 static ngx_int_t
+ngx_stream_ssl_handler(ngx_stream_session_t *s)
+{
+    ngx_connection_t       *c;
+    ngx_stream_ssl_conf_t  *sslcf;
+
+    c = s->connection;
+
+    sslcf = ngx_stream_get_module_srv_conf(s, ngx_stream_ssl_module);
+
+    if (s->ssl && c->ssl == NULL) {
+        c->log->action = "SSL handshaking";
+
+        if (sslcf->ssl.ctx == NULL) {
+            ngx_log_error(NGX_LOG_ERR, c->log, 0,
+                          "no \"ssl_certificate\" is defined "
+                          "in server listening on SSL port");
+            return NGX_ERROR;
+        }
+
+        return ngx_stream_ssl_init_connection(&sslcf->ssl, c);
+    }
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_stream_ssl_init_connection(ngx_ssl_t *ssl, ngx_connection_t *c)
+{
+    ngx_int_t               rc;
+    ngx_stream_session_t   *s;
+    ngx_stream_ssl_conf_t  *sslcf;
+
+    s = c->data;
+
+    if (ngx_ssl_create_connection(ssl, c, 0) == NGX_ERROR) {
+        return NGX_ERROR;
+    }
+
+    rc = ngx_ssl_handshake(c);
+
+    if (rc == NGX_ERROR) {
+        return NGX_ERROR;
+    }
+
+    if (rc == NGX_AGAIN) {
+        sslcf = ngx_stream_get_module_srv_conf(s, ngx_stream_ssl_module);
+
+        ngx_add_timer(c->read, sslcf->handshake_timeout);
+
+        c->ssl->handler = ngx_stream_ssl_handshake_handler;
+
+        return NGX_AGAIN;
+    }
+
+    /* rc == NGX_OK */
+
+    return NGX_OK;
+}
+
+
+static void
+ngx_stream_ssl_handshake_handler(ngx_connection_t *c)
+{
+    ngx_stream_session_t  *s;
+
+    s = c->data;
+
+    if (!c->ssl->handshaked) {
+        ngx_stream_finalize_session(s, NGX_STREAM_INTERNAL_SERVER_ERROR);
+        return;
+    }
+
+    if (c->read->timer_set) {
+        ngx_del_timer(c->read);
+    }
+
+    ngx_stream_core_run_phases(s);
+}
+
+
+static ngx_int_t
 ngx_stream_ssl_static_variable(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data)
 {
@@ -565,3 +652,22 @@ invalid:
 
     return NGX_CONF_ERROR;
 }
+
+
+static ngx_int_t
+ngx_stream_ssl_init(ngx_conf_t *cf)
+{
+    ngx_stream_handler_pt        *h;
+    ngx_stream_core_main_conf_t  *cmcf;
+
+    cmcf = ngx_stream_conf_get_module_main_conf(cf, ngx_stream_core_module);
+
+    h = ngx_array_push(&cmcf->phases[NGX_STREAM_SSL_PHASE].handlers);
+    if (h == NULL) {
+        return NGX_ERROR;
+    }
+
+    *h = ngx_stream_ssl_handler;
+
+    return NGX_OK;
+}


From vl at nginx.com  Thu Sep 15 11:57:51 2016
From: vl at nginx.com (Vladimir Homutov)
Date: Thu, 15 Sep 2016 11:57:51 +0000
Subject: [nginx] Stream: preread phase.
Message-ID: <hg.ea9dfe2f62e7.1473940671.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/ea9dfe2f62e7
branches:  
changeset: 6694:ea9dfe2f62e7
user:      Vladimir Homutov <vl at nginx.com>
date:      Thu Sep 15 14:56:02 2016 +0300
description:
Stream: preread phase.

In this phase, head of a stream is read and analysed before proceeding to the
content phase.  Amount of data read is controlled by the module implementing
the phase, but not more than defined by the "preread_buffer_size" directive.
The time spent on processing preread is controlled by the "preread_timeout"
directive.

The typical preread phase module will parse the beginning of a stream and set
variable that may be used by the content phase, for example to make routing
decision.

diffstat:

 src/stream/ngx_stream.c             |   11 +++
 src/stream/ngx_stream.h             |    5 +
 src/stream/ngx_stream_core_module.c |  130 +++++++++++++++++++++++++++++++++++-
 3 files changed, 145 insertions(+), 1 deletions(-)

diffs (225 lines):

diff -r 3908156a51fa -r ea9dfe2f62e7 src/stream/ngx_stream.c
--- a/src/stream/ngx_stream.c	Thu Sep 15 14:55:54 2016 +0300
+++ b/src/stream/ngx_stream.c	Thu Sep 15 14:56:02 2016 +0300
@@ -302,6 +302,13 @@ ngx_stream_init_phases(ngx_conf_t *cf, n
     }
 #endif
 
+    if (ngx_array_init(&cmcf->phases[NGX_STREAM_PREREAD_PHASE].handlers,
+                       cf->pool, 1, sizeof(ngx_stream_handler_pt))
+        != NGX_OK)
+    {
+        return NGX_ERROR;
+    }
+
     if (ngx_array_init(&cmcf->phases[NGX_STREAM_LOG_PHASE].handlers,
                        cf->pool, 1, sizeof(ngx_stream_handler_pt))
         != NGX_OK)
@@ -343,6 +350,10 @@ ngx_stream_init_phase_handlers(ngx_conf_
 
         switch (i) {
 
+        case NGX_STREAM_PREREAD_PHASE:
+            checker = ngx_stream_core_preread_phase;
+            break;
+
         case NGX_STREAM_CONTENT_PHASE:
             ph->checker = ngx_stream_core_content_phase;
             n++;
diff -r 3908156a51fa -r ea9dfe2f62e7 src/stream/ngx_stream.h
--- a/src/stream/ngx_stream.h	Thu Sep 15 14:55:54 2016 +0300
+++ b/src/stream/ngx_stream.h	Thu Sep 15 14:56:02 2016 +0300
@@ -122,6 +122,7 @@ typedef enum {
 #if (NGX_STREAM_SSL)
     NGX_STREAM_SSL_PHASE,
 #endif
+    NGX_STREAM_PREREAD_PHASE,
     NGX_STREAM_CONTENT_PHASE,
     NGX_STREAM_LOG_PHASE
 } ngx_stream_phases;
@@ -181,6 +182,8 @@ typedef struct {
     ngx_uint_t                     line;
 
     ngx_flag_t                     tcp_nodelay;
+    size_t                         preread_buffer_size;
+    ngx_msec_t                     preread_timeout;
 
     ngx_log_t                     *error_log;
 
@@ -280,6 +283,8 @@ typedef struct {
 void ngx_stream_core_run_phases(ngx_stream_session_t *s);
 ngx_int_t ngx_stream_core_generic_phase(ngx_stream_session_t *s,
     ngx_stream_phase_handler_t *ph);
+ngx_int_t ngx_stream_core_preread_phase(ngx_stream_session_t *s,
+    ngx_stream_phase_handler_t *ph);
 ngx_int_t ngx_stream_core_content_phase(ngx_stream_session_t *s,
     ngx_stream_phase_handler_t *ph);
 
diff -r 3908156a51fa -r ea9dfe2f62e7 src/stream/ngx_stream_core_module.c
--- a/src/stream/ngx_stream_core_module.c	Thu Sep 15 14:55:54 2016 +0300
+++ b/src/stream/ngx_stream_core_module.c	Thu Sep 15 14:56:02 2016 +0300
@@ -91,6 +91,20 @@ static ngx_command_t  ngx_stream_core_co
       offsetof(ngx_stream_core_srv_conf_t, tcp_nodelay),
       NULL },
 
+    { ngx_string("preread_buffer_size"),
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_TAKE1,
+      ngx_conf_set_size_slot,
+      NGX_STREAM_SRV_CONF_OFFSET,
+      offsetof(ngx_stream_core_srv_conf_t, preread_buffer_size),
+      NULL },
+
+    { ngx_string("preread_timeout"),
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_TAKE1,
+      ngx_conf_set_msec_slot,
+      NGX_STREAM_SRV_CONF_OFFSET,
+      offsetof(ngx_stream_core_srv_conf_t, preread_timeout),
+      NULL },
+
       ngx_null_command
 };
 
@@ -153,7 +167,7 @@ ngx_stream_core_generic_phase(ngx_stream
 
     /*
      * generic phase checker,
-     * used by all phases, except for content
+     * used by all phases, except for preread and content
      */
 
     ngx_log_debug1(NGX_LOG_DEBUG_STREAM, s->connection->log, 0,
@@ -186,6 +200,112 @@ ngx_stream_core_generic_phase(ngx_stream
 
 
 ngx_int_t
+ngx_stream_core_preread_phase(ngx_stream_session_t *s,
+    ngx_stream_phase_handler_t *ph)
+{
+    size_t                       size;
+    ssize_t                      n;
+    ngx_int_t                    rc;
+    ngx_connection_t            *c;
+    ngx_stream_core_srv_conf_t  *cscf;
+
+    c = s->connection;
+
+    c->log->action = "prereading client data";
+
+    cscf = ngx_stream_get_module_srv_conf(s, ngx_stream_core_module);
+
+    if (c->read->timedout) {
+        rc = NGX_STREAM_OK;
+
+    } else if (c->read->timer_set) {
+        rc = NGX_AGAIN;
+
+    } else {
+        rc = ph->handler(s);
+    }
+
+    while (rc == NGX_AGAIN) {
+
+        if (c->buffer == NULL) {
+            c->buffer = ngx_create_temp_buf(c->pool, cscf->preread_buffer_size);
+            if (c->buffer == NULL) {
+                rc = NGX_ERROR;
+                break;
+            }
+        }
+
+        size = c->buffer->end - c->buffer->last;
+
+        if (size == 0) {
+            ngx_log_error(NGX_LOG_ERR, c->log, 0, "preread buffer full");
+            rc = NGX_STREAM_BAD_REQUEST;
+            break;
+        }
+
+        if (c->read->eof) {
+            rc = NGX_STREAM_OK;
+            break;
+        }
+
+        if (!c->read->ready) {
+            if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+                rc = NGX_ERROR;
+                break;
+            }
+
+            if (!c->read->timer_set) {
+                ngx_add_timer(c->read, cscf->preread_timeout);
+            }
+
+            c->read->handler = ngx_stream_session_handler;
+
+            return NGX_OK;
+        }
+
+        n = c->recv(c, c->buffer->last, size);
+
+        if (n == NGX_ERROR) {
+            rc = NGX_STREAM_OK;
+            break;
+        }
+
+        if (n > 0) {
+            c->buffer->last += n;
+        }
+
+        rc = ph->handler(s);
+    }
+
+    if (c->read->timer_set) {
+        ngx_del_timer(c->read);
+    }
+
+    if (rc == NGX_OK) {
+        s->phase_handler = ph->next;
+        return NGX_AGAIN;
+    }
+
+    if (rc == NGX_DECLINED) {
+        s->phase_handler++;
+        return NGX_AGAIN;
+    }
+
+    if (rc == NGX_DONE) {
+        return NGX_OK;
+    }
+
+    if (rc == NGX_ERROR) {
+        rc = NGX_STREAM_INTERNAL_SERVER_ERROR;
+    }
+
+    ngx_stream_finalize_session(s, rc);
+
+    return NGX_OK;
+}
+
+
+ngx_int_t
 ngx_stream_core_content_phase(ngx_stream_session_t *s,
     ngx_stream_phase_handler_t *ph)
 {
@@ -303,6 +423,8 @@ ngx_stream_core_create_srv_conf(ngx_conf
     cscf->resolver_timeout = NGX_CONF_UNSET_MSEC;
     cscf->proxy_protocol_timeout = NGX_CONF_UNSET_MSEC;
     cscf->tcp_nodelay = NGX_CONF_UNSET;
+    cscf->preread_buffer_size = NGX_CONF_UNSET_SIZE;
+    cscf->preread_timeout = NGX_CONF_UNSET_MSEC;
 
     return cscf;
 }
@@ -355,6 +477,12 @@ ngx_stream_core_merge_srv_conf(ngx_conf_
 
     ngx_conf_merge_value(conf->tcp_nodelay, prev->tcp_nodelay, 1);
 
+    ngx_conf_merge_size_value(conf->preread_buffer_size,
+                              prev->preread_buffer_size, 16384);
+
+    ngx_conf_merge_msec_value(conf->preread_timeout,
+                              prev->preread_timeout, 30000);
+
     return NGX_CONF_OK;
 }
 


From vl at nginx.com  Thu Sep 15 11:57:54 2016
From: vl at nginx.com (Vladimir Homutov)
Date: Thu, 15 Sep 2016 11:57:54 +0000
Subject: [nginx] Stream: ssl_preread module.
Message-ID: <hg.060d71292b69.1473940674.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/060d71292b69
branches:  
changeset: 6695:060d71292b69
user:      Vladimir Homutov <vl at nginx.com>
date:      Thu Sep 15 14:56:51 2016 +0300
description:
Stream: ssl_preread module.

The ssl_preread module extracts information from the SSL Client Hello message
without terminating SSL.  Currently, only $ssl_preread_server_name variable
is supported, which contains server name from the SNI extension.

diffstat:

 auto/modules                               |   10 +
 auto/options                               |    4 +
 src/stream/ngx_stream_ssl_preread_module.c |  448 +++++++++++++++++++++++++++++
 3 files changed, 462 insertions(+), 0 deletions(-)

diffs (500 lines):

diff -r ea9dfe2f62e7 -r 060d71292b69 auto/modules
--- a/auto/modules	Thu Sep 15 14:56:02 2016 +0300
+++ b/auto/modules	Thu Sep 15 14:56:51 2016 +0300
@@ -1120,6 +1120,16 @@ if [ $STREAM != NO ]; then
 
         . auto/module
     fi
+
+    if [ $STREAM_SSL_PREREAD = YES ]; then
+        ngx_module_name=ngx_stream_ssl_preread_module
+        ngx_module_deps=
+        ngx_module_srcs=src/stream/ngx_stream_ssl_preread_module.c
+        ngx_module_libs=
+        ngx_module_link=$STREAM_SSL_PREREAD
+
+        . auto/module
+    fi
 fi
 
 
diff -r ea9dfe2f62e7 -r 060d71292b69 auto/options
--- a/auto/options	Thu Sep 15 14:56:02 2016 +0300
+++ b/auto/options	Thu Sep 15 14:56:51 2016 +0300
@@ -126,6 +126,7 @@ STREAM_RETURN=YES
 STREAM_UPSTREAM_HASH=YES
 STREAM_UPSTREAM_LEAST_CONN=YES
 STREAM_UPSTREAM_ZONE=YES
+STREAM_SSL_PREREAD=NO
 
 DYNAMIC_MODULES=
 
@@ -301,6 +302,8 @@ use the \"--with-mail_ssl_module\" optio
         --with-stream_geoip_module)      STREAM_GEOIP=YES           ;;
         --with-stream_geoip_module=dynamic)
                                          STREAM_GEOIP=DYNAMIC       ;;
+        --with-stream_ssl_preread_module)
+                                         STREAM_SSL_PREREAD=YES     ;;
         --without-stream_limit_conn_module)
                                          STREAM_LIMIT_CONN=NO       ;;
         --without-stream_access_module)  STREAM_ACCESS=NO           ;;
@@ -508,6 +511,7 @@ cat << END
   --with-stream_realip_module        enable ngx_stream_realip_module
   --with-stream_geoip_module         enable ngx_stream_geoip_module
   --with-stream_geoip_module=dynamic enable dynamic ngx_stream_geoip_module
+  --with-stream_ssl_preread_module   enable ngx_stream_ssl_preread_module
   --without-stream_limit_conn_module disable ngx_stream_limit_conn_module
   --without-stream_access_module     disable ngx_stream_access_module
   --without-stream_geo_module        disable ngx_stream_geo_module
diff -r ea9dfe2f62e7 -r 060d71292b69 src/stream/ngx_stream_ssl_preread_module.c
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/src/stream/ngx_stream_ssl_preread_module.c	Thu Sep 15 14:56:51 2016 +0300
@@ -0,0 +1,448 @@
+
+/*
+ * Copyright (C) Nginx, Inc.
+ */
+
+
+#include <ngx_config.h>
+#include <ngx_core.h>
+#include <ngx_stream.h>
+
+
+typedef struct {
+    ngx_flag_t      enabled;
+} ngx_stream_ssl_preread_srv_conf_t;
+
+
+typedef struct {
+    size_t          left;
+    size_t          size;
+    u_char         *pos;
+    u_char         *dst;
+    u_char          buf[4];
+    ngx_str_t       host;
+    ngx_log_t      *log;
+    ngx_pool_t     *pool;
+    ngx_uint_t      state;
+} ngx_stream_ssl_preread_ctx_t;
+
+
+static ngx_int_t ngx_stream_ssl_preread_handler(ngx_stream_session_t *s);
+static ngx_int_t ngx_stream_ssl_preread_parse_record(
+    ngx_stream_ssl_preread_ctx_t *ctx, u_char *pos, u_char *last);
+static ngx_int_t ngx_stream_ssl_preread_server_name_variable(
+    ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
+static ngx_int_t ngx_stream_ssl_preread_add_variables(ngx_conf_t *cf);
+static void *ngx_stream_ssl_preread_create_srv_conf(ngx_conf_t *cf);
+static char *ngx_stream_ssl_preread_merge_srv_conf(ngx_conf_t *cf, void *parent,
+    void *child);
+static ngx_int_t ngx_stream_ssl_preread_init(ngx_conf_t *cf);
+
+
+static ngx_command_t  ngx_stream_ssl_preread_commands[] = {
+
+    { ngx_string("ssl_preread"),
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_FLAG,
+      ngx_conf_set_flag_slot,
+      NGX_STREAM_SRV_CONF_OFFSET,
+      offsetof(ngx_stream_ssl_preread_srv_conf_t, enabled),
+      NULL },
+
+      ngx_null_command
+};
+
+
+static ngx_stream_module_t  ngx_stream_ssl_preread_module_ctx = {
+    ngx_stream_ssl_preread_add_variables,   /* preconfiguration */
+    ngx_stream_ssl_preread_init,            /* postconfiguration */
+
+    NULL,                                   /* create main configuration */
+    NULL,                                   /* init main configuration */
+
+    ngx_stream_ssl_preread_create_srv_conf, /* create server configuration */
+    ngx_stream_ssl_preread_merge_srv_conf   /* merge server configuration */
+};
+
+
+ngx_module_t  ngx_stream_ssl_preread_module = {
+    NGX_MODULE_V1,
+    &ngx_stream_ssl_preread_module_ctx,     /* module context */
+    ngx_stream_ssl_preread_commands,        /* module directives */
+    NGX_STREAM_MODULE,                      /* module type */
+    NULL,                                   /* init master */
+    NULL,                                   /* init module */
+    NULL,                                   /* init process */
+    NULL,                                   /* init thread */
+    NULL,                                   /* exit thread */
+    NULL,                                   /* exit process */
+    NULL,                                   /* exit master */
+    NGX_MODULE_V1_PADDING
+};
+
+
+static ngx_stream_variable_t  ngx_stream_ssl_preread_vars[] = {
+
+    { ngx_string("ssl_preread_server_name"), NULL,
+      ngx_stream_ssl_preread_server_name_variable, 0, 0, 0 },
+
+    { ngx_null_string, NULL, NULL, 0, 0, 0 }
+};
+
+
+static ngx_int_t
+ngx_stream_ssl_preread_handler(ngx_stream_session_t *s)
+{
+    u_char                             *last, *p;
+    size_t                              len;
+    ngx_int_t                           rc;
+    ngx_connection_t                   *c;
+    ngx_stream_ssl_preread_ctx_t       *ctx;
+    ngx_stream_ssl_preread_srv_conf_t  *sscf;
+
+    c = s->connection;
+
+    ngx_log_debug0(NGX_LOG_DEBUG_STREAM, c->log, 0, "ssl preread handler");
+
+    sscf = ngx_stream_get_module_srv_conf(s, ngx_stream_ssl_preread_module);
+
+    if (!sscf->enabled) {
+        return NGX_DECLINED;
+    }
+
+    if (c->type != SOCK_STREAM) {
+        return NGX_DECLINED;
+    }
+
+    if (c->buffer == NULL) {
+        return NGX_AGAIN;
+    }
+
+    ctx = ngx_stream_get_module_ctx(s, ngx_stream_ssl_preread_module);
+    if (ctx == NULL) {
+        ctx = ngx_pcalloc(c->pool, sizeof(ngx_stream_ssl_preread_ctx_t));
+        if (ctx == NULL) {
+            return NGX_ERROR;
+        }
+
+        ngx_stream_set_ctx(s, ctx, ngx_stream_ssl_preread_module);
+
+        ctx->pool = c->pool;
+        ctx->log = c->log;
+        ctx->pos = c->buffer->pos;
+    }
+
+    p = ctx->pos;
+    last = c->buffer->last;
+
+    while (last - p >= 5) {
+
+        if (p[0] != 0x16) {
+            ngx_log_debug(NGX_LOG_DEBUG_STREAM, ctx->log, 0,
+                          "ssl preread: not a handshake");
+            return NGX_DECLINED;
+        }
+
+        if (p[1] != 3 || p[2] == 0) {
+            ngx_log_debug(NGX_LOG_DEBUG_STREAM, ctx->log, 0,
+                          "ssl preread: unsupported SSL version");
+            return NGX_DECLINED;
+        }
+
+        len = (p[3] << 8) + p[4];
+
+        /* read the whole record before parsing */
+        if ((size_t) (last - p) < len + 5) {
+            break;
+        }
+
+        p += 5;
+
+        rc = ngx_stream_ssl_preread_parse_record(ctx, p, p + len);
+        if (rc != NGX_AGAIN) {
+            return rc;
+        }
+
+        p += len;
+    }
+
+    ctx->pos = p;
+
+    return NGX_AGAIN;
+}
+
+
+static ngx_int_t
+ngx_stream_ssl_preread_parse_record(ngx_stream_ssl_preread_ctx_t *ctx,
+    u_char *pos, u_char *last)
+{
+    size_t   left, n, size;
+    u_char  *dst, *p;
+
+    enum {
+        sw_start = 0,
+        sw_header,          /* handshake msg_type, length */
+        sw_head_tail,       /* version, random */
+        sw_sid_len,         /* session_id length */
+        sw_sid,             /* session_id */
+        sw_cs_len,          /* cipher_suites length */
+        sw_cs,              /* cipher_suites */
+        sw_cm_len,          /* compression_methods length */
+        sw_cm,              /* compression_methods */
+        sw_ext,             /* extension */
+        sw_ext_header,      /* extension_type, extension_data length */
+        sw_sni_len,         /* SNI length */
+        sw_sni_host_head,   /* SNI name_type, host_name length */
+        sw_sni_host         /* SNI host_name */
+    } state;
+
+    ngx_log_debug2(NGX_LOG_DEBUG_STREAM, ctx->log, 0,
+                   "ssl preread: state %ui left %z", ctx->state, ctx->left);
+
+    state = ctx->state;
+    size = ctx->size;
+    left = ctx->left;
+    dst = ctx->dst;
+    p = ctx->buf;
+
+    for ( ;; ) {
+        n = ngx_min((size_t) (last - pos), size);
+
+        if (dst) {
+            dst = ngx_cpymem(dst, pos, n);
+        }
+
+        pos += n;
+        size -= n;
+        left -= n;
+
+        if (size != 0) {
+            break;
+        }
+
+        switch (state) {
+
+        case sw_start:
+            state = sw_header;
+            dst = p;
+            size = 4;
+            left = size;
+            break;
+
+        case sw_header:
+            if (p[0] != 1) {
+                ngx_log_debug(NGX_LOG_DEBUG_STREAM, ctx->log, 0,
+                              "ssl preread: not a client hello");
+                return NGX_DECLINED;
+            }
+
+            state = sw_head_tail;
+            dst = NULL;
+            size = 34;
+            left = (p[1] << 16) + (p[2] << 8) + p[3];
+            break;
+
+        case sw_head_tail:
+            state = sw_sid_len;
+            dst = p;
+            size = 1;
+            break;
+
+        case sw_sid_len:
+            state = sw_sid;
+            dst = NULL;
+            size = p[0];
+            break;
+
+        case sw_sid:
+            state = sw_cs_len;
+            dst = p;
+            size = 2;
+            break;
+
+        case sw_cs_len:
+            state = sw_cs;
+            dst = NULL;
+            size = (p[0] << 8) + p[1];
+            break;
+
+        case sw_cs:
+            state = sw_cm_len;
+            dst = p;
+            size = 1;
+            break;
+
+        case sw_cm_len:
+            state = sw_cm;
+            dst = NULL;
+            size = p[0];
+            break;
+
+        case sw_cm:
+            if (left == 0) {
+                /* no extensions */
+                return NGX_OK;
+            }
+
+            state = sw_ext;
+            dst = p;
+            size = 2;
+            break;
+
+        case sw_ext:
+            if (left == 0) {
+                return NGX_OK;
+            }
+
+            state = sw_ext_header;
+            dst = p;
+            size = 4;
+            break;
+
+        case sw_ext_header:
+            if (p[0] == 0 && p[1] == 0) {
+                /* SNI extension */
+                state = sw_sni_len;
+                dst = NULL;
+                size = 2;
+                break;
+            }
+
+            state = sw_ext;
+            dst = NULL;
+            size = (p[2] << 8) + p[3];
+            break;
+
+        case sw_sni_len:
+            state = sw_sni_host_head;
+            dst = p;
+            size = 3;
+            break;
+
+        case sw_sni_host_head:
+            if (p[0] != 0) {
+                ngx_log_debug0(NGX_LOG_DEBUG_STREAM, ctx->log, 0,
+                               "ssl preread: SNI hostname type is not DNS");
+                return NGX_DECLINED;
+            }
+
+            state = sw_sni_host;
+            size = (p[1] << 8) + p[2];
+
+            ctx->host.data = ngx_pnalloc(ctx->pool, size);
+            if (ctx->host.data == NULL) {
+                return NGX_ERROR;
+            }
+
+            ctx->host.len = size;
+            dst = ctx->host.data;
+            break;
+
+        case sw_sni_host:
+            ngx_log_debug1(NGX_LOG_DEBUG_STREAM, ctx->log, 0,
+                           "ssl preread: SNI hostname \"%V\"", &ctx->host);
+            return NGX_OK;
+        }
+
+        if (left < size) {
+           ngx_log_debug0(NGX_LOG_DEBUG_STREAM, ctx->log, 0,
+                          "ssl preread: failed to parse handshake");
+           return NGX_DECLINED;
+        }
+    }
+
+    ctx->state = state;
+    ctx->size = size;
+    ctx->left = left;
+    ctx->dst = dst;
+
+    return NGX_AGAIN;
+}
+
+
+static ngx_int_t
+ngx_stream_ssl_preread_server_name_variable(ngx_stream_session_t *s,
+    ngx_variable_value_t *v, uintptr_t data)
+{
+    ngx_stream_ssl_preread_ctx_t  *ctx;
+
+    ctx = ngx_stream_get_module_ctx(s, ngx_stream_ssl_preread_module);
+
+    if (ctx == NULL) {
+        v->not_found = 1;
+        return NGX_OK;
+    }
+
+    v->valid = 1;
+    v->no_cacheable = 0;
+    v->not_found = 0;
+    v->len = ctx->host.len;
+    v->data = ctx->host.data;
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_stream_ssl_preread_add_variables(ngx_conf_t *cf)
+{
+    ngx_stream_variable_t  *var, *v;
+
+    for (v = ngx_stream_ssl_preread_vars; v->name.len; v++) {
+        var = ngx_stream_add_variable(cf, &v->name, v->flags);
+        if (var == NULL) {
+            return NGX_ERROR;
+        }
+
+        var->get_handler = v->get_handler;
+        var->data = v->data;
+    }
+
+    return NGX_OK;
+}
+
+
+static void *
+ngx_stream_ssl_preread_create_srv_conf(ngx_conf_t *cf)
+{
+    ngx_stream_ssl_preread_srv_conf_t  *conf;
+
+    conf = ngx_pcalloc(cf->pool, sizeof(ngx_stream_ssl_preread_srv_conf_t));
+    if (conf == NULL) {
+        return NULL;
+    }
+
+    conf->enabled = NGX_CONF_UNSET;
+
+    return conf;
+}
+
+
+static char *
+ngx_stream_ssl_preread_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child)
+{
+    ngx_stream_ssl_preread_srv_conf_t *prev = parent;
+    ngx_stream_ssl_preread_srv_conf_t *conf = child;
+
+    ngx_conf_merge_value(conf->enabled, prev->enabled, 0);
+
+    return NGX_CONF_OK;
+}
+
+
+static ngx_int_t
+ngx_stream_ssl_preread_init(ngx_conf_t *cf)
+{
+    ngx_stream_handler_pt        *h;
+    ngx_stream_core_main_conf_t  *cmcf;
+
+    cmcf = ngx_stream_conf_get_module_main_conf(cf, ngx_stream_core_module);
+
+    h = ngx_array_push(&cmcf->phases[NGX_STREAM_PREREAD_PHASE].handlers);
+    if (h == NULL) {
+        return NGX_ERROR;
+    }
+
+    *h = ngx_stream_ssl_preread_handler;
+
+    return NGX_OK;
+}


From vl at nginx.com  Thu Sep 15 13:18:39 2016
From: vl at nginx.com (Vladimir Homutov)
Date: Thu, 15 Sep 2016 13:18:39 +0000
Subject: [nginx] Stream ssl_preread: removed internal macro.
Message-ID: <hg.e83540f825cd.1473945519.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/e83540f825cd
branches:  
changeset: 6696:e83540f825cd
user:      Vladimir Homutov <vl at nginx.com>
date:      Thu Sep 15 15:36:02 2016 +0300
description:
Stream ssl_preread: removed internal macro.

The ngx_log_debug() macro is internal and should not be used.

diffstat:

 src/stream/ngx_stream_ssl_preread_module.c |  12 ++++++------
 1 files changed, 6 insertions(+), 6 deletions(-)

diffs (33 lines):

diff -r 060d71292b69 -r e83540f825cd src/stream/ngx_stream_ssl_preread_module.c
--- a/src/stream/ngx_stream_ssl_preread_module.c	Thu Sep 15 14:56:51 2016 +0300
+++ b/src/stream/ngx_stream_ssl_preread_module.c	Thu Sep 15 15:36:02 2016 +0300
@@ -137,14 +137,14 @@ ngx_stream_ssl_preread_handler(ngx_strea
     while (last - p >= 5) {
 
         if (p[0] != 0x16) {
-            ngx_log_debug(NGX_LOG_DEBUG_STREAM, ctx->log, 0,
-                          "ssl preread: not a handshake");
+            ngx_log_debug0(NGX_LOG_DEBUG_STREAM, ctx->log, 0,
+                           "ssl preread: not a handshake");
             return NGX_DECLINED;
         }
 
         if (p[1] != 3 || p[2] == 0) {
-            ngx_log_debug(NGX_LOG_DEBUG_STREAM, ctx->log, 0,
-                          "ssl preread: unsupported SSL version");
+            ngx_log_debug0(NGX_LOG_DEBUG_STREAM, ctx->log, 0,
+                           "ssl preread: unsupported SSL version");
             return NGX_DECLINED;
         }
 
@@ -230,8 +230,8 @@ ngx_stream_ssl_preread_parse_record(ngx_
 
         case sw_header:
             if (p[0] != 1) {
-                ngx_log_debug(NGX_LOG_DEBUG_STREAM, ctx->log, 0,
-                              "ssl preread: not a client hello");
+                ngx_log_debug0(NGX_LOG_DEBUG_STREAM, ctx->log, 0,
+                               "ssl preread: not a client hello");
                 return NGX_DECLINED;
             }
 


From arut at nginx.com  Thu Sep 15 14:38:18 2016
From: arut at nginx.com (Roman Arutyunyan)
Date: Thu, 15 Sep 2016 14:38:18 +0000
Subject: [njs] Fixed compilation of nginx njs module without stream or http.
Message-ID: <hg.16cc1aaf4e47.1473950298.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/16cc1aaf4e47
branches:  
changeset: 174:16cc1aaf4e47
user:      Roman Arutyunyan <arut at nginx.com>
date:      Thu Sep 15 17:34:34 2016 +0300
description:
Fixed compilation of nginx njs module without stream or http.

Additionally, support for nginx versions without dynamic modules is now dropped.

diffstat:

 nginx/config |  14 +++-----------
 1 files changed, 3 insertions(+), 11 deletions(-)

diffs (38 lines):

diff -r 1745fcf4a5cc -r 16cc1aaf4e47 nginx/config
--- a/nginx/config	Thu Sep 15 10:52:01 2016 +0300
+++ b/nginx/config	Thu Sep 15 17:34:34 2016 +0300
@@ -1,6 +1,6 @@
 ngx_addon_name="ngx_js_module"
 
-if test -n "$ngx_module_link"; then
+if [ $HTTP != NO ]; then
     ngx_module_type=HTTP
     ngx_module_name=ngx_http_js_module
     ngx_module_incs="$ngx_addon_dir/../nxt $ngx_addon_dir/../njs"
@@ -8,7 +8,9 @@ if test -n "$ngx_module_link"; then
     ngx_module_libs="PCRE $ngx_addon_dir/../build/libnjs.a -lm"
 
     . auto/module
+fi
 
+if [ $STREAM != NO ]; then
     ngx_module_type=STREAM
     ngx_module_name=ngx_stream_js_module
     ngx_module_incs="$ngx_addon_dir/../nxt $ngx_addon_dir/../njs"
@@ -16,16 +18,6 @@ if test -n "$ngx_module_link"; then
     ngx_module_libs="PCRE $ngx_addon_dir/../build/libnjs.a -lm"
 
     . auto/module
-else
-    USE_PCRE=YES
-
-    HTTP_MODULES="$HTTP_MODULES ngx_http_js_module"
-    STREAM_MODULES="$STREAM_MODULES ngx_stream_js_module"
-    CORE_INCS="$CORE_INCS $ngx_addon_dir/../nxt $ngx_addon_dir/../njs"
-    NGX_ADDON_SRCS="$NGX_ADDON_SRCS \
-                    $ngx_addon_dir/ngx_http_js_module.c \
-                    $ngx_addon_dir/ngx_stream_js_module.c"
-    CORE_LIBS="$CORE_LIBS $ngx_addon_dir/../build/libnjs.a -lm"
 fi
 
 LINK_DEPS="$LINK_DEPS $ngx_addon_dir/../build/libnjs.a"


From jeppojeps at gmail.com  Fri Sep 16 13:17:38 2016
From: jeppojeps at gmail.com (Antonio Nappa)
Date: Fri, 16 Sep 2016 15:17:38 +0200
Subject: Hook Event After Accept
Message-ID: <CAOJuYzVFnQyQL0xCGQqn+Hn+-iQ2+=XtS5H5rnP93s9Di=WG0Q@mail.gmail.com>

Hello,

I have a plugin doing the following:

- go through active connections and detect the one listening on a
particular port (e.g. 443)
- register a read event on a different fd that I got with dup(fd of
connection on 443) with a custom handler (I don't want to interrupt the
normal handshake just watch the data passing).

My issue is that when the event is fired there is no data to read on the
duped socket because the accept on the original one hasn't happened, how
can I register the event to fire up after the accept?
I have tried to restore the event in the handler but I got a file exists
error. I have also tried to have a while(1) waiting on the original
connection to become ready, but of course the while(1) is blocking so no
luck with that.

Thanks
Jeppo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160916/c566ff8b/attachment.html>

From alessandro at ghedini.me  Fri Sep 16 13:44:38 2016
From: alessandro at ghedini.me (Alessandro Ghedini)
Date: Fri, 16 Sep 2016 14:44:38 +0100
Subject: [PATCH] Use int64 and int32 types with GCC atomics
Message-ID: <64afed0e819b9af65e7a.1474033326@pinky.local>

# HG changeset patch
# User Alessandro Ghedini <alessandro at ghedini.me>
# Date 1474033303 -3600
#      Fri Sep 16 14:41:43 2016 +0100
# Node ID 64afed0e819b9af65e7afa89baad3313f1db18d4
# Parent  e83540f825cd8c936f4f7f1e0336279d66446606
Use int64 and int32 types with GCC atomics

It's not quite clear if this was done on purpose, but it seems a good
idea to use appropriately sized types when GCC atomics are used.

diff -r e83540f825cd -r 64afed0e819b src/os/unix/ngx_atomic.h
--- a/src/os/unix/ngx_atomic.h	Thu Sep 15 15:36:02 2016 +0300
+++ b/src/os/unix/ngx_atomic.h	Fri Sep 16 14:41:43 2016 +0100
@@ -94,13 +94,18 @@
 
 #define NGX_HAVE_ATOMIC_OPS  1
 
-typedef long                        ngx_atomic_int_t;
-typedef unsigned long               ngx_atomic_uint_t;
+#if (NGX_PTR_SIZE == 8)
 
-#if (NGX_PTR_SIZE == 8)
+typedef int64_t                     ngx_atomic_int_t;
+typedef uint64_t                    ngx_atomic_uint_t;
 #define NGX_ATOMIC_T_LEN            (sizeof("-9223372036854775808") - 1)
+
 #else
+
+typedef int32_t                     ngx_atomic_int_t;
+typedef uint32_t                    ngx_atomic_uint_t;
 #define NGX_ATOMIC_T_LEN            (sizeof("-2147483648") - 1)
+
 #endif
 
 typedef volatile ngx_atomic_uint_t  ngx_atomic_t;


From vl at nginx.com  Fri Sep 16 14:12:18 2016
From: vl at nginx.com (Vladimir Homutov)
Date: Fri, 16 Sep 2016 14:12:18 +0000
Subject: [nginx] Upstream hash: fixed missing upstream name initialization.
Message-ID: <hg.52367732bcbc.1474035138.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/52367732bcbc
branches:  
changeset: 6697:52367732bcbc
user:      Vladimir Homutov <vl at nginx.com>
date:      Fri Sep 16 15:13:24 2016 +0300
description:
Upstream hash: fixed missing upstream name initialization.

diffstat:

 src/http/modules/ngx_http_upstream_hash_module.c |  1 +
 src/stream/ngx_stream_upstream_hash_module.c     |  1 +
 2 files changed, 2 insertions(+), 0 deletions(-)

diffs (22 lines):

diff -r e83540f825cd -r 52367732bcbc src/http/modules/ngx_http_upstream_hash_module.c
--- a/src/http/modules/ngx_http_upstream_hash_module.c	Thu Sep 15 15:36:02 2016 +0300
+++ b/src/http/modules/ngx_http_upstream_hash_module.c	Fri Sep 16 15:13:24 2016 +0300
@@ -571,6 +571,7 @@ ngx_http_upstream_get_chash_peer(ngx_pee
         hp->tries++;
 
         if (hp->tries >= points->number) {
+            pc->name = hp->rrp.peers->name;
             ngx_http_upstream_rr_peers_unlock(hp->rrp.peers);
             return NGX_BUSY;
         }
diff -r e83540f825cd -r 52367732bcbc src/stream/ngx_stream_upstream_hash_module.c
--- a/src/stream/ngx_stream_upstream_hash_module.c	Thu Sep 15 15:36:02 2016 +0300
+++ b/src/stream/ngx_stream_upstream_hash_module.c	Fri Sep 16 15:13:24 2016 +0300
@@ -572,6 +572,7 @@ ngx_stream_upstream_get_chash_peer(ngx_p
         hp->tries++;
 
         if (hp->tries >= points->number) {
+            pc->name = hp->rrp.peers->name;
             ngx_stream_upstream_rr_peers_unlock(hp->rrp.peers);
             return NGX_BUSY;
         }


From igor at sysoev.ru  Fri Sep 16 14:54:27 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Fri, 16 Sep 2016 14:54:27 +0000
Subject: [njs] A fix of use-after-free memory bug.
Message-ID: <hg.70e3b2fc7a8d.1474037667.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/70e3b2fc7a8d
branches:  
changeset: 175:70e3b2fc7a8d
user:      Igor Sysoev <igor at sysoev.ru>
date:      Thu Sep 15 17:37:52 2016 +0300
description:
A fix of use-after-free memory bug.

diffstat:

 njs/njs_vm.c |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (19 lines):

diff -r 16cc1aaf4e47 -r 70e3b2fc7a8d njs/njs_vm.c
--- a/njs/njs_vm.c	Thu Sep 15 17:34:34 2016 +0300
+++ b/njs/njs_vm.c	Thu Sep 15 17:37:52 2016 +0300
@@ -2588,7 +2588,6 @@ njs_vmcode_continuation(njs_vm_t *vm, nj
         skip = frame->skip;
 
         vm->frame = njs_function_previous_frame(frame);
-        (void) njs_function_frame_free(vm, frame);
 
         /*
          * If a retval is in a callee arguments scope it
@@ -2606,6 +2605,7 @@ njs_vmcode_continuation(njs_vm_t *vm, nj
         }
 
         vm->current = cont->return_address;
+        (void) njs_function_frame_free(vm, frame);
 
         return 0;
 


From igor at sysoev.ru  Fri Sep 16 14:54:29 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Fri, 16 Sep 2016 14:54:29 +0000
Subject: [njs] Returning an intptr_t result by rbtree comparison function
Message-ID: <hg.28bc6acceb0e.1474037669.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/28bc6acceb0e
branches:  
changeset: 176:28bc6acceb0e
user:      Igor Sysoev <igor at sysoev.ru>
date:      Thu Sep 15 18:00:55 2016 +0300
description:
Returning an intptr_t result by rbtree comparison function
decreases overhead required to implement correct addresses
comparison without result truncation.

diffstat:

 nxt/nxt_mem_cache_pool.c    |  4 ++--
 nxt/nxt_rbtree.c            |  8 ++++----
 nxt/nxt_rbtree.h            |  2 +-
 nxt/test/rbtree_unit_test.c |  7 +++----
 4 files changed, 10 insertions(+), 11 deletions(-)

diffs (96 lines):

diff -r 70e3b2fc7a8d -r 28bc6acceb0e nxt/nxt_mem_cache_pool.c
--- a/nxt/nxt_mem_cache_pool.c	Thu Sep 15 17:37:52 2016 +0300
+++ b/nxt/nxt_mem_cache_pool.c	Thu Sep 15 18:00:55 2016 +0300
@@ -131,7 +131,7 @@ static nxt_mem_cache_block_t *
     nxt_mem_cache_alloc_cluster(nxt_mem_cache_pool_t *pool);
 static void *nxt_mem_cache_alloc_large(nxt_mem_cache_pool_t *pool,
     size_t alignment, size_t size);
-static nxt_int_t nxt_mem_cache_rbtree_compare(nxt_rbtree_node_t *node1,
+static intptr_t nxt_mem_cache_rbtree_compare(nxt_rbtree_node_t *node1,
     nxt_rbtree_node_t *node2);
 static nxt_mem_cache_block_t *nxt_mem_cache_find_block(nxt_rbtree_t *tree,
     u_char *p);
@@ -597,7 +597,7 @@ nxt_mem_cache_alloc_large(nxt_mem_cache_
 }
 
 
-static nxt_int_t
+static intptr_t
 nxt_mem_cache_rbtree_compare(nxt_rbtree_node_t *node1, nxt_rbtree_node_t *node2)
 {
     nxt_mem_cache_block_t  *block1, *block2;
diff -r 70e3b2fc7a8d -r 28bc6acceb0e nxt/nxt_rbtree.c
--- a/nxt/nxt_rbtree.c	Thu Sep 15 17:37:52 2016 +0300
+++ b/nxt/nxt_rbtree.c	Thu Sep 15 18:00:55 2016 +0300
@@ -176,7 +176,7 @@ nxt_rbtree_insert_fixup(nxt_rbtree_node_
 nxt_rbtree_node_t *
 nxt_rbtree_find(nxt_rbtree_t *tree, nxt_rbtree_part_t *part)
 {
-    nxt_int_t             n;
+    intptr_t              n;
     nxt_rbtree_node_t     *node, *next, *sentinel;
     nxt_rbtree_compare_t  compare;
 
@@ -210,7 +210,7 @@ nxt_rbtree_find(nxt_rbtree_t *tree, nxt_
 nxt_rbtree_node_t *
 nxt_rbtree_find_less_or_equal(nxt_rbtree_t *tree, nxt_rbtree_part_t *part)
 {
-    nxt_int_t             n;
+    intptr_t              n;
     nxt_rbtree_node_t     *node, *retval, *next, *sentinel;
     nxt_rbtree_compare_t  compare;
 
@@ -247,7 +247,7 @@ nxt_rbtree_find_less_or_equal(nxt_rbtree
 nxt_rbtree_node_t *
 nxt_rbtree_find_greater_or_equal(nxt_rbtree_t *tree, nxt_rbtree_part_t *part)
 {
-    nxt_int_t             n;
+    intptr_t              n;
     nxt_rbtree_node_t     *node, *retval, *next, *sentinel;
     nxt_rbtree_compare_t  compare;
 
@@ -284,7 +284,7 @@ nxt_rbtree_find_greater_or_equal(nxt_rbt
 void
 nxt_rbtree_delete(nxt_rbtree_t *tree, nxt_rbtree_part_t *part)
 {
-    nxt_uint_t         color;
+    uint8_t            color;
     nxt_rbtree_node_t  *node, *sentinel, *subst, *child;
 
     node = (nxt_rbtree_node_t *) part;
diff -r 70e3b2fc7a8d -r 28bc6acceb0e nxt/nxt_rbtree.h
--- a/nxt/nxt_rbtree.h	Thu Sep 15 17:37:52 2016 +0300
+++ b/nxt/nxt_rbtree.h	Thu Sep 15 18:00:55 2016 +0300
@@ -39,7 +39,7 @@ typedef struct {
 } nxt_rbtree_t;
 
 
-typedef nxt_int_t (*nxt_rbtree_compare_t)(nxt_rbtree_node_t *node1,
+typedef intptr_t (*nxt_rbtree_compare_t)(nxt_rbtree_node_t *node1,
     nxt_rbtree_node_t *node2);
 
 
diff -r 70e3b2fc7a8d -r 28bc6acceb0e nxt/test/rbtree_unit_test.c
--- a/nxt/test/rbtree_unit_test.c	Thu Sep 15 17:37:52 2016 +0300
+++ b/nxt/test/rbtree_unit_test.c	Thu Sep 15 18:00:55 2016 +0300
@@ -20,7 +20,7 @@ typedef struct {
 } nxt_rbtree_test_t;
 
 
-static nxt_int_t rbtree_unit_test_comparison(nxt_rbtree_node_t *node1,
+static intptr_t rbtree_unit_test_comparison(nxt_rbtree_node_t *node1,
     nxt_rbtree_node_t *node2);
 static nxt_int_t rbtree_unit_test_compare(uint32_t key1, uint32_t key2);
 static int nxt_cdecl rbtree_unit_test_sort_cmp(const void *one,
@@ -132,9 +132,8 @@ fail:
 }
 
 
-static nxt_int_t
-rbtree_unit_test_comparison(nxt_rbtree_node_t *node1,
-    nxt_rbtree_node_t *node2)
+static intptr_t
+rbtree_unit_test_comparison(nxt_rbtree_node_t *node1, nxt_rbtree_node_t *node2)
 {
     nxt_rbtree_test_t  *item1, *item2;
 


From mdounin at mdounin.ru  Fri Sep 16 17:16:41 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 16 Sep 2016 20:16:41 +0300
Subject: [PATCH] Use int64 and int32 types with GCC atomics
In-Reply-To: <64afed0e819b9af65e7a.1474033326@pinky.local>
References: <64afed0e819b9af65e7a.1474033326@pinky.local>
Message-ID: <20160916171641.GB35798@mdounin.ru>

Hello!

On Fri, Sep 16, 2016 at 02:44:38PM +0100, Alessandro Ghedini wrote:

> # HG changeset patch
> # User Alessandro Ghedini <alessandro at ghedini.me>
> # Date 1474033303 -3600
> #      Fri Sep 16 14:41:43 2016 +0100
> # Node ID 64afed0e819b9af65e7afa89baad3313f1db18d4
> # Parent  e83540f825cd8c936f4f7f1e0336279d66446606
> Use int64 and int32 types with GCC atomics
> 
> It's not quite clear if this was done on purpose, but it seems a good
> idea to use appropriately sized types when GCC atomics are used.

The "long" is expected to be one of the most native types 
available, and I see no problem with using it as long as it is 
understood by builtins used (it is).  It is also in line with the 
configure test we use to find it out if GCC atomic ops are 
present:

    ngx_feature_test="long  n = 0;
                      if (!__sync_bool_compare_and_swap(&n, 0, 1))
                          return 1;
                      if (__sync_fetch_and_add(&n, 1) != 1)
                          return 1;
                      if (n != 2)
                          return 1;
                      __sync_synchronize();"

Changing the code to diverge from the test as your patch doesn't 
looks like a good idea, and I would rather preserve this as 
is unless there are better reasons than "seems a good idea".

Note well that we anyway use the long for ngx_atomic*_t when using 
libatomic where this is the only option available.  That is, 
atomic types may not match pointer sizes, and nginx is expected to 
deal with this anyway.

-- 
Maxim Dounin
http://nginx.org/


From piotrsikora at google.com  Fri Sep 16 21:43:49 2016
From: piotrsikora at google.com (Piotr Sikora)
Date: Fri, 16 Sep 2016 14:43:49 -0700
Subject: [PATCH 2 of 2] Core: add ngx_atomic_store() and ngx_atomic_load()
In-Reply-To: <20160912155003.GB1527@mdounin.ru>
References: <2f2ec92c3af93c11e195.1471480171@piotrsikora.sfo.corp.google.com>
 <40765d8ee4dd29089b0e.1471480172@piotrsikora.sfo.corp.google.com>
 <20160912155003.GB1527@mdounin.ru>
Message-ID: <CAF-CG+LNctXvCh-B_RWFHEZO8_qrh+A-LmQ=p94j0T3=mtEHHg@mail.gmail.com>

Hey Maxim,

> The "*(lock) == 0" check here is just an optimization, it only
> ensures that the lock is likely to succed.

Yes, and use of the ngx_atomic_load() doesn't affect that.

Namely, in the micro-benchmarks I did (heavy contention - 100 threads
trying to acquire lock, update value, release lock in a loop), there
is no performance lose while using ngx_atomic_load() on x86_64,
whereas removing this optimization resulted in 3x worse performance.

> If the
> check returns a wrong result due to non-atomic load - this won't
> do any harm.

It's not just wrong result but a "data race", which leads to undefined
behavior (at least according to C++11).

Best regards,
Piotr Sikora


From hongzhidao at gmail.com  Sat Sep 17 00:25:54 2016
From: hongzhidao at gmail.com (=?UTF-8?B?5rSq5b+X6YGT?=)
Date: Sat, 17 Sep 2016 08:25:54 +0800
Subject: nginx timer in log module
Message-ID: <CAJuLd0how_J6e6iLraxmaw-Aonsd7jO9FrXn-wH2mPKpnVJ=fw@mail.gmail.com>

Hi

As the following codes.


buffer->event->handler = ngx_http_log_flush_handler;


static void
ngx_http_log_flush_handler(ngx_event_t *ev)
{
    ...
    if (ev->timedout) {
        ngx_http_log_flush(ev->data, ev->log);
        return;
    }
    ...
}


static void
ngx_http_log_flush(ngx_open_file_t *file, ngx_log_t *log)
{
    ...
    if (buffer->event && buffer->event->timer_set) {
        ngx_del_timer(buffer->event);
    }
}


I find there are two functions explicitly call event handler.
1. ngx_event_cancel_timers happened as long as worker process quits.
2. ngx_event_expire_timers in ngx_process_events_and_timers.

And they remove timer from timer rbtree, then set timer_set zero.
After that they call event->handler.

So why we call ngx_del_timer in ngx_http_log_flush again?

Thanks.

B.R~
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160917/4d7ae7b0/attachment.html>

From hongzhidao at gmail.com  Sat Sep 17 01:43:21 2016
From: hongzhidao at gmail.com (=?UTF-8?B?5rSq5b+X6YGT?=)
Date: Sat, 17 Sep 2016 09:43:21 +0800
Subject: nginx timer in log module
In-Reply-To: <CAJuLd0how_J6e6iLraxmaw-Aonsd7jO9FrXn-wH2mPKpnVJ=fw@mail.gmail.com>
References: <CAJuLd0how_J6e6iLraxmaw-Aonsd7jO9FrXn-wH2mPKpnVJ=fw@mail.gmail.com>
Message-ID: <CAJuLd0hAOo7V6nrQDRRyuac2GKh_y1jZWYbq0b8yFMnNzQxTHA@mail.gmail.com>

Well, I get it.

By the way, If I want to add a timer, it can do something like crontab in
worker process.
For example, we collect data in worker process and dump the data every 5
minutes.

event->cancelable = 1;

In event handler, the codes like the following:

static void
ngx_http_test_timer_handler(ngx_event_t *ev)
{

     if (event->timedout) {
         /* todo timedout */
         return;
     }

     if (ngx_terminate || ngx_exiting) {
         /* I'm not sure whether we need to determine the condition */
     }

     /* I think we needn't to explicitly remove the event timer */
     #if 0
     if (ev->timer_set) {
        ngx_del_timer(ev);
    }
    #endif
}

Thanks again.

B.R.~

2016-09-17 8:25 GMT+08:00 ??? <hongzhidao at gmail.com>:

> Hi
>
> As the following codes.
>
>
> buffer->event->handler = ngx_http_log_flush_handler;
>
>
> static void
> ngx_http_log_flush_handler(ngx_event_t *ev)
> {
>     ...
>     if (ev->timedout) {
>         ngx_http_log_flush(ev->data, ev->log);
>         return;
>     }
>     ...
> }
>
>
> static void
> ngx_http_log_flush(ngx_open_file_t *file, ngx_log_t *log)
> {
>     ...
>     if (buffer->event && buffer->event->timer_set) {
>         ngx_del_timer(buffer->event);
>     }
> }
>
>
> I find there are two functions explicitly call event handler.
> 1. ngx_event_cancel_timers happened as long as worker process quits.
> 2. ngx_event_expire_timers in ngx_process_events_and_timers.
>
> And they remove timer from timer rbtree, then set timer_set zero.
> After that they call event->handler.
>
> So why we call ngx_del_timer in ngx_http_log_flush again?
>
> Thanks.
>
> B.R~
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160917/cfae69c9/attachment.html>

From mdounin at mdounin.ru  Sat Sep 17 02:48:24 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sat, 17 Sep 2016 05:48:24 +0300
Subject: nginx timer in log module
In-Reply-To: <CAJuLd0how_J6e6iLraxmaw-Aonsd7jO9FrXn-wH2mPKpnVJ=fw@mail.gmail.com>
References: <CAJuLd0how_J6e6iLraxmaw-Aonsd7jO9FrXn-wH2mPKpnVJ=fw@mail.gmail.com>
Message-ID: <20160917024824.GE35798@mdounin.ru>

Hello!

On Sat, Sep 17, 2016 at 08:25:54AM +0800, ??? wrote:

[...]

> static void
> ngx_http_log_flush(ngx_open_file_t *file, ngx_log_t *log)
> {
>     ...
>     if (buffer->event && buffer->event->timer_set) {
>         ngx_del_timer(buffer->event);
>     }
> }
> 
> 
> I find there are two functions explicitly call event handler.
> 1. ngx_event_cancel_timers happened as long as worker process quits.
> 2. ngx_event_expire_timers in ngx_process_events_and_timers.
> 
> And they remove timer from timer rbtree, then set timer_set zero.
> After that they call event->handler.
> 
> So why we call ngx_del_timer in ngx_http_log_flush again?

The ngx_http_log_flush() function is also used as a flush handler 
for the log file, notably called while reopening logs.  In this 
case the timer can be set when ngx_http_log_flush() is called and 
should be deleted - and this is what the code in question does.

-- 
Maxim Dounin
http://nginx.org/


From hongzhidao at gmail.com  Sat Sep 17 06:42:25 2016
From: hongzhidao at gmail.com (=?UTF-8?B?5rSq5b+X6YGT?=)
Date: Sat, 17 Sep 2016 14:42:25 +0800
Subject: nginx timer in log module
In-Reply-To: <20160917024824.GE35798@mdounin.ru>
References: <CAJuLd0how_J6e6iLraxmaw-Aonsd7jO9FrXn-wH2mPKpnVJ=fw@mail.gmail.com>
 <20160917024824.GE35798@mdounin.ru>
Message-ID: <CAJuLd0hgshd5z8xLnR4k5FT5s2b+22CamMU5gojmfmgZn1VA3g@mail.gmail.com>

Thank you for your reply, I get it now.

In other case such as I showed above, it seems we needn't do like the
following ?

 if (ev->timer_set) {
      ngx_del_timer(ev);
}

2016-09-17 10:48 GMT+08:00 Maxim Dounin <mdounin at mdounin.ru>:

> Hello!
>
> On Sat, Sep 17, 2016 at 08:25:54AM +0800, ??? wrote:
>
> [...]
>
> > static void
> > ngx_http_log_flush(ngx_open_file_t *file, ngx_log_t *log)
> > {
> >     ...
> >     if (buffer->event && buffer->event->timer_set) {
> >         ngx_del_timer(buffer->event);
> >     }
> > }
> >
> >
> > I find there are two functions explicitly call event handler.
> > 1. ngx_event_cancel_timers happened as long as worker process quits.
> > 2. ngx_event_expire_timers in ngx_process_events_and_timers.
> >
> > And they remove timer from timer rbtree, then set timer_set zero.
> > After that they call event->handler.
> >
> > So why we call ngx_del_timer in ngx_http_log_flush again?
>
> The ngx_http_log_flush() function is also used as a flush handler
> for the log file, notably called while reopening logs.  In this
> case the timer can be set when ngx_http_log_flush() is called and
> should be deleted - and this is what the code in question does.
>
> --
> Maxim Dounin
> http://nginx.org/
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160917/cc01c6b5/attachment.html>

From mdounin at mdounin.ru  Sat Sep 17 06:59:20 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sat, 17 Sep 2016 09:59:20 +0300
Subject: [PATCH 2 of 2] Core: add ngx_atomic_store() and ngx_atomic_load()
In-Reply-To: <CAF-CG+LNctXvCh-B_RWFHEZO8_qrh+A-LmQ=p94j0T3=mtEHHg@mail.gmail.com>
References: <2f2ec92c3af93c11e195.1471480171@piotrsikora.sfo.corp.google.com>
 <40765d8ee4dd29089b0e.1471480172@piotrsikora.sfo.corp.google.com>
 <20160912155003.GB1527@mdounin.ru>
 <CAF-CG+LNctXvCh-B_RWFHEZO8_qrh+A-LmQ=p94j0T3=mtEHHg@mail.gmail.com>
Message-ID: <20160917065920.GG35798@mdounin.ru>

Hello!

On Fri, Sep 16, 2016 at 02:43:49PM -0700, Piotr Sikora wrote:

> Hey Maxim,
> 
> > The "*(lock) == 0" check here is just an optimization, it only
> > ensures that the lock is likely to succed.
> 
> Yes, and use of the ngx_atomic_load() doesn't affect that.
> 
> Namely, in the micro-benchmarks I did (heavy contention - 100 threads
> trying to acquire lock, update value, release lock in a loop), there
> is no performance lose while using ngx_atomic_load() on x86_64,
> whereas removing this optimization resulted in 3x worse performance.

The question is: why ngx_atomic_load() is at all needed.  For now 
the only reason I see is "because ThreadSanitizer complains".

> > If the
> > check returns a wrong result due to non-atomic load - this won't
> > do any harm.
> 
> It's not just wrong result but a "data race", which leads to undefined
> behavior (at least according to C++11).

As far as I understand, from C11 point of view undefined behaviour 
is still here even with ngx_atomic_load(), as it's not an atomic 
operation defined by C11.

I can understand introducing load/store as a part of introducing 
C11 atomic operations support.  But I see no reason why it should 
be done separately.

On the other hand, C11 doesn't seem to require explicit use of 
load/store operations, "*(lock)" will work just fine, though 
probably will not be optimal.

Just for the record, below is a quick-and-dirty patch to introduce 
?11 atomic operations support (mostly untested).

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1474094645 -10800
#      Sat Sep 17 09:44:05 2016 +0300
# Node ID b79d4fd920eaa056103f68d311452b8cd6da833c
# Parent  9a4934f07bb47fbcf154ce275a04eb5dd1ba16be
C11 atomic operations.

diff --git a/auto/cc/conf b/auto/cc/conf
--- a/auto/cc/conf
+++ b/auto/cc/conf
@@ -178,6 +178,27 @@ if [ "$NGX_PLATFORM" != win32 ]; then
     fi
 
 
+    ngx_feature="C11 atomic operations"
+    ngx_feature_name=NGX_HAVE_C11_ATOMIC
+    ngx_feature_run=yes
+    ngx_feature_incs="#include <stdatomic.h>"
+    ngx_feature_path=
+    ngx_feature_libs=
+    ngx_feature_test="#ifndef ATOMIC_LONG_LOCK_FREE
+                      #error atomic_long is not lock-free
+                      #endif
+                      atomic_long  n = ATOMIC_VAR_INIT(0);
+                      long         tmp = 0;
+                      if (!atomic_compare_exchange_strong(&n, &tmp, 1))
+                          return 1;
+                      if (atomic_fetch_add(&n, 1) != 1)
+                          return 1;
+                      if (atomic_load(&n) != 2)
+                          return 1;
+                      atomic_thread_fence(memory_order_acq_rel);"
+    . auto/feature
+
+
     ngx_feature="gcc builtin atomic operations"
     ngx_feature_name=NGX_HAVE_GCC_ATOMIC
     ngx_feature_run=yes
diff --git a/src/os/unix/ngx_atomic.h b/src/os/unix/ngx_atomic.h
--- a/src/os/unix/ngx_atomic.h
+++ b/src/os/unix/ngx_atomic.h
@@ -88,6 +88,44 @@ typedef uint32_t                    ngx_
 typedef volatile ngx_atomic_uint_t  ngx_atomic_t;
 
 
+#elif (NGX_HAVE_C11_ATOMIC)
+
+/* C11 atomic operations */
+
+#include <stdatomic.h>
+
+#define NGX_HAVE_ATOMIC_OPS  1
+
+typedef long                        ngx_atomic_int_t;
+typedef unsigned long               ngx_atomic_uint_t;
+
+#if (NGX_PTR_SIZE == 8)
+#define NGX_ATOMIC_T_LEN            (sizeof("-9223372036854775808") - 1)
+#else
+#define NGX_ATOMIC_T_LEN            (sizeof("-2147483648") - 1)
+#endif
+
+typedef volatile atomic_ulong       ngx_atomic_t;
+
+ngx_inline ngx_atomic_uint_t
+ngx_atomic_cmp_set(ngx_atomic_t *lock, ngx_atomic_uint_t old,
+    ngx_atomic_uint_t set)
+{
+    return atomic_compare_exchange_strong(lock, &old, set);
+}
+
+#define ngx_atomic_fetch_add(value, add)                                      \
+    atomic_fetch_add(value, add)
+
+#define ngx_memory_barrier()        atomic_thread_fence(memory_order_seq_cst)
+
+#if ( __i386__ || __i386 || __amd64__ || __amd64 )
+#define ngx_cpu_pause()             __asm__ ("pause")
+#else
+#define ngx_cpu_pause()
+#endif
+
+
 #elif (NGX_HAVE_GCC_ATOMIC)
 
 /* GCC 4.1 builtin atomic operations */

-- 
Maxim Dounin
http://nginx.org/


From mdounin at mdounin.ru  Sat Sep 17 07:06:20 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sat, 17 Sep 2016 10:06:20 +0300
Subject: nginx timer in log module
In-Reply-To: <CAJuLd0hgshd5z8xLnR4k5FT5s2b+22CamMU5gojmfmgZn1VA3g@mail.gmail.com>
References: <CAJuLd0how_J6e6iLraxmaw-Aonsd7jO9FrXn-wH2mPKpnVJ=fw@mail.gmail.com>
 <20160917024824.GE35798@mdounin.ru>
 <CAJuLd0hgshd5z8xLnR4k5FT5s2b+22CamMU5gojmfmgZn1VA3g@mail.gmail.com>
Message-ID: <20160917070620.GH35798@mdounin.ru>

Hello!

On Sat, Sep 17, 2016 at 02:42:25PM +0800, ??? wrote:

> Thank you for your reply, I get it now.
> 
> In other case such as I showed above, it seems we needn't do like the
> following ?
> 
>  if (ev->timer_set) {
>       ngx_del_timer(ev);
> }

Yes, as long as the code is in the timer handler it is not needed.  
Timers are called only once and they are always deleted before a 
timer handler is called.

-- 
Maxim Dounin
http://nginx.org/


From bjornar.ness at gmail.com  Sun Sep 18 12:11:14 2016
From: bjornar.ness at gmail.com (=?UTF-8?q?Bj=C3=B8rnar=20Ness?=)
Date: Sun, 18 Sep 2016 14:11:14 +0200
Subject: [PATCH] proxy-protocol dst variables and proxy-proxy-protocol
Message-ID: <1474200674-9906-1-git-send-email-bjornar.ness@gmail.com>

Introduce proxy_protocol_src/dst_addr/port and store proxy_protocol
header in ngx_connection struct. This enables proxy-proxy-protocol
in stream module if proxy_protocol is enabled on both listen and
outgoing, which should be the logical default.

Work in progress if/when this patch is accepted is proxy protocol
support for the mail module.
---
 src/core/ngx_connection.h                 |   7 +-
 src/core/ngx_proxy_protocol.c             | 111 ++++++++++++++++--------------
 src/http/modules/ngx_http_realip_module.c |   6 +-
 src/http/ngx_http_variables.c             |  60 ++++++++--------
 src/stream/ngx_stream_realip_module.c     |  10 +--
 src/stream/ngx_stream_variables.c         |  60 ++++++++--------
 6 files changed, 132 insertions(+), 122 deletions(-)

diff --git a/src/core/ngx_connection.h b/src/core/ngx_connection.h
index e484c81..c6e2337 100644
--- a/src/core/ngx_connection.h
+++ b/src/core/ngx_connection.h
@@ -148,8 +148,11 @@ struct ngx_connection_s {
     socklen_t           socklen;
     ngx_str_t           addr_text;
 
-    ngx_str_t           proxy_protocol_addr;
-    in_port_t           proxy_protocol_port;
+    ngx_str_t           proxy_protocol;
+    ngx_str_t           proxy_protocol_src_addr;
+    ngx_str_t           proxy_protocol_src_port;
+    ngx_str_t           proxy_protocol_dst_addr;
+    ngx_str_t           proxy_protocol_dst_port;
 
 #if (NGX_SSL)
     ngx_ssl_connection_t  *ssl;
diff --git a/src/core/ngx_proxy_protocol.c b/src/core/ngx_proxy_protocol.c
index 523ec35..04dae8b 100644
--- a/src/core/ngx_proxy_protocol.c
+++ b/src/core/ngx_proxy_protocol.c
@@ -13,8 +13,9 @@ u_char *
 ngx_proxy_protocol_read(ngx_connection_t *c, u_char *buf, u_char *last)
 {
     size_t     len;
-    u_char     ch, *p, *addr, *port;
-    ngx_int_t  n;
+    ngx_uint_t i, n;
+    u_char     ch, *p, *pp;
+    ngx_str_t  addr[2], port[2];
 
     p = buf;
     len = last - buf;
@@ -40,77 +41,79 @@ ngx_proxy_protocol_read(ngx_connection_t *c, u_char *buf, u_char *last)
     }
 
     p += 5;
-    addr = p;
 
-    for ( ;; ) {
-        if (p == last) {
-            goto invalid;
-        }
+    for (i=0;i<2;i++) {
+	addr[i].data = p;
 
-        ch = *p++;
+        for ( ;; ) {
+            if (p == last) {
+                goto invalid;
+            }
 
-        if (ch == ' ') {
-            break;
-        }
+            ch = *p++;
 
-        if (ch != ':' && ch != '.'
-            && (ch < 'a' || ch > 'f')
-            && (ch < 'A' || ch > 'F')
-            && (ch < '0' || ch > '9'))
-        {
-            goto invalid;
-        }
-    }
+            if (ch == ' ') {
+                break;
+            }
 
-    len = p - addr - 1;
-    c->proxy_protocol_addr.data = ngx_pnalloc(c->pool, len);
+            if (ch != ':' && ch != '.'
+                && (ch < 'a' || ch > 'f')
+                && (ch < 'A' || ch > 'F')
+                && (ch < '0' || ch > '9'))
+            {
+                goto invalid;
+            }
+        }
 
-    if (c->proxy_protocol_addr.data == NULL) {
-        return NULL;
+        addr[i].len = p - addr[i].data - 1;
     }
 
-    ngx_memcpy(c->proxy_protocol_addr.data, addr, len);
-    c->proxy_protocol_addr.len = len;
+    for (i=0;i<2;i++) {
+        port[i].data = p;
 
-    for ( ;; ) {
-        if (p == last) {
-            goto invalid;
-        }
+        for ( ;; ) {
+            if (p == last) {
+                goto invalid;
+            }
 
-        if (*p++ == ' ') {
-            break;
+            if ((*p++ == ' ' && i == 0) || (i == 1 && p != last && *p == CR)) {
+                break;
+            }
         }
-    }
 
-    port = p;
+        port[i].len = p - port[i].data - (1-i);
+        n = ngx_atoi(port[i].data, port[i].len);
 
-    for ( ;; ) {
-        if (p == last) {
+        if (n < 1 || n > 65535) {
             goto invalid;
         }
-
-        if (*p++ == ' ') {
-            break;
-        }
-    }
-
-    len = p - port - 1;
-
-    n = ngx_atoi(port, len);
-
-    if (n < 0 || n > 65535) {
-        goto invalid;
     }
 
-    c->proxy_protocol_port = (in_port_t) n;
-
-    ngx_log_debug2(NGX_LOG_DEBUG_CORE, c->log, 0,
-                   "PROXY protocol address: %V %i", &c->proxy_protocol_addr, n);
-
 skip:
 
     for ( /* void */ ; p < last - 1; p++) {
         if (p[0] == CR && p[1] == LF) {
+            len = p - buf - 6;
+            c->proxy_protocol.data = ngx_pnalloc(c->pool, len);
+            ngx_memcpy(c->proxy_protocol.data, buf + 6, len);
+            c->proxy_protocol.len = len;
+
+            if (i) { /* not UNKNOWN */
+                pp = c->proxy_protocol.data;
+                c->proxy_protocol_src_addr.data = pp + (addr[0].data - buf - 6);
+                c->proxy_protocol_src_port.data = pp + (port[0].data - buf - 6);
+                c->proxy_protocol_dst_addr.data = pp + (addr[1].data - buf - 6);
+                c->proxy_protocol_dst_port.data = pp + (port[1].data - buf - 6);
+                c->proxy_protocol_src_addr.len = addr[0].len;
+                c->proxy_protocol_src_port.len = port[0].len;
+                c->proxy_protocol_dst_addr.len = addr[1].len;
+                c->proxy_protocol_dst_port.len = port[1].len;
+
+                ngx_log_debug2(NGX_LOG_DEBUG_CORE, c->log, 0,
+                               "PROXY protocol address: %V %V",
+                               &addr[0], &port[0]);
+            }
+
             return p + 2;
         }
     }
@@ -133,6 +136,10 @@ ngx_proxy_protocol_write(ngx_connection_t *c, u_char *buf, u_char *last)
         return NULL;
     }
 
+    if (c->proxy_protocol.len) {
+	return ngx_slprintf(buf, last, "PROXY %V" CRLF, &c->proxy_protocol);
+    }
+
     if (ngx_connection_local_sockaddr(c, NULL, 0) != NGX_OK) {
         return NULL;
     }
diff --git a/src/http/modules/ngx_http_realip_module.c b/src/http/modules/ngx_http_realip_module.c
index dba3c52..e2f7bdd 100644
--- a/src/http/modules/ngx_http_realip_module.c
+++ b/src/http/modules/ngx_http_realip_module.c
@@ -180,7 +180,7 @@ ngx_http_realip_handler(ngx_http_request_t *r)
 
     case NGX_HTTP_REALIP_PROXY:
 
-        value = &r->connection->proxy_protocol_addr;
+        value = &r->connection->proxy_protocol_src_addr;
 
         if (value->len == 0) {
             return NGX_DECLINED;
@@ -238,7 +238,9 @@ found:
         != NGX_DECLINED)
     {
         if (rlcf->type == NGX_HTTP_REALIP_PROXY) {
-            ngx_inet_set_port(addr.sockaddr, c->proxy_protocol_port);
+            ngx_inet_set_port(addr.sockaddr, (in_port_t)
+                              ngx_atoi(c->proxy_protocol_src_port.data,
+                                       c->proxy_protocol_src_port.len));
         }
 
         return ngx_http_realip_set_addr(r, &addr);
diff --git a/src/http/ngx_http_variables.c b/src/http/ngx_http_variables.c
index 7e65b2e..6071874 100644
--- a/src/http/ngx_http_variables.c
+++ b/src/http/ngx_http_variables.c
@@ -56,9 +56,7 @@ static ngx_int_t ngx_http_variable_remote_addr(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_http_variable_remote_port(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
-static ngx_int_t ngx_http_variable_proxy_protocol_addr(ngx_http_request_t *r,
-    ngx_http_variable_value_t *v, uintptr_t data);
-static ngx_int_t ngx_http_variable_proxy_protocol_port(ngx_http_request_t *r,
+static ngx_int_t ngx_http_variable_proxy_protocol_value(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_http_variable_server_addr(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data);
@@ -194,10 +192,28 @@ static ngx_http_variable_t  ngx_http_core_variables[] = {
     { ngx_string("remote_port"), NULL, ngx_http_variable_remote_port, 0, 0, 0 },
 
     { ngx_string("proxy_protocol_addr"), NULL,
-      ngx_http_variable_proxy_protocol_addr, 0, 0, 0 },
+      ngx_http_variable_proxy_protocol_value,
+      offsetof(ngx_connection_t, proxy_protocol_src_addr), 0, 0 },
+
+    { ngx_string("proxy_protocol_src_addr"), NULL,
+      ngx_http_variable_proxy_protocol_value,
+      offsetof(ngx_connection_t, proxy_protocol_src_addr), 0, 0 },
+
+    { ngx_string("proxy_protocol_dst_addr"), NULL,
+      ngx_http_variable_proxy_protocol_value,
+      offsetof(ngx_connection_t, proxy_protocol_dst_addr), 0, 0 },
 
     { ngx_string("proxy_protocol_port"), NULL,
-      ngx_http_variable_proxy_protocol_port, 0, 0, 0 },
+      ngx_http_variable_proxy_protocol_value,
+      offsetof(ngx_connection_t, proxy_protocol_src_port), 0, 0 },
+
+    { ngx_string("proxy_protocol_src_port"), NULL,
+      ngx_http_variable_proxy_protocol_value,
+      offsetof(ngx_connection_t, proxy_protocol_src_port), 0, 0 },
+
+    { ngx_string("proxy_protocol_dst_port"), NULL,
+      ngx_http_variable_proxy_protocol_value,
+      offsetof(ngx_connection_t, proxy_protocol_dst_port), 0, 0 },
 
     { ngx_string("server_addr"), NULL, ngx_http_variable_server_addr, 0, 0, 0 },
 
@@ -1224,40 +1240,22 @@ ngx_http_variable_remote_port(ngx_http_request_t *r,
 
 
 static ngx_int_t
-ngx_http_variable_proxy_protocol_addr(ngx_http_request_t *r,
+ngx_http_variable_proxy_protocol_value(ngx_http_request_t *r,
     ngx_http_variable_value_t *v, uintptr_t data)
 {
-    v->len = r->connection->proxy_protocol_addr.len;
-    v->valid = 1;
-    v->no_cacheable = 0;
-    v->not_found = 0;
-    v->data = r->connection->proxy_protocol_addr.data;
-
-    return NGX_OK;
-}
+    ngx_str_t   *a;
 
+    a = (ngx_str_t *) ((char *) r->connection + data);
 
-static ngx_int_t
-ngx_http_variable_proxy_protocol_port(ngx_http_request_t *r,
-    ngx_http_variable_value_t *v, uintptr_t data)
-{
-    ngx_uint_t  port;
+    if (a->data == NULL) {
+        return NGX_ERROR;
+    }
 
-    v->len = 0;
+    v->len = a->len;
     v->valid = 1;
     v->no_cacheable = 0;
     v->not_found = 0;
-
-    v->data = ngx_pnalloc(r->pool, sizeof("65535") - 1);
-    if (v->data == NULL) {
-        return NGX_ERROR;
-    }
-
-    port = r->connection->proxy_protocol_port;
-
-    if (port > 0 && port < 65536) {
-        v->len = ngx_sprintf(v->data, "%ui", port) - v->data;
-    }
+    v->data = a->data;
 
     return NGX_OK;
 }
diff --git a/src/stream/ngx_stream_realip_module.c b/src/stream/ngx_stream_realip_module.c
index 0740431..3380aab 100644
--- a/src/stream/ngx_stream_realip_module.c
+++ b/src/stream/ngx_stream_realip_module.c
@@ -108,7 +108,7 @@ ngx_stream_realip_handler(ngx_stream_session_t *s)
 
     c = s->connection;
 
-    if (c->proxy_protocol_addr.len == 0) {
+    if (c->proxy_protocol_src_addr.len == 0) {
         return NGX_DECLINED;
     }
 
@@ -116,14 +116,16 @@ ngx_stream_realip_handler(ngx_stream_session_t *s)
         return NGX_DECLINED;
     }
 
-    if (ngx_parse_addr(c->pool, &addr, c->proxy_protocol_addr.data,
-                       c->proxy_protocol_addr.len)
+    if (ngx_parse_addr(c->pool, &addr, c->proxy_protocol_src_addr.data,
+                       c->proxy_protocol_src_addr.len)
         != NGX_OK)
     {
         return NGX_DECLINED;
     }
 
-    ngx_inet_set_port(addr.sockaddr, c->proxy_protocol_port);
+    ngx_inet_set_port(addr.sockaddr, 
+                      (in_port_t) ngx_atoi(c->proxy_protocol_src_port.data,
+                                           c->proxy_protocol_src_port.len));
 
     return ngx_stream_realip_set_addr(s, &addr);
 }
diff --git a/src/stream/ngx_stream_variables.c b/src/stream/ngx_stream_variables.c
index aa5361d..0aa4f73 100644
--- a/src/stream/ngx_stream_variables.c
+++ b/src/stream/ngx_stream_variables.c
@@ -17,9 +17,7 @@ static ngx_int_t ngx_stream_variable_remote_addr(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_stream_variable_remote_port(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
-static ngx_int_t ngx_stream_variable_proxy_protocol_addr(
-    ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
-static ngx_int_t ngx_stream_variable_proxy_protocol_port(
+static ngx_int_t ngx_stream_variable_proxy_protocol_value(
     ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
 static ngx_int_t ngx_stream_variable_server_addr(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
@@ -62,10 +60,28 @@ static ngx_stream_variable_t  ngx_stream_core_variables[] = {
       ngx_stream_variable_remote_port, 0, 0, 0 },
 
     { ngx_string("proxy_protocol_addr"), NULL,
-      ngx_stream_variable_proxy_protocol_addr, 0, 0, 0 },
+      ngx_stream_variable_proxy_protocol_value,
+      offsetof(ngx_connection_t, proxy_protocol_src_addr), 0, 0 },
+
+    { ngx_string("proxy_protocol_src_addr"), NULL,
+      ngx_stream_variable_proxy_protocol_value,
+      offsetof(ngx_connection_t, proxy_protocol_src_addr), 0, 0 },
+
+    { ngx_string("proxy_protocol_dst_addr"), NULL,
+      ngx_stream_variable_proxy_protocol_value,
+      offsetof(ngx_connection_t, proxy_protocol_dst_addr), 0, 0 },
 
     { ngx_string("proxy_protocol_port"), NULL,
-      ngx_stream_variable_proxy_protocol_port, 0, 0, 0 },
+      ngx_stream_variable_proxy_protocol_value,
+      offsetof(ngx_connection_t, proxy_protocol_src_port), 0, 0 },
+
+    { ngx_string("proxy_protocol_src_port"), NULL,
+      ngx_stream_variable_proxy_protocol_value,
+      offsetof(ngx_connection_t, proxy_protocol_src_port), 0, 0 },
+
+    { ngx_string("proxy_protocol_dst_port"), NULL,
+      ngx_stream_variable_proxy_protocol_value,
+      offsetof(ngx_connection_t, proxy_protocol_dst_port), 0, 0 },
 
     { ngx_string("server_addr"), NULL,
       ngx_stream_variable_server_addr, 0, 0, 0 },
@@ -428,40 +444,22 @@ ngx_stream_variable_remote_port(ngx_stream_session_t *s,
 
 
 static ngx_int_t
-ngx_stream_variable_proxy_protocol_addr(ngx_stream_session_t *s,
+ngx_stream_variable_proxy_protocol_value(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data)
 {
-    v->len = s->connection->proxy_protocol_addr.len;
-    v->valid = 1;
-    v->no_cacheable = 0;
-    v->not_found = 0;
-    v->data = s->connection->proxy_protocol_addr.data;
-
-    return NGX_OK;
-}
+    ngx_str_t  *a;
 
+    a = (ngx_str_t *) ((char *) s->connection + data);
 
-static ngx_int_t
-ngx_stream_variable_proxy_protocol_port(ngx_stream_session_t *s,
-    ngx_stream_variable_value_t *v, uintptr_t data)
-{
-    ngx_uint_t  port;
+    if (a->data == NULL) {
+      return NGX_ERROR;
+    }
 
-    v->len = 0;
+    v->len = a->len;
     v->valid = 1;
     v->no_cacheable = 0;
     v->not_found = 0;
-
-    v->data = ngx_pnalloc(s->connection->pool, sizeof("65535") - 1);
-    if (v->data == NULL) {
-        return NGX_ERROR;
-    }
-
-    port = s->connection->proxy_protocol_port;
-
-    if (port > 0 && port < 65536) {
-        v->len = ngx_sprintf(v->data, "%ui", port) - v->data;
-    }
+    v->data = a->data;
 
     return NGX_OK;
 }
-- 
2.7.4


From bjornar.ness at gmail.com  Mon Sep 19 19:48:28 2016
From: bjornar.ness at gmail.com (=?UTF-8?Q?Bj=C3=B8rnar_Ness?=)
Date: Mon, 19 Sep 2016 21:48:28 +0200
Subject: [PATCH] proxy-protocol dst variables and proxy-proxy-protocol
In-Reply-To: <1474200674-9906-1-git-send-email-bjornar.ness@gmail.com>
References: <1474200674-9906-1-git-send-email-bjornar.ness@gmail.com>
Message-ID: <CAJO99Tm_MNQFhYMGUgr-2TAbfqqgUJXG+JXdzG4Mh4O0Ys=90Q@mail.gmail.com>

Came to my attention that i can be used uninitialized ((thanks, Johnny), so:

diff --git a/src/core/ngx_proxy_protocol.c b/src/core/ngx_proxy_protocol.c
index 04dae8b..21c710a 100644
--- a/src/core/ngx_proxy_protocol.c
+++ b/src/core/ngx_proxy_protocol.c
@@ -13,7 +13,7 @@ u_char *
  ngx_proxy_protocol_read(ngx_connection_t *c, u_char *buf, u_char *last)
  {
      size_t     len;
-    ngx_uint_t i, n;
+    ngx_uint_t i=0, n;
      u_char     ch, *p, *pp;
      ngx_str_t  addr[2], port[2];


From ru at nginx.com  Tue Sep 20 09:35:43 2016
From: ru at nginx.com (Ruslan Ermilov)
Date: Tue, 20 Sep 2016 09:35:43 +0000
Subject: [nginx] Removed influence of some options on structures.
Message-ID: <hg.e8d4c9e9682a.1474364143.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/e8d4c9e9682a
branches:  
changeset: 6698:e8d4c9e9682a
user:      Ruslan Ermilov <ru at nginx.com>
date:      Tue Sep 20 12:30:52 2016 +0300
description:
Removed influence of some options on structures.

diffstat:

 src/core/ngx_buf.h            |  2 --
 src/event/ngx_event_connect.h |  2 --
 src/http/ngx_http_request.h   |  5 +----
 3 files changed, 1 insertions(+), 8 deletions(-)

diffs (50 lines):

diff -r 52367732bcbc -r e8d4c9e9682a src/core/ngx_buf.h
--- a/src/core/ngx_buf.h	Fri Sep 16 15:13:24 2016 +0300
+++ b/src/core/ngx_buf.h	Tue Sep 20 12:30:52 2016 +0300
@@ -85,9 +85,7 @@ struct ngx_output_chain_ctx_s {
 
     unsigned                     sendfile:1;
     unsigned                     directio:1;
-#if (NGX_HAVE_ALIGNED_DIRECTIO)
     unsigned                     unaligned:1;
-#endif
     unsigned                     need_in_memory:1;
     unsigned                     need_in_temp:1;
 #if (NGX_HAVE_FILE_AIO || NGX_THREADS)
diff -r 52367732bcbc -r e8d4c9e9682a src/event/ngx_event_connect.h
--- a/src/event/ngx_event_connect.h	Fri Sep 16 15:13:24 2016 +0300
+++ b/src/event/ngx_event_connect.h	Tue Sep 20 12:30:52 2016 +0300
@@ -61,9 +61,7 @@ struct ngx_peer_connection_s {
     ngx_log_t                       *log;
 
     unsigned                         cached:1;
-#if (NGX_HAVE_TRANSPARENT_PROXY)
     unsigned                         transparent:1;
-#endif
 
                                      /* ngx_connection_log_error_e */
     unsigned                         log_error:2;
diff -r 52367732bcbc -r e8d4c9e9682a src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h	Fri Sep 16 15:13:24 2016 +0300
+++ b/src/http/ngx_http_request.h	Tue Sep 20 12:30:52 2016 +0300
@@ -302,7 +302,7 @@ typedef struct {
     ngx_http_addr_conf_t             *addr_conf;
     ngx_http_conf_ctx_t              *conf_ctx;
 
-#if (NGX_HTTP_SSL && defined SSL_CTRL_SET_TLSEXT_HOSTNAME)
+#if (NGX_HTTP_SSL)
     ngx_str_t                        *ssl_servername;
 #if (NGX_PCRE)
     ngx_http_regex_t                 *ssl_servername_regex;
@@ -539,11 +539,8 @@ struct ngx_http_request_s {
     unsigned                          subrequest_ranges:1;
     unsigned                          single_range:1;
     unsigned                          disable_not_modified:1;
-
-#if (NGX_STAT_STUB)
     unsigned                          stat_reading:1;
     unsigned                          stat_writing:1;
-#endif
 
     /* used to parse HTTP headers */
 


From xeioex at nginx.com  Tue Sep 20 11:16:55 2016
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 20 Sep 2016 14:16:55 +0300
Subject: [PATCH] proxy-protocol dst variables and proxy-proxy-protocol
In-Reply-To: <1474200674-9906-1-git-send-email-bjornar.ness@gmail.com>
References: <1474200674-9906-1-git-send-email-bjornar.ness@gmail.com>
Message-ID: <57E11AA7.5030706@nginx.com>

Could you please clarify what a problem are you trying to solve? Any 
real world scenario?

http://nginx.org/en/docs/contributing_changes.html
 > Try to make it clear why the suggested change is needed, and provide 
a use case, if possible.

On 18.09.2016 15:11, Bj?rnar Ness wrote:
> Introduce proxy_protocol_src/dst_addr/port and store proxy_protocol
> header in ngx_connection struct. This enables proxy-proxy-protocol
> in stream module if proxy_protocol is enabled on both listen and
> outgoing, which should be the logical default.
>
> Work in progress if/when this patch is accepted is proxy protocol
> support for the mail module.
> ---
>   src/core/ngx_connection.h                 |   7 +-
>   src/core/ngx_proxy_protocol.c             | 111 ++++++++++++++++--------------
>   src/http/modules/ngx_http_realip_module.c |   6 +-
>   src/http/ngx_http_variables.c             |  60 ++++++++--------
>   src/stream/ngx_stream_realip_module.c     |  10 +--
>   src/stream/ngx_stream_variables.c         |  60 ++++++++--------
>   6 files changed, 132 insertions(+), 122 deletions(-)
>
> diff --git a/src/core/ngx_connection.h b/src/core/ngx_connection.h
> index e484c81..c6e2337 100644
> --- a/src/core/ngx_connection.h
> +++ b/src/core/ngx_connection.h
> @@ -148,8 +148,11 @@ struct ngx_connection_s {
>       socklen_t           socklen;
>       ngx_str_t           addr_text;
>
> -    ngx_str_t           proxy_protocol_addr;
> -    in_port_t           proxy_protocol_port;
> +    ngx_str_t           proxy_protocol;
> +    ngx_str_t           proxy_protocol_src_addr;
> +    ngx_str_t           proxy_protocol_src_port;
> +    ngx_str_t           proxy_protocol_dst_addr;
> +    ngx_str_t           proxy_protocol_dst_port;
>
>   #if (NGX_SSL)
>       ngx_ssl_connection_t  *ssl;
> diff --git a/src/core/ngx_proxy_protocol.c b/src/core/ngx_proxy_protocol.c
> index 523ec35..04dae8b 100644
> --- a/src/core/ngx_proxy_protocol.c
> +++ b/src/core/ngx_proxy_protocol.c
> @@ -13,8 +13,9 @@ u_char *
>   ngx_proxy_protocol_read(ngx_connection_t *c, u_char *buf, u_char *last)
>   {
>       size_t     len;
> -    u_char     ch, *p, *addr, *port;
> -    ngx_int_t  n;
> +    ngx_uint_t i, n;
> +    u_char     ch, *p, *pp;
> +    ngx_str_t  addr[2], port[2];
>
>       p = buf;
>       len = last - buf;
> @@ -40,77 +41,79 @@ ngx_proxy_protocol_read(ngx_connection_t *c, u_char *buf, u_char *last)
>       }
>
>       p += 5;
> -    addr = p;
>
> -    for ( ;; ) {
> -        if (p == last) {
> -            goto invalid;
> -        }
> +    for (i=0;i<2;i++) {
> +	addr[i].data = p;
>
> -        ch = *p++;
> +        for ( ;; ) {
> +            if (p == last) {
> +                goto invalid;
> +            }
>
> -        if (ch == ' ') {
> -            break;
> -        }
> +            ch = *p++;
>
> -        if (ch != ':' && ch != '.'
> -            && (ch < 'a' || ch > 'f')
> -            && (ch < 'A' || ch > 'F')
> -            && (ch < '0' || ch > '9'))
> -        {
> -            goto invalid;
> -        }
> -    }
> +            if (ch == ' ') {
> +                break;
> +            }
>
> -    len = p - addr - 1;
> -    c->proxy_protocol_addr.data = ngx_pnalloc(c->pool, len);
> +            if (ch != ':' && ch != '.'
> +                && (ch < 'a' || ch > 'f')
> +                && (ch < 'A' || ch > 'F')
> +                && (ch < '0' || ch > '9'))
> +            {
> +                goto invalid;
> +            }
> +        }
>
> -    if (c->proxy_protocol_addr.data == NULL) {
> -        return NULL;
> +        addr[i].len = p - addr[i].data - 1;
>       }
>
> -    ngx_memcpy(c->proxy_protocol_addr.data, addr, len);
> -    c->proxy_protocol_addr.len = len;
> +    for (i=0;i<2;i++) {
> +        port[i].data = p;
>
> -    for ( ;; ) {
> -        if (p == last) {
> -            goto invalid;
> -        }
> +        for ( ;; ) {
> +            if (p == last) {
> +                goto invalid;
> +            }
>
> -        if (*p++ == ' ') {
> -            break;
> +            if ((*p++ == ' ' && i == 0) || (i == 1 && p != last && *p == CR)) {
> +                break;
> +            }
>           }
> -    }
>
> -    port = p;
> +        port[i].len = p - port[i].data - (1-i);
> +        n = ngx_atoi(port[i].data, port[i].len);
>
> -    for ( ;; ) {
> -        if (p == last) {
> +        if (n < 1 || n > 65535) {
>               goto invalid;
>           }
> -
> -        if (*p++ == ' ') {
> -            break;
> -        }
> -    }
> -
> -    len = p - port - 1;
> -
> -    n = ngx_atoi(port, len);
> -
> -    if (n < 0 || n > 65535) {
> -        goto invalid;
>       }
>
> -    c->proxy_protocol_port = (in_port_t) n;
> -
> -    ngx_log_debug2(NGX_LOG_DEBUG_CORE, c->log, 0,
> -                   "PROXY protocol address: %V %i", &c->proxy_protocol_addr, n);
> -
>   skip:
>
>       for ( /* void */ ; p < last - 1; p++) {
>           if (p[0] == CR && p[1] == LF) {
> +            len = p - buf - 6;
> +            c->proxy_protocol.data = ngx_pnalloc(c->pool, len);
> +            ngx_memcpy(c->proxy_protocol.data, buf + 6, len);
> +            c->proxy_protocol.len = len;
> +
> +            if (i) { /* not UNKNOWN */
> +                pp = c->proxy_protocol.data;
> +                c->proxy_protocol_src_addr.data = pp + (addr[0].data - buf - 6);
> +                c->proxy_protocol_src_port.data = pp + (port[0].data - buf - 6);
> +                c->proxy_protocol_dst_addr.data = pp + (addr[1].data - buf - 6);
> +                c->proxy_protocol_dst_port.data = pp + (port[1].data - buf - 6);
> +                c->proxy_protocol_src_addr.len = addr[0].len;
> +                c->proxy_protocol_src_port.len = port[0].len;
> +                c->proxy_protocol_dst_addr.len = addr[1].len;
> +                c->proxy_protocol_dst_port.len = port[1].len;
> +
> +                ngx_log_debug2(NGX_LOG_DEBUG_CORE, c->log, 0,
> +                               "PROXY protocol address: %V %V",
> +                               &addr[0], &port[0]);
> +            }
> +
>               return p + 2;
>           }
>       }
> @@ -133,6 +136,10 @@ ngx_proxy_protocol_write(ngx_connection_t *c, u_char *buf, u_char *last)
>           return NULL;
>       }
>
> +    if (c->proxy_protocol.len) {
> +	return ngx_slprintf(buf, last, "PROXY %V" CRLF, &c->proxy_protocol);
> +    }
> +
>       if (ngx_connection_local_sockaddr(c, NULL, 0) != NGX_OK) {
>           return NULL;
>       }
> diff --git a/src/http/modules/ngx_http_realip_module.c b/src/http/modules/ngx_http_realip_module.c
> index dba3c52..e2f7bdd 100644
> --- a/src/http/modules/ngx_http_realip_module.c
> +++ b/src/http/modules/ngx_http_realip_module.c
> @@ -180,7 +180,7 @@ ngx_http_realip_handler(ngx_http_request_t *r)
>
>       case NGX_HTTP_REALIP_PROXY:
>
> -        value = &r->connection->proxy_protocol_addr;
> +        value = &r->connection->proxy_protocol_src_addr;
>
>           if (value->len == 0) {
>               return NGX_DECLINED;
> @@ -238,7 +238,9 @@ found:
>           != NGX_DECLINED)
>       {
>           if (rlcf->type == NGX_HTTP_REALIP_PROXY) {
> -            ngx_inet_set_port(addr.sockaddr, c->proxy_protocol_port);
> +            ngx_inet_set_port(addr.sockaddr, (in_port_t)
> +                              ngx_atoi(c->proxy_protocol_src_port.data,
> +                                       c->proxy_protocol_src_port.len));
>           }
>
>           return ngx_http_realip_set_addr(r, &addr);
> diff --git a/src/http/ngx_http_variables.c b/src/http/ngx_http_variables.c
> index 7e65b2e..6071874 100644
> --- a/src/http/ngx_http_variables.c
> +++ b/src/http/ngx_http_variables.c
> @@ -56,9 +56,7 @@ static ngx_int_t ngx_http_variable_remote_addr(ngx_http_request_t *r,
>       ngx_http_variable_value_t *v, uintptr_t data);
>   static ngx_int_t ngx_http_variable_remote_port(ngx_http_request_t *r,
>       ngx_http_variable_value_t *v, uintptr_t data);
> -static ngx_int_t ngx_http_variable_proxy_protocol_addr(ngx_http_request_t *r,
> -    ngx_http_variable_value_t *v, uintptr_t data);
> -static ngx_int_t ngx_http_variable_proxy_protocol_port(ngx_http_request_t *r,
> +static ngx_int_t ngx_http_variable_proxy_protocol_value(ngx_http_request_t *r,
>       ngx_http_variable_value_t *v, uintptr_t data);
>   static ngx_int_t ngx_http_variable_server_addr(ngx_http_request_t *r,
>       ngx_http_variable_value_t *v, uintptr_t data);
> @@ -194,10 +192,28 @@ static ngx_http_variable_t  ngx_http_core_variables[] = {
>       { ngx_string("remote_port"), NULL, ngx_http_variable_remote_port, 0, 0, 0 },
>
>       { ngx_string("proxy_protocol_addr"), NULL,
> -      ngx_http_variable_proxy_protocol_addr, 0, 0, 0 },
> +      ngx_http_variable_proxy_protocol_value,
> +      offsetof(ngx_connection_t, proxy_protocol_src_addr), 0, 0 },
> +
> +    { ngx_string("proxy_protocol_src_addr"), NULL,
> +      ngx_http_variable_proxy_protocol_value,
> +      offsetof(ngx_connection_t, proxy_protocol_src_addr), 0, 0 },
> +
> +    { ngx_string("proxy_protocol_dst_addr"), NULL,
> +      ngx_http_variable_proxy_protocol_value,
> +      offsetof(ngx_connection_t, proxy_protocol_dst_addr), 0, 0 },
>
>       { ngx_string("proxy_protocol_port"), NULL,
> -      ngx_http_variable_proxy_protocol_port, 0, 0, 0 },
> +      ngx_http_variable_proxy_protocol_value,
> +      offsetof(ngx_connection_t, proxy_protocol_src_port), 0, 0 },
> +
> +    { ngx_string("proxy_protocol_src_port"), NULL,
> +      ngx_http_variable_proxy_protocol_value,
> +      offsetof(ngx_connection_t, proxy_protocol_src_port), 0, 0 },
> +
> +    { ngx_string("proxy_protocol_dst_port"), NULL,
> +      ngx_http_variable_proxy_protocol_value,
> +      offsetof(ngx_connection_t, proxy_protocol_dst_port), 0, 0 },
>
>       { ngx_string("server_addr"), NULL, ngx_http_variable_server_addr, 0, 0, 0 },
>
> @@ -1224,40 +1240,22 @@ ngx_http_variable_remote_port(ngx_http_request_t *r,
>
>
>   static ngx_int_t
> -ngx_http_variable_proxy_protocol_addr(ngx_http_request_t *r,
> +ngx_http_variable_proxy_protocol_value(ngx_http_request_t *r,
>       ngx_http_variable_value_t *v, uintptr_t data)
>   {
> -    v->len = r->connection->proxy_protocol_addr.len;
> -    v->valid = 1;
> -    v->no_cacheable = 0;
> -    v->not_found = 0;
> -    v->data = r->connection->proxy_protocol_addr.data;
> -
> -    return NGX_OK;
> -}
> +    ngx_str_t   *a;
>
> +    a = (ngx_str_t *) ((char *) r->connection + data);
>
> -static ngx_int_t
> -ngx_http_variable_proxy_protocol_port(ngx_http_request_t *r,
> -    ngx_http_variable_value_t *v, uintptr_t data)
> -{
> -    ngx_uint_t  port;
> +    if (a->data == NULL) {
> +        return NGX_ERROR;
> +    }
>
> -    v->len = 0;
> +    v->len = a->len;
>       v->valid = 1;
>       v->no_cacheable = 0;
>       v->not_found = 0;
> -
> -    v->data = ngx_pnalloc(r->pool, sizeof("65535") - 1);
> -    if (v->data == NULL) {
> -        return NGX_ERROR;
> -    }
> -
> -    port = r->connection->proxy_protocol_port;
> -
> -    if (port > 0 && port < 65536) {
> -        v->len = ngx_sprintf(v->data, "%ui", port) - v->data;
> -    }
> +    v->data = a->data;
>
>       return NGX_OK;
>   }
> diff --git a/src/stream/ngx_stream_realip_module.c b/src/stream/ngx_stream_realip_module.c
> index 0740431..3380aab 100644
> --- a/src/stream/ngx_stream_realip_module.c
> +++ b/src/stream/ngx_stream_realip_module.c
> @@ -108,7 +108,7 @@ ngx_stream_realip_handler(ngx_stream_session_t *s)
>
>       c = s->connection;
>
> -    if (c->proxy_protocol_addr.len == 0) {
> +    if (c->proxy_protocol_src_addr.len == 0) {
>           return NGX_DECLINED;
>       }
>
> @@ -116,14 +116,16 @@ ngx_stream_realip_handler(ngx_stream_session_t *s)
>           return NGX_DECLINED;
>       }
>
> -    if (ngx_parse_addr(c->pool, &addr, c->proxy_protocol_addr.data,
> -                       c->proxy_protocol_addr.len)
> +    if (ngx_parse_addr(c->pool, &addr, c->proxy_protocol_src_addr.data,
> +                       c->proxy_protocol_src_addr.len)
>           != NGX_OK)
>       {
>           return NGX_DECLINED;
>       }
>
> -    ngx_inet_set_port(addr.sockaddr, c->proxy_protocol_port);
> +    ngx_inet_set_port(addr.sockaddr,
> +                      (in_port_t) ngx_atoi(c->proxy_protocol_src_port.data,
> +                                           c->proxy_protocol_src_port.len));
>
>       return ngx_stream_realip_set_addr(s, &addr);
>   }
> diff --git a/src/stream/ngx_stream_variables.c b/src/stream/ngx_stream_variables.c
> index aa5361d..0aa4f73 100644
> --- a/src/stream/ngx_stream_variables.c
> +++ b/src/stream/ngx_stream_variables.c
> @@ -17,9 +17,7 @@ static ngx_int_t ngx_stream_variable_remote_addr(ngx_stream_session_t *s,
>       ngx_stream_variable_value_t *v, uintptr_t data);
>   static ngx_int_t ngx_stream_variable_remote_port(ngx_stream_session_t *s,
>       ngx_stream_variable_value_t *v, uintptr_t data);
> -static ngx_int_t ngx_stream_variable_proxy_protocol_addr(
> -    ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
> -static ngx_int_t ngx_stream_variable_proxy_protocol_port(
> +static ngx_int_t ngx_stream_variable_proxy_protocol_value(
>       ngx_stream_session_t *s, ngx_stream_variable_value_t *v, uintptr_t data);
>   static ngx_int_t ngx_stream_variable_server_addr(ngx_stream_session_t *s,
>       ngx_stream_variable_value_t *v, uintptr_t data);
> @@ -62,10 +60,28 @@ static ngx_stream_variable_t  ngx_stream_core_variables[] = {
>         ngx_stream_variable_remote_port, 0, 0, 0 },
>
>       { ngx_string("proxy_protocol_addr"), NULL,
> -      ngx_stream_variable_proxy_protocol_addr, 0, 0, 0 },
> +      ngx_stream_variable_proxy_protocol_value,
> +      offsetof(ngx_connection_t, proxy_protocol_src_addr), 0, 0 },
> +
> +    { ngx_string("proxy_protocol_src_addr"), NULL,
> +      ngx_stream_variable_proxy_protocol_value,
> +      offsetof(ngx_connection_t, proxy_protocol_src_addr), 0, 0 },
> +
> +    { ngx_string("proxy_protocol_dst_addr"), NULL,
> +      ngx_stream_variable_proxy_protocol_value,
> +      offsetof(ngx_connection_t, proxy_protocol_dst_addr), 0, 0 },
>
>       { ngx_string("proxy_protocol_port"), NULL,
> -      ngx_stream_variable_proxy_protocol_port, 0, 0, 0 },
> +      ngx_stream_variable_proxy_protocol_value,
> +      offsetof(ngx_connection_t, proxy_protocol_src_port), 0, 0 },
> +
> +    { ngx_string("proxy_protocol_src_port"), NULL,
> +      ngx_stream_variable_proxy_protocol_value,
> +      offsetof(ngx_connection_t, proxy_protocol_src_port), 0, 0 },
> +
> +    { ngx_string("proxy_protocol_dst_port"), NULL,
> +      ngx_stream_variable_proxy_protocol_value,
> +      offsetof(ngx_connection_t, proxy_protocol_dst_port), 0, 0 },
>
>       { ngx_string("server_addr"), NULL,
>         ngx_stream_variable_server_addr, 0, 0, 0 },
> @@ -428,40 +444,22 @@ ngx_stream_variable_remote_port(ngx_stream_session_t *s,
>
>
>   static ngx_int_t
> -ngx_stream_variable_proxy_protocol_addr(ngx_stream_session_t *s,
> +ngx_stream_variable_proxy_protocol_value(ngx_stream_session_t *s,
>       ngx_stream_variable_value_t *v, uintptr_t data)
>   {
> -    v->len = s->connection->proxy_protocol_addr.len;
> -    v->valid = 1;
> -    v->no_cacheable = 0;
> -    v->not_found = 0;
> -    v->data = s->connection->proxy_protocol_addr.data;
> -
> -    return NGX_OK;
> -}
> +    ngx_str_t  *a;
>
> +    a = (ngx_str_t *) ((char *) s->connection + data);
>
> -static ngx_int_t
> -ngx_stream_variable_proxy_protocol_port(ngx_stream_session_t *s,
> -    ngx_stream_variable_value_t *v, uintptr_t data)
> -{
> -    ngx_uint_t  port;
> +    if (a->data == NULL) {
> +      return NGX_ERROR;
> +    }
>
> -    v->len = 0;
> +    v->len = a->len;
>       v->valid = 1;
>       v->no_cacheable = 0;
>       v->not_found = 0;
> -
> -    v->data = ngx_pnalloc(s->connection->pool, sizeof("65535") - 1);
> -    if (v->data == NULL) {
> -        return NGX_ERROR;
> -    }
> -
> -    port = s->connection->proxy_protocol_port;
> -
> -    if (port > 0 && port < 65536) {
> -        v->len = ngx_sprintf(v->data, "%ui", port) - v->data;
> -    }
> +    v->data = a->data;
>
>       return NGX_OK;
>   }
>


From vbart at nginx.com  Tue Sep 20 12:10:36 2016
From: vbart at nginx.com (Valentin Bartenev)
Date: Tue, 20 Sep 2016 12:10:36 +0000
Subject: [nginx] Fixed log levels of configuration parsing errors.
Message-ID: <hg.9cf2dce316e5.1474373436.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/9cf2dce316e5
branches:  
changeset: 6699:9cf2dce316e5
user:      Valentin Bartenev <vbart at nginx.com>
date:      Tue Sep 20 15:07:16 2016 +0300
description:
Fixed log levels of configuration parsing errors.

All the errors that prevent loading configuration must be printed on the "emerg"
log level.  Previously, nginx might silently fail to load configuration in some
cases as the default log level is "error".

diffstat:

 src/core/ngx_conf_file.c         |  4 ++--
 src/event/ngx_event_openssl.c    |  4 ++--
 src/http/ngx_http_upstream.c     |  4 ++--
 src/mail/ngx_mail_ssl_module.c   |  4 ++--
 src/stream/ngx_stream_upstream.c |  4 ++--
 5 files changed, 10 insertions(+), 10 deletions(-)

diffs (101 lines):

diff -r e8d4c9e9682a -r 9cf2dce316e5 src/core/ngx_conf_file.c
--- a/src/core/ngx_conf_file.c	Tue Sep 20 12:30:52 2016 +0300
+++ b/src/core/ngx_conf_file.c	Tue Sep 20 15:07:16 2016 +0300
@@ -1336,7 +1336,7 @@ ngx_conf_set_enum_slot(ngx_conf_t *cf, n
         return NGX_CONF_OK;
     }
 
-    ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
+    ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                        "invalid value \"%s\"", value[1].data);
 
     return NGX_CONF_ERROR;
@@ -1378,7 +1378,7 @@ ngx_conf_set_bitmask_slot(ngx_conf_t *cf
         }
 
         if (mask[m].name.len == 0) {
-            ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
+            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                                "invalid value \"%s\"", value[i].data);
 
             return NGX_CONF_ERROR;
diff -r e8d4c9e9682a -r 9cf2dce316e5 src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c	Tue Sep 20 12:30:52 2016 +0300
+++ b/src/event/ngx_event_openssl.c	Tue Sep 20 15:07:16 2016 +0300
@@ -3656,13 +3656,13 @@ ngx_openssl_engine(ngx_conf_t *cf, ngx_c
     engine = ENGINE_by_id((char *) value[1].data);
 
     if (engine == NULL) {
-        ngx_ssl_error(NGX_LOG_WARN, cf->log, 0,
+        ngx_ssl_error(NGX_LOG_EMERG, cf->log, 0,
                       "ENGINE_by_id(\"%V\") failed", &value[1]);
         return NGX_CONF_ERROR;
     }
 
     if (ENGINE_set_default(engine, ENGINE_METHOD_ALL) == 0) {
-        ngx_ssl_error(NGX_LOG_WARN, cf->log, 0,
+        ngx_ssl_error(NGX_LOG_EMERG, cf->log, 0,
                       "ENGINE_set_default(\"%V\", ENGINE_METHOD_ALL) failed",
                       &value[1]);
 
diff -r e8d4c9e9682a -r 9cf2dce316e5 src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Tue Sep 20 12:30:52 2016 +0300
+++ b/src/http/ngx_http_upstream.c	Tue Sep 20 15:07:16 2016 +0300
@@ -5717,14 +5717,14 @@ ngx_http_upstream_add(ngx_conf_t *cf, ng
         }
 
         if ((uscfp[i]->flags & NGX_HTTP_UPSTREAM_CREATE) && !u->no_port) {
-            ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
+            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                                "upstream \"%V\" may not have port %d",
                                &u->host, u->port);
             return NULL;
         }
 
         if ((flags & NGX_HTTP_UPSTREAM_CREATE) && !uscfp[i]->no_port) {
-            ngx_log_error(NGX_LOG_WARN, cf->log, 0,
+            ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                           "upstream \"%V\" may not have port %d in %s:%ui",
                           &u->host, uscfp[i]->port,
                           uscfp[i]->file_name, uscfp[i]->line);
diff -r e8d4c9e9682a -r 9cf2dce316e5 src/mail/ngx_mail_ssl_module.c
--- a/src/mail/ngx_mail_ssl_module.c	Tue Sep 20 12:30:52 2016 +0300
+++ b/src/mail/ngx_mail_ssl_module.c	Tue Sep 20 15:07:16 2016 +0300
@@ -488,7 +488,7 @@ ngx_mail_ssl_enable(ngx_conf_t *cf, ngx_
     }
 
     if (scf->enable && (ngx_int_t) scf->starttls > NGX_MAIL_STARTTLS_OFF) {
-        ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "\"starttls\" directive conflicts with \"ssl on\"");
         return NGX_CONF_ERROR;
     }
@@ -514,7 +514,7 @@ ngx_mail_ssl_starttls(ngx_conf_t *cf, ng
     }
 
     if (scf->enable == 1 && (ngx_int_t) scf->starttls > NGX_MAIL_STARTTLS_OFF) {
-        ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
+        ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                            "\"ssl\" directive conflicts with \"starttls\"");
         return NGX_CONF_ERROR;
     }
diff -r e8d4c9e9682a -r 9cf2dce316e5 src/stream/ngx_stream_upstream.c
--- a/src/stream/ngx_stream_upstream.c	Tue Sep 20 12:30:52 2016 +0300
+++ b/src/stream/ngx_stream_upstream.c	Tue Sep 20 15:07:16 2016 +0300
@@ -586,14 +586,14 @@ ngx_stream_upstream_add(ngx_conf_t *cf, 
         }
 
         if ((uscfp[i]->flags & NGX_STREAM_UPSTREAM_CREATE) && !u->no_port) {
-            ngx_conf_log_error(NGX_LOG_WARN, cf, 0,
+            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
                                "upstream \"%V\" may not have port %d",
                                &u->host, u->port);
             return NULL;
         }
 
         if ((flags & NGX_STREAM_UPSTREAM_CREATE) && !uscfp[i]->no_port) {
-            ngx_log_error(NGX_LOG_WARN, cf->log, 0,
+            ngx_log_error(NGX_LOG_EMERG, cf->log, 0,
                           "upstream \"%V\" may not have port %d in %s:%ui",
                           &u->host, uscfp[i]->port,
                           uscfp[i]->file_name, uscfp[i]->line);


From bjornar.ness at gmail.com  Tue Sep 20 12:50:56 2016
From: bjornar.ness at gmail.com (=?UTF-8?Q?Bj=C3=B8rnar_Ness?=)
Date: Tue, 20 Sep 2016 14:50:56 +0200
Subject: [PATCH] proxy-protocol dst variables and proxy-proxy-protocol
In-Reply-To: <57E11AA7.5030706@nginx.com>
References: <1474200674-9906-1-git-send-email-bjornar.ness@gmail.com>
 <57E11AA7.5030706@nginx.com>
Message-ID: <CAJO99Tnt==gBvjUCaDBWprvpOYwZqLuOXSZPRKefp3z5YFQY3g@mail.gmail.com>

2016-09-20 13:16 GMT+02:00 Dmitry Volyntsev <xeioex at nginx.com>:
> Could you please clarify what a problem are you trying to solve? Any real
> world scenario?

Hello, Dmitry, thanks for responding.

The first problem I am trying to solve is the case where we have:

LB -> nginx_proxy -> (something_with_proxy_protocol_support)

Here, if proxy_protocol is enabled on both listen and outgoing, it
is logical to assume the user wants the incoming header passed
on directly.

Also, I want access to the _dst part of the proxy protocol to make
decisions based on the original destination address, which is
currently unavailable.

I chose to expose both $proxy_protocol_(src|dst)_(addr|port) and keep
backwards compability with the original $proxy_protocol_(addr|port),
which I suggest removing since its naming is confusing wrt src/dst.

Storing these as ngx_str_t can definately be discussed, atleast if
you plan/want proxyprotocol v2 support. In that case it would
perhaps make sense to use something like:

struct proxy_protocol_data {
    int socket_type;
    struct sockaddr src;
    struct sockaddr dst;
}

and in ngx_connection_t

struct proxy_protocol_data *proxy_protocol;

What do you think? I feel storing as strings makes sense until the need
for v2 arises, also, it makes the proxy-proxy-protocol support simple.

> http://nginx.org/en/docs/contributing_changes.html
>> Try to make it clear why the suggested change is needed, and provide a use
>> case, if possible.

Thanks, I will keep to this standard in future updates.

--
Bj(/)rnar


From xeioex at nginx.com  Tue Sep 20 15:18:48 2016
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Tue, 20 Sep 2016 18:18:48 +0300
Subject: [PATCH] proxy-protocol dst variables and proxy-proxy-protocol
In-Reply-To: <CAJO99Tnt==gBvjUCaDBWprvpOYwZqLuOXSZPRKefp3z5YFQY3g@mail.gmail.com>
References: <1474200674-9906-1-git-send-email-bjornar.ness@gmail.com>
 <57E11AA7.5030706@nginx.com>
 <CAJO99Tnt==gBvjUCaDBWprvpOYwZqLuOXSZPRKefp3z5YFQY3g@mail.gmail.com>
Message-ID: <57E15358.6080809@nginx.com>


On 20.09.2016 15:50, Bj?rnar Ness wrote:
> 2016-09-20 13:16 GMT+02:00 Dmitry Volyntsev <xeioex at nginx.com>:
>> Could you please clarify what a problem are you trying to solve? Any real
>> world scenario?
>
> Hello, Dmitry, thanks for responding.
>
> The first problem I am trying to solve is the case where we have:
>
> LB -> nginx_proxy -> (something_with_proxy_protocol_support)
>
> Here, if proxy_protocol is enabled on both listen and outgoing, it
> is logical to assume the user wants the incoming header passed
> on directly.

Transparent proxy protocol bypass sounds like a valid use case.

>
> Also, I want access to the _dst part of the proxy protocol to make
> decisions based on the original destination address, which is
> currently unavailable.

Why do you need to know the destination address? could you elaborate
what do you want to know on the 
"(something_with_proxy_protocol_support)" side?

If you have a fixed set of LBs you could configure a separate listen
in nginx configuration per each LB.

>
> I chose to expose both $proxy_protocol_(src|dst)_(addr|port) and keep
> backwards compability with the original $proxy_protocol_(addr|port),
> which I suggest removing since its naming is confusing wrt src/dst.
>
> Storing these as ngx_str_t can definately be discussed, atleast if
> you plan/want proxyprotocol v2 support. In that case it would
> perhaps make sense to use something like:
>
> struct proxy_protocol_data {
>      int socket_type;
>      struct sockaddr src;
>      struct sockaddr dst;
> }
>
> and in ngx_connection_t
>
> struct proxy_protocol_data *proxy_protocol;
>
> What do you think? I feel storing as strings makes sense until the need
> for v2 arises, also, it makes the proxy-proxy-protocol support simple.
>
>> http://nginx.org/en/docs/contributing_changes.html
>>> Try to make it clear why the suggested change is needed, and provide a use
>>> case, if possible.
>
> Thanks, I will keep to this standard in future updates.
>
> --
> Bj(/)rnar
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>


From bjornar.ness at gmail.com  Tue Sep 20 17:18:16 2016
From: bjornar.ness at gmail.com (=?UTF-8?Q?Bj=C3=B8rnar_Ness?=)
Date: Tue, 20 Sep 2016 19:18:16 +0200
Subject: [PATCH] proxy-protocol dst variables and proxy-proxy-protocol
In-Reply-To: <57E15358.6080809@nginx.com>
References: <1474200674-9906-1-git-send-email-bjornar.ness@gmail.com>
 <57E11AA7.5030706@nginx.com>
 <CAJO99Tnt==gBvjUCaDBWprvpOYwZqLuOXSZPRKefp3z5YFQY3g@mail.gmail.com>
 <57E15358.6080809@nginx.com>
Message-ID: <CAJO99TkAqahrZXeaesFEuoKaa-jwB_vQPQzBVFE2Oj0UcKCa6A@mail.gmail.com>

2016-09-20 17:18 GMT+02:00 Dmitry Volyntsev <xeioex at nginx.com>:
> On 20.09.2016 15:50, Bj?rnar Ness wrote:
>> [ ... ]
>> Also, I want access to the _dst part of the proxy protocol to make
>> decisions based on the original destination address, which is
>> currently unavailable.
>
> Why do you need to know the destination address? could you elaborate
> what do you want to know on the "(something_with_proxy_protocol_support)"
> side?

$proxy_protocol_dst_addr is the original destination of the request. This
is useful to be able to provide dynamic responses based on destination
address, or in cases where the lookup key for a decision is a function on
the original destination address.

Say lookup-key in for example a database is $proxy_protocol_dst_addr/$http_host

> If you have a fixed set of LBs you could configure a separate listen
> in nginx configuration per each LB.

This is true, but requires duplicated configurations and configuration
generation/
reloads for a scaleup/down

-- 
Bj(/)rnar


From nginx at rastos.org  Wed Sep 21 18:55:33 2016
From: nginx at rastos.org (Rastislav Stanik)
Date: Wed, 21 Sep 2016 20:55:33 +0200
Subject: [PATCH] - flastmod SSI command
Message-ID: <57E2D7A5.2040304@rastos.org>

Hi.

I'm new here and I subscribed because recently I wanted to switch
some little personal web server from apache to nginx and found
out that "flastmod" SSI command is not supported by nginx.
This command is supported by Apache
http://httpd.apache.org/docs/current/mod/mod_include.html#element.flastmod
as well as IIS
https://msdn.microsoft.com/en-us/library/ms525105%28v=vs.90%29.aspx

So I tried to implement that. The patch/changeset is in the attachment.
I would appreciate any feedback which would help to get that eventually
included in official nginx code.
--
	bye
		rastos
-------------- next part --------------
# HG changeset patch
# User Rastislav Stanik <nginx at rastos.org>
# Date 1474145684 -7200
#      Sat Sep 17 22:54:44 2016 +0200
# Node ID f12b9f072e00182c8e12f24a4b8fa323c0d28080
# Parent  52367732bcbc87430e8276424356aaec3c2b6a87
add handling of flastmod SSI directive

Add handling of SSI directive 'flastmod'. The directive
takes a single argument 'file' and is replaced with
modification time of the specified file. The filename
is accepted/rejected based on the same rules as for the
SSI directive 'include'.
The time string is constructed based on SSI directive config timefmt.
Failure to obtain the modification time is logged.

diff -r 52367732bcbc -r f12b9f072e00 src/http/modules/ngx_http_ssi_filter_module.c
--- a/src/http/modules/ngx_http_ssi_filter_module.c	Fri Sep 16 15:13:24 2016 +0300
+++ b/src/http/modules/ngx_http_ssi_filter_module.c	Sat Sep 17 22:54:44 2016 +0200
@@ -85,6 +85,8 @@
 
 static ngx_int_t ngx_http_ssi_include(ngx_http_request_t *r,
     ngx_http_ssi_ctx_t *ctx, ngx_str_t **params);
+static ngx_int_t ngx_http_ssi_flastmod(ngx_http_request_t *r,
+    ngx_http_ssi_ctx_t *ctx, ngx_str_t **params);
 static ngx_int_t ngx_http_ssi_stub_output(ngx_http_request_t *r, void *data,
     ngx_int_t rc);
 static ngx_int_t ngx_http_ssi_set_variable(ngx_http_request_t *r, void *data,
@@ -223,6 +225,8 @@
 #define  NGX_HTTP_SSI_INCLUDE_SET      3
 #define  NGX_HTTP_SSI_INCLUDE_STUB     4
 
+#define  NGX_HTTP_SSI_FLASTMOD_FILE    0
+
 #define  NGX_HTTP_SSI_ECHO_VAR         0
 #define  NGX_HTTP_SSI_ECHO_DEFAULT     1
 #define  NGX_HTTP_SSI_ECHO_ENCODING    2
@@ -247,6 +251,10 @@
     { ngx_null_string, 0, 0, 0 }
 };
 
+static ngx_http_ssi_param_t  ngx_http_ssi_flastmod_params[] = {
+    { ngx_string("file"), NGX_HTTP_SSI_FLASTMOD_FILE, 0, 0 }
+};
+
 
 static ngx_http_ssi_param_t  ngx_http_ssi_echo_params[] = {
     { ngx_string("var"), NGX_HTTP_SSI_ECHO_VAR, 1, 0 },
@@ -308,7 +316,8 @@
                        ngx_http_ssi_block_params, 0, 0, 0 },
     { ngx_string("endblock"), ngx_http_ssi_endblock,
                        ngx_http_ssi_no_params, 0, 1, 0 },
-
+    { ngx_string("flastmod"), ngx_http_ssi_flastmod,
+                       ngx_http_ssi_flastmod_params, 0, 0, 0 },
     { ngx_null_string, NULL, NULL, 0, 0, 0 }
 };
 
@@ -2197,6 +2206,97 @@
     return NGX_OK;
 }
 
+static ngx_int_t
+ngx_http_ssi_flastmod(ngx_http_request_t *r, ngx_http_ssi_ctx_t *ctx,
+    ngx_str_t **params)
+{
+    ngx_str_t   *file;
+    ngx_str_t   current_path;
+    ngx_uint_t  flags;
+    ngx_int_t   rc;
+    ngx_str_t   args;
+    size_t      root;
+    struct stat st_buf;
+    u_char *endpos;
+    char *statpath;
+    struct tm mod_time;
+    
+    file = params[NGX_HTTP_SSI_FLASTMOD_FILE];
+
+    if (file == NULL) {
+        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                      "no parameter in \"flastmod\" SSI command");
+        return NGX_HTTP_SSI_ERROR;
+    }
+
+    rc = ngx_http_ssi_evaluate_string(r, ctx, file, 0);
+
+    if (rc != NGX_OK) {
+        return rc;
+    }
+
+    ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
+                   "ssi flastmod: \"%V\"", file);
+
+    ngx_str_null(&args);
+    flags = NGX_HTTP_LOG_UNSAFE;
+    
+    if (ngx_http_parse_unsafe_uri(r, file, &args, &flags) != NGX_OK) {
+        return NGX_HTTP_SSI_ERROR;
+    }
+    
+    root=0;
+    ngx_http_map_uri_to_path(r, &current_path, &root, 0);
+    // in worst case we will append the flastmod file at the end of the current pathname and add terminating zero
+    statpath=ngx_pcalloc(r->pool,current_path.len+file->len+1);
+    if (!statpath) {
+        return NGX_HTTP_SSI_ERROR;
+    }
+    // write the current pathname to 'statpath'
+    strncpy(statpath,current_path.data,current_path.len);
+    endpos=statpath+current_path.len;
+    // find the last slash
+    for (;endpos>current_path.data && !ngx_path_separator(*endpos);endpos--)
+            ;
+    // append the flastmod file after last slash
+    strncpy(endpos+1,file->data,file->len);
+    *(endpos+1+file->len)='\0'; // zero terminate the string so that we can pass it to stat(2)
+    
+    if (0==stat(statpath,&st_buf)) {
+        if (gmtime_r(&st_buf.st_mtim.tv_sec,&mod_time)) {
+            char *result=NULL;
+            ngx_uint_t len=0;
+            ngx_buf_t *b=NULL;
+            
+            result = ngx_pcalloc(r->pool, NGX_HTTP_SSI_MAX_FLASTMOD);
+            if (result) {
+                // convert the modification time to string:
+                len=strftime(result,NGX_HTTP_SSI_MAX_FLASTMOD,ctx->timefmt.data,&mod_time);
+                b = ngx_calloc_buf(r->pool);
+                if (b) {
+                    ngx_chain_t *cl = ngx_alloc_chain_link(r->pool);
+                    if (cl) {
+                        // insert the time string into the http response:
+                        b->memory = 1;
+                        b->pos = result;
+                        b->last = result + len;
+                        
+                        cl->buf = b;
+                        cl->next = NULL;
+                        *ctx->last_out = cl;
+                        ctx->last_out = &cl->next;
+                        return NGX_OK;
+                    }
+                }
+            }
+        }
+    }
+    else
+       ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+                     "could not stat() \"%s\" file. err=%d",statpath,ngx_errno);
+
+    return NGX_HTTP_SSI_ERROR;
+}
 
 static ngx_int_t
 ngx_http_ssi_stub_output(ngx_http_request_t *r, void *data, ngx_int_t rc)
diff -r 52367732bcbc -r f12b9f072e00 src/http/modules/ngx_http_ssi_filter_module.h
--- a/src/http/modules/ngx_http_ssi_filter_module.h	Fri Sep 16 15:13:24 2016 +0300
+++ b/src/http/modules/ngx_http_ssi_filter_module.h	Sat Sep 17 22:54:44 2016 +0200
@@ -30,6 +30,9 @@
 #define NGX_HTTP_SSI_ENTITY_ENCODING  2
 
 
+#define  NGX_HTTP_SSI_MAX_FLASTMOD     100
+
+
 typedef struct {
     ngx_hash_t                hash;
     ngx_hash_keys_arrays_t    commands;

From thresh at nginx.com  Thu Sep 22 14:50:30 2016
From: thresh at nginx.com (Konstantin Pavlov)
Date: Thu, 22 Sep 2016 14:50:30 +0000
Subject: [nginx] Perl: pass additional linker options to perl module.
Message-ID: <hg.0d2956dfc4e6.1474555830.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/0d2956dfc4e6
branches:  
changeset: 6700:0d2956dfc4e6
user:      Konstantin Pavlov <thresh at nginx.com>
date:      Tue Sep 20 22:11:23 2016 +0300
description:
Perl: pass additional linker options to perl module.

Previously flags passed by --with-ld-opt were not used when building perl
module, which meant hardening flags provided by package build systems were not
applied.

diffstat:

 auto/lib/perl/conf                |  4 +++-
 auto/lib/perl/make                |  1 +
 auto/make                         |  1 +
 src/http/modules/perl/Makefile.PL |  2 ++
 4 files changed, 7 insertions(+), 1 deletions(-)

diffs (49 lines):

diff -r 9cf2dce316e5 -r 0d2956dfc4e6 auto/lib/perl/conf
--- a/auto/lib/perl/conf	Tue Sep 20 15:07:16 2016 +0300
+++ b/auto/lib/perl/conf	Tue Sep 20 22:11:23 2016 +0300
@@ -28,8 +28,10 @@ if test -n "$NGX_PERL_VER"; then
         exit 1;
     fi
 
+    NGX_PM_CFLAGS=`$NGX_PERL -MExtUtils::Embed -e ccopts`
+    NGX_PM_LDFLAGS=`$NGX_PERL -MConfig -e 'print $Config{lddlflags}'`
+
     NGX_PERL_CFLAGS="$CFLAGS `$NGX_PERL -MExtUtils::Embed -e ccopts`"
-    NGX_PM_CFLAGS=`$NGX_PERL -MExtUtils::Embed -e ccopts`
 
     # gcc 4.1/4.2 warn about unused values in pTHX_
     NGX_PERL_CFLAGS=`echo $NGX_PERL_CFLAGS \
diff -r 9cf2dce316e5 -r 0d2956dfc4e6 auto/lib/perl/make
--- a/auto/lib/perl/make	Tue Sep 20 15:07:16 2016 +0300
+++ b/auto/lib/perl/make	Tue Sep 20 22:11:23 2016 +0300
@@ -35,6 +35,7 @@ cat << END                              
 
 	cd $NGX_OBJS/src/http/modules/perl \\
 		&& NGX_PM_CFLAGS="\$(NGX_PM_CFLAGS) -g $NGX_CC_OPT" \\
+			NGX_PM_LDFLAGS="$NGX_LD_OPT \$(NGX_PM_LDFLAGS)" \\
 			NGX_INCS="$CORE_INCS $NGX_OBJS $HTTP_INCS" \\
 			NGX_DEPS="\$(CORE_DEPS) \$(HTTP_DEPS)" \\
 		$NGX_PERL Makefile.PL \\
diff -r 9cf2dce316e5 -r 0d2956dfc4e6 auto/make
--- a/auto/make	Tue Sep 20 15:07:16 2016 +0300
+++ b/auto/make	Tue Sep 20 22:11:23 2016 +0300
@@ -31,6 +31,7 @@ END
 if test -n "$NGX_PERL_CFLAGS"; then
     echo NGX_PERL_CFLAGS = $NGX_PERL_CFLAGS                   >> $NGX_MAKEFILE
     echo NGX_PM_CFLAGS = $NGX_PM_CFLAGS                       >> $NGX_MAKEFILE
+    echo NGX_PM_LDFLAGS = $NGX_PM_LDFLAGS                     >> $NGX_MAKEFILE
 fi
 
 
diff -r 9cf2dce316e5 -r 0d2956dfc4e6 src/http/modules/perl/Makefile.PL
--- a/src/http/modules/perl/Makefile.PL	Tue Sep 20 15:07:16 2016 +0300
+++ b/src/http/modules/perl/Makefile.PL	Tue Sep 20 22:11:23 2016 +0300
@@ -16,6 +16,8 @@ WriteMakefile(
     CCFLAGS           => "$ENV{NGX_PM_CFLAGS}",
     OPTIMIZE          => '-O',
 
+    LDDLFLAGS         => "$ENV{NGX_PM_LDFLAGS}",
+
     INC               => join(" ", map {
                              m#^/# ? "-I $_" : "-I ../../../../../$_"
                          } (split /\s+/, $ENV{NGX_INCS})),


From mdounin at mdounin.ru  Thu Sep 22 19:59:44 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 22 Sep 2016 19:59:44 +0000
Subject: [nginx] Upstream: style, ngx_http_upstream_rr_peer_t.next moved.
Message-ID: <hg.d69964eb8335.1474574384.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/d69964eb8335
branches:  
changeset: 6701:d69964eb8335
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Jul 25 16:23:35 2016 +0300
description:
Upstream: style, ngx_http_upstream_rr_peer_t.next moved.

diffstat:

 src/http/ngx_http_upstream_round_robin.h     |  4 ++--
 src/stream/ngx_stream_upstream_round_robin.h |  4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diffs (34 lines):

diff --git a/src/http/ngx_http_upstream_round_robin.h b/src/http/ngx_http_upstream_round_robin.h
--- a/src/http/ngx_http_upstream_round_robin.h
+++ b/src/http/ngx_http_upstream_round_robin.h
@@ -42,11 +42,11 @@ struct ngx_http_upstream_rr_peer_s {
     int                             ssl_session_len;
 #endif
 
-    ngx_http_upstream_rr_peer_t    *next;
-
 #if (NGX_HTTP_UPSTREAM_ZONE)
     ngx_atomic_t                    lock;
 #endif
+
+    ngx_http_upstream_rr_peer_t    *next;
 };
 
 
diff --git a/src/stream/ngx_stream_upstream_round_robin.h b/src/stream/ngx_stream_upstream_round_robin.h
--- a/src/stream/ngx_stream_upstream_round_robin.h
+++ b/src/stream/ngx_stream_upstream_round_robin.h
@@ -42,11 +42,11 @@ struct ngx_stream_upstream_rr_peer_s {
     int                              ssl_session_len;
 #endif
 
-    ngx_stream_upstream_rr_peer_t   *next;
-
 #if (NGX_STREAM_UPSTREAM_ZONE)
     ngx_atomic_t                     lock;
 #endif
+
+    ngx_stream_upstream_rr_peer_t   *next;
 };
 
 


From mdounin at mdounin.ru  Thu Sep 22 19:59:47 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 22 Sep 2016 19:59:47 +0000
Subject: [nginx] Upstream: style.
Message-ID: <hg.914d8a63dbad.1474574387.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/914d8a63dbad
branches:  
changeset: 6702:914d8a63dbad
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Sep 22 19:31:08 2016 +0300
description:
Upstream: style.

diffstat:

 src/http/modules/ngx_http_upstream_hash_module.c       |  1 -
 src/http/modules/ngx_http_upstream_least_conn_module.c |  1 -
 src/http/ngx_http_upstream_round_robin.c               |  1 -
 src/stream/ngx_stream_upstream_hash_module.c           |  1 -
 src/stream/ngx_stream_upstream_least_conn_module.c     |  1 -
 src/stream/ngx_stream_upstream_round_robin.c           |  1 -
 6 files changed, 0 insertions(+), 6 deletions(-)

diffs (66 lines):

diff --git a/src/http/modules/ngx_http_upstream_hash_module.c b/src/http/modules/ngx_http_upstream_hash_module.c
--- a/src/http/modules/ngx_http_upstream_hash_module.c
+++ b/src/http/modules/ngx_http_upstream_hash_module.c
@@ -523,7 +523,6 @@ ngx_http_upstream_get_chash_peer(ngx_pee
              peer;
              peer = peer->next, i++)
         {
-
             n = i / (8 * sizeof(uintptr_t));
             m = (uintptr_t) 1 << i % (8 * sizeof(uintptr_t));
 
diff --git a/src/http/modules/ngx_http_upstream_least_conn_module.c b/src/http/modules/ngx_http_upstream_least_conn_module.c
--- a/src/http/modules/ngx_http_upstream_least_conn_module.c
+++ b/src/http/modules/ngx_http_upstream_least_conn_module.c
@@ -136,7 +136,6 @@ ngx_http_upstream_get_least_conn_peer(ng
          peer;
          peer = peer->next, i++)
     {
-
         n = i / (8 * sizeof(uintptr_t));
         m = (uintptr_t) 1 << i % (8 * sizeof(uintptr_t));
 
diff --git a/src/http/ngx_http_upstream_round_robin.c b/src/http/ngx_http_upstream_round_robin.c
--- a/src/http/ngx_http_upstream_round_robin.c
+++ b/src/http/ngx_http_upstream_round_robin.c
@@ -521,7 +521,6 @@ ngx_http_upstream_get_peer(ngx_http_upst
          peer;
          peer = peer->next, i++)
     {
-
         n = i / (8 * sizeof(uintptr_t));
         m = (uintptr_t) 1 << i % (8 * sizeof(uintptr_t));
 
diff --git a/src/stream/ngx_stream_upstream_hash_module.c b/src/stream/ngx_stream_upstream_hash_module.c
--- a/src/stream/ngx_stream_upstream_hash_module.c
+++ b/src/stream/ngx_stream_upstream_hash_module.c
@@ -524,7 +524,6 @@ ngx_stream_upstream_get_chash_peer(ngx_p
              peer;
              peer = peer->next, i++)
         {
-
             n = i / (8 * sizeof(uintptr_t));
             m = (uintptr_t) 1 << i % (8 * sizeof(uintptr_t));
 
diff --git a/src/stream/ngx_stream_upstream_least_conn_module.c b/src/stream/ngx_stream_upstream_least_conn_module.c
--- a/src/stream/ngx_stream_upstream_least_conn_module.c
+++ b/src/stream/ngx_stream_upstream_least_conn_module.c
@@ -132,7 +132,6 @@ ngx_stream_upstream_get_least_conn_peer(
          peer;
          peer = peer->next, i++)
     {
-
         n = i / (8 * sizeof(uintptr_t));
         m = (uintptr_t) 1 << i % (8 * sizeof(uintptr_t));
 
diff --git a/src/stream/ngx_stream_upstream_round_robin.c b/src/stream/ngx_stream_upstream_round_robin.c
--- a/src/stream/ngx_stream_upstream_round_robin.c
+++ b/src/stream/ngx_stream_upstream_round_robin.c
@@ -527,7 +527,6 @@ ngx_stream_upstream_get_peer(ngx_stream_
          peer;
          peer = peer->next, i++)
     {
-
         n = i / (8 * sizeof(uintptr_t));
         m = (uintptr_t) 1 << i % (8 * sizeof(uintptr_t));
 


From mdounin at mdounin.ru  Thu Sep 22 19:59:49 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 22 Sep 2016 19:59:49 +0000
Subject: [nginx] Upstream: introduced u->upstream.
Message-ID: <hg.edcd9303a4d3.1474574389.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/edcd9303a4d3
branches:  
changeset: 6703:edcd9303a4d3
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Sep 22 19:32:26 2016 +0300
description:
Upstream: introduced u->upstream.

It holds upstream{} block configuration, including ones selected via
run-time lookup using variables.

diffstat:

 src/http/ngx_http_upstream.c         |  2 ++
 src/http/ngx_http_upstream.h         |  1 +
 src/stream/ngx_stream_proxy_module.c |  2 ++
 src/stream/ngx_stream_upstream.h     |  1 +
 4 files changed, 6 insertions(+), 0 deletions(-)

diffs (46 lines):

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -748,6 +748,8 @@ found:
         return;
     }
 
+    u->upstream = uscf;
+
 #if (NGX_HTTP_SSL)
     u->ssl_name = uscf->host;
 #endif
diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h
+++ b/src/http/ngx_http_upstream.h
@@ -313,6 +313,7 @@ struct ngx_http_upstream_s {
     ngx_chain_writer_ctx_t           writer;
 
     ngx_http_upstream_conf_t        *conf;
+    ngx_http_upstream_srv_conf_t    *upstream;
 #if (NGX_HTTP_CACHE)
     ngx_array_t                     *caches;
 #endif
diff --git a/src/stream/ngx_stream_proxy_module.c b/src/stream/ngx_stream_proxy_module.c
--- a/src/stream/ngx_stream_proxy_module.c
+++ b/src/stream/ngx_stream_proxy_module.c
@@ -521,6 +521,8 @@ found:
         return;
     }
 
+    u->upstream = uscf;
+
 #if (NGX_STREAM_SSL)
     u->ssl_name = uscf->host;
 #endif
diff --git a/src/stream/ngx_stream_upstream.h b/src/stream/ngx_stream_upstream.h
--- a/src/stream/ngx_stream_upstream.h
+++ b/src/stream/ngx_stream_upstream.h
@@ -124,6 +124,7 @@ typedef struct {
     ngx_str_t                          ssl_name;
 #endif
 
+    ngx_stream_upstream_srv_conf_t    *upstream;
     ngx_stream_upstream_resolved_t    *resolved;
     ngx_stream_upstream_state_t       *state;
     unsigned                           connected:1;


From mdounin at mdounin.ru  Thu Sep 22 19:59:52 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 22 Sep 2016 19:59:52 +0000
Subject: [nginx] Upstream: removed the quick recovery mechanism.
Message-ID: <hg.a44ba757851d.1474574392.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/a44ba757851d
branches:  
changeset: 6704:a44ba757851d
user:      Ruslan Ermilov <ru at nginx.com>
date:      Thu Sep 22 19:32:45 2016 +0300
description:
Upstream: removed the quick recovery mechanism.

Its usefulness it questionable, and it interacts badly with max_conns.

diffstat:

 src/http/modules/ngx_http_upstream_least_conn_module.c |  6 ------
 src/http/ngx_http_upstream_round_robin.c               |  6 ------
 src/stream/ngx_stream_upstream_least_conn_module.c     |  6 ------
 src/stream/ngx_stream_upstream_round_robin.c           |  6 ------
 4 files changed, 0 insertions(+), 24 deletions(-)

diffs (64 lines):

diff --git a/src/http/modules/ngx_http_upstream_least_conn_module.c b/src/http/modules/ngx_http_upstream_least_conn_module.c
--- a/src/http/modules/ngx_http_upstream_least_conn_module.c
+++ b/src/http/modules/ngx_http_upstream_least_conn_module.c
@@ -272,12 +272,6 @@ failed:
         ngx_http_upstream_rr_peers_wlock(peers);
     }
 
-    /* all peers failed, mark them as live for quick recovery */
-
-    for (peer = peers->peer; peer; peer = peer->next) {
-        peer->fails = 0;
-    }
-
     ngx_http_upstream_rr_peers_unlock(peers);
 
     pc->name = peers->name;
diff --git a/src/http/ngx_http_upstream_round_robin.c b/src/http/ngx_http_upstream_round_robin.c
--- a/src/http/ngx_http_upstream_round_robin.c
+++ b/src/http/ngx_http_upstream_round_robin.c
@@ -485,12 +485,6 @@ failed:
         ngx_http_upstream_rr_peers_wlock(peers);
     }
 
-    /* all peers failed, mark them as live for quick recovery */
-
-    for (peer = peers->peer; peer; peer = peer->next) {
-        peer->fails = 0;
-    }
-
     ngx_http_upstream_rr_peers_unlock(peers);
 
     pc->name = peers->name;
diff --git a/src/stream/ngx_stream_upstream_least_conn_module.c b/src/stream/ngx_stream_upstream_least_conn_module.c
--- a/src/stream/ngx_stream_upstream_least_conn_module.c
+++ b/src/stream/ngx_stream_upstream_least_conn_module.c
@@ -268,12 +268,6 @@ failed:
         ngx_stream_upstream_rr_peers_wlock(peers);
     }
 
-    /* all peers failed, mark them as live for quick recovery */
-
-    for (peer = peers->peer; peer; peer = peer->next) {
-        peer->fails = 0;
-    }
-
     ngx_stream_upstream_rr_peers_unlock(peers);
 
     pc->name = peers->name;
diff --git a/src/stream/ngx_stream_upstream_round_robin.c b/src/stream/ngx_stream_upstream_round_robin.c
--- a/src/stream/ngx_stream_upstream_round_robin.c
+++ b/src/stream/ngx_stream_upstream_round_robin.c
@@ -491,12 +491,6 @@ failed:
         ngx_stream_upstream_rr_peers_wlock(peers);
     }
 
-    /* all peers failed, mark them as live for quick recovery */
-
-    for (peer = peers->peer; peer; peer = peer->next) {
-        peer->fails = 0;
-    }
-
     ngx_stream_upstream_rr_peers_unlock(peers);
 
     pc->name = peers->name;


From mdounin at mdounin.ru  Thu Sep 22 19:59:54 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 22 Sep 2016 19:59:54 +0000
Subject: [nginx] Upstream: max_conns.
Message-ID: <hg.29bf0dbc0a77.1474574394.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/29bf0dbc0a77
branches:  
changeset: 6705:29bf0dbc0a77
user:      Ruslan Ermilov <ru at nginx.com>
date:      Thu Sep 22 19:32:47 2016 +0300
description:
Upstream: max_conns.

diffstat:

 src/http/modules/ngx_http_upstream_hash_module.c       |   9 ++++++++
 src/http/modules/ngx_http_upstream_ip_hash_module.c    |   5 ++++
 src/http/modules/ngx_http_upstream_least_conn_module.c |   9 ++++++++
 src/http/ngx_http_upstream.c                           |  20 +++++++++++++++++-
 src/http/ngx_http_upstream.h                           |   2 +
 src/http/ngx_http_upstream_round_robin.c               |  13 +++++++++++
 src/http/ngx_http_upstream_round_robin.h               |   1 +
 src/stream/ngx_stream_upstream.c                       |  20 +++++++++++++++++-
 src/stream/ngx_stream_upstream.h                       |   2 +
 src/stream/ngx_stream_upstream_hash_module.c           |   9 ++++++++
 src/stream/ngx_stream_upstream_least_conn_module.c     |   9 ++++++++
 src/stream/ngx_stream_upstream_round_robin.c           |  11 +++++++++
 src/stream/ngx_stream_upstream_round_robin.h           |   1 +
 13 files changed, 109 insertions(+), 2 deletions(-)

diffs (444 lines):

diff --git a/src/http/modules/ngx_http_upstream_hash_module.c b/src/http/modules/ngx_http_upstream_hash_module.c
--- a/src/http/modules/ngx_http_upstream_hash_module.c
+++ b/src/http/modules/ngx_http_upstream_hash_module.c
@@ -242,6 +242,10 @@ ngx_http_upstream_get_hash_peer(ngx_peer
             goto next;
         }
 
+        if (peer->max_conns && peer->conns >= peer->max_conns) {
+            goto next;
+        }
+
         break;
 
     next:
@@ -548,6 +552,10 @@ ngx_http_upstream_get_chash_peer(ngx_pee
                 continue;
             }
 
+            if (peer->max_conns && peer->conns >= peer->max_conns) {
+                continue;
+            }
+
             peer->current_weight += peer->effective_weight;
             total += peer->effective_weight;
 
@@ -647,6 +655,7 @@ ngx_http_upstream_hash(ngx_conf_t *cf, n
 
     uscf->flags = NGX_HTTP_UPSTREAM_CREATE
                   |NGX_HTTP_UPSTREAM_WEIGHT
+                  |NGX_HTTP_UPSTREAM_MAX_CONNS
                   |NGX_HTTP_UPSTREAM_MAX_FAILS
                   |NGX_HTTP_UPSTREAM_FAIL_TIMEOUT
                   |NGX_HTTP_UPSTREAM_DOWN;
diff --git a/src/http/modules/ngx_http_upstream_ip_hash_module.c b/src/http/modules/ngx_http_upstream_ip_hash_module.c
--- a/src/http/modules/ngx_http_upstream_ip_hash_module.c
+++ b/src/http/modules/ngx_http_upstream_ip_hash_module.c
@@ -212,6 +212,10 @@ ngx_http_upstream_get_ip_hash_peer(ngx_p
             goto next;
         }
 
+        if (peer->max_conns && peer->conns >= peer->max_conns) {
+            goto next;
+        }
+
         break;
 
     next:
@@ -259,6 +263,7 @@ ngx_http_upstream_ip_hash(ngx_conf_t *cf
 
     uscf->flags = NGX_HTTP_UPSTREAM_CREATE
                   |NGX_HTTP_UPSTREAM_WEIGHT
+                  |NGX_HTTP_UPSTREAM_MAX_CONNS
                   |NGX_HTTP_UPSTREAM_MAX_FAILS
                   |NGX_HTTP_UPSTREAM_FAIL_TIMEOUT
                   |NGX_HTTP_UPSTREAM_DOWN;
diff --git a/src/http/modules/ngx_http_upstream_least_conn_module.c b/src/http/modules/ngx_http_upstream_least_conn_module.c
--- a/src/http/modules/ngx_http_upstream_least_conn_module.c
+++ b/src/http/modules/ngx_http_upstream_least_conn_module.c
@@ -154,6 +154,10 @@ ngx_http_upstream_get_least_conn_peer(ng
             continue;
         }
 
+        if (peer->max_conns && peer->conns >= peer->max_conns) {
+            continue;
+        }
+
         /*
          * select peer with least number of connections; if there are
          * multiple peers with the same number of connections, select
@@ -209,6 +213,10 @@ ngx_http_upstream_get_least_conn_peer(ng
                 continue;
             }
 
+            if (peer->max_conns && peer->conns >= peer->max_conns) {
+                continue;
+            }
+
             peer->current_weight += peer->effective_weight;
             total += peer->effective_weight;
 
@@ -296,6 +304,7 @@ ngx_http_upstream_least_conn(ngx_conf_t 
 
     uscf->flags = NGX_HTTP_UPSTREAM_CREATE
                   |NGX_HTTP_UPSTREAM_WEIGHT
+                  |NGX_HTTP_UPSTREAM_MAX_CONNS
                   |NGX_HTTP_UPSTREAM_MAX_FAILS
                   |NGX_HTTP_UPSTREAM_FAIL_TIMEOUT
                   |NGX_HTTP_UPSTREAM_DOWN
diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -5444,6 +5444,7 @@ ngx_http_upstream(ngx_conf_t *cf, ngx_co
 
     uscf = ngx_http_upstream_add(cf, &u, NGX_HTTP_UPSTREAM_CREATE
                                          |NGX_HTTP_UPSTREAM_WEIGHT
+                                         |NGX_HTTP_UPSTREAM_MAX_CONNS
                                          |NGX_HTTP_UPSTREAM_MAX_FAILS
                                          |NGX_HTTP_UPSTREAM_FAIL_TIMEOUT
                                          |NGX_HTTP_UPSTREAM_DOWN
@@ -5545,7 +5546,7 @@ ngx_http_upstream_server(ngx_conf_t *cf,
     time_t                       fail_timeout;
     ngx_str_t                   *value, s;
     ngx_url_t                    u;
-    ngx_int_t                    weight, max_fails;
+    ngx_int_t                    weight, max_conns, max_fails;
     ngx_uint_t                   i;
     ngx_http_upstream_server_t  *us;
 
@@ -5559,6 +5560,7 @@ ngx_http_upstream_server(ngx_conf_t *cf,
     value = cf->args->elts;
 
     weight = 1;
+    max_conns = 0;
     max_fails = 1;
     fail_timeout = 10;
 
@@ -5579,6 +5581,21 @@ ngx_http_upstream_server(ngx_conf_t *cf,
             continue;
         }
 
+        if (ngx_strncmp(value[i].data, "max_conns=", 10) == 0) {
+
+            if (!(uscf->flags & NGX_HTTP_UPSTREAM_MAX_CONNS)) {
+                goto not_supported;
+            }
+
+            max_conns = ngx_atoi(&value[i].data[10], value[i].len - 10);
+
+            if (max_conns == NGX_ERROR) {
+                goto invalid;
+            }
+
+            continue;
+        }
+
         if (ngx_strncmp(value[i].data, "max_fails=", 10) == 0) {
 
             if (!(uscf->flags & NGX_HTTP_UPSTREAM_MAX_FAILS)) {
@@ -5655,6 +5672,7 @@ ngx_http_upstream_server(ngx_conf_t *cf,
     us->addrs = u.addrs;
     us->naddrs = u.naddrs;
     us->weight = weight;
+    us->max_conns = max_conns;
     us->max_fails = max_fails;
     us->fail_timeout = fail_timeout;
 
diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h
+++ b/src/http/ngx_http_upstream.h
@@ -95,6 +95,7 @@ typedef struct {
     ngx_addr_t                      *addrs;
     ngx_uint_t                       naddrs;
     ngx_uint_t                       weight;
+    ngx_uint_t                       max_conns;
     ngx_uint_t                       max_fails;
     time_t                           fail_timeout;
 
@@ -109,6 +110,7 @@ typedef struct {
 #define NGX_HTTP_UPSTREAM_FAIL_TIMEOUT  0x0008
 #define NGX_HTTP_UPSTREAM_DOWN          0x0010
 #define NGX_HTTP_UPSTREAM_BACKUP        0x0020
+#define NGX_HTTP_UPSTREAM_MAX_CONNS     0x0100
 
 
 struct ngx_http_upstream_srv_conf_s {
diff --git a/src/http/ngx_http_upstream_round_robin.c b/src/http/ngx_http_upstream_round_robin.c
--- a/src/http/ngx_http_upstream_round_robin.c
+++ b/src/http/ngx_http_upstream_round_robin.c
@@ -92,6 +92,7 @@ ngx_http_upstream_init_round_robin(ngx_c
                 peer[n].weight = server[i].weight;
                 peer[n].effective_weight = server[i].weight;
                 peer[n].current_weight = 0;
+                peer[n].max_conns = server[i].max_conns;
                 peer[n].max_fails = server[i].max_fails;
                 peer[n].fail_timeout = server[i].fail_timeout;
                 peer[n].down = server[i].down;
@@ -155,6 +156,7 @@ ngx_http_upstream_init_round_robin(ngx_c
                 peer[n].weight = server[i].weight;
                 peer[n].effective_weight = server[i].weight;
                 peer[n].current_weight = 0;
+                peer[n].max_conns = server[i].max_conns;
                 peer[n].max_fails = server[i].max_fails;
                 peer[n].fail_timeout = server[i].fail_timeout;
                 peer[n].down = server[i].down;
@@ -223,6 +225,7 @@ ngx_http_upstream_init_round_robin(ngx_c
         peer[i].weight = 1;
         peer[i].effective_weight = 1;
         peer[i].current_weight = 0;
+        peer[i].max_conns = 0;
         peer[i].max_fails = 1;
         peer[i].fail_timeout = 10;
         *peerp = &peer[i];
@@ -337,6 +340,7 @@ ngx_http_upstream_create_round_robin_pee
         peer[0].weight = 1;
         peer[0].effective_weight = 1;
         peer[0].current_weight = 0;
+        peer[0].max_conns = 0;
         peer[0].max_fails = 1;
         peer[0].fail_timeout = 10;
         peers->peer = peer;
@@ -370,6 +374,7 @@ ngx_http_upstream_create_round_robin_pee
             peer[i].weight = 1;
             peer[i].effective_weight = 1;
             peer[i].current_weight = 0;
+            peer[i].max_conns = 0;
             peer[i].max_fails = 1;
             peer[i].fail_timeout = 10;
             *peerp = &peer[i];
@@ -432,6 +437,10 @@ ngx_http_upstream_get_round_robin_peer(n
             goto failed;
         }
 
+        if (peer->max_conns && peer->conns >= peer->max_conns) {
+            goto failed;
+        }
+
         rrp->current = peer;
 
     } else {
@@ -533,6 +542,10 @@ ngx_http_upstream_get_peer(ngx_http_upst
             continue;
         }
 
+        if (peer->max_conns && peer->conns >= peer->max_conns) {
+            continue;
+        }
+
         peer->current_weight += peer->effective_weight;
         total += peer->effective_weight;
 
diff --git a/src/http/ngx_http_upstream_round_robin.h b/src/http/ngx_http_upstream_round_robin.h
--- a/src/http/ngx_http_upstream_round_robin.h
+++ b/src/http/ngx_http_upstream_round_robin.h
@@ -27,6 +27,7 @@ struct ngx_http_upstream_rr_peer_s {
     ngx_int_t                       weight;
 
     ngx_uint_t                      conns;
+    ngx_uint_t                      max_conns;
 
     ngx_uint_t                      fails;
     time_t                          accessed;
diff --git a/src/stream/ngx_stream_upstream.c b/src/stream/ngx_stream_upstream.c
--- a/src/stream/ngx_stream_upstream.c
+++ b/src/stream/ngx_stream_upstream.c
@@ -322,6 +322,7 @@ ngx_stream_upstream(ngx_conf_t *cf, ngx_
 
     uscf = ngx_stream_upstream_add(cf, &u, NGX_STREAM_UPSTREAM_CREATE
                                            |NGX_STREAM_UPSTREAM_WEIGHT
+                                           |NGX_STREAM_UPSTREAM_MAX_CONNS
                                            |NGX_STREAM_UPSTREAM_MAX_FAILS
                                            |NGX_STREAM_UPSTREAM_FAIL_TIMEOUT
                                            |NGX_STREAM_UPSTREAM_DOWN
@@ -407,7 +408,7 @@ ngx_stream_upstream_server(ngx_conf_t *c
     time_t                         fail_timeout;
     ngx_str_t                     *value, s;
     ngx_url_t                      u;
-    ngx_int_t                      weight, max_fails;
+    ngx_int_t                      weight, max_conns, max_fails;
     ngx_uint_t                     i;
     ngx_stream_upstream_server_t  *us;
 
@@ -421,6 +422,7 @@ ngx_stream_upstream_server(ngx_conf_t *c
     value = cf->args->elts;
 
     weight = 1;
+    max_conns = 0;
     max_fails = 1;
     fail_timeout = 10;
 
@@ -441,6 +443,21 @@ ngx_stream_upstream_server(ngx_conf_t *c
             continue;
         }
 
+        if (ngx_strncmp(value[i].data, "max_conns=", 10) == 0) {
+
+            if (!(uscf->flags & NGX_STREAM_UPSTREAM_MAX_CONNS)) {
+                goto not_supported;
+            }
+
+            max_conns = ngx_atoi(&value[i].data[10], value[i].len - 10);
+
+            if (max_conns == NGX_ERROR) {
+                goto invalid;
+            }
+
+            continue;
+        }
+
         if (ngx_strncmp(value[i].data, "max_fails=", 10) == 0) {
 
             if (!(uscf->flags & NGX_STREAM_UPSTREAM_MAX_FAILS)) {
@@ -522,6 +539,7 @@ ngx_stream_upstream_server(ngx_conf_t *c
     us->addrs = u.addrs;
     us->naddrs = u.naddrs;
     us->weight = weight;
+    us->max_conns = max_conns;
     us->max_fails = max_fails;
     us->fail_timeout = fail_timeout;
 
diff --git a/src/stream/ngx_stream_upstream.h b/src/stream/ngx_stream_upstream.h
--- a/src/stream/ngx_stream_upstream.h
+++ b/src/stream/ngx_stream_upstream.h
@@ -21,6 +21,7 @@
 #define NGX_STREAM_UPSTREAM_FAIL_TIMEOUT  0x0008
 #define NGX_STREAM_UPSTREAM_DOWN          0x0010
 #define NGX_STREAM_UPSTREAM_BACKUP        0x0020
+#define NGX_STREAM_UPSTREAM_MAX_CONNS     0x0100
 
 
 typedef struct {
@@ -50,6 +51,7 @@ typedef struct {
     ngx_addr_t                        *addrs;
     ngx_uint_t                         naddrs;
     ngx_uint_t                         weight;
+    ngx_uint_t                         max_conns;
     ngx_uint_t                         max_fails;
     time_t                             fail_timeout;
 
diff --git a/src/stream/ngx_stream_upstream_hash_module.c b/src/stream/ngx_stream_upstream_hash_module.c
--- a/src/stream/ngx_stream_upstream_hash_module.c
+++ b/src/stream/ngx_stream_upstream_hash_module.c
@@ -241,6 +241,10 @@ ngx_stream_upstream_get_hash_peer(ngx_pe
             goto next;
         }
 
+        if (peer->max_conns && peer->conns >= peer->max_conns) {
+            goto next;
+        }
+
         break;
 
     next:
@@ -549,6 +553,10 @@ ngx_stream_upstream_get_chash_peer(ngx_p
                 continue;
             }
 
+            if (peer->max_conns && peer->conns >= peer->max_conns) {
+                continue;
+            }
+
             peer->current_weight += peer->effective_weight;
             total += peer->effective_weight;
 
@@ -646,6 +654,7 @@ ngx_stream_upstream_hash(ngx_conf_t *cf,
 
     uscf->flags = NGX_STREAM_UPSTREAM_CREATE
                   |NGX_STREAM_UPSTREAM_WEIGHT
+                  |NGX_STREAM_UPSTREAM_MAX_CONNS
                   |NGX_STREAM_UPSTREAM_MAX_FAILS
                   |NGX_STREAM_UPSTREAM_FAIL_TIMEOUT
                   |NGX_STREAM_UPSTREAM_DOWN;
diff --git a/src/stream/ngx_stream_upstream_least_conn_module.c b/src/stream/ngx_stream_upstream_least_conn_module.c
--- a/src/stream/ngx_stream_upstream_least_conn_module.c
+++ b/src/stream/ngx_stream_upstream_least_conn_module.c
@@ -150,6 +150,10 @@ ngx_stream_upstream_get_least_conn_peer(
             continue;
         }
 
+        if (peer->max_conns && peer->conns >= peer->max_conns) {
+            continue;
+        }
+
         /*
          * select peer with least number of connections; if there are
          * multiple peers with the same number of connections, select
@@ -205,6 +209,10 @@ ngx_stream_upstream_get_least_conn_peer(
                 continue;
             }
 
+            if (peer->max_conns && peer->conns >= peer->max_conns) {
+                continue;
+            }
+
             peer->current_weight += peer->effective_weight;
             total += peer->effective_weight;
 
@@ -292,6 +300,7 @@ ngx_stream_upstream_least_conn(ngx_conf_
 
     uscf->flags = NGX_STREAM_UPSTREAM_CREATE
                   |NGX_STREAM_UPSTREAM_WEIGHT
+                  |NGX_STREAM_UPSTREAM_MAX_CONNS
                   |NGX_STREAM_UPSTREAM_MAX_FAILS
                   |NGX_STREAM_UPSTREAM_FAIL_TIMEOUT
                   |NGX_STREAM_UPSTREAM_DOWN
diff --git a/src/stream/ngx_stream_upstream_round_robin.c b/src/stream/ngx_stream_upstream_round_robin.c
--- a/src/stream/ngx_stream_upstream_round_robin.c
+++ b/src/stream/ngx_stream_upstream_round_robin.c
@@ -96,6 +96,7 @@ ngx_stream_upstream_init_round_robin(ngx
                 peer[n].weight = server[i].weight;
                 peer[n].effective_weight = server[i].weight;
                 peer[n].current_weight = 0;
+                peer[n].max_conns = server[i].max_conns;
                 peer[n].max_fails = server[i].max_fails;
                 peer[n].fail_timeout = server[i].fail_timeout;
                 peer[n].down = server[i].down;
@@ -159,6 +160,7 @@ ngx_stream_upstream_init_round_robin(ngx
                 peer[n].weight = server[i].weight;
                 peer[n].effective_weight = server[i].weight;
                 peer[n].current_weight = 0;
+                peer[n].max_conns = server[i].max_conns;
                 peer[n].max_fails = server[i].max_fails;
                 peer[n].fail_timeout = server[i].fail_timeout;
                 peer[n].down = server[i].down;
@@ -227,6 +229,7 @@ ngx_stream_upstream_init_round_robin(ngx
         peer[i].weight = 1;
         peer[i].effective_weight = 1;
         peer[i].current_weight = 0;
+        peer[i].max_conns = 0;
         peer[i].max_fails = 1;
         peer[i].fail_timeout = 10;
         *peerp = &peer[i];
@@ -438,6 +441,10 @@ ngx_stream_upstream_get_round_robin_peer
             goto failed;
         }
 
+        if (peer->max_conns && peer->conns >= peer->max_conns) {
+            goto failed;
+        }
+
         rrp->current = peer;
 
     } else {
@@ -539,6 +546,10 @@ ngx_stream_upstream_get_peer(ngx_stream_
             continue;
         }
 
+        if (peer->max_conns && peer->conns >= peer->max_conns) {
+            continue;
+        }
+
         peer->current_weight += peer->effective_weight;
         total += peer->effective_weight;
 
diff --git a/src/stream/ngx_stream_upstream_round_robin.h b/src/stream/ngx_stream_upstream_round_robin.h
--- a/src/stream/ngx_stream_upstream_round_robin.h
+++ b/src/stream/ngx_stream_upstream_round_robin.h
@@ -27,6 +27,7 @@ struct ngx_stream_upstream_rr_peer_s {
     ngx_int_t                        weight;
 
     ngx_uint_t                       conns;
+    ngx_uint_t                       max_conns;
 
     ngx_uint_t                       fails;
     time_t                           accessed;


From igor at sysoev.ru  Fri Sep 23 11:00:13 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Fri, 23 Sep 2016 11:00:13 +0000
Subject: [njs] String.replace() function.
Message-ID: <hg.04aee54864c3.1474628413.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/04aee54864c3
branches:  
changeset: 177:04aee54864c3
user:      Igor Sysoev <igor at sysoev.ru>
date:      Fri Sep 23 11:59:48 2016 +0300
description:
String.replace() function.

diffstat:

 njs/njs_function.h       |   11 +-
 njs/njs_regexp.c         |    4 +-
 njs/njs_regexp_pattern.h |    6 +
 njs/njs_string.c         |  739 ++++++++++++++++++++++++++++++++++++++++++++++-
 njs/njs_string.h         |   29 +
 njs/njs_vm.c             |   14 +
 njs/test/njs_unit_test.c |   90 +++++
 nxt/auto/clang           |    2 +-
 nxt/nxt_array.c          |   66 ++-
 nxt/nxt_array.h          |   20 +-
 nxt/nxt_pcre.c           |   23 +-
 nxt/nxt_pcre.h           |    4 +-
 nxt/nxt_regex.h          |    1 +
 13 files changed, 949 insertions(+), 60 deletions(-)

diffs (truncated from 1296 to 1000 lines):

diff -r 28bc6acceb0e -r 04aee54864c3 njs/njs_function.h
--- a/njs/njs_function.h	Thu Sep 15 18:00:55 2016 +0300
+++ b/njs/njs_function.h	Fri Sep 23 11:59:48 2016 +0300
@@ -11,11 +11,12 @@
 #define NJS_SKIP_ARG               1
 #define NJS_NUMBER_ARG             2
 #define NJS_INTEGER_ARG            3
-#define NJS_STRING_OBJECT_ARG      4
-#define NJS_STRING_ARG             5
-#define NJS_OBJECT_ARG             6
-#define NJS_REGEXP_ARG             7
-#define NJS_DATE_ARG               8
+#define NJS_STRING_ARG             4
+#define NJS_OBJECT_ARG             5
+#define NJS_STRING_OBJECT_ARG      6
+#define NJS_FUNCTION_ARG           7
+#define NJS_REGEXP_ARG             8
+#define NJS_DATE_ARG               9
 
 
 struct njs_function_lambda_s {
diff -r 28bc6acceb0e -r 04aee54864c3 njs/njs_regexp.c
--- a/njs/njs_regexp.c	Thu Sep 15 18:00:55 2016 +0300
+++ b/njs/njs_regexp.c	Fri Sep 23 11:59:48 2016 +0300
@@ -609,7 +609,7 @@ njs_regexp_prototype_test(njs_vm_t *vm, 
         if (ret >= 0) {
             retval = &njs_value_true;
 
-        } else if (ret != NGX_REGEX_NOMATCH) {
+        } else if (ret != NXT_REGEX_NOMATCH) {
             return NXT_ERROR;
         }
     }
@@ -683,7 +683,7 @@ njs_regexp_prototype_exec(njs_vm_t *vm, 
                                           utf8);
         }
 
-        if (nxt_slow_path(ret != NGX_REGEX_NOMATCH)) {
+        if (nxt_slow_path(ret != NXT_REGEX_NOMATCH)) {
             nxt_regex_match_data_free(match_data, vm->regex_context);
 
             return NXT_ERROR;
diff -r 28bc6acceb0e -r 04aee54864c3 njs/njs_regexp_pattern.h
--- a/njs/njs_regexp_pattern.h	Thu Sep 15 18:00:55 2016 +0300
+++ b/njs/njs_regexp_pattern.h	Fri Sep 23 11:59:48 2016 +0300
@@ -10,6 +10,12 @@
 #include <pcre.h>
 
 
+typedef enum {
+    NJS_REGEXP_BYTE = 0,
+    NJS_REGEXP_UTF8,
+} njs_regexp_utf8_t;
+
+
 struct njs_regexp_pattern_s {
     nxt_regex_t           regex[2];
 
diff -r 28bc6acceb0e -r 04aee54864c3 njs/njs_string.c
--- a/njs/njs_string.c	Thu Sep 15 18:00:55 2016 +0300
+++ b/njs/njs_string.c	Fri Sep 23 11:59:48 2016 +0300
@@ -33,6 +33,50 @@
 #include <string.h>
 
 
+typedef struct {
+    u_char                     *start;
+    size_t                     size;
+    njs_value_t                value;
+} njs_string_replace_part_t;
+
+
+#define NJS_SUBST_COPY        255
+#define NJS_SUBST_PRECEDING   254
+#define NJS_SUBST_FOLLOWING   253
+
+
+typedef struct {
+     uint32_t  type;
+     uint32_t  size;
+     u_char    *start;
+} njs_string_subst_t;
+
+
+typedef struct {
+    union {
+        njs_continuation_t     cont;
+        u_char                 padding[NJS_CONTINUATION_SIZE];
+    } u;
+    /*
+     * This retval value must be aligned so the continuation
+     * is padded to aligned size.
+     */
+    njs_value_t                retval;
+
+    nxt_array_t                parts;
+    njs_string_replace_part_t  array[3];
+    njs_string_replace_part_t  *part;
+
+    nxt_array_t                *substitutions;
+    njs_function_t             *function;
+
+    nxt_regex_match_data_t     *match_data;
+
+    njs_utf8_t                 utf8:8;
+    njs_regexp_utf8_t          type:8;
+} njs_string_replace_t;
+
+
 static nxt_noinline void njs_string_slice_prop(njs_string_prop_t *string,
     njs_slice_prop_t *slice, njs_value_t *args, nxt_uint_t nargs);
 static nxt_noinline void njs_string_slice_args(njs_slice_prop_t *slice,
@@ -45,6 +89,28 @@ static njs_ret_t njs_string_match_multip
     njs_regexp_pattern_t *pattern);
 static njs_ret_t njs_string_split_part_add(njs_vm_t *vm, njs_array_t *array,
     u_char *start, size_t size, nxt_uint_t utf8);
+static njs_ret_t njs_string_replace_regexp(njs_vm_t *vm, njs_value_t *args,
+    njs_string_replace_t *r);
+static njs_ret_t njs_string_replace_regexp_function(njs_vm_t *vm,
+    njs_value_t *args, njs_string_replace_t *r, int *captures, nxt_uint_t n);
+static njs_ret_t njs_string_replace_regexp_continuation(njs_vm_t *vm,
+    njs_value_t *args, nxt_uint_t nargs, njs_index_t unused);
+static njs_ret_t njs_string_replace_regexp_join(njs_vm_t *vm,
+    njs_string_replace_t *r);
+static njs_ret_t njs_string_replace_search(njs_vm_t *vm, njs_value_t *args,
+    njs_string_replace_t *r);
+static njs_ret_t njs_string_replace_search_function(njs_vm_t *vm,
+    njs_value_t *args, njs_string_replace_t *r);
+static njs_ret_t njs_string_replace_search_continuation(njs_vm_t *vm,
+    njs_value_t *args, nxt_uint_t nargs, njs_index_t unused);
+static njs_ret_t njs_string_replace_parse(njs_vm_t *vm,
+    njs_string_replace_t *r, u_char *p, u_char *end, size_t size,
+    nxt_uint_t ncaptures);
+static njs_ret_t njs_string_replace_substitute(njs_vm_t *vm,
+    njs_string_replace_t *r, int *captures);
+static njs_ret_t njs_string_replace_join(njs_vm_t *vm, njs_string_replace_t *r);
+static void njs_string_replacement_copy(njs_string_replace_part_t *string,
+    const njs_value_t *value);
 static njs_ret_t njs_string_encode(njs_vm_t *vm, njs_value_t *value,
     const uint32_t *escape);
 static njs_ret_t njs_string_decode(njs_vm_t *vm, njs_value_t *value,
@@ -1471,7 +1537,7 @@ njs_string_prototype_search(njs_vm_t *vm
                 captures = nxt_regex_captures(vm->single_match_data);
                 index = njs_string_index(&string, captures[0]);
 
-            } else if (ret != NGX_REGEX_NOMATCH) {
+            } else if (ret != NXT_REGEX_NOMATCH) {
                 return NXT_ERROR;
             }
         }
@@ -1629,7 +1695,7 @@ njs_string_match_multiple(njs_vm_t *vm, 
 
                 array->length++;
 
-            } else if (ret == NGX_REGEX_NOMATCH) {
+            } else if (ret == NXT_REGEX_NOMATCH) {
                 break;
 
             } else {
@@ -1760,7 +1826,7 @@ njs_string_prototype_split(njs_vm_t *vm,
                     p = start + captures[0];
                     next = start + captures[1];
 
-                } else if (ret == NGX_REGEX_NOMATCH) {
+                } else if (ret == NXT_REGEX_NOMATCH) {
                     p = (u_char *) end;
                     next = (u_char *) end + 1;
 
@@ -1837,6 +1903,665 @@ njs_string_split_part_add(njs_vm_t *vm, 
 }
 
 
+/*
+ * String.replace([regexp|string[, string|function]])
+ */
+
+static njs_ret_t
+njs_string_prototype_replace(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
+    njs_index_t unused)
+{
+    u_char                *p, *start, *end;
+    njs_ret_t             ret;
+    nxt_uint_t            ncaptures;
+    nxt_regex_t           *regex;
+    njs_string_prop_t     string;
+    njs_string_replace_t  *r;
+
+    if (nargs == 1) {
+        goto original;
+    }
+
+    (void) njs_string_prop(&string, &args[0]);
+
+    if (string.size == 0) {
+        goto original;
+    }
+
+    r = njs_continuation(vm->frame);
+
+    r->utf8 = NJS_STRING_BYTE;
+    r->type = NJS_REGEXP_BYTE;
+
+    if (string.length != 0) {
+        r->utf8 = NJS_STRING_ASCII;
+        r->type = NJS_REGEXP_UTF8;
+
+        if (string.length != string.size) {
+            r->utf8 = NJS_STRING_UTF8;
+        }
+    }
+
+    if (njs_is_regexp(&args[1])) {
+        regex = &args[1].data.u.regexp->pattern->regex[r->type];
+
+        if (!nxt_regex_is_valid(regex)) {
+            goto original;
+        }
+
+        ncaptures = nxt_regex_ncaptures(regex);
+
+    } else {
+        regex = NULL;
+        ncaptures = 1;
+    }
+
+    /* This cannot fail. */
+    r->part = nxt_array_init(&r->parts, &r->array,
+                             3, sizeof(njs_string_replace_part_t),
+                             &njs_array_mem_proto, vm->mem_cache_pool);
+
+    r->substitutions = NULL;
+    r->function = NULL;
+
+    /* A literal replacement is stored in the second part. */
+
+    if (nargs == 2) {
+        njs_string_replacement_copy(&r->part[1], &njs_string_void);
+
+    } else if (njs_is_string(&args[2])) {
+        njs_string_replacement_copy(&r->part[1], &args[2]);
+
+        start = r->part[1].start;
+
+        if (start == NULL) {
+            start = r->part[1].value.short_string.start;
+        }
+
+        end = start + r->part[1].size;
+
+        for (p = start; p < end; p++) {
+            if (*p == '$') {
+                ret = njs_string_replace_parse(vm, r, p, end, p - start,
+                                               ncaptures);
+                if (nxt_slow_path(ret != NXT_OK)) {
+                    return ret;
+                }
+
+                /* Reset parts array to the subject string only. */
+                r->parts.items = 1;
+
+                break;
+            }
+        }
+
+    } else {
+        r->function = args[2].data.u.function;
+    }
+
+    r->part[0].start = string.start;
+    r->part[0].size = string.size;
+    njs_set_invalid(&r->part[0].value);
+
+    if (regex != NULL) {
+        r->match_data = nxt_regex_match_data(regex, vm->regex_context);
+        if (nxt_slow_path(r->match_data == NULL)) {
+            return NXT_ERROR;
+        }
+
+        return njs_string_replace_regexp(vm, args, r);
+    }
+
+    return njs_string_replace_search(vm, args, r);
+
+original:
+
+    njs_string_copy(&vm->retval, &args[0]);
+
+    return NXT_OK;
+}
+
+
+static njs_ret_t
+njs_string_replace_regexp(njs_vm_t *vm, njs_value_t *args,
+    njs_string_replace_t *r)
+{
+    int                        *captures;
+    njs_ret_t                  ret;
+    njs_regexp_pattern_t       *pattern;
+    njs_string_replace_part_t  *part;
+
+    pattern = args[1].data.u.regexp->pattern;
+
+    do {
+        ret = njs_regexp_match(vm, &pattern->regex[r->type],
+                               r->part[0].start, r->part[0].size,
+                               r->match_data);
+
+        if (ret >= 0) {
+            captures = nxt_regex_captures(r->match_data);
+
+            if (r->substitutions != NULL) {
+                ret = njs_string_replace_substitute(vm, r, captures);
+                if (nxt_slow_path(ret != NXT_OK)) {
+                    return ret;
+                }
+
+                if (!pattern->global) {
+                    return njs_string_replace_regexp_join(vm, r);
+                }
+
+            } else {
+                if (r->part != r->parts.start) {
+                    r->part = nxt_array_add(&r->parts, &njs_array_mem_proto,
+                                            vm->mem_cache_pool);
+                    if (nxt_slow_path(r->part == NULL)) {
+                        return NXT_ERROR;
+                    }
+
+                    r->part = nxt_array_add(&r->parts, &njs_array_mem_proto,
+                                            vm->mem_cache_pool);
+                    if (nxt_slow_path(r->part == NULL)) {
+                        return NXT_ERROR;
+                    }
+
+                    r->part -= 2;
+                }
+
+                r->part[2].start = r->part[0].start + captures[1];
+                r->part[2].size = r->part[0].size - captures[1];
+                njs_set_invalid(&r->part[2].value);
+
+                if (r->function != NULL) {
+                    return njs_string_replace_regexp_function(vm, args, r,
+                                                              captures, ret);
+                }
+
+                r->part[0].size = captures[0];
+
+                if (!pattern->global) {
+                    return njs_string_replace_regexp_join(vm, r);
+                }
+
+                /* A literal replacement is stored in the second part. */
+                part = r->parts.start;
+                r->part[1] = part[1];
+
+                r->part += 2;
+            }
+
+        } else if (ret == NXT_REGEX_NOMATCH) {
+            break;
+
+        } else {
+            return NXT_ERROR;
+        }
+
+    } while (r->part[0].size > 0);
+
+    if (r->part != r->parts.start) {
+        return njs_string_replace_regexp_join(vm, r);
+    }
+
+    nxt_regex_match_data_free(r->match_data, vm->regex_context);
+
+    nxt_array_destroy(&r->parts, &njs_array_mem_proto, vm->mem_cache_pool);
+
+    njs_string_copy(&vm->retval, &args[0]);
+
+    return NXT_OK;
+}
+
+
+static njs_ret_t
+njs_string_replace_regexp_function(njs_vm_t *vm, njs_value_t *args,
+    njs_string_replace_t *r, int *captures, nxt_uint_t n)
+{
+    u_char       *start;
+    size_t       size, length;
+    njs_ret_t    ret;
+    nxt_uint_t   i, k;
+    njs_value_t  *arguments;
+
+    r->u.cont.function = njs_string_replace_regexp_continuation;
+    njs_set_invalid(&r->retval);
+
+    arguments = nxt_mem_cache_alloc(vm->mem_cache_pool,
+                                    (n + 3) * sizeof(njs_value_t));
+    if (nxt_slow_path(arguments == NULL)) {
+        return NXT_ERROR;
+    }
+
+    arguments[0] = njs_value_void;
+
+    /* Matched substring and parenthesized submatch strings. */
+    for (k = 0, i = 1; i <= n; i++) {
+
+        start = r->part[0].start + captures[k];
+        size = captures[k + 1] - captures[k];
+        k += 2;
+
+        length = njs_string_length(start, size, r->utf8);
+
+        ret = njs_string_create(vm, &arguments[i], start, size, length);
+        if (nxt_slow_path(ret != NXT_OK)) {
+            return NXT_ERROR;
+        }
+    }
+
+    /* The offset of the matched substring. */
+    njs_number_set(&arguments[n + 1], captures[0]);
+
+    /* The whole string being examined. */
+    length = njs_string_length(r->part[0].start, r->part[0].size, r->utf8);
+
+    ret = njs_string_create(vm, &arguments[n + 2], r->part[0].start,
+                            r->part[0].size, length);
+
+    if (nxt_slow_path(ret != NXT_OK)) {
+        return NXT_ERROR;
+    }
+
+    r->part[0].size = captures[0];
+
+    return njs_function_apply(vm, r->function, arguments, n + 3,
+                              (njs_index_t) &r->retval);
+}
+
+
+static njs_ret_t
+njs_string_replace_regexp_continuation(njs_vm_t *vm, njs_value_t *args,
+    nxt_uint_t nargs, njs_index_t unused)
+{
+    njs_string_replace_t  *r;
+
+    r = njs_continuation(vm->frame);
+
+    if (njs_is_string(&r->retval)) {
+        njs_string_replacement_copy(&r->part[1], &r->retval);
+
+        if (args[1].data.u.regexp->pattern->global) {
+            r->part += 2;
+            return njs_string_replace_regexp(vm, args, r);
+        }
+
+        return njs_string_replace_regexp_join(vm, r);
+    }
+
+    nxt_regex_match_data_free(r->match_data, vm->regex_context);
+
+    vm->exception = &njs_exception_type_error;
+
+    return NXT_ERROR;
+}
+
+
+static njs_ret_t
+njs_string_replace_regexp_join(njs_vm_t *vm, njs_string_replace_t *r)
+{
+    nxt_regex_match_data_free(r->match_data, vm->regex_context);
+
+    return njs_string_replace_join(vm, r);
+}
+
+
+static njs_ret_t
+njs_string_replace_search(njs_vm_t *vm, njs_value_t *args,
+    njs_string_replace_t *r)
+{
+    int                captures[2];
+    u_char             *p, *end;
+    size_t             size;
+    njs_ret_t          ret;
+    njs_string_prop_t  search;
+
+    (void) njs_string_prop(&search, &args[1]);
+
+    p = r->part[0].start;
+    end = (p + r->part[0].size) - (search.size - 1);
+
+    do {
+        if (memcmp(p, search.start, search.size) == 0) {
+
+            if (r->substitutions != NULL) {
+                captures[0] = p - r->part[0].start;
+                captures[1] = captures[0] + search.size;
+
+                ret = njs_string_replace_substitute(vm, r, captures);
+                if (nxt_slow_path(ret != NXT_OK)) {
+                    return ret;
+                }
+
+            } else {
+                r->part[2].start = p + search.size;
+                size = p - r->part[0].start;
+                r->part[2].size = r->part[0].size - size - search.size;
+                r->part[0].size = size;
+                njs_set_invalid(&r->part[2].value);
+
+                if (r->function != NULL) {
+                    return njs_string_replace_search_function(vm, args, r);
+                }
+            }
+
+            return njs_string_replace_join(vm, r);
+        }
+
+        if (r->utf8 < 2) {
+            p++;
+
+        } else {
+            p = (u_char *) nxt_utf8_next(p, end);
+        }
+
+    } while (p < end);
+
+    njs_string_copy(&vm->retval, &args[0]);
+
+    return NXT_OK;
+}
+
+
+static njs_ret_t
+njs_string_replace_search_function(njs_vm_t *vm, njs_value_t *args,
+    njs_string_replace_t *r)
+{
+    njs_value_t  arguments[4];
+
+    r->u.cont.function = njs_string_replace_search_continuation;
+
+    arguments[0] = njs_value_void;
+
+    /* GC, args[0], args[1] */
+
+    /* Matched substring, it is the same as the args[1]. */
+    arguments[1] = args[1];
+
+    /* The offset of the matched substring. */
+    njs_number_set(&arguments[2], r->part[0].size);
+
+    /* The whole string being examined. */
+    arguments[3] = args[0];
+
+    return njs_function_apply(vm, r->function, arguments, 4,
+                              (njs_index_t) &r->retval);
+}
+
+
+static njs_ret_t
+njs_string_replace_search_continuation(njs_vm_t *vm, njs_value_t *args,
+    nxt_uint_t nargs, njs_index_t unused)
+{
+    njs_string_replace_t  *r;
+
+    r = njs_continuation(vm->frame);
+
+    if (njs_is_string(&r->retval)) {
+        njs_string_replacement_copy(&r->part[1], &r->retval);
+
+        return njs_string_replace_join(vm, r);
+    }
+
+    vm->exception = &njs_exception_type_error;
+
+    return NXT_ERROR;
+}
+
+
+static njs_ret_t
+njs_string_replace_parse(njs_vm_t *vm, njs_string_replace_t *r, u_char *p,
+    u_char *end, size_t size, nxt_uint_t ncaptures)
+{
+    u_char              c;
+    uint32_t            type;
+    njs_string_subst_t  *s;
+
+    r->substitutions = nxt_array_create(4, sizeof(njs_string_subst_t),
+                                     &njs_array_mem_proto, vm->mem_cache_pool);
+
+    if (nxt_slow_path(r->substitutions == NULL)) {
+        return NXT_ERROR;
+    }
+
+    s = NULL;
+
+    if (size == 0) {
+        goto skip;
+    }
+
+copy:
+
+    if (s == NULL) {
+        s = nxt_array_add(r->substitutions, &njs_array_mem_proto,
+                          vm->mem_cache_pool);
+        if (nxt_slow_path(s == NULL)) {
+            return NXT_ERROR;
+        }
+
+        s->type = NJS_SUBST_COPY;
+        s->size = size;
+        s->start = p - size;
+
+    } else {
+        s->size += size;
+    }
+
+skip:
+
+    while (p < end) {
+        size = 1;
+        c = *p++;
+
+        if (c != '$' || p == end) {
+            goto copy;
+        }
+
+        c = *p++;
+
+        if (c == '$') {
+            s = NULL;
+            goto copy;
+        }
+
+        size = 2;
+
+        if (c >= '0' && c <= '9') {
+            type = c - '0';
+
+            if (p < end) {
+                c = *p;
+
+                if (c >= '0' && c <= '9') {
+                    type = type * 10 + (c - '0');
+                    p++;
+                    size = 3;
+                }
+            }
+
+            if (type >= ncaptures) {
+                goto copy;
+            }
+
+            type *= 2;
+
+        } else if (c == '&') {
+            type = 0;
+
+        } else if (c == '`') {
+            type = NJS_SUBST_PRECEDING;
+
+        } else if (c == '\'') {
+            type = NJS_SUBST_FOLLOWING;
+
+        } else {
+            goto copy;
+        }
+
+        s = nxt_array_add(r->substitutions, &njs_array_mem_proto,
+                          vm->mem_cache_pool);
+        if (nxt_slow_path(s == NULL)) {
+            return NXT_ERROR;
+        }
+
+        s->type = type;
+        s = NULL;
+    }
+
+    return NXT_OK;
+}
+
+
+static njs_ret_t
+njs_string_replace_substitute(njs_vm_t *vm, njs_string_replace_t *r,
+    int *captures)
+{
+    uint32_t                   i, n, last;
+    njs_string_subst_t         *s;
+    njs_string_replace_part_t  *part, *subject;
+
+    last = r->substitutions->items;
+
+    part = nxt_array_add_multiple(&r->parts, &njs_array_mem_proto,
+                                  vm->mem_cache_pool, last + 1);
+    if (nxt_slow_path(part == NULL)) {
+        return NXT_ERROR;
+    }
+
+    r->part = &part[-1];
+
+    part[last].start = r->part[0].start + captures[1];
+    part[last].size = r->part[0].size - captures[1];
+    njs_set_invalid(&part[last].value);
+
+    r->part[0].size = captures[0];
+
+    s = r->substitutions->start;
+
+    for (i = 0; i < last; i++) {
+        n = s[i].type;
+
+        switch (n) {
+
+        /* Literal text, "$$", and out of range "$n" substitutions. */
+        case NJS_SUBST_COPY:
+            part->start = s[i].start;
+            part->size = s[i].size;
+            break;
+
+        /* "$`" substitution. */
+        case NJS_SUBST_PRECEDING:
+            subject = r->parts.start;
+            part->start = subject->start;
+            part->size = (r->part[0].start - subject->start) + r->part[0].size;
+            break;
+
+        /* "$'" substitution. */
+        case NJS_SUBST_FOLLOWING:
+            part->start = r->part[last + 1].start;
+            part->size = r->part[last + 1].size;
+            break;
+
+        /*
+         * "$n" substitutions.
+         * "$&" is the same as "$0", the "$0" however is not supported.
+         */
+        default:
+            part->start = r->part[0].start + captures[n];
+            part->size = captures[n + 1] - captures[n];
+            break;
+        }
+
+        njs_set_invalid(&part->value);
+        part++;
+    }
+
+    r->part = part;
+
+    return NXT_OK;
+}
+
+
+static njs_ret_t
+njs_string_replace_join(njs_vm_t *vm, njs_string_replace_t *r)
+{
+    u_char                     *p, *string;
+    size_t                     size, length, mask;
+    ssize_t                    len;
+    nxt_uint_t                 i, n;
+    njs_string_replace_part_t  *part;
+
+    size = 0;
+    length = 0;
+    mask = -1;
+
+    part = r->parts.start;
+    n = r->parts.items;
+
+    for (i = 0; i < n; i++) {
+        if (part[i].start == NULL) {
+            part[i].start = part[i].value.short_string.start;
+        }
+
+        size += part[i].size;
+
+        len = nxt_utf8_length(part[i].start, part[i].size);
+
+        if (len >= 0) {
+            length += len;
+
+        } else {
+            mask = 0;
+        }
+    }
+
+    length &= mask;
+
+    string = njs_string_alloc(vm, &vm->retval, size, length);
+    if (nxt_slow_path(string == NULL)) {
+        return NXT_ERROR;
+    }
+
+    p = string;
+
+    for (i = 0; i < n; i++) {
+        p = memcpy(p, part[i].start, part[i].size);
+        p += part[i].size;
+
+        /* GC: release valid values. */
+    }
+
+    if (length >= NJS_STRING_MAP_OFFSET && size != length) {
+        njs_string_offset_map_init(string, size);
+    }
+
+    nxt_array_destroy(&r->parts, &njs_array_mem_proto, vm->mem_cache_pool);
+
+    return NXT_OK;
+}
+
+
+static void
+njs_string_replacement_copy(njs_string_replace_part_t *string,
+    const njs_value_t *value)
+{
+    size_t  size;
+
+    string->value = *value;
+
+    size = value->short_string.size;
+
+    if (size != NJS_STRING_LONG) {
+        string->start = NULL;
+
+    } else {
+        string->start = value->data.u.string->start;
+        size = value->data.string_size;
+    }
+
+    string->size = size;
+}
+
+
 njs_ret_t
 njs_primitive_value_to_string(njs_vm_t *vm, njs_value_t *dst,
     const njs_value_t *src)
@@ -2100,6 +2825,14 @@ static const njs_object_prop_t  njs_stri
         .value = njs_native_function(njs_string_prototype_split, 0,
                      NJS_STRING_OBJECT_ARG, NJS_REGEXP_ARG, NJS_INTEGER_ARG),
     },
+
+    {
+        .type = NJS_METHOD,
+        .name = njs_string("replace"),
+        .value = njs_native_function(njs_string_prototype_replace,
+                     njs_continuation_size(njs_string_replace_t),
+                     NJS_STRING_OBJECT_ARG, NJS_REGEXP_ARG, NJS_FUNCTION_ARG),
+    },
 };
 
 
diff -r 28bc6acceb0e -r 04aee54864c3 njs/njs_string.h
--- a/njs/njs_string.h	Thu Sep 15 18:00:55 2016 +0300
+++ b/njs/njs_string.h	Fri Sep 23 11:59:48 2016 +0300
@@ -7,6 +7,7 @@
 #ifndef _NJS_STRING_H_INCLUDED_
 #define _NJS_STRING_H_INCLUDED_
 
+#include <nxt_utf8.h>
 
 /*
  * nJSVM supports two string variants:
@@ -80,6 +81,34 @@ typedef struct {
 } njs_slice_prop_t;
 
 
+typedef enum {
+    NJS_STRING_BYTE = 0,
+    NJS_STRING_ASCII,
+    NJS_STRING_UTF8,
+} njs_utf8_t;
+
+
+nxt_inline uint32_t
+njs_string_length(u_char *start, size_t size, njs_utf8_t utf8)
+{
+    ssize_t  length;
+
+    switch (utf8) {
+
+    case NJS_STRING_BYTE:
+        return 0;
+
+    case NJS_STRING_ASCII:
+        return size;
+
+    default:  /* NJS_STRING_UTF8 */
+        length = nxt_utf8_length(start, size);
+
+        return (length >= 0) ? length : 0;
+    }
+}
+
+
 njs_ret_t njs_string_new(njs_vm_t *vm, njs_value_t *value, const u_char *start,
     uint32_t size, uint32_t length);
 u_char *njs_string_alloc(njs_vm_t *vm, njs_value_t *value, uint32_t size,
diff -r 28bc6acceb0e -r 04aee54864c3 njs/njs_vm.c
--- a/njs/njs_vm.c	Thu Sep 15 18:00:55 2016 +0300
+++ b/njs/njs_vm.c	Fri Sep 23 11:59:48 2016 +0300
@@ -2452,6 +2452,20 @@ njs_normalize_args(njs_vm_t *vm, njs_val
             trap = NJS_TRAP_NUMBER_ARG;
             goto trap;
 
+        case NJS_FUNCTION_ARG:
+
+            switch (args->type) {
+            case NJS_STRING:
+            case NJS_FUNCTION:
+                break;
+
+            default:
+                trap = NJS_TRAP_STRING_ARG;
+                goto trap;
+            }
+
+            break;
+
         case NJS_REGEXP_ARG:
 
             switch (args->type) {
diff -r 28bc6acceb0e -r 04aee54864c3 njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Thu Sep 15 18:00:55 2016 +0300
+++ b/njs/test/njs_unit_test.c	Fri Sep 23 11:59:48 2016 +0300
@@ -3176,6 +3176,96 @@ static njs_unit_test_t  njs_test[] =
                  "'123456'.search(r)"),
       nxt_string("3") },
 
+    { nxt_string("'abcdefgh'.replace()"),
+      nxt_string("abcdefgh") },
+
+    { nxt_string("'abcdefgh'.replace('d')"),
+      nxt_string("abcundefinedefgh") },
+
+    { nxt_string("'abcdefgh'.replace('d', undefined)"),
+      nxt_string("abcundefinedefgh") },
+
+    { nxt_string("'abcdefgh'.replace('d', null)"),
+      nxt_string("abcnullefgh") },
+
+    { nxt_string("'abcdefgh'.replace('d', 1)"),
+      nxt_string("abc1efgh") },
+
+    { nxt_string("'abcdefghdijklm'.replace('d', 'X')"),
+      nxt_string("abcXefghdijklm") },
+
+    { nxt_string("'?????????????'.replace('?', '?')"),
+      nxt_string("?????????????") },
+
+    { nxt_string("'abcdefghdijklm'.replace('d',"
+                 "   function(m, o, s) { return '|'+s+'|'+o+'|'+m+'|' })"),
+      nxt_string("abc|abcdefghdijklm|3|d|efghdijklm") },
+
+    { nxt_string("'abcdefgh'.replace('', 'X')"),
+      nxt_string("Xabcdefgh") },
+
+    { nxt_string("'abcdefghdijklm'.replace(/d/, 'X')"),
+      nxt_string("abcXefghdijklm") },
+
+    { nxt_string("'abcdefghdijklm'.replace(/d/,"
+                 "   function(m, o, s) { return '|'+s+'|'+o+'|'+m+'|' })"),
+      nxt_string("abc|abcdefghdijklm|3|d|efghdijklm") },
+
+    { nxt_string("'abcdefghdijklm'.replace(/(d)/,"
+                 "   function(m, p, o, s)"
+                       "{ return '|'+s+'|'+o+'|'+m+'|'+p+'|' })"),
+      nxt_string("abc|abcdefghdijklm|3|d|d|efghdijklm") },
+
+    { nxt_string("'abcdefghdijklm'.replace(/x/, 'X')"),
+      nxt_string("abcdefghdijklm") },
+
+    { nxt_string("'abcdefghdijklm'.replace(/x/,"
+                 "   function(m, o, s) { return '|'+s+'|'+o+'|'+m+'|' })"),
+      nxt_string("abcdefghdijklm") },
+
+    { nxt_string("'?????????????'.replace(/?/, '?')"),
+      nxt_string("?????????????") },
+
+    { nxt_string("'abcdefghdijklm'.replace(/d/g, 'X')"),
+      nxt_string("abcXefghXijklm") },
+
+    { nxt_string("'?????????????'.replace(/?/g, '?')"),
+      nxt_string("?????????????") },
+
+    { nxt_string("'abc12345#$*%'.replace(/([^\\d]*)(\\d*)([^\\w]*)/,"
+                 "   function(match, p1, p2, p3) {"
+                 "     return [p1, p2, p3].join('-')})"),
+      nxt_string("abc-12345-#$*%") },
+
+    { nxt_string("'ABCDEFGHDIJKLM'.replace(/[A-Z]/g,"
+                 "   function(match) { return '-' + match.toLowerCase() })"),
+      nxt_string("-a-b-c-d-e-f-g-h-d-i-j-k-l-m") },
+
+    { nxt_string("'abcdbe'.replace(/(b)/g, '$')"),
+      nxt_string("a$cd$e") },
+
+    { nxt_string("'abcdbe'.replace(/(b)/g, '$2$23')"),
+      nxt_string("a$2$23cd$2$23e") },
+
+    { nxt_string("'abcdbe'.replace(/(b)/g, '$2$23X$$Y')"),
+      nxt_string("a$2$23X$Ycd$2$23X$Ye") },
+

From igor at sysoev.ru  Fri Sep 23 11:00:21 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Fri, 23 Sep 2016 11:00:21 +0000
Subject: [njs] Style and small miscellaneous fixes.
Message-ID: <hg.9d09f9a522d6.1474628421.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/9d09f9a522d6
branches:  
changeset: 178:9d09f9a522d6
user:      Igor Sysoev <igor at sysoev.ru>
date:      Fri Sep 23 11:59:56 2016 +0300
description:
Style and small miscellaneous fixes.

diffstat:

 njs/njs_builtin.c           |  18 +++++++++---------
 njs/njs_function.h          |   8 ++++++++
 njs/njs_regexp.c            |   1 -
 njs/njs_regexp_pattern.h    |   1 -
 njs/njs_vm.c                |   2 +-
 njs/njs_vm.h                |   9 ---------
 njs/test/njs_unit_test.c    |   6 +++---
 nxt/nxt_mem_cache_pool.c    |  16 ++++++++--------
 nxt/nxt_types.h             |   8 ++++----
 nxt/test/lvlhsh_unit_test.c |   6 +++---
 nxt/test/utf8_unit_test.c   |  17 +++++++++++++----
 11 files changed, 49 insertions(+), 43 deletions(-)

diffs (279 lines):

diff -r 04aee54864c3 -r 9d09f9a522d6 njs/njs_builtin.c
--- a/njs/njs_builtin.c	Fri Sep 23 11:59:48 2016 +0300
+++ b/njs/njs_builtin.c	Fri Sep 23 11:59:56 2016 +0300
@@ -43,7 +43,7 @@ njs_builtin_objects_create(njs_vm_t *vm)
     njs_function_t          *functions, *constructors;
     njs_object_prototype_t  *prototypes;
 
-    static const njs_object_init_t    *prototype_init[] = {
+    static const njs_object_init_t  *prototype_init[] = {
         &njs_object_prototype_init,
         &njs_array_prototype_init,
         &njs_boolean_prototype_init,
@@ -56,7 +56,7 @@ njs_builtin_objects_create(njs_vm_t *vm)
 
     static const njs_object_prototype_t  prototype_values[] = {
         /*
-         * GCC 4 complains about unitialized .shared field,
+         * GCC 4 complains about uninitialized .shared field,
          * if the .type field is initialized as .object.type.
          */
         { .object =       { .type = NJS_OBJECT } },
@@ -95,15 +95,15 @@ njs_builtin_objects_create(njs_vm_t *vm)
 
     static const njs_function_init_t  native_constructors[] = {
         /* SunC does not allow empty array initialization. */
-        { njs_object_constructor,   { 0 } },
-        { njs_array_constructor,    { 0 } },
-        { njs_boolean_constructor,  { 0 } },
-        { njs_number_constructor,   { NJS_SKIP_ARG, NJS_NUMBER_ARG } },
-        { njs_string_constructor,   { NJS_SKIP_ARG, NJS_STRING_ARG } },
-        { njs_function_constructor, { 0 } },
+        { njs_object_constructor,     { 0 } },
+        { njs_array_constructor,      { 0 } },
+        { njs_boolean_constructor,    { 0 } },
+        { njs_number_constructor,     { NJS_SKIP_ARG, NJS_NUMBER_ARG } },
+        { njs_string_constructor,     { NJS_SKIP_ARG, NJS_STRING_ARG } },
+        { njs_function_constructor,   { 0 } },
         { njs_regexp_constructor,
           { NJS_SKIP_ARG, NJS_STRING_ARG, NJS_STRING_ARG } },
-        { njs_date_constructor,     { 0 } },
+        { njs_date_constructor,       { 0 } },
     };
 
     static const njs_object_init_t    *object_init[] = {
diff -r 04aee54864c3 -r 9d09f9a522d6 njs/njs_function.h
--- a/njs/njs_function.h	Fri Sep 23 11:59:48 2016 +0300
+++ b/njs/njs_function.h	Fri Sep 23 11:59:56 2016 +0300
@@ -47,6 +47,14 @@ struct njs_function_lambda_s {
 
 #define NJS_FRAME_SPARE_SIZE       512
 
+
+typedef struct {
+    njs_function_native_t          function;
+    u_char                         *return_address;
+    njs_index_t                    retval;
+} njs_continuation_t;
+
+
 #define njs_continuation(frame)                                               \
     (void *) ((u_char *) frame + NJS_NATIVE_FRAME_SIZE)
 
diff -r 04aee54864c3 -r 9d09f9a522d6 njs/njs_regexp.c
--- a/njs/njs_regexp.c	Fri Sep 23 11:59:48 2016 +0300
+++ b/njs/njs_regexp.c	Fri Sep 23 11:59:56 2016 +0300
@@ -292,7 +292,6 @@ njs_regexp_pattern_create(njs_vm_t *vm, 
     }
 
     pattern->flags = size;
-    pattern->next = NULL;
 
     p = (u_char *) pattern + sizeof(njs_regexp_pattern_t);
     pattern->source = p;
diff -r 04aee54864c3 -r 9d09f9a522d6 njs/njs_regexp_pattern.h
--- a/njs/njs_regexp_pattern.h	Fri Sep 23 11:59:48 2016 +0300
+++ b/njs/njs_regexp_pattern.h	Fri Sep 23 11:59:56 2016 +0300
@@ -19,7 +19,6 @@ typedef enum {
 struct njs_regexp_pattern_s {
     nxt_regex_t           regex[2];
 
-    njs_regexp_pattern_t  *next;
     /*
      * A pattern source is used by RegExp.toString() method and
      * RegExp.source property.  So it is is stored in form "/pattern/flags"
diff -r 04aee54864c3 -r 9d09f9a522d6 njs/njs_vm.c
--- a/njs/njs_vm.c	Fri Sep 23 11:59:48 2016 +0300
+++ b/njs/njs_vm.c	Fri Sep 23 11:59:56 2016 +0300
@@ -162,7 +162,7 @@ njs_vmcode_interpreter(njs_vm_t *vm)
     njs_native_frame_t    *previous;
     njs_vmcode_generic_t  *vmcode;
 
-    start:
+start:
 
     for ( ;; ) {
 
diff -r 04aee54864c3 -r 9d09f9a522d6 njs/njs_vm.h
--- a/njs/njs_vm.h	Fri Sep 23 11:59:48 2016 +0300
+++ b/njs/njs_vm.h	Fri Sep 23 11:59:56 2016 +0300
@@ -124,15 +124,6 @@ typedef struct njs_native_frame_s     nj
 typedef struct njs_property_next_s    njs_property_next_t;
 
 
-typedef struct njs_continuation_s     njs_continuation_t;
-
-struct njs_continuation_s {
-    njs_function_native_t             function;
-    u_char                            *return_address;
-    njs_index_t                       retval;
-};
-
-
 union njs_value_s {
     /*
      * The njs_value_t size is 16 bytes and must be aligned to 16 bytes
diff -r 04aee54864c3 -r 9d09f9a522d6 njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Fri Sep 23 11:59:48 2016 +0300
+++ b/njs/test/njs_unit_test.c	Fri Sep 23 11:59:56 2016 +0300
@@ -5572,11 +5572,11 @@ njs_unit_test_benchmark(nxt_str_t *scrip
          + usage.ru_stime.tv_sec * 1000000 + usage.ru_stime.tv_usec;
 
     if (n == 1) {
-        us /= 1000;
-        printf("%s: %d.%03ds\n", msg, (int) (us / 1000), (int) (us % 1000));
+        printf("%s: %.3fs\n", msg, (double) us / 1000000);
 
     } else {
-        printf("%s: %d\n", msg, (int) ((uint64_t) n * 1000000 / us));
+        printf("%s: %.3f?s, %d times/s\n",
+               msg, (double) us / n, (int) ((uint64_t) n * 1000000 / us));
     }
 
     return NXT_OK;
diff -r 04aee54864c3 -r 9d09f9a522d6 nxt/nxt_mem_cache_pool.c
--- a/nxt/nxt_mem_cache_pool.c	Fri Sep 23 11:59:48 2016 +0300
+++ b/nxt/nxt_mem_cache_pool.c	Fri Sep 23 11:59:56 2016 +0300
@@ -52,12 +52,12 @@ struct nxt_mem_cache_page_s {
      * Used to link pages with free chunks in pool chunk slot list
      * or to link free pages in clusters.
      */
-    nxt_queue_link_t             link;
+    nxt_queue_link_t            link;
 };
 
 
 typedef struct {
-    NXT_RBTREE_NODE              (node);
+    NXT_RBTREE_NODE             (node);
     uint8_t                     type;
     uint32_t                    size;
 
@@ -67,7 +67,7 @@ typedef struct {
 
 
 typedef struct {
-    nxt_queue_t                  pages;
+    nxt_queue_t                 pages;
 #if (NXT_64BIT)
     uint32_t                    size;
     uint32_t                    chunks;
@@ -80,9 +80,9 @@ typedef struct {
 
 struct nxt_mem_cache_pool_s {
     /* rbtree of nxt_mem_cache_block_t. */
-    nxt_rbtree_t                 blocks;
+    nxt_rbtree_t                blocks;
 
-    nxt_queue_t                  free_pages;
+    nxt_queue_t                 free_pages;
 
     uint8_t                     chunk_size_shift;
     uint8_t                     page_size_shift;
@@ -90,11 +90,11 @@ struct nxt_mem_cache_pool_s {
     uint32_t                    page_alignment;
     uint32_t                    cluster_size;
 
-    const nxt_mem_proto_t        *proto;
+    const nxt_mem_proto_t       *proto;
     void                        *mem;
     void                        *trace;
 
-    nxt_mem_cache_slot_t         slots[];
+    nxt_mem_cache_slot_t        slots[];
 };
 
 
@@ -515,7 +515,7 @@ nxt_mem_cache_alloc_cluster(nxt_mem_cach
     n = pool->cluster_size >> pool->page_size_shift;
 
     cluster = pool->proto->zalloc(pool->mem, sizeof(nxt_mem_cache_block_t)
-                                             + n * sizeof(nxt_mem_cache_page_t));
+                                           + n * sizeof(nxt_mem_cache_page_t));
 
     if (nxt_slow_path(cluster == NULL)) {
         return NULL;
diff -r 04aee54864c3 -r 9d09f9a522d6 nxt/nxt_types.h
--- a/nxt/nxt_types.h	Fri Sep 23 11:59:48 2016 +0300
+++ b/nxt/nxt_types.h	Fri Sep 23 11:59:56 2016 +0300
@@ -40,13 +40,13 @@
  * instructions are longer.
  */
 #define NXT_INT_T_SIZE  4
-typedef int            nxt_int_t;
-typedef u_int          nxt_uint_t;
+typedef int             nxt_int_t;
+typedef u_int           nxt_uint_t;
 
 #else
 #define NXT_INT_T_SIZE  NXT_PTR_SIZE
-typedef intptr_t       nxt_int_t;
-typedef uintptr_t      nxt_uint_t;
+typedef intptr_t        nxt_int_t;
+typedef uintptr_t       nxt_uint_t;
 #endif
 
 
diff -r 04aee54864c3 -r 9d09f9a522d6 nxt/test/lvlhsh_unit_test.c
--- a/nxt/test/lvlhsh_unit_test.c	Fri Sep 23 11:59:48 2016 +0300
+++ b/nxt/test/lvlhsh_unit_test.c	Fri Sep 23 11:59:56 2016 +0300
@@ -88,7 +88,7 @@ lvlhsh_unit_test_get(nxt_lvlhsh_t *lh, c
 
     lhq.key_hash = key;
     lhq.key.length = sizeof(uintptr_t);
-    lhq.key.start =  (u_char *) &key;
+    lhq.key.start = (u_char *) &key;
     lhq.proto = proto;
 
     if (nxt_lvlhsh_find(lh, &lhq) == NXT_OK) {
@@ -206,8 +206,8 @@ lvlhsh_unit_test(nxt_uint_t n)
     const size_t          cluster_size = 4096;
 
     pool = nxt_mem_cache_pool_create(&mem_cache_pool_proto, NULL, NULL,
-                                    cluster_size, page_alignment,
-                                    page_size, min_chunk_size);
+                                     cluster_size, page_alignment,
+                                     page_size, min_chunk_size);
     if (pool == NULL) {
         return NXT_ERROR;
     }
diff -r 04aee54864c3 -r 9d09f9a522d6 nxt/test/utf8_unit_test.c
--- a/nxt/test/utf8_unit_test.c	Fri Sep 23 11:59:48 2016 +0300
+++ b/nxt/test/utf8_unit_test.c	Fri Sep 23 11:59:56 2016 +0300
@@ -82,7 +82,7 @@ utf8_overlong(u_char *overlong, size_t l
 
 
 static nxt_int_t
-utf8_unit_test(void)
+utf8_unit_test(nxt_uint_t start)
 {
     u_char        *p, utf8[4];
     size_t        len;
@@ -142,7 +142,7 @@ utf8_unit_test(void)
 
     /* Test all overlong UTF-8. */
 
-    for (i = NXT_UTF8_START_TEST; i < 256; i++) {
+    for (i = start; i < 256; i++) {
         utf8[0] = i;
 
         if (utf8_overlong(utf8, 1) != NXT_OK) {
@@ -190,7 +190,16 @@ utf8_unit_test(void)
 
 
 int
-main(void)
+main(int argc, char **argv)
 {
-    return utf8_unit_test();
+    nxt_uint_t  start;
+
+    if (argc > 1 && argv[1][0] == 'a') {
+        start = NXT_UTF8_START_TEST;
+
+    } else {
+        start = 256;
+    }
+
+    return utf8_unit_test(start);
 }

From igor at sysoev.ru  Fri Sep 23 11:00:22 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Fri, 23 Sep 2016 11:00:22 +0000
Subject: [njs] The visibility attribute disabling in the commit af6c17324584
Message-ID: <hg.6c65289c3696.1474628422.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/6c65289c3696
branches:  
changeset: 179:6c65289c3696
user:      Igor Sysoev <igor at sysoev.ru>
date:      Fri Sep 23 11:59:58 2016 +0300
description:
The visibility attribute disabling in the commit af6c17324584
had not resovled the issue.  So now it is enabled and the aligment
attribute has been disabled instead for Linux/PPC64.

diffstat:

 nxt/auto/clang  |  13 +++++++++++++
 nxt/nxt_clang.h |  22 ++++++++++++++--------
 2 files changed, 27 insertions(+), 8 deletions(-)

diffs (62 lines):

diff -r 9d09f9a522d6 -r 6c65289c3696 nxt/auto/clang
--- a/nxt/auto/clang	Fri Sep 23 11:59:56 2016 +0300
+++ b/nxt/auto/clang	Fri Sep 23 11:59:58 2016 +0300
@@ -202,6 +202,19 @@ nxt_feature_test="int main() {
 . ${NXT_AUTO}feature
 
 
+nxt_feature="GCC __attribute__ visibility"
+nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_VISIBILITY
+nxt_feature_run=
+nxt_feature_path=
+nxt_feature_libs=
+nxt_feature_test="int n __attribute__ ((visibility(\"default\")));
+
+                  int main() {
+                      return 1;
+                  }"
+. ${NXT_AUTO}feature
+
+
 nxt_feature="GCC __attribute__ aligned"
 nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_ALIGNED
 nxt_feature_run=
diff -r 9d09f9a522d6 -r 6c65289c3696 nxt/nxt_clang.h
--- a/nxt/nxt_clang.h	Fri Sep 23 11:59:56 2016 +0300
+++ b/nxt/nxt_clang.h	Fri Sep 23 11:59:58 2016 +0300
@@ -60,6 +60,20 @@
 #endif
 
 
+#if (NXT_HAVE_GCC_ATTRIBUTE_ALIGNED)
+
+#if (NXT_LINUX && __PPC64__)
+/* Old GNU ld linker may hang on Linux ppc64le platform. */
+#define nxt_aligned(x)
+#else
+#define nxt_aligned(x)     __attribute__((aligned(x)))
+#endif
+
+#else
+#define nxt_aligned(x)
+#endif
+
+
 #if (NXT_HAVE_GCC_ATTRIBUTE_MALLOC)
 #define NXT_MALLOC_LIKE    __attribute__((__malloc__))
 
@@ -68,14 +82,6 @@
 #endif
 
 
-#if (NXT_HAVE_GCC_ATTRIBUTE_ALIGNED)
-#define nxt_aligned(x)     __attribute__((aligned(x)))
-
-#else
-#define nxt_aligned(x)
-#endif
-
-
 #if (NXT_CLANG)
 /* Any __asm__ directive disables loop vectorization in GCC and Clang. */
 #define nxt_pragma_loop_disable_vectorization  __asm__("")


From igor at sysoev.ru  Sun Sep 25 16:11:26 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Sun, 25 Sep 2016 16:11:26 +0000
Subject: [njs] All C feature attributes have been disabled for Linux/PPC64.
Message-ID: <hg.6a983847db14.1474819886.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/6a983847db14
branches:  
changeset: 180:6a983847db14
user:      Igor Sysoev <igor at sysoev.ru>
date:      Sat Sep 24 22:13:31 2016 +0300
description:
All C feature attributes have been disabled for Linux/PPC64.

diffstat:

 nxt/auto/clang  |  155 ++++++++++++++++++++++++++++++-------------------------
 nxt/nxt_clang.h |    6 --
 2 files changed, 84 insertions(+), 77 deletions(-)

diffs (190 lines):

diff -r 6c65289c3696 -r 6a983847db14 nxt/auto/clang
--- a/nxt/auto/clang	Fri Sep 23 11:59:58 2016 +0300
+++ b/nxt/auto/clang	Sat Sep 24 22:13:31 2016 +0300
@@ -166,85 +166,98 @@ END
 
 # C language features.
 
-nxt_feature="GCC __builtin_expect()"
-nxt_feature_name=NXT_HAVE_BUILTIN_EXPECT
-nxt_feature_run=no
-nxt_feature_incs=
-nxt_feature_libs=
-nxt_feature_test="int main(int argc, char *const *argv) {
-                      if ((__typeof__(argc == 0))
-                                   __builtin_expect((argc == 0), 0))
-                          return 0;
-                      return 1;
-                  }"
-. ${NXT_AUTO}feature
-
+nxt_os="$NXT_SYSTEM/$NXT_SYSTEM_PLATFORM"
 
-nxt_feature="GCC __builtin_unreachable()"
-nxt_feature_name=NXT_HAVE_BUILTIN_UNREACHABLE
-nxt_feature_run=no
-nxt_feature_incs=
-nxt_feature_libs=
-nxt_feature_test="int main() {
-                      __builtin_unreachable();
-                  }"
-. ${NXT_AUTO}feature
+if [ "$nxt_os" = "Linux/ppc64le" ]; then
 
+    # Old GNU ld linker may hang on Linux ppc64le platform
+    # if some of these features are enabled.
 
-nxt_feature="GCC __builtin_prefetch()"
-nxt_feature_name=NXT_HAVE_BUILTIN_PREFETCH
-nxt_feature_run=no
-nxt_feature_incs=
-nxt_feature_libs=
-nxt_feature_test="int main() {
-                      __builtin_prefetch(0);
-                  }"
-. ${NXT_AUTO}feature
+    echo "checking for C language features is disabled for $nxt_os."
+
+else
+
+    nxt_feature="GCC __builtin_expect()"
+    nxt_feature_name=NXT_HAVE_BUILTIN_EXPECT
+    nxt_feature_run=no
+    nxt_feature_incs=
+    nxt_feature_libs=
+    nxt_feature_test="int main(int argc, char *const *argv) {
+			  if ((__typeof__(argc == 0))
+				       __builtin_expect((argc == 0), 0))
+			      return 0;
+			  return 1;
+		      }"
+    . ${NXT_AUTO}feature
 
 
-nxt_feature="GCC __attribute__ visibility"
-nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_VISIBILITY
-nxt_feature_run=
-nxt_feature_path=
-nxt_feature_libs=
-nxt_feature_test="int n __attribute__ ((visibility(\"default\")));
-
-                  int main() {
-                      return 1;
-                  }"
-. ${NXT_AUTO}feature
+    nxt_feature="GCC __builtin_unreachable()"
+    nxt_feature_name=NXT_HAVE_BUILTIN_UNREACHABLE
+    nxt_feature_run=no
+    nxt_feature_incs=
+    nxt_feature_libs=
+    nxt_feature_test="int main() {
+			  __builtin_unreachable();
+		      }"
+    . ${NXT_AUTO}feature
 
 
-nxt_feature="GCC __attribute__ aligned"
-nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_ALIGNED
-nxt_feature_run=
-nxt_feature_path=
-nxt_feature_libs=
-nxt_feature_test="int n __attribute__ ((aligned(64)));
-
-                  int main() {
-                      return 1;
-                  }"
-. ${NXT_AUTO}feature
+    nxt_feature="GCC __builtin_prefetch()"
+    nxt_feature_name=NXT_HAVE_BUILTIN_PREFETCH
+    nxt_feature_run=no
+    nxt_feature_incs=
+    nxt_feature_libs=
+    nxt_feature_test="int main() {
+			  __builtin_prefetch(0);
+		      }"
+    . ${NXT_AUTO}feature
 
 
-nxt_feature="GCC __attribute__ malloc"
-nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_MALLOC
-nxt_feature_run=
-nxt_feature_path=
-nxt_feature_libs=
-nxt_feature_test="#include <stdlib.h>
-
-                  void *f(void) __attribute__ ((__malloc__));
+    nxt_feature="GCC __attribute__ visibility"
+    nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_VISIBILITY
+    nxt_feature_run=
+    nxt_feature_path=
+    nxt_feature_libs=
+    nxt_feature_test="int n __attribute__ ((visibility(\"default\")));
 
-                  void *f(void) {
-                      return malloc(1);
-                  }
+		      int main() {
+			  return 1;
+		      }"
+    . ${NXT_AUTO}feature
 
-                  int main() {
-                      if (f() != NULL) {
-                          return 1;
-                      }
-                      return 0;
-                  }"
-. ${NXT_AUTO}feature
+
+    nxt_feature="GCC __attribute__ aligned"
+    nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_ALIGNED
+    nxt_feature_run=
+    nxt_feature_path=
+    nxt_feature_libs=
+    nxt_feature_test="int n __attribute__ ((aligned(64)));
+
+		      int main() {
+			  return 1;
+		      }"
+    . ${NXT_AUTO}feature
+
+
+    nxt_feature="GCC __attribute__ malloc"
+    nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_MALLOC
+    nxt_feature_run=
+    nxt_feature_path=
+    nxt_feature_libs=
+    nxt_feature_test="#include <stdlib.h>
+
+		      void *f(void) __attribute__ ((__malloc__));
+
+		      void *f(void) {
+			  return malloc(1);
+		      }
+
+		      int main() {
+			  if (f() != NULL) {
+			      return 1;
+			  }
+			  return 0;
+		      }"
+    . ${NXT_AUTO}feature
+
+fi
diff -r 6c65289c3696 -r 6a983847db14 nxt/nxt_clang.h
--- a/nxt/nxt_clang.h	Fri Sep 23 11:59:58 2016 +0300
+++ b/nxt/nxt_clang.h	Sat Sep 24 22:13:31 2016 +0300
@@ -61,13 +61,7 @@
 
 
 #if (NXT_HAVE_GCC_ATTRIBUTE_ALIGNED)
-
-#if (NXT_LINUX && __PPC64__)
-/* Old GNU ld linker may hang on Linux ppc64le platform. */
-#define nxt_aligned(x)
-#else
 #define nxt_aligned(x)     __attribute__((aligned(x)))
-#endif
 
 #else
 #define nxt_aligned(x)


From igor at sysoev.ru  Sun Sep 25 16:11:28 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Sun, 25 Sep 2016 16:11:28 +0000
Subject: [njs] A fix of possible sign extension overflow,
Message-ID: <hg.5ef1091ecfaf.1474819888.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/5ef1091ecfaf
branches:  
changeset: 181:5ef1091ecfaf
user:      Igor Sysoev <igor at sysoev.ru>
date:      Sun Sep 25 10:54:51 2016 +0300
description:
A fix of possible sign extension overflow,
the issue has been found by Coverity Scan.

diffstat:

 nxt/nxt_array.c |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (21 lines):

diff -r 6a983847db14 -r 5ef1091ecfaf nxt/nxt_array.c
--- a/nxt/nxt_array.c	Sat Sep 24 22:13:31 2016 +0300
+++ b/nxt/nxt_array.c	Sun Sep 25 10:54:51 2016 +0300
@@ -113,7 +113,7 @@ nxt_array_add_multiple(nxt_array_t *arra
         old = array->start;
         array->start = start;
 
-        memcpy(start, old, array->items * array->item_size);
+        memcpy(start, old, (uint32_t) array->items * array->item_size);
 
         if (array->separate == 0) {
             array->separate = 1;
@@ -123,7 +123,7 @@ nxt_array_add_multiple(nxt_array_t *arra
         }
     }
 
-    item = (char *) array->start + array->items * array->item_size;
+    item = (char *) array->start + (uint32_t) array->items * array->item_size;
 
     array->items = items;
 


From igor at sysoev.ru  Mon Sep 26 15:42:53 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Mon, 26 Sep 2016 15:42:53 +0000
Subject: [njs] The cause of linker hanging on Linux/PPC64 has been
Message-ID: <hg.eed097d72d5e.1474904573.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/eed097d72d5e
branches:  
changeset: 182:eed097d72d5e
user:      Igor Sysoev <igor at sysoev.ru>
date:      Mon Sep 26 14:01:39 2016 +0300
description:
The cause of linker hanging on Linux/PPC64 has been
finally tracked down and thus only the aligment attribute
has been correctly disabled on this platform.

diffstat:

 nxt/auto/clang |  150 ++++++++++++++++++++++++++++----------------------------
 1 files changed, 75 insertions(+), 75 deletions(-)

diffs (177 lines):

diff -r 5ef1091ecfaf -r eed097d72d5e nxt/auto/clang
--- a/nxt/auto/clang	Sun Sep 25 10:54:51 2016 +0300
+++ b/nxt/auto/clang	Mon Sep 26 14:01:39 2016 +0300
@@ -166,6 +166,77 @@ END
 
 # C language features.
 
+nxt_feature="GCC __builtin_expect()"
+nxt_feature_name=NXT_HAVE_BUILTIN_EXPECT
+nxt_feature_run=no
+nxt_feature_incs=
+nxt_feature_libs=
+nxt_feature_test="int main(int argc, char *const *argv) {
+                      if ((__typeof__(argc == 0))
+                                   __builtin_expect((argc == 0), 0))
+                          return 0;
+                      return 1;
+                  }"
+. ${NXT_AUTO}feature
+
+
+nxt_feature="GCC __builtin_unreachable()"
+nxt_feature_name=NXT_HAVE_BUILTIN_UNREACHABLE
+nxt_feature_run=no
+nxt_feature_incs=
+nxt_feature_libs=
+nxt_feature_test="int main() {
+                      __builtin_unreachable();
+                  }"
+. ${NXT_AUTO}feature
+
+
+nxt_feature="GCC __builtin_prefetch()"
+nxt_feature_name=NXT_HAVE_BUILTIN_PREFETCH
+nxt_feature_run=no
+nxt_feature_incs=
+nxt_feature_libs=
+nxt_feature_test="int main() {
+                      __builtin_prefetch(0);
+                  }"
+. ${NXT_AUTO}feature
+
+
+nxt_feature="GCC __attribute__ visibility"
+nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_VISIBILITY
+nxt_feature_run=
+nxt_feature_path=
+nxt_feature_libs=
+nxt_feature_test="int n __attribute__ ((visibility(\"default\")));
+
+                  int main() {
+                      return 1;
+                  }"
+. ${NXT_AUTO}feature
+
+
+nxt_feature="GCC __attribute__ malloc"
+nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_MALLOC
+nxt_feature_run=
+nxt_feature_path=
+nxt_feature_libs=
+nxt_feature_test="#include <stdlib.h>
+
+                  void *f(void) __attribute__ ((__malloc__));
+
+                  void *f(void) {
+                      return malloc(1);
+                  }
+
+                  int main() {
+                      if (f() != NULL) {
+                          return 1;
+                      }
+                      return 0;
+                  }"
+. ${NXT_AUTO}feature
+
+
 nxt_os="$NXT_SYSTEM/$NXT_SYSTEM_PLATFORM"
 
 if [ "$nxt_os" = "Linux/ppc64le" ]; then
@@ -173,59 +244,10 @@ if [ "$nxt_os" = "Linux/ppc64le" ]; then
     # Old GNU ld linker may hang on Linux ppc64le platform
     # if some of these features are enabled.
 
-    echo "checking for C language features is disabled for $nxt_os."
+    echo "checking for GCC __attribute__ aligned is disabled for $nxt_os."
 
 else
 
-    nxt_feature="GCC __builtin_expect()"
-    nxt_feature_name=NXT_HAVE_BUILTIN_EXPECT
-    nxt_feature_run=no
-    nxt_feature_incs=
-    nxt_feature_libs=
-    nxt_feature_test="int main(int argc, char *const *argv) {
-			  if ((__typeof__(argc == 0))
-				       __builtin_expect((argc == 0), 0))
-			      return 0;
-			  return 1;
-		      }"
-    . ${NXT_AUTO}feature
-
-
-    nxt_feature="GCC __builtin_unreachable()"
-    nxt_feature_name=NXT_HAVE_BUILTIN_UNREACHABLE
-    nxt_feature_run=no
-    nxt_feature_incs=
-    nxt_feature_libs=
-    nxt_feature_test="int main() {
-			  __builtin_unreachable();
-		      }"
-    . ${NXT_AUTO}feature
-
-
-    nxt_feature="GCC __builtin_prefetch()"
-    nxt_feature_name=NXT_HAVE_BUILTIN_PREFETCH
-    nxt_feature_run=no
-    nxt_feature_incs=
-    nxt_feature_libs=
-    nxt_feature_test="int main() {
-			  __builtin_prefetch(0);
-		      }"
-    . ${NXT_AUTO}feature
-
-
-    nxt_feature="GCC __attribute__ visibility"
-    nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_VISIBILITY
-    nxt_feature_run=
-    nxt_feature_path=
-    nxt_feature_libs=
-    nxt_feature_test="int n __attribute__ ((visibility(\"default\")));
-
-		      int main() {
-			  return 1;
-		      }"
-    . ${NXT_AUTO}feature
-
-
     nxt_feature="GCC __attribute__ aligned"
     nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_ALIGNED
     nxt_feature_run=
@@ -233,31 +255,9 @@ else
     nxt_feature_libs=
     nxt_feature_test="int n __attribute__ ((aligned(64)));
 
-		      int main() {
-			  return 1;
-		      }"
-    . ${NXT_AUTO}feature
-
-
-    nxt_feature="GCC __attribute__ malloc"
-    nxt_feature_name=NXT_HAVE_GCC_ATTRIBUTE_MALLOC
-    nxt_feature_run=
-    nxt_feature_path=
-    nxt_feature_libs=
-    nxt_feature_test="#include <stdlib.h>
-
-		      void *f(void) __attribute__ ((__malloc__));
-
-		      void *f(void) {
-			  return malloc(1);
-		      }
-
-		      int main() {
-			  if (f() != NULL) {
-			      return 1;
-			  }
-			  return 0;
-		      }"
+                      int main() {
+                          return 1;
+                      }"
     . ${NXT_AUTO}feature
 
 fi


From igor at sysoev.ru  Mon Sep 26 15:42:55 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Mon, 26 Sep 2016 15:42:55 +0000
Subject: [njs] String processing unification using njs_string_length(),
Message-ID: <hg.fec0d8dfa38c.1474904575.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/fec0d8dfa38c
branches:  
changeset: 183:fec0d8dfa38c
user:      Igor Sysoev <igor at sysoev.ru>
date:      Mon Sep 26 14:01:45 2016 +0300
description:
String processing unification using njs_string_length(),
njs_utf8_t, and njs_regexp_utf8_t.

diffstat:

 njs/njs_regexp.c |  62 ++++++++++++++-----------------------
 njs/njs_string.c |  92 ++++++++++++++++---------------------------------------
 njs/njs_string.h |   2 +-
 njs/njs_vm.c     |   3 +-
 4 files changed, 54 insertions(+), 105 deletions(-)

diffs (383 lines):

diff -r eed097d72d5e -r fec0d8dfa38c njs/njs_regexp.c
--- a/njs/njs_regexp.c	Mon Sep 26 14:01:39 2016 +0300
+++ b/njs/njs_regexp.c	Mon Sep 26 14:01:45 2016 +0300
@@ -44,7 +44,7 @@ static u_char *njs_regexp_compile_trace_
 static u_char *njs_regexp_match_trace_handler(nxt_trace_t *trace,
     nxt_trace_data_t *td, u_char *start);
 static njs_ret_t njs_regexp_exec_result(njs_vm_t *vm, njs_regexp_t *regexp,
-    u_char *string, nxt_regex_match_data_t *match_data, nxt_uint_t utf8);
+    njs_utf8_t utf8, u_char *string, nxt_regex_match_data_t *match_data);
 static njs_ret_t njs_regexp_string_create(njs_vm_t *vm, njs_value_t *value,
     u_char *start, uint32_t size, int32_t length);
 
@@ -539,7 +539,8 @@ static njs_ret_t
 njs_regexp_prototype_source(njs_vm_t *vm, njs_value_t *value)
 {
     u_char                *source;
-    size_t                length, size;
+    int32_t               length;
+    uint32_t              size;
     njs_regexp_pattern_t  *pattern;
 
     pattern = value->data.u.regexp->pattern;
@@ -558,7 +559,8 @@ njs_regexp_prototype_to_string(njs_vm_t 
     nxt_uint_t nargs, njs_index_t unused)
 {
     u_char                *source;
-    size_t                length, size;
+    int32_t               length;
+    uint32_t              size;
     njs_regexp_pattern_t  *pattern;
 
     pattern = args[0].data.u.regexp->pattern;
@@ -624,10 +626,11 @@ njs_regexp_prototype_exec(njs_vm_t *vm, 
     njs_index_t unused)
 {
     njs_ret_t               ret;
-    nxt_uint_t              n, utf8;
+    njs_utf8_t              utf8;
     njs_value_t             *value;
     njs_regexp_t            *regexp;
     njs_string_prop_t       string;
+    njs_regexp_utf8_t       type;
     njs_regexp_pattern_t    *pattern;
     nxt_regex_match_data_t  *match_data;
 
@@ -648,38 +651,35 @@ njs_regexp_prototype_exec(njs_vm_t *vm, 
 
     (void) njs_string_prop(&string, value);
 
-    /* Byte string. */
-    utf8 = 0;
-    n = 0;
+    utf8 = NJS_STRING_BYTE;
+    type = NJS_REGEXP_BYTE;
 
     if (string.length != 0) {
-        /* ASCII string. */
-        utf8 = 1;
-        n = 1;
+        utf8 = NJS_STRING_ASCII;
+        type = NJS_REGEXP_UTF8;
 
         if (string.length != string.size) {
-            /* UTF-8 string. */
-            utf8 = 2;
+            utf8 = NJS_STRING_UTF8;
         }
     }
 
     pattern = regexp->pattern;
 
-    if (nxt_regex_is_valid(&pattern->regex[n])) {
+    if (nxt_regex_is_valid(&pattern->regex[type])) {
         string.start += regexp->last_index;
         string.size -= regexp->last_index;
 
-        match_data = nxt_regex_match_data(&pattern->regex[n],
+        match_data = nxt_regex_match_data(&pattern->regex[type],
                                           vm->regex_context);
         if (nxt_slow_path(match_data == NULL)) {
             return NXT_ERROR;
         }
 
-        ret = njs_regexp_match(vm, &pattern->regex[n], string.start,
+        ret = njs_regexp_match(vm, &pattern->regex[type], string.start,
                                string.size, match_data);
         if (ret >= 0) {
-            return njs_regexp_exec_result(vm, regexp, string.start, match_data,
-                                          utf8);
+            return njs_regexp_exec_result(vm, regexp, utf8, string.start,
+                                          match_data);
         }
 
         if (nxt_slow_path(ret != NXT_REGEX_NOMATCH)) {
@@ -697,8 +697,8 @@ njs_regexp_prototype_exec(njs_vm_t *vm, 
 
 
 static njs_ret_t
-njs_regexp_exec_result(njs_vm_t *vm, njs_regexp_t *regexp, u_char *string,
-    nxt_regex_match_data_t *match_data, nxt_uint_t utf8)
+njs_regexp_exec_result(njs_vm_t *vm, njs_regexp_t *regexp, njs_utf8_t utf8,
+    u_char *string, nxt_regex_match_data_t *match_data)
 {
     int                 *captures;
     u_char              *start;
@@ -726,20 +726,10 @@ njs_regexp_exec_result(njs_vm_t *vm, njs
             start = &string[captures[n]];
             size = captures[n + 1] - captures[n];
 
-            switch (utf8) {
-            case 0:
-                length = 0;
-                break;
-            case 1:
-                length = size;
-                break;
-            default:
-                length = nxt_utf8_length(start, size);
-                break;
-            }
+            length = njs_string_length(utf8, start, size);
 
-            ret = njs_regexp_string_create(vm, &array->start[i],
-                                           start, size, length);
+            ret = njs_regexp_string_create(vm, &array->start[i], start, size,
+                                           length);
             if (nxt_slow_path(ret != NXT_OK)) {
                 goto fail;
             }
@@ -812,13 +802,9 @@ static njs_ret_t
 njs_regexp_string_create(njs_vm_t *vm, njs_value_t *value, u_char *start,
     uint32_t size, int32_t length)
 {
-    if (nxt_fast_path(length >= 0)) {
-        return njs_string_create(vm, value, start, size, length);
-    }
+    length = (length >= 0) ? length : 0;
 
-    vm->exception = &njs_exception_internal_error;
-
-    return NXT_ERROR;
+    return njs_string_create(vm, value, start, size, length);
 }
 
 
diff -r eed097d72d5e -r fec0d8dfa38c njs/njs_string.c
--- a/njs/njs_string.c	Mon Sep 26 14:01:39 2016 +0300
+++ b/njs/njs_string.c	Mon Sep 26 14:01:45 2016 +0300
@@ -88,7 +88,7 @@ static nxt_noinline ssize_t njs_string_i
 static njs_ret_t njs_string_match_multiple(njs_vm_t *vm, njs_value_t *args,
     njs_regexp_pattern_t *pattern);
 static njs_ret_t njs_string_split_part_add(njs_vm_t *vm, njs_array_t *array,
-    u_char *start, size_t size, nxt_uint_t utf8);
+    njs_utf8_t utf8, u_char *start, size_t size);
 static njs_ret_t njs_string_replace_regexp(njs_vm_t *vm, njs_value_t *args,
     njs_string_replace_t *r);
 static njs_ret_t njs_string_replace_regexp_function(njs_vm_t *vm,
@@ -1609,8 +1609,9 @@ njs_string_match_multiple(njs_vm_t *vm, 
     u_char             *start;
     int32_t            size, length;
     njs_ret_t          ret;
-    nxt_uint_t         n, utf8;
+    njs_utf8_t         utf8;
     njs_array_t        *array;
+    njs_regexp_utf8_t  type;
     njs_string_prop_t  string;
 
     args[1].data.u.regexp->last_index = 0;
@@ -1618,26 +1619,23 @@ njs_string_match_multiple(njs_vm_t *vm, 
 
     (void) njs_string_prop(&string, &args[0]);
 
-    /* Byte string. */
-    utf8 = 0;
-    n = 0;
+    utf8 = NJS_STRING_BYTE;
+    type = NJS_REGEXP_BYTE;
 
     if (string.length != 0) {
-        /* ASCII string. */
-        utf8 = 1;
-        n = 1;
+        utf8 = NJS_STRING_ASCII;
+        type = NJS_REGEXP_UTF8;
 
         if (string.length != string.size) {
-            /* UTF-8 string. */
-            utf8 = 2;
+            utf8 = NJS_STRING_UTF8;
         }
     }
 
-    if (nxt_regex_is_valid(&pattern->regex[n])) {
+    if (nxt_regex_is_valid(&pattern->regex[type])) {
         array = NULL;
 
         do {
-            ret = njs_regexp_match(vm, &pattern->regex[n], string.start,
+            ret = njs_regexp_match(vm, &pattern->regex[type], string.start,
                                    string.size, vm->single_match_data);
             if (ret >= 0) {
                 if (array != NULL) {
@@ -1667,25 +1665,7 @@ njs_string_match_multiple(njs_vm_t *vm, 
 
                 size = captures[1] - captures[0];
 
-                switch (utf8) {
-                case 0:
-                    length = 0;
-                    break;
-
-                case 1:
-                    length = size;
-                    break;
-
-                default:
-                    length = nxt_utf8_length(start, size);
-
-                    if (nxt_slow_path(length < 0)) {
-                        vm->exception = &njs_exception_internal_error;
-                        return NXT_ERROR;
-                    }
-
-                    break;
-                }
+                length = njs_string_length(utf8, start, size);
 
                 ret = njs_string_create(vm, &array->start[array->length],
                                         start, size, length);
@@ -1721,9 +1701,10 @@ njs_string_prototype_split(njs_vm_t *vm,
     u_char                *p, *start, *next;
     size_t                size;
     uint32_t              limit;
-    nxt_uint_t            n, utf8;
+    njs_utf8_t            utf8;
     njs_array_t           *array;
     const u_char          *end;
+    njs_regexp_utf8_t     type;
     njs_string_prop_t     string, split;
     njs_regexp_pattern_t  *pattern;
 
@@ -1751,18 +1732,15 @@ njs_string_prototype_split(njs_vm_t *vm,
             goto single;
         }
 
-        /* Byte string. */
-        utf8 = 0;
-        n = 0;
+        utf8 = NJS_STRING_BYTE;
+        type = NJS_REGEXP_BYTE;
 
         if (string.length != 0) {
-            /* ASCII string. */
-            utf8 = 1;
+            utf8 = NJS_STRING_ASCII;
+            type = NJS_REGEXP_UTF8;
 
             if (string.length != string.size) {
-                /* UTF-8 string. */
-                utf8 = 2;
-                n = 1;
+                utf8 = NJS_STRING_UTF8;
             }
         }
 
@@ -1795,7 +1773,7 @@ njs_string_prototype_split(njs_vm_t *vm,
 
                 size = p - start;
 
-                ret = njs_string_split_part_add(vm, array, start, size, utf8);
+                ret = njs_string_split_part_add(vm, array, utf8, start, size);
                 if (nxt_slow_path(ret != NXT_OK)) {
                     return ret;
                 }
@@ -1810,7 +1788,7 @@ njs_string_prototype_split(njs_vm_t *vm,
         case NJS_REGEXP:
             pattern = args[1].data.u.regexp->pattern;
 
-            if (!nxt_regex_is_valid(&pattern->regex[n])) {
+            if (!nxt_regex_is_valid(&pattern->regex[type])) {
                 goto single;
             }
 
@@ -1818,7 +1796,7 @@ njs_string_prototype_split(njs_vm_t *vm,
             end = string.start + string.size;
 
             do {
-                ret = njs_regexp_match(vm, &pattern->regex[n], start,
+                ret = njs_regexp_match(vm, &pattern->regex[type], start,
                                        end - start, vm->single_match_data);
                 if (ret >= 0) {
                     captures = nxt_regex_captures(vm->single_match_data);
@@ -1842,7 +1820,7 @@ njs_string_prototype_split(njs_vm_t *vm,
 
                 size = p - start;
 
-                ret = njs_string_split_part_add(vm, array, start, size, utf8);
+                ret = njs_string_split_part_add(vm, array, utf8, start, size);
                 if (nxt_slow_path(ret != NXT_OK)) {
                     return ret;
                 }
@@ -1876,28 +1854,12 @@ done:
 
 
 static njs_ret_t
-njs_string_split_part_add(njs_vm_t *vm, njs_array_t *array, u_char *start,
-    size_t size, nxt_uint_t utf8)
+njs_string_split_part_add(njs_vm_t *vm, njs_array_t *array, njs_utf8_t utf8,
+    u_char *start, size_t size)
 {
     ssize_t  length;
 
-    switch (utf8) {
-    case 0:
-        length = 0;
-        break;
-
-    case 1:
-        length = size;
-        break;
-
-    default:
-        length = nxt_utf8_length(start, size);
-
-        if (nxt_slow_path(length < 0)) {
-            vm->exception = &njs_exception_internal_error;
-            return NXT_ERROR;
-        }
-    }
+    length = njs_string_length(utf8, start, size);
 
     return njs_array_string_add(vm, array, start, size, length);
 }
@@ -2141,7 +2103,7 @@ njs_string_replace_regexp_function(njs_v
         size = captures[k + 1] - captures[k];
         k += 2;
 
-        length = njs_string_length(start, size, r->utf8);
+        length = njs_string_length(r->utf8, start, size);
 
         ret = njs_string_create(vm, &arguments[i], start, size, length);
         if (nxt_slow_path(ret != NXT_OK)) {
@@ -2153,7 +2115,7 @@ njs_string_replace_regexp_function(njs_v
     njs_number_set(&arguments[n + 1], captures[0]);
 
     /* The whole string being examined. */
-    length = njs_string_length(r->part[0].start, r->part[0].size, r->utf8);
+    length = njs_string_length(r->utf8, r->part[0].start, r->part[0].size);
 
     ret = njs_string_create(vm, &arguments[n + 2], r->part[0].start,
                             r->part[0].size, length);
diff -r eed097d72d5e -r fec0d8dfa38c njs/njs_string.h
--- a/njs/njs_string.h	Mon Sep 26 14:01:39 2016 +0300
+++ b/njs/njs_string.h	Mon Sep 26 14:01:45 2016 +0300
@@ -89,7 +89,7 @@ typedef enum {
 
 
 nxt_inline uint32_t
-njs_string_length(u_char *start, size_t size, njs_utf8_t utf8)
+njs_string_length(njs_utf8_t utf8, u_char *start, size_t size)
 {
     ssize_t  length;
 
diff -r eed097d72d5e -r fec0d8dfa38c njs/njs_vm.c
--- a/njs/njs_vm.c	Mon Sep 26 14:01:39 2016 +0300
+++ b/njs/njs_vm.c	Mon Sep 26 14:01:45 2016 +0300
@@ -3334,7 +3334,7 @@ njs_value_string_copy(njs_vm_t *vm, nxt_
 void
 njs_vm_throw_exception(njs_vm_t *vm, u_char *buf, uint32_t size)
 {
-    uint32_t     length;
+    int32_t      length;
     njs_value_t  *value;
 
     value = nxt_mem_cache_alloc(vm->mem_cache_pool, sizeof(njs_value_t));
@@ -3343,6 +3343,7 @@ njs_vm_throw_exception(njs_vm_t *vm, u_c
         vm->exception = value;
 
         length = nxt_utf8_length(buf, size);
+        length = (length >= 0) ? length : 0;
 
         (void) njs_string_new(vm, value, buf, size, length);
     }


From igor at sysoev.ru  Mon Sep 26 15:42:56 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Mon, 26 Sep 2016 15:42:56 +0000
Subject: [njs] A function stored in array could not be called.
Message-ID: <hg.cfa17c3e25da.1474904576.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/cfa17c3e25da
branches:  
changeset: 184:cfa17c3e25da
user:      Igor Sysoev <igor at sysoev.ru>
date:      Mon Sep 26 18:41:57 2016 +0300
description:
A function stored in array could not be called.

diffstat:

 njs/njs_vm.c             |  116 ++++++++++++++++++++++++++--------------------
 njs/test/njs_unit_test.c |    3 +
 2 files changed, 68 insertions(+), 51 deletions(-)

diffs (167 lines):

diff -r fec0d8dfa38c -r cfa17c3e25da njs/njs_vm.c
--- a/njs/njs_vm.c	Mon Sep 26 14:01:45 2016 +0300
+++ b/njs/njs_vm.c	Mon Sep 26 18:41:57 2016 +0300
@@ -83,6 +83,8 @@ static nxt_noinline njs_ret_t njs_values
 static nxt_noinline njs_ret_t njs_values_compare(njs_value_t *val1,
     njs_value_t *val2);
 static njs_object_t *njs_function_new_object(njs_vm_t *vm, njs_value_t *value);
+static njs_ret_t njs_vmcode_method_call(njs_vm_t *vm, njs_value_t *object,
+    njs_value_t *value);
 static njs_ret_t njs_vmcode_continuation(njs_vm_t *vm, njs_value_t *invld1,
     njs_value_t *invld2);
 static njs_native_frame_t *
@@ -2231,9 +2233,8 @@ njs_ret_t
 njs_vmcode_method_frame(njs_vm_t *vm, njs_value_t *object, njs_value_t *name)
 {
     njs_ret_t                  ret;
-    njs_value_t                this;
+    njs_value_t                this, *value;
     njs_extern_t               *ext;
-    njs_function_t             *function;
     njs_object_prop_t          *prop;
     njs_property_query_t       pq;
     njs_vmcode_method_frame_t  *method;
@@ -2246,13 +2247,35 @@ njs_vmcode_method_frame(njs_vm_t *vm, nj
         prop = pq.lhq.value;
 
         if (njs_is_function(&prop->value)) {
-
+            return njs_vmcode_method_call(vm, object, &prop->value);
+        }
+
+        break;
+
+    case NJS_ARRAY_VALUE:
+        value = pq.lhq.value;
+
+        if (njs_is_function(value)) {
+            return njs_vmcode_method_call(vm, object, value);
+        }
+
+        break;
+
+    case NJS_EXTERNAL_VALUE:
+        ext = object->data.u.external;
+
+        ret = nxt_lvlhsh_find(&ext->hash, &pq.lhq);
+
+        if (ret == NXT_OK) {
             method = (njs_vmcode_method_frame_t *) vm->current;
-            function = prop->value.data.u.function;
-
-            if (!function->native) {
-                ret = njs_function_frame(vm, function, object, NULL,
-                                         method->nargs, method->code.ctor);
+            ext = pq.lhq.value;
+
+            if (ext->type == NJS_EXTERN_METHOD) {
+                this.data.u.data = vm->external[ext->object];
+
+                ret = njs_function_native_frame(vm, ext->function, &this, NULL,
+                                                method->nargs, 0,
+                                                method->code.ctor);
 
                 if (nxt_fast_path(ret == NXT_OK)) {
                     return sizeof(njs_vmcode_method_frame_t);
@@ -2260,54 +2283,45 @@ njs_vmcode_method_frame(njs_vm_t *vm, nj
 
                 return ret;
             }
-
-            ret = njs_function_native_frame(vm, function, object, NULL,
-                                            method->nargs, 0,
-                                            method->code.ctor);
-
-            if (nxt_fast_path(ret == NXT_OK)) {
-                njs_retain(object);
-                return sizeof(njs_vmcode_method_frame_t);
-            }
-
-            return ret;
         }
-
-        break;
-
-    case NJS_EXTERNAL_VALUE:
-        ext = object->data.u.external;
-
-        ret = nxt_lvlhsh_find(&ext->hash, &pq.lhq);
-
-        if (ret == NXT_OK) {
-            method = (njs_vmcode_method_frame_t *) vm->current;
-            ext = pq.lhq.value;
-
-            if (ext->type == NJS_EXTERN_METHOD) {
-                this.data.u.data = vm->external[ext->object];
-
-                ret = njs_function_native_frame(vm, ext->function, &this, NULL,
-                                                method->nargs, 0,
-                                                method->code.ctor);
-
-                if (nxt_fast_path(ret == NXT_OK)) {
-                    return sizeof(njs_vmcode_method_frame_t);
-                }
-
-                return ret;
-            }
+    }
+
+    vm->exception = &njs_exception_type_error;
+
+    return NXT_ERROR;
+}
+
+
+static njs_ret_t
+njs_vmcode_method_call(njs_vm_t *vm, njs_value_t *object, njs_value_t *value)
+{
+    njs_ret_t                  ret;
+    njs_function_t             *function;
+    njs_vmcode_method_frame_t  *method;
+
+    method = (njs_vmcode_method_frame_t *) vm->current;
+    function = value->data.u.function;
+
+    if (!function->native) {
+        ret = njs_function_frame(vm, function, object, NULL, method->nargs,
+                                 method->code.ctor);
+
+        if (nxt_fast_path(ret == NXT_OK)) {
+            return sizeof(njs_vmcode_method_frame_t);
         }
 
-        break;
-
-    default:
-        break;
+        return ret;
     }
 
-    vm->exception = &njs_exception_type_error;
-
-    return NXT_ERROR;
+    ret = njs_function_native_frame(vm, function, object, NULL, method->nargs,
+                                    0, method->code.ctor);
+
+    if (nxt_fast_path(ret == NXT_OK)) {
+        njs_retain(object);
+        return sizeof(njs_vmcode_method_frame_t);
+    }
+
+    return ret;
 }
 
 
diff -r fec0d8dfa38c -r cfa17c3e25da njs/test/njs_unit_test.c
--- a/njs/test/njs_unit_test.c	Mon Sep 26 14:01:45 2016 +0300
+++ b/njs/test/njs_unit_test.c	Mon Sep 26 18:41:57 2016 +0300
@@ -2092,6 +2092,9 @@ static njs_unit_test_t  njs_test[] =
     { nxt_string("a = [1, 2]; delete a[0]; 0 in a"),
       nxt_string("false") },
 
+    { nxt_string("var a = [ function(a) {return a + 1} ]; a[0](5)"),
+      nxt_string("6") },
+
     { nxt_string("var s = '', a = [5,1,2];"
                  "a[null] = null;"
                  "a[undefined] = 'defined';"


From cloud.gate at outlook.com  Mon Sep 26 18:09:07 2016
From: cloud.gate at outlook.com (Cloud Gate)
Date: Mon, 26 Sep 2016 18:09:07 +0000
Subject: Handling Large Number of Dynamic Hosts in Custom Module Subrequests
Message-ID: <BLUPR0101MB14901D8215C1F9539E4A131C94CD0@BLUPR0101MB1490.prod.exchangelabs.com>

Hello nginx developers,

Background: Our custom C++ module for nginx 1.11.4 uses subrequests to perform REST calls to our server during the course of processing incoming proxy requests. The subrequest / server URL is dynamic and varies by request e.g. <host1>.domain.foo, <host2>.domain.foo, etc. There may be thousands of variations of host. As best we can tell, subrequests require explicit configuration in nginx.conf, where the host / URL is hard-coded. We'd rather not add thousands of config sections, which bloat the configuration and require constant maintenance.

Is there a way to configure nginx to proxy a large number of hosts, where the specific host is determined dynamically at runtime?

We are basing our development on these sample resources: http://nginx.org/en/docs/http/ngx_http_auth_request_module.html or https://github.com/openresty/echo-nginx-module. Configuration for those modules hard-codes a small number of known locations for upstream servers, e.g. auth request module (additional authentication servers would require additional location blocks):

location /private/ {
    auth_request /auth;
    ...
}

location = /auth {
    proxy_pass http://authserver
    ...

The first alternative approach we've come up with is to set an HTTP header in the subrequest containing the target URL (x_our_proxy_url=http://host1.domain.foo/...), target the subrequest at a static /ourproxy location, and in nginx.conf pass the URL from the header to proxy_pass:

location /ourproxy {
   proxy_pass $http_x_our_proxy_url;
}

While this works, it seems somewhat ad-hoc, and we're wondering if there are better options for a subrequest to target arbitrary hosts. Note that use of third-party modules such as OpenResty is not really an option at this time, as we are strictly a C++ module.

Thanks


--

CG Development
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160926/d5384879/attachment.html>

From mdounin at mdounin.ru  Tue Sep 27 12:40:51 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 27 Sep 2016 15:40:51 +0300
Subject: Handling Large Number of Dynamic Hosts in Custom Module
 Subrequests
In-Reply-To: <BLUPR0101MB14901D8215C1F9539E4A131C94CD0@BLUPR0101MB1490.prod.exchangelabs.com>
References: <BLUPR0101MB14901D8215C1F9539E4A131C94CD0@BLUPR0101MB1490.prod.exchangelabs.com>
Message-ID: <20160927124051.GI73038@mdounin.ru>

Hello!

On Mon, Sep 26, 2016 at 06:09:07PM +0000, Cloud Gate wrote:

[...]

> The first alternative approach we've come up with is to set an 
> HTTP header in the subrequest containing the target URL 
> (x_our_proxy_url=http://host1.domain.foo/...), target the 
> subrequest at a static /ourproxy location, and in nginx.conf 
> pass the URL from the header to proxy_pass:
> 
> location /ourproxy {
>    proxy_pass $http_x_our_proxy_url;
> }
> 
> While this works, it seems somewhat ad-hoc, and we're wondering 
> if there are better options for a subrequest to target arbitrary 
> hosts.
> module.

In general, using proxy_pass with a variable is correct.  Though 
instead of abusing HTTP headers (in general, this is not something 
nginx allows, and you are likely doing it wrong) I would recommend 
using part of URI instead, or just providing appropriate variable 
from your module.

-- 
Maxim Dounin
http://nginx.org/


From nginx-devel at hodor.cz  Tue Sep 27 21:23:00 2016
From: nginx-devel at hodor.cz (Jan Seda)
Date: Tue, 27 Sep 2016 23:23:00 +0200
Subject: [PATCH] Core: fixed uninitialized memory access
Message-ID: <20160927212259.GT19300@manetheren.hodor.cz>

Struct sockaddr was allocated without zeroing. As getsockname() does not
necessarily fill all the bytes, this left potentially uninitialized
memory, which caused me a weird failure in the on-the-fly upgrade
mechanism, where the new master failed to pair the config file AF_UNIX
sockets with the inherited ones. ngx_cmp_sockaddr() would return
NGX_DECLINED because the inherited one would contain uninitialized
garbage after the actual name. This in turn would cause nginx to try to
bind() the unix socket again, and obviously fail, because the listening
socket was bound already (and inherited).

Easy to reproduce on debian wheezy, debian jessie and CentOS 7 systems
(all x86_64) with statically linked openssl 1.1.0. The attached patch
seems to fix the problem.

Also harmonized socklen calculation for AF_UNIX sockets to the way
ngx_parse_unix_domain_url() does it (sizeof(struct sockaddr_un)), so
ngx_cmp_sockaddr() is now less dangerous.

Applies cleanly to 1.11.4.


commit 6f2a907b0dc870314e1d1ecc24fd59c60553152e
Author: Jan Seda <hodor at hodor.cz>
Date:   Tue Sep 27 21:52:06 2016 +0200

    Core: fixed uninitialized memory access
    
    Struct sockaddr was allocated without zeroing.
    
    Also harmonized socklen calculation for AF_UNIX sockets.

diff --git a/src/core/ngx_connection.c b/src/core/ngx_connection.c
index 16ba630..fe6e64f 100644
--- a/src/core/ngx_connection.c
+++ b/src/core/ngx_connection.c
@@ -151,7 +151,7 @@ ngx_set_inherited_sockets(ngx_cycle_t *cycle)
     ls = cycle->listening.elts;
     for (i = 0; i < cycle->listening.nelts; i++) {
 
-        ls[i].sockaddr = ngx_palloc(cycle->pool, sizeof(ngx_sockaddr_t));
+        ls[i].sockaddr = ngx_pcalloc(cycle->pool, sizeof(ngx_sockaddr_t));
         if (ls[i].sockaddr == NULL) {
             return NGX_ERROR;
         }
@@ -178,6 +178,7 @@ ngx_set_inherited_sockets(ngx_cycle_t *cycle)
         case AF_UNIX:
             ls[i].addr_text_max_len = NGX_UNIX_ADDRSTRLEN;
             len = NGX_UNIX_ADDRSTRLEN;
+            ls[i].socklen = sizeof(struct sockaddr_un);
             break;
 #endif
 
@@ -1323,7 +1324,7 @@ ngx_connection_local_sockaddr(ngx_connection_t *c, ngx_str_t *s,
             return NGX_ERROR;
         }
 
-        c->local_sockaddr = ngx_palloc(c->pool, len);
+        c->local_sockaddr = ngx_pcalloc(c->pool, sizeof(ngx_sockaddr_t));
         if (c->local_sockaddr == NULL) {
             return NGX_ERROR;
         }
@@ -1331,6 +1332,11 @@ ngx_connection_local_sockaddr(ngx_connection_t *c, ngx_str_t *s,
         ngx_memcpy(c->local_sockaddr, &sa, len);
 
         c->local_socklen = len;
+#if (NGX_HAVE_UNIX_DOMAIN)
+        if(sa.sockaddr.sa_family == AF_UNIX) {
+            c->local_socklen = sizeof(struct sockaddr_un);
+        }
+#endif
     }
 
     if (s == NULL) {


-- 
Jan Seda
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nginx_uninitialized_sockaddr.diff
Type: text/x-diff
Size: 1815 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160927/4a324def/attachment.bin>

From alessandro at cloudflare.com  Wed Sep 28 14:10:46 2016
From: alessandro at cloudflare.com (Alessandro Ghedini)
Date: Wed, 28 Sep 2016 15:10:46 +0100
Subject: [PATCH 0 of 1] Upstream: fix warning when building with BoringSSL
Message-ID: <patchbomb.1475071846@mandy.local>

Hello,

I don't now what your current plans for supporting BoringSSL are, but its
API has been fairly stable for a while and this is the only change required
to make NGINX build with it again (the other issue with error definitions was
fixed in BoringSSL itself).

I don't think BoringSSL is going to change the API back, so NGINX migh want
to fix this if support for BoringSSL is desired (again, don't know your
opinion on this).

Please have a look and let me know what you think.

Cheers


From alessandro at cloudflare.com  Wed Sep 28 14:10:47 2016
From: alessandro at cloudflare.com (Alessandro Ghedini)
Date: Wed, 28 Sep 2016 15:10:47 +0100
Subject: [PATCH 1 of 1] Upstream: fix warning when building with BoringSSL
In-Reply-To: <patchbomb.1475071846@mandy.local>
References: <patchbomb.1475071846@mandy.local>
Message-ID: <8c3706ffdc9bde38ef50.1475071847@mandy.local>

# HG changeset patch
# User Alessandro Ghedini <alessandro at cloudflare.com>
# Date 1475070884 -3600
#      Wed Sep 28 14:54:44 2016 +0100
# Node ID 8c3706ffdc9bde38ef5035b5f2fa9979cf1ea4da
# Parent  29bf0dbc0a77914bc94bd001a2b17d364e8e50d9
Upstream: fix warning when building with BoringSSL

BoringSSL takes a const u_char * for SSL_set_tlsext_host_name but
OpenSSL only takes a u_char *. Since NGINX is built with -Werror by
default this breaks the build.

diff -r 29bf0dbc0a77 -r 8c3706ffdc9b src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Thu Sep 22 19:32:47 2016 +0300
+++ b/src/http/ngx_http_upstream.c	Wed Sep 28 14:54:44 2016 +0100
@@ -1696,7 +1696,12 @@
     ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                    "upstream SSL server name: \"%s\"", name.data);
 
+#ifdef OPENSSL_IS_BORINGSSL
+    if (SSL_set_tlsext_host_name(c->ssl->connection, 
+                                 (const char *) name.data) == 0) {
+#else
     if (SSL_set_tlsext_host_name(c->ssl->connection, name.data) == 0) {
+#endif
         ngx_ssl_error(NGX_LOG_ERR, r->connection->log, 0,
                       "SSL_set_tlsext_host_name(\"%s\") failed", name.data);
         return NGX_ERROR;


From mdounin at mdounin.ru  Wed Sep 28 14:19:02 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 28 Sep 2016 17:19:02 +0300
Subject: [PATCH 0 of 1] Upstream: fix warning when building with BoringSSL
In-Reply-To: <patchbomb.1475071846@mandy.local>
References: <patchbomb.1475071846@mandy.local>
Message-ID: <20160928141902.GN73038@mdounin.ru>

Hello!

On Wed, Sep 28, 2016 at 03:10:46PM +0100, Alessandro Ghedini wrote:

> Hello,
> 
> I don't now what your current plans for supporting BoringSSL are, but its
> API has been fairly stable for a while and this is the only change required
> to make NGINX build with it again (the other issue with error definitions was
> fixed in BoringSSL itself).
> 
> I don't think BoringSSL is going to change the API back, so NGINX migh want
> to fix this if support for BoringSSL is desired (again, don't know your
> opinion on this).
> 
> Please have a look and let me know what you think.

Quoting 
http://mailman.nginx.org/pipermail/nginx-devel/2016-August/008680.html:

: Ok, this looks like the real reason for the patch.  This looks 
: like an API change in BoringSSL, and should be threated 
: accordingly.

: Given the number of various API changes BoringSSL introduces here 
: and there - we probably don't want to follow, at least till some 
: version is actually released.

-- 
Maxim Dounin
http://nginx.org/


From alessandro at cloudflare.com  Wed Sep 28 14:37:48 2016
From: alessandro at cloudflare.com (Alessandro Ghedini)
Date: Wed, 28 Sep 2016 15:37:48 +0100
Subject: [PATCH 0 of 1] Upstream: fix warning when building with BoringSSL
In-Reply-To: <20160928141902.GN73038@mdounin.ru>
References: <patchbomb.1475071846@mandy.local>
 <20160928141902.GN73038@mdounin.ru>
Message-ID: <20160928143748.7mb5aiw5pqsnb56w@mandy.local>

On Wed, Sep 28, 2016 at 05:19:02PM +0300, Maxim Dounin wrote:
> Hello!
> 
> On Wed, Sep 28, 2016 at 03:10:46PM +0100, Alessandro Ghedini wrote:
> 
> > Hello,
> > 
> > I don't now what your current plans for supporting BoringSSL are, but its
> > API has been fairly stable for a while and this is the only change required
> > to make NGINX build with it again (the other issue with error definitions was
> > fixed in BoringSSL itself).
> > 
> > I don't think BoringSSL is going to change the API back, so NGINX migh want
> > to fix this if support for BoringSSL is desired (again, don't know your
> > opinion on this).
> > 
> > Please have a look and let me know what you think.
> 
> Quoting 
> http://mailman.nginx.org/pipermail/nginx-devel/2016-August/008680.html:
> 
> : Ok, this looks like the real reason for the patch.  This looks 
> : like an API change in BoringSSL, and should be threated 
> : accordingly.
> 
> : Given the number of various API changes BoringSSL introduces here 
> : and there - we probably don't want to follow, at least till some 
> : version is actually released.

Ok, thanks, I missed that. TBH I don't think the BoringSSL team intends to
release "proper" versions like OpenSSL does, so what you propose to wait for
might not actually ever happen.

I understand your concern of wanting to target a fixed release, but as I
mentioned (and Piotr as well) BoringSSL's API seems to have been fairly stable
for a while (except for fixes like the one for the problem mentioned in the
patch you linked, which was worked around in BoringSSL itself), and AFAIK there
aren't other similar compatibility problems left except for this build warning
(but maybe Piotr could prove me wrong on that), so it might make sense to start
looking at supporting BoringSSL again.

Cheers


From mdounin at mdounin.ru  Wed Sep 28 16:00:00 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 28 Sep 2016 19:00:00 +0300
Subject: [PATCH 0 of 1] Upstream: fix warning when building with BoringSSL
In-Reply-To: <20160928143748.7mb5aiw5pqsnb56w@mandy.local>
References: <patchbomb.1475071846@mandy.local>
 <20160928141902.GN73038@mdounin.ru>
 <20160928143748.7mb5aiw5pqsnb56w@mandy.local>
Message-ID: <20160928160000.GP73038@mdounin.ru>

Hello!

On Wed, Sep 28, 2016 at 03:37:48PM +0100, Alessandro Ghedini wrote:

> On Wed, Sep 28, 2016 at 05:19:02PM +0300, Maxim Dounin wrote:
> > Hello!
> > 
> > On Wed, Sep 28, 2016 at 03:10:46PM +0100, Alessandro Ghedini wrote:
> > 
> > > Hello,
> > > 
> > > I don't now what your current plans for supporting BoringSSL are, but its
> > > API has been fairly stable for a while and this is the only change required
> > > to make NGINX build with it again (the other issue with error definitions was
> > > fixed in BoringSSL itself).
> > > 
> > > I don't think BoringSSL is going to change the API back, so NGINX migh want
> > > to fix this if support for BoringSSL is desired (again, don't know your
> > > opinion on this).
> > > 
> > > Please have a look and let me know what you think.
> > 
> > Quoting 
> > http://mailman.nginx.org/pipermail/nginx-devel/2016-August/008680.html:
> > 
> > : Ok, this looks like the real reason for the patch.  This looks 
> > : like an API change in BoringSSL, and should be threated 
> > : accordingly.
> > 
> > : Given the number of various API changes BoringSSL introduces here 
> > : and there - we probably don't want to follow, at least till some 
> > : version is actually released.
> 
> Ok, thanks, I missed that. TBH I don't think the BoringSSL team intends to
> release "proper" versions like OpenSSL does, so what you propose to wait for
> might not actually ever happen.

Sure, and I'm fine with it.

> I understand your concern of wanting to target a fixed release, but as I
> mentioned (and Piotr as well) BoringSSL's API seems to have been fairly stable
> for a while (except for fixes like the one for the problem mentioned in the
> patch you linked, which was worked around in BoringSSL itself), and AFAIK there
> aren't other similar compatibility problems left except for this build warning
> (but maybe Piotr could prove me wrong on that), so it might make sense to start
> looking at supporting BoringSSL again.

Last time I looked into BoringSSL code due to ticket #993 several 
months ago (https://trac.nginx.org/nginx/ticket/993), and my 
impression wasn't that positive.

-- 
Maxim Dounin
http://nginx.org/


From alessandro at cloudflare.com  Wed Sep 28 16:28:48 2016
From: alessandro at cloudflare.com (Alessandro Ghedini)
Date: Wed, 28 Sep 2016 17:28:48 +0100
Subject: [PATCH 0 of 1] Upstream: fix warning when building with BoringSSL
In-Reply-To: <20160928160000.GP73038@mdounin.ru>
References: <patchbomb.1475071846@mandy.local>
 <20160928141902.GN73038@mdounin.ru>
 <20160928143748.7mb5aiw5pqsnb56w@mandy.local>
 <20160928160000.GP73038@mdounin.ru>
Message-ID: <20160928162848.c5nx2pfpbnx2n56h@mandy.local>

On Wed, Sep 28, 2016 at 07:00:00PM +0300, Maxim Dounin wrote:
> Hello!
> 
> On Wed, Sep 28, 2016 at 03:37:48PM +0100, Alessandro Ghedini wrote:
> 
> > On Wed, Sep 28, 2016 at 05:19:02PM +0300, Maxim Dounin wrote:
> > > Hello!
> > > 
> > > On Wed, Sep 28, 2016 at 03:10:46PM +0100, Alessandro Ghedini wrote:
> > > 
> > > > Hello,
> > > > 
> > > > I don't now what your current plans for supporting BoringSSL are, but its
> > > > API has been fairly stable for a while and this is the only change required
> > > > to make NGINX build with it again (the other issue with error definitions was
> > > > fixed in BoringSSL itself).
> > > > 
> > > > I don't think BoringSSL is going to change the API back, so NGINX migh want
> > > > to fix this if support for BoringSSL is desired (again, don't know your
> > > > opinion on this).
> > > > 
> > > > Please have a look and let me know what you think.
> > > 
> > > Quoting 
> > > http://mailman.nginx.org/pipermail/nginx-devel/2016-August/008680.html:
> > > 
> > > : Ok, this looks like the real reason for the patch.  This looks 
> > > : like an API change in BoringSSL, and should be threated 
> > > : accordingly.
> > > 
> > > : Given the number of various API changes BoringSSL introduces here 
> > > : and there - we probably don't want to follow, at least till some 
> > > : version is actually released.
> > 
> > Ok, thanks, I missed that. TBH I don't think the BoringSSL team intends to
> > release "proper" versions like OpenSSL does, so what you propose to wait for
> > might not actually ever happen.
> 
> Sure, and I'm fine with it.
> 
> > I understand your concern of wanting to target a fixed release, but as I
> > mentioned (and Piotr as well) BoringSSL's API seems to have been fairly stable
> > for a while (except for fixes like the one for the problem mentioned in the
> > patch you linked, which was worked around in BoringSSL itself), and AFAIK there
> > aren't other similar compatibility problems left except for this build warning
> > (but maybe Piotr could prove me wrong on that), so it might make sense to start
> > looking at supporting BoringSSL again.
> 
> Last time I looked into BoringSSL code due to ticket #993 several 
> months ago (https://trac.nginx.org/nginx/ticket/993), and my 
> impression wasn't that positive.

Ah, BoringSSL actually supports the new SSL_CTX_set1_curves_list() API that
NGINX already uses, but it doesn't seem to define SSL_CTRL_SET_CURVES_LIST
(yet) so the other API is picked. I'll make a patch for BoringSSL to fix this.

Cheers


From mdounin at mdounin.ru  Wed Sep 28 16:40:13 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 28 Sep 2016 19:40:13 +0300
Subject: [PATCH 0 of 1] Upstream: fix warning when building with BoringSSL
In-Reply-To: <20160928162848.c5nx2pfpbnx2n56h@mandy.local>
References: <patchbomb.1475071846@mandy.local>
 <20160928141902.GN73038@mdounin.ru>
 <20160928143748.7mb5aiw5pqsnb56w@mandy.local>
 <20160928160000.GP73038@mdounin.ru>
 <20160928162848.c5nx2pfpbnx2n56h@mandy.local>
Message-ID: <20160928164013.GR73038@mdounin.ru>

Hello!

On Wed, Sep 28, 2016 at 05:28:48PM +0100, Alessandro Ghedini wrote:

> On Wed, Sep 28, 2016 at 07:00:00PM +0300, Maxim Dounin wrote:
> > Hello!
> > 
> > On Wed, Sep 28, 2016 at 03:37:48PM +0100, Alessandro Ghedini wrote:
> > 
> > > On Wed, Sep 28, 2016 at 05:19:02PM +0300, Maxim Dounin wrote:
> > > > Hello!
> > > > 
> > > > On Wed, Sep 28, 2016 at 03:10:46PM +0100, Alessandro Ghedini wrote:
> > > > 
> > > > > Hello,
> > > > > 
> > > > > I don't now what your current plans for supporting BoringSSL are, but its
> > > > > API has been fairly stable for a while and this is the only change required
> > > > > to make NGINX build with it again (the other issue with error definitions was
> > > > > fixed in BoringSSL itself).
> > > > > 
> > > > > I don't think BoringSSL is going to change the API back, so NGINX migh want
> > > > > to fix this if support for BoringSSL is desired (again, don't know your
> > > > > opinion on this).
> > > > > 
> > > > > Please have a look and let me know what you think.
> > > > 
> > > > Quoting 
> > > > http://mailman.nginx.org/pipermail/nginx-devel/2016-August/008680.html:
> > > > 
> > > > : Ok, this looks like the real reason for the patch.  This looks 
> > > > : like an API change in BoringSSL, and should be threated 
> > > > : accordingly.
> > > > 
> > > > : Given the number of various API changes BoringSSL introduces here 
> > > > : and there - we probably don't want to follow, at least till some 
> > > > : version is actually released.
> > > 
> > > Ok, thanks, I missed that. TBH I don't think the BoringSSL team intends to
> > > release "proper" versions like OpenSSL does, so what you propose to wait for
> > > might not actually ever happen.
> > 
> > Sure, and I'm fine with it.
> > 
> > > I understand your concern of wanting to target a fixed release, but as I
> > > mentioned (and Piotr as well) BoringSSL's API seems to have been fairly stable
> > > for a while (except for fixes like the one for the problem mentioned in the
> > > patch you linked, which was worked around in BoringSSL itself), and AFAIK there
> > > aren't other similar compatibility problems left except for this build warning
> > > (but maybe Piotr could prove me wrong on that), so it might make sense to start
> > > looking at supporting BoringSSL again.
> > 
> > Last time I looked into BoringSSL code due to ticket #993 several 
> > months ago (https://trac.nginx.org/nginx/ticket/993), and my 
> > impression wasn't that positive.
> 
> Ah, BoringSSL actually supports the new SSL_CTX_set1_curves_list() API that
> NGINX already uses, but it doesn't seem to define SSL_CTRL_SET_CURVES_LIST
> (yet) so the other API is picked. I'll make a patch for BoringSSL to fix this.

I'm afraid you are wrong here,

$ grep -r SSL_CTX_set1_curves_list boringssl/ | wc -l
       0

Instead, BoringSSL introduced its own API to work with curves.

-- 
Maxim Dounin
http://nginx.org/


From arut at nginx.com  Wed Sep 28 16:59:23 2016
From: arut at nginx.com (Roman Arutyunyan)
Date: Wed, 28 Sep 2016 16:59:23 +0000
Subject: [njs] Stream js: js_access, js_preread, js_filter.
Message-ID: <hg.322359ec0913.1475081963.5965299922797593991@dev.nginx.com>

details:   http://hg.nginx.org/njs/rev/322359ec0913
branches:  
changeset: 185:322359ec0913
user:      Roman Arutyunyan <arut at nginx.com>
date:      Wed Sep 28 19:52:05 2016 +0300
description:
Stream js: js_access, js_preread, js_filter.

diffstat:

 README                       |  100 +++++++-
 nginx/ngx_stream_js_module.c |  550 ++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 642 insertions(+), 8 deletions(-)

diffs (792 lines):

diff -r cfa17c3e25da -r 322359ec0913 README
--- a/README	Mon Sep 26 18:41:57 2016 +0300
+++ b/README	Wed Sep 28 19:52:05 2016 +0300
@@ -144,8 +144,16 @@ Each Stream JavaScript handler receives 
 The following properties are available in the session object:
 
      - remoteAddress
+     - eof
+     - fromUpstream
+     - buffer
      - variables{}
      - log()
+     - OK
+     - DECLINED
+     - AGAIN
+     - ERROR
+     - ABORT
 
 
 Example nginx.conf:
@@ -167,22 +175,60 @@ Example nginx.conf:
         server {
             listen 8000;
 
+            # preread data with qux()
+            js_preread qux;
+
             # create $foo and $bar variables and set JavaScript
             # functions foo() and bar() from the included JavaScript
             # file as their handlers
             js_set $foo foo;
             js_set $bar bar;
 
-            return $foo-$bar;
+            # echo-server: return preread data
+            return foo;
+        }
+
+        server {
+            listen 8001;
+
+            # check access in xyz()
+            js_access xyz;
+
+            proxy_pass 127.0.0.1:9000;
+
+            # add JavaScript filter baz() from the included
+            # JavaScript file
+            js_filter baz;
+        }
+    }
+
+    http {
+        server {
+            listen 9000;
+            location / {
+                return 200 $http_foo\n;
+            }
         }
     }
 
 
 stream.js:
 
+    var req = '';
+    var matched = 0;
+    var line = '';
+
+    function qux(s) {
+        n = s.buffer.indexOf('\n');
+        if (n == -1) {
+            return s.AGAIN;
+        }
+
+        line = s.buffer.substr(0, n);
+    }
+
     function foo(s) {
-        s.log("hello from foo() handler!");
-        return s.remoteAddress;
+        return line;
     }
 
     function bar(s) {
@@ -191,6 +237,54 @@ stream.js:
         return "foo-var" + v.remote_port + "; pid=" + v.pid;
     }
 
+    // The filter processes one buffer per call.
+    // The buffer is available in s.buffer both for
+    // reading and writing.  Called for both directions.
+
+    function baz(s) {
+        if (s.fromUpstream || matched) {
+            return;
+        }
+
+        // Disable certain addresses.
+
+        if (s.remoteAddress.match('^192.*')) {
+            return s.ERROR;
+        }
+
+        // Read HTTP request line.
+        // Collect bytes in 'req' until request
+        // line is read.  Clear current buffer to
+        // disable output.
+
+        req = req + s.buffer;
+        s.buffer = '';
+
+        n = req.search('\n');
+
+        if (n != -1) {
+            // Inject a new HTTP header.
+            var rest = req.substr(n + 1);
+            req = req.substr(0, n + 1);
+
+            addr = s.remoteAddress;
+
+            s.log('req:' + req);
+            s.log('rest:' + rest);
+
+            // Output the result and skip further
+            // processing.
+
+            s.buffer = req + 'Foo: addr_' + addr + '\r\n' + rest;
+            matched = 1;
+        }
+    }
+
+    function xyz(s) {
+        if (s.remoteAddress.match('^192.*')) {
+            return s.ABORT;
+        }
+    }
 
 --
 NGINX, Inc., http://nginx.com
diff -r cfa17c3e25da -r 322359ec0913 nginx/ngx_stream_js_module.c
--- a/nginx/ngx_stream_js_module.c	Mon Sep 26 18:41:57 2016 +0300
+++ b/nginx/ngx_stream_js_module.c	Wed Sep 28 19:52:05 2016 +0300
@@ -40,15 +40,30 @@
 typedef struct {
     njs_vm_t              *vm;
     njs_opaque_value_t     arg;
+    ngx_str_t              access;
+    ngx_str_t              preread;
+    ngx_str_t              filter;
 } ngx_stream_js_srv_conf_t;
 
 
 typedef struct {
     njs_vm_t              *vm;
     njs_opaque_value_t    *arg;
+    ngx_buf_t             *buf;
+    ngx_chain_t           *free;
+    ngx_chain_t           *busy;
+    ngx_stream_session_t  *session;
+    unsigned               from_upstream:1;
+    unsigned               filter:1;
 } ngx_stream_js_ctx_t;
 
 
+static ngx_int_t ngx_stream_js_access_handler(ngx_stream_session_t *s);
+static ngx_int_t ngx_stream_js_preread_handler(ngx_stream_session_t *s);
+static ngx_int_t ngx_stream_js_phase_handler(ngx_stream_session_t *s,
+    ngx_str_t *name);
+static ngx_int_t ngx_stream_js_body_filter(ngx_stream_session_t *s,
+    ngx_chain_t *in, ngx_uint_t from_upstream);
 static ngx_int_t ngx_stream_js_variable(ngx_stream_session_t *s,
     ngx_stream_variable_value_t *v, uintptr_t data);
 static void ngx_stream_js_cleanup_mem_cache_pool(void *data);
@@ -61,10 +76,20 @@ static void ngx_stream_js_free(void *mem
 
 static njs_ret_t ngx_stream_js_ext_get_remote_address(njs_vm_t *vm,
     njs_value_t *value, void *obj, uintptr_t data);
-static njs_ret_t ngx_stream_js_ext_log(njs_vm_t *vm, njs_value_t *args,
-    nxt_uint_t nargs, njs_index_t unused);
+static njs_ret_t ngx_stream_js_ext_get_eof(njs_vm_t *vm, njs_value_t *value,
+    void *obj, uintptr_t data);
+static njs_ret_t ngx_stream_js_ext_get_from_upstream(njs_vm_t *vm,
+    njs_value_t *value, void *obj, uintptr_t data);
+static njs_ret_t ngx_stream_js_ext_get_buffer(njs_vm_t *vm, njs_value_t *value,
+    void *obj, uintptr_t data);
+static njs_ret_t ngx_stream_js_ext_set_buffer(njs_vm_t *vm, void *obj,
+    uintptr_t data, nxt_str_t *value);
+ static njs_ret_t ngx_stream_js_ext_log(njs_vm_t *vm, njs_value_t *args,
+     nxt_uint_t nargs, njs_index_t unused);
 static njs_ret_t ngx_stream_js_ext_get_variable(njs_vm_t *vm,
     njs_value_t *value, void *obj, uintptr_t data);
+static njs_ret_t ngx_stream_js_ext_get_code(njs_vm_t *vm,
+    njs_value_t *value, void *obj, uintptr_t data);
 
 static char *ngx_stream_js_include(ngx_conf_t *cf, ngx_command_t *cmd,
     void *conf);
@@ -73,6 +98,7 @@ static char *ngx_stream_js_set(ngx_conf_
 static void *ngx_stream_js_create_srv_conf(ngx_conf_t *cf);
 static char *ngx_stream_js_merge_srv_conf(ngx_conf_t *cf, void *parent,
     void *child);
+static ngx_int_t ngx_stream_js_init(ngx_conf_t *cf);
 
 
 static ngx_command_t  ngx_stream_js_commands[] = {
@@ -91,13 +117,34 @@ static ngx_command_t  ngx_stream_js_comm
       0,
       NULL },
 
+    { ngx_string("js_access"),
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_TAKE1,
+      ngx_conf_set_str_slot,
+      NGX_STREAM_SRV_CONF_OFFSET,
+      offsetof(ngx_stream_js_srv_conf_t, access),
+      NULL },
+
+    { ngx_string("js_preread"),
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_TAKE1,
+      ngx_conf_set_str_slot,
+      NGX_STREAM_SRV_CONF_OFFSET,
+      offsetof(ngx_stream_js_srv_conf_t, preread),
+      NULL },
+
+    { ngx_string("js_filter"),
+      NGX_STREAM_MAIN_CONF|NGX_STREAM_SRV_CONF|NGX_CONF_TAKE1,
+      ngx_conf_set_str_slot,
+      NGX_STREAM_SRV_CONF_OFFSET,
+      offsetof(ngx_stream_js_srv_conf_t, filter),
+      NULL },
+
     ngx_null_command
 };
 
 
 static ngx_stream_module_t  ngx_stream_js_module_ctx = {
     NULL,                          /* preconfiguration */
-    NULL,                          /* postconfiguration */
+    ngx_stream_js_init,            /* postconfiguration */
 
     NULL,                          /* create main configuration */
     NULL,                          /* init main configuration */
@@ -148,6 +195,42 @@ static njs_external_t  ngx_stream_js_ext
       NULL,
       0 },
 
+    { nxt_string("eof"),
+      NJS_EXTERN_PROPERTY,
+      NULL,
+      0,
+      ngx_stream_js_ext_get_eof,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      0 },
+
+    { nxt_string("fromUpstream"),
+      NJS_EXTERN_PROPERTY,
+      NULL,
+      0,
+      ngx_stream_js_ext_get_from_upstream,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      0 },
+
+    { nxt_string("buffer"),
+      NJS_EXTERN_PROPERTY,
+      NULL,
+      0,
+      ngx_stream_js_ext_get_buffer,
+      ngx_stream_js_ext_set_buffer,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      0 },
+
     { nxt_string("log"),
       NJS_EXTERN_METHOD,
       NULL,
@@ -171,6 +254,66 @@ static njs_external_t  ngx_stream_js_ext
       NULL,
       NULL,
       0 },
+
+    { nxt_string("OK"),
+      NJS_EXTERN_PROPERTY,
+      NULL,
+      0,
+      ngx_stream_js_ext_get_code,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      -NGX_OK },
+
+    { nxt_string("DECLINED"),
+      NJS_EXTERN_PROPERTY,
+      NULL,
+      0,
+      ngx_stream_js_ext_get_code,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      -NGX_DECLINED },
+
+    { nxt_string("AGAIN"),
+      NJS_EXTERN_PROPERTY,
+      NULL,
+      0,
+      ngx_stream_js_ext_get_code,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      -NGX_AGAIN },
+
+    { nxt_string("ERROR"),
+      NJS_EXTERN_PROPERTY,
+      NULL,
+      0,
+      ngx_stream_js_ext_get_code,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      -NGX_ERROR },
+
+    { nxt_string("ABORT"),
+      NJS_EXTERN_PROPERTY,
+      NULL,
+      0,
+      ngx_stream_js_ext_get_code,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      NULL,
+      -NGX_ABORT },
 };
 
 
@@ -190,6 +333,224 @@ static njs_external_t  ngx_stream_js_ext
 };
 
 
+static ngx_stream_filter_pt  ngx_stream_next_filter;
+
+
+static ngx_int_t
+ngx_stream_js_access_handler(ngx_stream_session_t *s)
+{
+    ngx_int_t                  rc;
+    ngx_stream_js_srv_conf_t  *jscf;
+
+    ngx_log_debug0(NGX_LOG_DEBUG_STREAM, s->connection->log, 0,
+                   "js access handler");
+
+    jscf = ngx_stream_get_module_srv_conf(s, ngx_stream_js_module);
+
+    rc = ngx_stream_js_phase_handler(s, &jscf->access);
+
+    if (rc == NGX_ABORT) {
+        ngx_log_error(NGX_LOG_ERR, s->connection->log, 0,
+                      "access forbidden by js");
+        rc = NGX_STREAM_FORBIDDEN;
+    }
+
+    return rc;
+}
+
+
+static ngx_int_t
+ngx_stream_js_preread_handler(ngx_stream_session_t *s)
+{
+    ngx_stream_js_srv_conf_t  *jscf;
+
+    ngx_log_debug0(NGX_LOG_DEBUG_STREAM, s->connection->log, 0,
+                   "js preread handler");
+
+    jscf = ngx_stream_get_module_srv_conf(s, ngx_stream_js_module);
+
+    return ngx_stream_js_phase_handler(s, &jscf->preread);
+}
+
+
+static ngx_int_t
+ngx_stream_js_phase_handler(ngx_stream_session_t *s, ngx_str_t *name)
+{
+    nxt_str_t                  fname, value, exception;
+    ngx_int_t                  rc;
+    njs_function_t            *func;
+    ngx_connection_t          *c;
+    ngx_stream_js_ctx_t       *ctx;
+
+    if (name->len == 0) {
+        return NGX_DECLINED;
+    }
+
+    c = s->connection;
+
+    rc = ngx_stream_js_init_vm(s);
+    if (rc != NGX_OK) {
+        return rc;
+    }
+
+    ctx = ngx_stream_get_module_ctx(s, ngx_stream_js_module);
+
+    fname.start = name->data;
+    fname.length = name->len;
+
+    func = njs_vm_function(ctx->vm, &fname);
+
+    if (func == NULL) {
+        ngx_log_error(NGX_LOG_ERR, c->log, 0, "js function \"%V\" not found",
+                      name);
+        return NGX_ERROR;
+    }
+
+    if (njs_vm_call(ctx->vm, func, ctx->arg, 1) != NJS_OK) {
+        njs_vm_exception(ctx->vm, &exception);
+
+        ngx_log_error(NGX_LOG_ERR, c->log, 0, "js exception: %*s",
+                      exception.length, exception.start);
+
+        return NGX_ERROR;
+    }
+
+    if (ctx->vm->retval.type == NJS_VOID) {
+        return NGX_OK;
+    }
+
+    if (njs_vm_retval(ctx->vm, &value) != NJS_OK) {
+        return NGX_ERROR;
+    }
+
+    ngx_log_debug2(NGX_LOG_DEBUG_STREAM, c->log, 0, "js return value: \"%*s\"",
+                   value.length, value.start);
+
+    rc = ngx_atoi(value.start, value.length);
+
+    if (rc == NGX_ERROR) {
+        ngx_log_error(NGX_LOG_ERR, c->log, 0,
+                      "unexpected js return code: \"%*s\"",
+                      value.length, value.start);
+        return NGX_ERROR;
+    }
+
+    return -rc;
+}
+
+
+static ngx_int_t
+ngx_stream_js_body_filter(ngx_stream_session_t *s, ngx_chain_t *in,
+    ngx_uint_t from_upstream)
+{
+    nxt_str_t                  name, value, exception;
+    ngx_int_t                  rc;
+    ngx_chain_t               *out, *cl, **ll;
+    njs_function_t            *func;
+    ngx_connection_t          *c;
+    ngx_stream_js_ctx_t       *ctx;
+    ngx_stream_js_srv_conf_t  *jscf;
+
+    jscf = ngx_stream_get_module_srv_conf(s, ngx_stream_js_module);
+    if (jscf->filter.len == 0) {
+        return ngx_stream_next_filter(s, in, from_upstream);
+    }
+
+    c = s->connection;
+
+    ngx_log_debug1(NGX_LOG_DEBUG_STREAM, c->log, 0, "stream js filter u:%ui",
+                   from_upstream);
+
+    rc = ngx_stream_js_init_vm(s);
+
+    if (rc == NGX_ERROR) {
+        return NGX_ERROR;
+    }
+
+    if (rc == NGX_DECLINED) {
+        return ngx_stream_next_filter(s, in, from_upstream);
+    }
+
+    ctx = ngx_stream_get_module_ctx(s, ngx_stream_js_module);
+
+    ctx->filter = 1;
+
+    name.start = jscf->filter.data;
+    name.length = jscf->filter.len;
+
+    func = njs_vm_function(ctx->vm, &name);
+
+    if (func == NULL) {
+        ngx_log_error(NGX_LOG_ERR, c->log, 0, "js function \"%V\" not found",
+                      &jscf->filter);
+        return NGX_ERROR;
+    }
+
+    ctx->from_upstream = from_upstream;
+
+    ll = &out;
+
+    while (in) {
+        ctx->buf = in->buf;
+
+        if (njs_vm_call(ctx->vm, func, ctx->arg, 1) != NJS_OK) {
+            njs_vm_exception(ctx->vm, &exception);
+
+            ngx_log_error(NGX_LOG_ERR, c->log, 0, "js exception: %*s",
+                          exception.length, exception.start);
+
+            return NGX_ERROR;
+        }
+
+        if (njs_vm_retval(ctx->vm, &value) != NJS_OK) {
+            return NGX_ERROR;
+        }
+
+        ngx_log_debug2(NGX_LOG_DEBUG_STREAM, c->log, 0,
+                       "js return value: \"%*s\"",
+                       value.length, value.start);
+
+        if (value.length) {
+            rc = ngx_atoi(value.start, value.length);
+
+            if (rc != NGX_OK && rc != -NGX_ERROR) {
+                ngx_log_error(NGX_LOG_ERR, c->log, 0,
+                              "unexpected js return code: \"%*s\"",
+                              value.length, value.start);
+                return NGX_ERROR;
+            }
+
+            rc = -rc;
+
+            if (rc == NGX_ERROR) {
+                return NGX_ERROR;
+            }
+        }
+
+        cl = ngx_alloc_chain_link(c->pool);
+        if (cl == NULL) {
+            return NGX_ERROR;
+        }
+
+        cl->buf = ctx->buf;
+
+        *ll = cl;
+        ll = &cl->next;
+
+        in = in->next;
+    }
+
+    *ll = NULL;
+
+    rc = ngx_stream_next_filter(s, out, from_upstream);
+
+    ngx_chain_update_chains(c->pool, &ctx->free, &ctx->busy, &out,
+                            (ngx_buf_tag_t) &ngx_stream_js_module);
+
+    return rc;
+}
+
+
 static ngx_int_t
 ngx_stream_js_variable(ngx_stream_session_t *s, ngx_stream_variable_value_t *v,
     uintptr_t data)
@@ -364,6 +725,137 @@ ngx_stream_js_ext_get_remote_address(njs
 
 
 static njs_ret_t
+ngx_stream_js_ext_get_eof(njs_vm_t *vm, njs_value_t *value, void *obj,
+    uintptr_t data)
+{
+    ngx_buf_t             *b;
+    ngx_connection_t      *c;
+    ngx_stream_js_ctx_t   *ctx;
+    ngx_stream_session_t  *s;
+
+    s = (ngx_stream_session_t *) obj;
+    c = s->connection;
+    ctx = ngx_stream_get_module_ctx(s, ngx_stream_js_module);
+
+    b = ctx->filter ? ctx->buf : c->buffer;
+
+    *value = (b && b->last_buf ? njs_value_true : njs_value_false);
+
+    return NJS_OK;
+}
+
+
+static njs_ret_t
+ngx_stream_js_ext_get_from_upstream(njs_vm_t *vm, njs_value_t *value, void *obj,
+    uintptr_t data)
+{
+    ngx_stream_js_ctx_t   *ctx;
+    ngx_stream_session_t  *s;
+
+    s = (ngx_stream_session_t *) obj;
+    ctx = ngx_stream_get_module_ctx(s, ngx_stream_js_module);
+
+    *value = (ctx->from_upstream ? njs_value_true : njs_value_false);
+
+    return NJS_OK;
+}
+
+
+static njs_ret_t
+ngx_stream_js_ext_get_buffer(njs_vm_t *vm, njs_value_t *value, void *obj,
+    uintptr_t data)
+{
+    size_t                 len;
+    u_char                *p;
+    ngx_buf_t             *b;
+    ngx_connection_t      *c;
+    ngx_stream_js_ctx_t   *ctx;
+    ngx_stream_session_t  *s;
+
+    s = (ngx_stream_session_t *) obj;
+    c = s->connection;
+    ctx = ngx_stream_get_module_ctx(s, ngx_stream_js_module);
+
+    b = ctx->filter ? ctx->buf : c->buffer;
+
+    len = b ? b->last - b->pos : 0;
+
+    p = njs_string_alloc(vm, value, len, 0);
+    if (p == NULL) {
+        return NJS_ERROR;
+    }
+
+    if (len) {
+        ngx_memcpy(p, b->pos, len);
+    }
+
+    return NJS_OK;
+}
+
+
+static njs_ret_t
+ngx_stream_js_ext_set_buffer(njs_vm_t *vm, void *obj, uintptr_t data,
+    nxt_str_t *value)
+{
+    ngx_buf_t             *b;
+    ngx_chain_t           *cl;
+    ngx_connection_t      *c;
+    ngx_stream_js_ctx_t   *ctx;
+    ngx_stream_session_t  *s;
+
+    s = (ngx_stream_session_t *) obj;
+    c = s->connection;
+    ctx = ngx_stream_get_module_ctx(s, ngx_stream_js_module);
+
+    ngx_log_debug2(NGX_LOG_DEBUG_STREAM, c->log, 0,
+                   "stream js set buffer \"%*s\"", value->length, value->start);
+
+    if (!ctx->filter) {
+        ngx_log_error(NGX_LOG_WARN, c->log, 0,
+                      "cannot set buffer in this handler");
+        return NJS_OK;
+    }
+
+    cl = ngx_chain_get_free_buf(c->pool, &ctx->free);
+    if (cl == NULL) {
+        return NJS_ERROR;
+    }
+
+    b = cl->buf;
+
+    ngx_free_chain(c->pool, cl);
+
+    b->last_buf = ctx->buf->last_buf;
+    b->memory = (value->length ? 1 : 0);
+    b->sync = (value->length ? 0 : 1);
+    b->tag = (ngx_buf_tag_t) &ngx_stream_js_module;
+
+    b->start = value->start;
+    b->end = value->start + value->length;
+    b->pos = b->start;
+    b->last = b->end;
+
+    if (ctx->buf->tag != (ngx_buf_tag_t) &ngx_stream_js_module) {
+        ctx->buf->pos = ctx->buf->last;
+
+    } else {
+        cl = ngx_alloc_chain_link(c->pool);
+        if (cl == NULL) {
+            return NJS_ERROR;
+        }
+
+        cl->buf = ctx->buf;
+        cl->next = ctx->free;
+        ctx->free = cl;
+    }
+
+    ctx->buf = b;
+
+    return NJS_OK;
+}
+
+
+static njs_ret_t
 ngx_stream_js_ext_log(njs_vm_t *vm, njs_value_t *args, nxt_uint_t nargs,
     njs_index_t unused)
 {
@@ -375,8 +867,7 @@ ngx_stream_js_ext_log(njs_vm_t *vm, njs_
     s = njs_value_data(njs_argument(args, 0));
     c = s->connection;
 
-    if (njs_value_to_ext_string(vm, &msg, njs_argument(args, 1)) == NJS_ERROR)
-    {
+    if (njs_value_to_ext_string(vm, &msg, njs_argument(args, 1)) == NJS_ERROR) {
         return NJS_ERROR;
     }
 
@@ -418,6 +909,19 @@ ngx_stream_js_ext_get_variable(njs_vm_t 
 }
 
 
+static njs_ret_t
+ngx_stream_js_ext_get_code(njs_vm_t *vm, njs_value_t *value, void *obj,
+    uintptr_t data)
+{
+    ngx_memzero(value, sizeof(njs_value_t));
+
+    value->data.type = NJS_NUMBER;
+    value->data.u.number = data;
+
+    return NJS_OK;
+}
+
+
 static char *
 ngx_stream_js_include(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
 {
@@ -606,6 +1110,9 @@ ngx_stream_js_create_srv_conf(ngx_conf_t
      *
      *     conf->vm = NULL;
      *     conf->arg = NULL;
+     *     conf->access = { 0, NULL };
+     *     conf->preread = { 0, NULL };
+     *     conf->filter = { 0, NULL };
      */
 
     return conf;
@@ -623,5 +1130,38 @@ ngx_stream_js_merge_srv_conf(ngx_conf_t 
         conf->arg = prev->arg;
     }
 
+    ngx_conf_merge_str_value(conf->access, prev->access, "");
+    ngx_conf_merge_str_value(conf->preread, prev->preread, "");
+    ngx_conf_merge_str_value(conf->filter, prev->filter, "");
+
     return NGX_CONF_OK;
 }
+
+
+static ngx_int_t
+ngx_stream_js_init(ngx_conf_t *cf)
+{
+    ngx_stream_handler_pt        *h;
+    ngx_stream_core_main_conf_t  *cmcf;
+
+    ngx_stream_next_filter = ngx_stream_top_filter;
+    ngx_stream_top_filter = ngx_stream_js_body_filter;
+
+    cmcf = ngx_stream_conf_get_module_main_conf(cf, ngx_stream_core_module);
+
+    h = ngx_array_push(&cmcf->phases[NGX_STREAM_ACCESS_PHASE].handlers);
+    if (h == NULL) {
+        return NGX_ERROR;
+    }
+
+    *h = ngx_stream_js_access_handler;
+
+    h = ngx_array_push(&cmcf->phases[NGX_STREAM_PREREAD_PHASE].handlers);
+    if (h == NULL) {
+        return NGX_ERROR;
+    }
+
+    *h = ngx_stream_js_preread_handler;
+
+    return NGX_OK;
+}


From alessandro at cloudflare.com  Wed Sep 28 17:02:40 2016
From: alessandro at cloudflare.com (Alessandro Ghedini)
Date: Wed, 28 Sep 2016 18:02:40 +0100
Subject: [PATCH 0 of 1] Upstream: fix warning when building with BoringSSL
In-Reply-To: <20160928164013.GR73038@mdounin.ru>
References: <patchbomb.1475071846@mandy.local>
 <20160928141902.GN73038@mdounin.ru>
 <20160928143748.7mb5aiw5pqsnb56w@mandy.local>
 <20160928160000.GP73038@mdounin.ru>
 <20160928162848.c5nx2pfpbnx2n56h@mandy.local>
 <20160928164013.GR73038@mdounin.ru>
Message-ID: <20160928170240.zfaqlapyjuxdxyks@mandy.local>

On Wed, Sep 28, 2016 at 07:40:13PM +0300, Maxim Dounin wrote:
> Hello!
> 
> On Wed, Sep 28, 2016 at 05:28:48PM +0100, Alessandro Ghedini wrote:
> 
> > On Wed, Sep 28, 2016 at 07:00:00PM +0300, Maxim Dounin wrote:
> > > Hello!
> > > 
> > > On Wed, Sep 28, 2016 at 03:37:48PM +0100, Alessandro Ghedini wrote:
> > > 
> > > > On Wed, Sep 28, 2016 at 05:19:02PM +0300, Maxim Dounin wrote:
> > > > > Hello!
> > > > > 
> > > > > On Wed, Sep 28, 2016 at 03:10:46PM +0100, Alessandro Ghedini wrote:
> > > > > 
> > > > > > Hello,
> > > > > > 
> > > > > > I don't now what your current plans for supporting BoringSSL are, but its
> > > > > > API has been fairly stable for a while and this is the only change required
> > > > > > to make NGINX build with it again (the other issue with error definitions was
> > > > > > fixed in BoringSSL itself).
> > > > > > 
> > > > > > I don't think BoringSSL is going to change the API back, so NGINX migh want
> > > > > > to fix this if support for BoringSSL is desired (again, don't know your
> > > > > > opinion on this).
> > > > > > 
> > > > > > Please have a look and let me know what you think.
> > > > > 
> > > > > Quoting 
> > > > > http://mailman.nginx.org/pipermail/nginx-devel/2016-August/008680.html:
> > > > > 
> > > > > : Ok, this looks like the real reason for the patch.  This looks 
> > > > > : like an API change in BoringSSL, and should be threated 
> > > > > : accordingly.
> > > > > 
> > > > > : Given the number of various API changes BoringSSL introduces here 
> > > > > : and there - we probably don't want to follow, at least till some 
> > > > > : version is actually released.
> > > > 
> > > > Ok, thanks, I missed that. TBH I don't think the BoringSSL team intends to
> > > > release "proper" versions like OpenSSL does, so what you propose to wait for
> > > > might not actually ever happen.
> > > 
> > > Sure, and I'm fine with it.
> > > 
> > > > I understand your concern of wanting to target a fixed release, but as I
> > > > mentioned (and Piotr as well) BoringSSL's API seems to have been fairly stable
> > > > for a while (except for fixes like the one for the problem mentioned in the
> > > > patch you linked, which was worked around in BoringSSL itself), and AFAIK there
> > > > aren't other similar compatibility problems left except for this build warning
> > > > (but maybe Piotr could prove me wrong on that), so it might make sense to start
> > > > looking at supporting BoringSSL again.
> > > 
> > > Last time I looked into BoringSSL code due to ticket #993 several 
> > > months ago (https://trac.nginx.org/nginx/ticket/993), and my 
> > > impression wasn't that positive.
> > 
> > Ah, BoringSSL actually supports the new SSL_CTX_set1_curves_list() API that
> > NGINX already uses, but it doesn't seem to define SSL_CTRL_SET_CURVES_LIST
> > (yet) so the other API is picked. I'll make a patch for BoringSSL to fix this.
> 
> I'm afraid you are wrong here,
> 
> $ grep -r SSL_CTX_set1_curves_list boringssl/ | wc -l
>        0
> 
> Instead, BoringSSL introduced its own API to work with curves.

Ah, you are right, I was actually thinking of SSL_CTX_set1_curves() which does
seem to be defined by both OpenSSL and BoringSSL. I'll see if the BoringSSL
people are interested in having SSL_CTX_set1_curves_list() as well.

Cheers


From mdounin at mdounin.ru  Thu Sep 29 06:02:16 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 29 Sep 2016 09:02:16 +0300
Subject: [PATCH] Core: fixed uninitialized memory access
In-Reply-To: <20160927212259.GT19300@manetheren.hodor.cz>
References: <20160927212259.GT19300@manetheren.hodor.cz>
Message-ID: <20160929060215.GV73038@mdounin.ru>

Hello!

On Tue, Sep 27, 2016 at 11:23:00PM +0200, Jan Seda wrote:

> Struct sockaddr was allocated without zeroing. As getsockname() does not
> necessarily fill all the bytes, this left potentially uninitialized
> memory, which caused me a weird failure in the on-the-fly upgrade
> mechanism, where the new master failed to pair the config file AF_UNIX
> sockets with the inherited ones. ngx_cmp_sockaddr() would return
> NGX_DECLINED because the inherited one would contain uninitialized
> garbage after the actual name. This in turn would cause nginx to try to
> bind() the unix socket again, and obviously fail, because the listening
> socket was bound already (and inherited).
> 
> Easy to reproduce on debian wheezy, debian jessie and CentOS 7 systems
> (all x86_64) with statically linked openssl 1.1.0. The attached patch
> seems to fix the problem.
> 
> Also harmonized socklen calculation for AF_UNIX sockets to the way
> ngx_parse_unix_domain_url() does it (sizeof(struct sockaddr_un)), so
> ngx_cmp_sockaddr() is now less dangerous.

Thanks for catching this.

Dropping the length returned by getsockname() doesn't look like 
a correct solution though.  Instead, teaching ngx_cmp_sockaddr() to 
compare sockaddrs with not-completely-filled sun_path - that is, 
respecting socklen - should be the right way to go.

Please try the following patch:

diff --git a/src/core/ngx_inet.c b/src/core/ngx_inet.c
--- a/src/core/ngx_inet.c
+++ b/src/core/ngx_inet.c
@@ -1364,6 +1364,7 @@ ngx_cmp_sockaddr(struct sockaddr *sa1, s
     struct sockaddr_in6  *sin61, *sin62;
 #endif
 #if (NGX_HAVE_UNIX_DOMAIN)
+    size_t                len;
     struct sockaddr_un   *saun1, *saun2;
 #endif
 
@@ -1393,15 +1394,45 @@ ngx_cmp_sockaddr(struct sockaddr *sa1, s
 #if (NGX_HAVE_UNIX_DOMAIN)
     case AF_UNIX:
 
-        /* TODO length */
-
         saun1 = (struct sockaddr_un *) sa1;
         saun2 = (struct sockaddr_un *) sa2;
 
-        if (ngx_memcmp(&saun1->sun_path, &saun2->sun_path,
-                       sizeof(saun1->sun_path))
-            != 0)
-        {
+        if (slen1 <= (socklen_t) offsetof(struct sockaddr_un, sun_path)) {
+            return NGX_DECLINED;
+        }
+
+        if (slen2 <= (socklen_t) offsetof(struct sockaddr_un, sun_path)) {
+            return NGX_DECLINED;
+        }
+
+        if (saun1->sun_path[0] == '\0' || saun2->sun_path[0] == '\0') {
+
+            /* an abstract socket address */
+
+            if (ngx_memn2cmp((u_char *) saun1->sun_path,
+                             (u_char *) saun2->sun_path,
+                             slen1 - offsetof(struct sockaddr_un, sun_path),
+                             slen2 - offsetof(struct sockaddr_un, sun_path))
+                != 0)
+            {
+                return NGX_DECLINED;
+            }
+
+            break;
+        }
+
+        if (slen1 < slen2) {
+            len = slen1 - offsetof(struct sockaddr_un, sun_path);
+
+        } else {
+            len = slen2 - offsetof(struct sockaddr_un, sun_path);
+        }
+
+        if (len > sizeof(saun1->sun_path)) {
+            len = sizeof(saun1->sun_path);
+        }
+
+        if (ngx_strncmp(saun1->sun_path, saun2->sun_path, len) != 0) {
             return NGX_DECLINED;
         }
 

-- 
Maxim Dounin
http://nginx.org/


From ru at nginx.com  Thu Sep 29 10:05:12 2016
From: ru at nginx.com (Ruslan Ermilov)
Date: Thu, 29 Sep 2016 10:05:12 +0000
Subject: [nginx] Stream: style.
Message-ID: <hg.829468da49d6.1475143512.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/829468da49d6
branches:  
changeset: 6706:829468da49d6
user:      Ruslan Ermilov <ru at nginx.com>
date:      Thu Sep 29 12:59:13 2016 +0300
description:
Stream: style.

Explicitly initialized peer's max_conns for upstreams created with
variables similar to how it's done in http.

diffstat:

 src/stream/ngx_stream_upstream_round_robin.c |  2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diffs (19 lines):

diff -r 29bf0dbc0a77 -r 829468da49d6 src/stream/ngx_stream_upstream_round_robin.c
--- a/src/stream/ngx_stream_upstream_round_robin.c	Thu Sep 22 19:32:47 2016 +0300
+++ b/src/stream/ngx_stream_upstream_round_robin.c	Thu Sep 29 12:59:13 2016 +0300
@@ -347,6 +347,7 @@ ngx_stream_upstream_create_round_robin_p
         peer[0].weight = 1;
         peer[0].effective_weight = 1;
         peer[0].current_weight = 0;
+        peer[0].max_conns = 0;
         peer[0].max_fails = 1;
         peer[0].fail_timeout = 10;
         peers->peer = peer;
@@ -380,6 +381,7 @@ ngx_stream_upstream_create_round_robin_p
             peer[i].weight = 1;
             peer[i].effective_weight = 1;
             peer[i].current_weight = 0;
+            peer[i].max_conns = 0;
             peer[i].max_fails = 1;
             peer[i].fail_timeout = 10;
             *peerp = &peer[i];


From mdounin at mdounin.ru  Thu Sep 29 17:27:09 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 29 Sep 2016 17:27:09 +0000
Subject: [nginx] Modules compatibility: peer.notify.
Message-ID: <hg.1c9863f9592e.1475170029.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/1c9863f9592e
branches:  
changeset: 6707:1c9863f9592e
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Sep 29 18:05:59 2016 +0300
description:
Modules compatibility: peer.notify.

This callback can be used to notify balancer about various events.
For now, it is only used in nginx-plus.

diffstat:

 src/event/ngx_event_connect.h |  3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diffs (20 lines):

diff --git a/src/event/ngx_event_connect.h b/src/event/ngx_event_connect.h
--- a/src/event/ngx_event_connect.h
+++ b/src/event/ngx_event_connect.h
@@ -25,6 +25,8 @@ typedef ngx_int_t (*ngx_event_get_peer_p
     void *data);
 typedef void (*ngx_event_free_peer_pt)(ngx_peer_connection_t *pc, void *data,
     ngx_uint_t state);
+typedef void (*ngx_event_notify_peer_pt)(ngx_peer_connection_t *pc,
+    void *data, ngx_uint_t type);
 #if (NGX_SSL)
 
 typedef ngx_int_t (*ngx_event_set_peer_session_pt)(ngx_peer_connection_t *pc,
@@ -46,6 +48,7 @@ struct ngx_peer_connection_s {
 
     ngx_event_get_peer_pt            get;
     ngx_event_free_peer_pt           free;
+    ngx_event_notify_peer_pt         notify;
     void                            *data;
 
 #if (NGX_SSL)


From mdounin at mdounin.ru  Thu Sep 29 17:27:12 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 29 Sep 2016 17:27:12 +0000
Subject: [nginx] Modules compatibility: slow start fields.
Message-ID: <hg.4080f94a996f.1475170032.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/4080f94a996f
branches:  
changeset: 6708:4080f94a996f
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Sep 29 18:06:00 2016 +0300
description:
Modules compatibility: slow start fields.

diffstat:

 src/http/ngx_http_upstream.h                 |  1 +
 src/http/ngx_http_upstream_round_robin.h     |  2 ++
 src/stream/ngx_stream_upstream.h             |  1 +
 src/stream/ngx_stream_upstream_round_robin.h |  2 ++
 4 files changed, 6 insertions(+), 0 deletions(-)

diffs (46 lines):

diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h
+++ b/src/http/ngx_http_upstream.h
@@ -98,6 +98,7 @@ typedef struct {
     ngx_uint_t                       max_conns;
     ngx_uint_t                       max_fails;
     time_t                           fail_timeout;
+    ngx_msec_t                       slow_start;
 
     unsigned                         down:1;
     unsigned                         backup:1;
diff --git a/src/http/ngx_http_upstream_round_robin.h b/src/http/ngx_http_upstream_round_robin.h
--- a/src/http/ngx_http_upstream_round_robin.h
+++ b/src/http/ngx_http_upstream_round_robin.h
@@ -35,6 +35,8 @@ struct ngx_http_upstream_rr_peer_s {
 
     ngx_uint_t                      max_fails;
     time_t                          fail_timeout;
+    ngx_msec_t                      slow_start;
+    ngx_msec_t                      start_time;
 
     ngx_uint_t                      down;          /* unsigned  down:1; */
 
diff --git a/src/stream/ngx_stream_upstream.h b/src/stream/ngx_stream_upstream.h
--- a/src/stream/ngx_stream_upstream.h
+++ b/src/stream/ngx_stream_upstream.h
@@ -54,6 +54,7 @@ typedef struct {
     ngx_uint_t                         max_conns;
     ngx_uint_t                         max_fails;
     time_t                             fail_timeout;
+    ngx_msec_t                         slow_start;
 
     unsigned                           down:1;
     unsigned                           backup:1;
diff --git a/src/stream/ngx_stream_upstream_round_robin.h b/src/stream/ngx_stream_upstream_round_robin.h
--- a/src/stream/ngx_stream_upstream_round_robin.h
+++ b/src/stream/ngx_stream_upstream_round_robin.h
@@ -35,6 +35,8 @@ struct ngx_stream_upstream_rr_peer_s {
 
     ngx_uint_t                       max_fails;
     time_t                           fail_timeout;
+    ngx_msec_t                       slow_start;
+    ngx_msec_t                       start_time;
 
     ngx_uint_t                       down;         /* unsigned  down:1; */
 


From mdounin at mdounin.ru  Thu Sep 29 17:27:15 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 29 Sep 2016 17:27:15 +0000
Subject: [nginx] Modules compatibility: cache purge fields.
Message-ID: <hg.e08e741f74cd.1475170035.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/e08e741f74cd
branches:  
changeset: 6709:e08e741f74cd
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Sep 29 18:06:01 2016 +0300
description:
Modules compatibility: cache purge fields.

diffstat:

 src/core/ngx_file.h          |  2 ++
 src/http/ngx_http_cache.h    |  5 ++++-
 src/http/ngx_http_upstream.h |  1 +
 3 files changed, 7 insertions(+), 1 deletions(-)

diffs (59 lines):

diff --git a/src/core/ngx_file.h b/src/core/ngx_file.h
--- a/src/core/ngx_file.h
+++ b/src/core/ngx_file.h
@@ -43,6 +43,7 @@ struct ngx_file_s {
 
 
 typedef time_t (*ngx_path_manager_pt) (void *data);
+typedef ngx_msec_t (*ngx_path_purger_pt) (void *data);
 typedef void (*ngx_path_loader_pt) (void *data);
 
 
@@ -52,6 +53,7 @@ typedef struct {
     size_t                     level[NGX_MAX_PATH_LEVEL];
 
     ngx_path_manager_pt        manager;
+    ngx_path_purger_pt         purger;
     ngx_path_loader_pt         loader;
     void                      *data;
 
diff --git a/src/http/ngx_http_cache.h b/src/http/ngx_http_cache.h
--- a/src/http/ngx_http_cache.h
+++ b/src/http/ngx_http_cache.h
@@ -50,7 +50,8 @@ typedef struct {
     unsigned                         exists:1;
     unsigned                         updating:1;
     unsigned                         deleting:1;
-                                     /* 11 unused bits */
+    unsigned                         purged:1;
+                                     /* 10 unused bits */
 
     ngx_file_uniq_t                  uniq;
     time_t                           expire;
@@ -85,6 +86,7 @@ struct ngx_http_cache_s {
     ngx_uint_t                       min_uses;
     ngx_uint_t                       error;
     ngx_uint_t                       valid_msec;
+    ngx_uint_t                       vary_tag;
 
     ngx_buf_t                       *buf;
 
@@ -109,6 +111,7 @@ struct ngx_http_cache_s {
     unsigned                         updating:1;
     unsigned                         exists:1;
     unsigned                         temp_file:1;
+    unsigned                         purged:1;
     unsigned                         reading:1;
     unsigned                         secondary:1;
 };
diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h
+++ b/src/http/ngx_http_upstream.h
@@ -205,6 +205,7 @@ typedef struct {
 
     ngx_array_t                     *cache_valid;
     ngx_array_t                     *cache_bypass;
+    ngx_array_t                     *cache_purge;
     ngx_array_t                     *no_cache;
 #endif
 


From mdounin at mdounin.ru  Thu Sep 29 17:27:17 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 29 Sep 2016 17:27:17 +0000
Subject: [nginx] Modules compatibility: health check fields.
Message-ID: <hg.3ab8736958cb.1475170037.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/3ab8736958cb
branches:  
changeset: 6710:3ab8736958cb
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Sep 29 18:06:02 2016 +0300
description:
Modules compatibility: health check fields.

diffstat:

 src/http/ngx_http_request.h |  2 ++
 src/stream/ngx_stream.h     |  4 +++-
 2 files changed, 5 insertions(+), 1 deletions(-)

diffs (27 lines):

diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -542,6 +542,8 @@ struct ngx_http_request_s {
     unsigned                          stat_reading:1;
     unsigned                          stat_writing:1;
 
+    unsigned                          health_check:1;
+
     /* used to parse HTTP headers */
 
     ngx_uint_t                        state;
diff --git a/src/stream/ngx_stream.h b/src/stream/ngx_stream.h
--- a/src/stream/ngx_stream.h
+++ b/src/stream/ngx_stream.h
@@ -226,8 +226,10 @@ struct ngx_stream_session_s {
     ngx_uint_t                     status;
 
 #if (NGX_STREAM_SSL)
-    ngx_uint_t                     ssl;  /* unsigned  ssl:1; */
+    unsigned                       ssl:1;
 #endif
+
+    unsigned                       health_check:1;
 };
 
 


From mdounin at mdounin.ru  Thu Sep 29 17:27:20 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 29 Sep 2016 17:27:20 +0000
Subject: [nginx] Modules compatibility: status fields.
Message-ID: <hg.90a03b1dc4db.1475170040.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/90a03b1dc4db
branches:  
changeset: 6711:90a03b1dc4db
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Sep 29 18:06:03 2016 +0300
description:
Modules compatibility: status fields.

diffstat:

 src/http/ngx_http_request.h |  1 +
 src/stream/ngx_stream.h     |  2 ++
 2 files changed, 3 insertions(+), 0 deletions(-)

diffs (23 lines):

diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h
--- a/src/http/ngx_http_request.h
+++ b/src/http/ngx_http_request.h
@@ -541,6 +541,7 @@ struct ngx_http_request_s {
     unsigned                          disable_not_modified:1;
     unsigned                          stat_reading:1;
     unsigned                          stat_writing:1;
+    unsigned                          stat_processing:1;
 
     unsigned                          health_check:1;
 
diff --git a/src/stream/ngx_stream.h b/src/stream/ngx_stream.h
--- a/src/stream/ngx_stream.h
+++ b/src/stream/ngx_stream.h
@@ -229,6 +229,8 @@ struct ngx_stream_session_s {
     unsigned                       ssl:1;
 #endif
 
+    unsigned                       stat_processing:1;
+
     unsigned                       health_check:1;
 };
 


From mdounin at mdounin.ru  Thu Sep 29 17:27:24 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 29 Sep 2016 17:27:24 +0000
Subject: [nginx] Modules compatibility: down flag promoted to a bitmask.
Message-ID: <hg.63b8b157b776.1475170044.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/63b8b157b776
branches:  
changeset: 6713:63b8b157b776
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Sep 29 18:06:05 2016 +0300
description:
Modules compatibility: down flag promoted to a bitmask.

It is to be used as a bitmask with various bits set/reset when appropriate.
Any bit set means that the peer should not be used, that is, exactly what
current checks do, no additional changes required.

diffstat:

 src/http/ngx_http_upstream_round_robin.h     |  2 +-
 src/stream/ngx_stream_upstream_round_robin.h |  2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diffs (24 lines):

diff --git a/src/http/ngx_http_upstream_round_robin.h b/src/http/ngx_http_upstream_round_robin.h
--- a/src/http/ngx_http_upstream_round_robin.h
+++ b/src/http/ngx_http_upstream_round_robin.h
@@ -38,7 +38,7 @@ struct ngx_http_upstream_rr_peer_s {
     ngx_msec_t                      slow_start;
     ngx_msec_t                      start_time;
 
-    ngx_uint_t                      down;          /* unsigned  down:1; */
+    ngx_uint_t                      down;
 
 #if (NGX_HTTP_SSL)
     void                           *ssl_session;
diff --git a/src/stream/ngx_stream_upstream_round_robin.h b/src/stream/ngx_stream_upstream_round_robin.h
--- a/src/stream/ngx_stream_upstream_round_robin.h
+++ b/src/stream/ngx_stream_upstream_round_robin.h
@@ -38,7 +38,7 @@ struct ngx_stream_upstream_rr_peer_s {
     ngx_msec_t                       slow_start;
     ngx_msec_t                       start_time;
 
-    ngx_uint_t                       down;         /* unsigned  down:1; */
+    ngx_uint_t                       down;
 
 #if (NGX_STREAM_SSL)
     void                            *ssl_session;


From mdounin at mdounin.ru  Thu Sep 29 17:27:22 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 29 Sep 2016 17:27:22 +0000
Subject: [nginx] Modules compatibility: upstream config field.
Message-ID: <hg.fd5c2781460b.1475170042.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/fd5c2781460b
branches:  
changeset: 6712:fd5c2781460b
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Thu Sep 29 18:06:04 2016 +0300
description:
Modules compatibility: upstream config field.

It is to be used to track version of an upstream configuration used for
request processing.

diffstat:

 src/http/ngx_http_upstream_round_robin.c     |  2 ++
 src/http/ngx_http_upstream_round_robin.h     |  1 +
 src/stream/ngx_stream_upstream_round_robin.c |  2 ++
 src/stream/ngx_stream_upstream_round_robin.h |  1 +
 4 files changed, 6 insertions(+), 0 deletions(-)

diffs (60 lines):

diff --git a/src/http/ngx_http_upstream_round_robin.c b/src/http/ngx_http_upstream_round_robin.c
--- a/src/http/ngx_http_upstream_round_robin.c
+++ b/src/http/ngx_http_upstream_round_robin.c
@@ -260,6 +260,7 @@ ngx_http_upstream_init_round_robin_peer(
 
     rrp->peers = us->peer.data;
     rrp->current = NULL;
+    rrp->config = 0;
 
     n = rrp->peers->number;
 
@@ -384,6 +385,7 @@ ngx_http_upstream_create_round_robin_pee
 
     rrp->peers = peers;
     rrp->current = NULL;
+    rrp->config = 0;
 
     if (rrp->peers->number <= 8 * sizeof(uintptr_t)) {
         rrp->tried = &rrp->data;
diff --git a/src/http/ngx_http_upstream_round_robin.h b/src/http/ngx_http_upstream_round_robin.h
--- a/src/http/ngx_http_upstream_round_robin.h
+++ b/src/http/ngx_http_upstream_round_robin.h
@@ -122,6 +122,7 @@ struct ngx_http_upstream_rr_peers_s {
 
 
 typedef struct {
+    ngx_uint_t                      config;
     ngx_http_upstream_rr_peers_t   *peers;
     ngx_http_upstream_rr_peer_t    *current;
     uintptr_t                      *tried;
diff --git a/src/stream/ngx_stream_upstream_round_robin.c b/src/stream/ngx_stream_upstream_round_robin.c
--- a/src/stream/ngx_stream_upstream_round_robin.c
+++ b/src/stream/ngx_stream_upstream_round_robin.c
@@ -265,6 +265,7 @@ ngx_stream_upstream_init_round_robin_pee
 
     rrp->peers = us->peer.data;
     rrp->current = NULL;
+    rrp->config = 0;
 
     n = rrp->peers->number;
 
@@ -391,6 +392,7 @@ ngx_stream_upstream_create_round_robin_p
 
     rrp->peers = peers;
     rrp->current = NULL;
+    rrp->config = 0;
 
     if (rrp->peers->number <= 8 * sizeof(uintptr_t)) {
         rrp->tried = &rrp->data;
diff --git a/src/stream/ngx_stream_upstream_round_robin.h b/src/stream/ngx_stream_upstream_round_robin.h
--- a/src/stream/ngx_stream_upstream_round_robin.h
+++ b/src/stream/ngx_stream_upstream_round_robin.h
@@ -122,6 +122,7 @@ struct ngx_stream_upstream_rr_peers_s {
 
 
 typedef struct {
+    ngx_uint_t                       config;
     ngx_stream_upstream_rr_peers_t  *peers;
     ngx_stream_upstream_rr_peer_t   *current;
     uintptr_t                       *tried;


From nginx-devel at hodor.cz  Thu Sep 29 22:09:12 2016
From: nginx-devel at hodor.cz (Jan Seda)
Date: Fri, 30 Sep 2016 00:09:12 +0200
Subject: [PATCH] Core: fixed uninitialized memory access
In-Reply-To: <20160929060215.GV73038@mdounin.ru>
References: <20160927212259.GT19300@manetheren.hodor.cz>
 <20160929060215.GV73038@mdounin.ru>
Message-ID: <20160929220911.GU19300@manetheren.hodor.cz>

Hello.

On 2016-09-29, 09:02:16, Maxim Dounin wrote:
> Thanks for catching this.
> 
> Dropping the length returned by getsockname() doesn't look like 
> a correct solution though.  Instead, teaching ngx_cmp_sockaddr() to 
> compare sockaddrs with not-completely-filled sun_path - that is, 
> respecting socklen - should be the right way to go.
> 
> Please try the following patch:

Seems to work OK. Cannot reproduce the problem anymore. Thanks.

BTW, wouldn't s/ngx_palloc/ngx_pcalloc/ in ngx_set_inherited_sockets()
be prudent anyway?

Also, I see your patch is prepared for abstract namespace sockets. Is
this feature planned soon? I cobbled up a patch for that (attached) and
such sockets now interop with haproxy (abns@ scheme) and socat (with
unix-tightsocklen=0). But it probably is not production-ready.

-- 
Jan Seda
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nginx_abns.diff
Type: text/x-diff
Size: 3691 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160930/cc7faa25/attachment.bin>

From alessandro at cloudflare.com  Fri Sep 30 11:37:00 2016
From: alessandro at cloudflare.com (Alessandro Ghedini)
Date: Fri, 30 Sep 2016 12:37:00 +0100
Subject: [PATCH 0 of 1] Upstream: fix warning when building with BoringSSL
In-Reply-To: <20160928164013.GR73038@mdounin.ru>
References: <patchbomb.1475071846@mandy.local>
 <20160928141902.GN73038@mdounin.ru>
 <20160928143748.7mb5aiw5pqsnb56w@mandy.local>
 <20160928160000.GP73038@mdounin.ru>
 <20160928162848.c5nx2pfpbnx2n56h@mandy.local>
 <20160928164013.GR73038@mdounin.ru>
Message-ID: <20160930113700.bwdbbw237verishu@mandy.local>

Hey Maxim,

On Wed, Sep 28, 2016 at 07:40:13PM +0300, Maxim Dounin wrote:
> Hello!
> 
> On Wed, Sep 28, 2016 at 05:28:48PM +0100, Alessandro Ghedini wrote:
> 
> > On Wed, Sep 28, 2016 at 07:00:00PM +0300, Maxim Dounin wrote:
> > > Hello!
> > > 
> > > On Wed, Sep 28, 2016 at 03:37:48PM +0100, Alessandro Ghedini wrote:
> > > 
> > > > On Wed, Sep 28, 2016 at 05:19:02PM +0300, Maxim Dounin wrote:
> > > > > Hello!
> > > > > 
> > > > > On Wed, Sep 28, 2016 at 03:10:46PM +0100, Alessandro Ghedini wrote:
> > > > > 
> > > > > > Hello,
> > > > > > 
> > > > > > I don't now what your current plans for supporting BoringSSL are, but its
> > > > > > API has been fairly stable for a while and this is the only change required
> > > > > > to make NGINX build with it again (the other issue with error definitions was
> > > > > > fixed in BoringSSL itself).
> > > > > > 
> > > > > > I don't think BoringSSL is going to change the API back, so NGINX migh want
> > > > > > to fix this if support for BoringSSL is desired (again, don't know your
> > > > > > opinion on this).
> > > > > > 
> > > > > > Please have a look and let me know what you think.
> > > > > 
> > > > > Quoting 
> > > > > http://mailman.nginx.org/pipermail/nginx-devel/2016-August/008680.html:
> > > > > 
> > > > > : Ok, this looks like the real reason for the patch.  This looks 
> > > > > : like an API change in BoringSSL, and should be threated 
> > > > > : accordingly.
> > > > > 
> > > > > : Given the number of various API changes BoringSSL introduces here 
> > > > > : and there - we probably don't want to follow, at least till some 
> > > > > : version is actually released.
> > > > 
> > > > Ok, thanks, I missed that. TBH I don't think the BoringSSL team intends to
> > > > release "proper" versions like OpenSSL does, so what you propose to wait for
> > > > might not actually ever happen.
> > > 
> > > Sure, and I'm fine with it.
> > > 
> > > > I understand your concern of wanting to target a fixed release, but as I
> > > > mentioned (and Piotr as well) BoringSSL's API seems to have been fairly stable
> > > > for a while (except for fixes like the one for the problem mentioned in the
> > > > patch you linked, which was worked around in BoringSSL itself), and AFAIK there
> > > > aren't other similar compatibility problems left except for this build warning
> > > > (but maybe Piotr could prove me wrong on that), so it might make sense to start
> > > > looking at supporting BoringSSL again.
> > > 
> > > Last time I looked into BoringSSL code due to ticket #993 several 
> > > months ago (https://trac.nginx.org/nginx/ticket/993), and my 
> > > impression wasn't that positive.
> > 
> > Ah, BoringSSL actually supports the new SSL_CTX_set1_curves_list() API that
> > NGINX already uses, but it doesn't seem to define SSL_CTRL_SET_CURVES_LIST
> > (yet) so the other API is picked. I'll make a patch for BoringSSL to fix this.
> 
> I'm afraid you are wrong here,
> 
> $ grep -r SSL_CTX_set1_curves_list boringssl/ | wc -l
>        0
> 
> Instead, BoringSSL introduced its own API to work with curves.

BoringSSL now provides SSL_CTX_set1_curves_list() as well:
https://github.com/google/boringssl/commit/5fd1807d95f75895ae99e336ac21b422f3cc6bd3

Both me and Piotr verified that it works with NGINX.

David Benjamin also suggested to modify my NGINX patch to always cast to
(const char *) the argument of SSL_set_tlsext_host_name(), even when using
OpenSSL since it uses macros and force-cast the argument back to (char *):
https://github.com/openssl/openssl/blob/master/include/openssl/ssl.h#L1231

This way there would be no need for the #ifdef, which makes the whole thing
nicer. I'll send an updated patch in a bit so you can pick the one you like
the most if you decide to add BoringSSL compatibility.

Cheers


From alessandro at cloudflare.com  Fri Sep 30 11:31:06 2016
From: alessandro at cloudflare.com (Alessandro Ghedini)
Date: Fri, 30 Sep 2016 12:31:06 +0100
Subject: [PATCH] Upstream: fix warning when building with BoringSSL
In-Reply-To: <8c3706ffdc9bde38ef50.1475071847@mandy.local>
References: <8c3706ffdc9bde38ef50.1475071847@mandy.local>
Message-ID: <fe7d9e3987d40f16d86f.1475235066@mandy.local>

# HG changeset patch
# User Alessandro Ghedini <alessandro at cloudflare.com>
# Date 1475070884 -3600
#      Wed Sep 28 14:54:44 2016 +0100
# Node ID fe7d9e3987d40f16d86fd01d94ad16ff58467af2
# Parent  29bf0dbc0a77914bc94bd001a2b17d364e8e50d9
Upstream: fix warning when building with BoringSSL

BoringSSL takes a const u_char * for SSL_set_tlsext_host_name but
OpenSSL only takes a u_char *. Since NGINX is built with -Werror by
default this breaks the build.

diff -r 29bf0dbc0a77 -r fe7d9e3987d4 src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Thu Sep 22 19:32:47 2016 +0300
+++ b/src/http/ngx_http_upstream.c	Wed Sep 28 14:54:44 2016 +0100
@@ -1696,7 +1696,8 @@
     ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,
                    "upstream SSL server name: \"%s\"", name.data);
 
-    if (SSL_set_tlsext_host_name(c->ssl->connection, name.data) == 0) {
+    if (SSL_set_tlsext_host_name(c->ssl->connection, 
+                                 (const char *) name.data) == 0) {
         ngx_ssl_error(NGX_LOG_ERR, r->connection->log, 0,
                       "SSL_set_tlsext_host_name(\"%s\") failed", name.data);
         return NGX_ERROR;


From piotrsikora at google.com  Fri Sep 30 12:50:27 2016
From: piotrsikora at google.com (Piotr Sikora)
Date: Fri, 30 Sep 2016 05:50:27 -0700
Subject: [PATCH] Upstream: fix warning when building with BoringSSL
In-Reply-To: <fe7d9e3987d40f16d86f.1475235066@mandy.local>
References: <8c3706ffdc9bde38ef50.1475071847@mandy.local>
 <fe7d9e3987d40f16d86f.1475235066@mandy.local>
Message-ID: <CAF-CG+LEohHweqf_4Sq6=HFHojSfDOejbVNhKA_Z1C-6yLJiow@mail.gmail.com>

Hey Alessandro,

> # HG changeset patch
> # User Alessandro Ghedini <alessandro at cloudflare.com>
> # Date 1475070884 -3600
> #      Wed Sep 28 14:54:44 2016 +0100
> # Node ID fe7d9e3987d40f16d86fd01d94ad16ff58467af2
> # Parent  29bf0dbc0a77914bc94bd001a2b17d364e8e50d9
> Upstream: fix warning when building with BoringSSL
>
> BoringSSL takes a const u_char * for SSL_set_tlsext_host_name but
> OpenSSL only takes a u_char *. Since NGINX is built with -Werror by
> default this breaks the build.

You need to apply the same fix to ngx_stream_proxy_module.c.

btw: I've sent exactly the same patch in the past, so good luck:
http://mailman.nginx.org/pipermail/nginx-devel/2015-November/007499.html

Best regards,
Piotr Sikora