From hongzhidao at gmail.com  Sat Jan  2 14:17:30 2016
From: hongzhidao at gmail.com (=?UTF-8?B?5rSq5b+X6YGT?=)
Date: Sat, 2 Jan 2016 22:17:30 +0800
Subject: njs isuse -- ext->set is null
Message-ID: <CAJuLd0gMWnV_GWkhr1RU8S+ikV793ZqYC5KCC2mC3xx+a7dSrA@mail.gmail.com>

js_set $bar "
  $r.error = "this will cause coredump file since ext->set is null";
";
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160102/dfe91af6/attachment.html>

From igor at sysoev.ru  Sat Jan  2 17:34:30 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Sat, 2 Jan 2016 20:34:30 +0300
Subject: njs isuse -- ext->set is null
In-Reply-To: <CAJuLd0gMWnV_GWkhr1RU8S+ikV793ZqYC5KCC2mC3xx+a7dSrA@mail.gmail.com>
References: <CAJuLd0gMWnV_GWkhr1RU8S+ikV793ZqYC5KCC2mC3xx+a7dSrA@mail.gmail.com>
Message-ID: <F78C95D1-FB6B-49EF-BD78-9AD3A67BC356@sysoev.ru>

On 02 Jan 2016, at 17:17, ??? <hongzhidao at gmail.com> wrote:

> js_set $bar "
>   $r.error = "this will cause coredump file since ext->set is null";
> ?;

Thank you, I will this after new year vacation.


-- 
Igor Sysoev
http://nginx.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160102/1f1433eb/attachment.html>

From agentzh at gmail.com  Sun Jan  3 04:06:28 2016
From: agentzh at gmail.com (Yichun Zhang (agentzh))
Date: Sat, 2 Jan 2016 20:06:28 -0800
Subject: [PATCH] SSL: handled SSL_CTX_set_cert_cb() callback yielding.
Message-ID: <CAB4Tn6NGax5KBGm=V+L3=--CLgxE98x-YksK-YQBRWWXefp02Q@mail.gmail.com>

# HG changeset patch
# User Yichun Zhang <agentzh at openresty.org>
# Date 1451762084 28800
#      Sat Jan 02 11:14:44 2016 -0800
# Node ID 449f0461859c16e95bdb18e8be6b94401545d3dd
# Parent  78b4e10b4367b31367aad3c83c9c3acdd42397c4
SSL: handled SSL_CTX_set_cert_cb() callback yielding.

OpenSSL 1.0.2+ introduces SSL_CTX_set_cert_cb() to allow custom
callbacks to serve the SSL certificiates and private keys dynamically
and lazily. The callbacks may yield for nonblocking I/O or sleeping.
Here we added support for such usage in NGINX 3rd-party modules
(like ngx_lua) in NGINX's event handlers for downstream SSL
connections.

diff -r 78b4e10b4367 -r 449f0461859c src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c Thu Dec 17 16:39:15 2015 +0300
+++ b/src/event/ngx_event_openssl.c Sat Jan 02 11:14:44 2016 -0800
@@ -1210,6 +1210,23 @@
         return NGX_AGAIN;
     }

+#if OPENSSL_VERSION_NUMBER >= 0x10002000L
+    if (sslerr == SSL_ERROR_WANT_X509_LOOKUP) {
+        c->read->handler = ngx_ssl_handshake_handler;
+        c->write->handler = ngx_ssl_handshake_handler;
+
+        if (ngx_handle_read_event(c->read, 0) != NGX_OK) {
+            return NGX_ERROR;
+        }
+
+        if (ngx_handle_write_event(c->write, 0) != NGX_OK) {
+            return NGX_ERROR;
+        }
+
+        return NGX_AGAIN;
+    }
+#endif
+
     err = (sslerr == SSL_ERROR_SYSCALL) ? ngx_errno : 0;

     c->ssl->no_wait_shutdown = 1;
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ssl_cert_cb_yield.patch
Type: application/octet-stream
Size: 1439 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160102/c88c28e8/attachment.obj>

From hongzhidao at gmail.com  Sun Jan  3 05:18:11 2016
From: hongzhidao at gmail.com (=?UTF-8?B?5rSq5b+X6YGT?=)
Date: Sun, 3 Jan 2016 13:18:11 +0800
Subject: njs isuse -- ext->set is null
In-Reply-To: <F78C95D1-FB6B-49EF-BD78-9AD3A67BC356@sysoev.ru>
References: <CAJuLd0gMWnV_GWkhr1RU8S+ikV793ZqYC5KCC2mC3xx+a7dSrA@mail.gmail.com>
 <F78C95D1-FB6B-49EF-BD78-9AD3A67BC356@sysoev.ru>
Message-ID: <CAJuLd0jqKfTz1uJTCKWNQJ99-LenE5RFXeK31GENDA9kwYsZDA@mail.gmail.com>

yep, happy new year.

i found anthoer bug.

js_run "
            function foo()
            {
                return 'bug while there is empty statement such ; after
return statement';
            }
        ";

i think in the source:
njs/njs_parser.c/njs_parser_function_lambda: 520

/*
     * There is no function body or the last function body statement is not
     * "return" statement.  If function has body then the body->right node
is
     * always present and it is a NJS_TOKEN_STATEMENT link node.
     */
    body = parser->node;

if function has body but the last statement is empty such simicolon, the
body->right node is also null.


2016-01-03 1:34 GMT+08:00 Igor Sysoev <igor at sysoev.ru>:

> On 02 Jan 2016, at 17:17, ??? <hongzhidao at gmail.com> wrote:
>
> js_set $bar "
>   $r.error = "this will cause coredump file since ext->set is null";
> ?;
>
>
> Thank you, I will this after new year vacation.
>
>
> --
> Igor Sysoev
> http://nginx.com
>
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160103/19687421/attachment.html>

From peterchencs at gmail.com  Tue Jan  5 05:30:32 2016
From: peterchencs at gmail.com (Peter Chen)
Date: Tue, 5 Jan 2016 00:30:32 -0500
Subject: Nginx Vulnerability on FreeBSD
Message-ID: <CAHF3bU8YfHHjio2SoJfLCDUbyQqwznxw8E_faY5s6T3y5dEcDw@mail.gmail.com>

Hi,

I am trying to do a security research experiment on FreeBSD.
I try to test the Nginx Vulnerability CVE-2013-2028 on FreeBSD 10.1 x86-64,
with Nginx 1.3.9/1.4.0.
(https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-2028)

However, most exploit samples can succeed on Linux, but not FreeBSD.
The basic idea for the exploit, is to send a packet with a very large chunk
size, making the victim process stack-overflow. After Nginx's many crashes,
the attacker can find enough gadgets to launch a return-oriented
programming attack.

However, it is hard to let Nginx worker process crash (due to overwritten
return address) on FreeBSD. Process crash is the first step of the whole
exploit. I do the experiment on both local and remote (LAN) machines.

This exploit requires:
-----------------------------------------------------------------
This also includes an IP fragmentation router to make the attack possible
on WANs. Nginx does a non-blocking read on a 4096 byte buffer, and typical
MTUs are 1500, so IP fragmentation is needed to deliver a large TCP segment
that will result in a single read of over 4096 bytes.
------------------------------------------------------------------

Any comments/suggestions on this, just to make the victim process crash?

Here are two exploit code examples, which can run against Linux target, but
fail to make the Nginx worker process crash on FreeBSD:

http://www.scs.stanford.edu/brop/
http://www.scs.stanford.edu/brop/nginx-1.4.0-exp.tgz

https://www.exploit-db.com/docs/27074.pdf
http://seclists.org/fulldisclosure/2013/Jul/att-90/ngxunlock_pl.bin


Thanks very much for your time!!

Best,
Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160105/b0e84905/attachment.html>

From teward at dark-net.net  Tue Jan  5 13:08:52 2016
From: teward at dark-net.net (Thomas Ward)
Date: Tue, 5 Jan 2016 08:08:52 -0500
Subject: Mainline HTTP/2 and Dependencies?
Message-ID: <568BC064.3000701@dark-net.net>

Hello!

Being ever vigilant downstream, it was asked of me as the packaging
maintainer for Ubuntu's nginx package by the Ubuntu Security Team and
the Server Team contact at Canonical whether HTTP/2 has any external
dependencies, such as 'libnghttp2', to make HTTP/2 work here in NGINX.

As far as I know, there are no external dependencies other than an SSL
library (such as an OpenSSL library, with ALPN TLS support), and the
HTTP/2 implementation done by NGINX is in fact NGINX's implementation
and does not rely on the external implementations in other libraries.  I
would, however, like a confirmation on this.

Can someone please confirm whether what I think I know is in fact the case?


Thanks.



Thomas



From maxim at nginx.com  Tue Jan  5 13:15:09 2016
From: maxim at nginx.com (Maxim Konovalov)
Date: Tue, 5 Jan 2016 16:15:09 +0300
Subject: Mainline HTTP/2 and Dependencies?
In-Reply-To: <568BC064.3000701@dark-net.net>
References: <568BC064.3000701@dark-net.net>
Message-ID: <568BC1DD.9030304@nginx.com>

Hi Thomas,

On 1/5/16 4:08 PM, Thomas Ward wrote:
> Hello!
> 
> Being ever vigilant downstream, it was asked of me as the packaging
> maintainer for Ubuntu's nginx package by the Ubuntu Security Team and
> the Server Team contact at Canonical whether HTTP/2 has any external
> dependencies, such as 'libnghttp2', to make HTTP/2 work here in NGINX.
> 
> As far as I know, there are no external dependencies other than an SSL
> library (such as an OpenSSL library, with ALPN TLS support), and the
> HTTP/2 implementation done by NGINX is in fact NGINX's implementation
> and does not rely on the external implementations in other libraries.  I
> would, however, like a confirmation on this.
> 
> Can someone please confirm whether what I think I know is in fact the case?
> 
Confirmed.

-- 
Maxim Konovalov


From teward at dark-net.net  Tue Jan  5 13:24:05 2016
From: teward at dark-net.net (Thomas Ward)
Date: Tue, 5 Jan 2016 08:24:05 -0500
Subject: Mainline HTTP/2 and Dependencies?
In-Reply-To: <568BC1DD.9030304@nginx.com>
References: <568BC064.3000701@dark-net.net> <568BC1DD.9030304@nginx.com>
Message-ID: <568BC3F5.1050706@dark-net.net>

Hello Maxim,

On 01/05/2016 08:15 AM, Maxim Konovalov wrote:
> Hi Thomas,
>
> On 1/5/16 4:08 PM, Thomas Ward wrote:
>> Hello!
>>
>> Being ever vigilant downstream, it was asked of me as the packaging
>> maintainer for Ubuntu's nginx package by the Ubuntu Security Team and
>> the Server Team contact at Canonical whether HTTP/2 has any external
>> dependencies, such as 'libnghttp2', to make HTTP/2 work here in NGINX.
>>
>> As far as I know, there are no external dependencies other than an SSL
>> library (such as an OpenSSL library, with ALPN TLS support), and the
>> HTTP/2 implementation done by NGINX is in fact NGINX's implementation
>> and does not rely on the external implementations in other libraries.  I
>> would, however, like a confirmation on this.
>>
>> Can someone please confirm whether what I think I know is in fact the case?
>>
> Confirmed.
>
That's what I thought, but thank you for confirming for me!


Hope you and everyone (both at NGINX, and indirectly everyone on this
list) had a happy new year!


Thomas



From agentzh at gmail.com  Tue Jan  5 22:27:08 2016
From: agentzh at gmail.com (Yichun Zhang (agentzh))
Date: Tue, 5 Jan 2016 14:27:08 -0800
Subject: [PATCH] SSL: handled SSL_CTX_set_cert_cb() callback yielding.
In-Reply-To: <CAB4Tn6NGax5KBGm=V+L3=--CLgxE98x-YksK-YQBRWWXefp02Q@mail.gmail.com>
References: <CAB4Tn6NGax5KBGm=V+L3=--CLgxE98x-YksK-YQBRWWXefp02Q@mail.gmail.com>
Message-ID: <CAB4Tn6NjS0h0nmps+xTMaqMWzd0cOAd=T0KHRDpkpfimLzBJMg@mail.gmail.com>

Hello!

On Sat, Jan 2, 2016 at 8:06 PM, Yichun Zhang (agentzh) wrote:
> SSL: handled SSL_CTX_set_cert_cb() callback yielding.
>
> OpenSSL 1.0.2+ introduces SSL_CTX_set_cert_cb() to allow custom
> callbacks to serve the SSL certificiates and private keys dynamically
> and lazily. The callbacks may yield for nonblocking I/O or sleeping.
> Here we added support for such usage in NGINX 3rd-party modules
> (like ngx_lua) in NGINX's event handlers for downstream SSL
> connections.
>

FYI, the new ssl_certificate_by_lua* directives of ngx_http_lua_module
relies on this NGINX core patch:

    https://github.com/openresty/lua-nginx-module#ssl_certificate_by_lua_block

This allows users to use Lua to dynamically load and configure SSL
certificates and private keys used by downstream https connections,
for example.

It'll be great if the mainline nginx core can have this patch applied.
And this would also be a wonderful feature for the NGINX world as
well.

Regards,
-agentzh


From chencw1982 at gmail.com  Thu Jan  7 05:16:55 2016
From: chencw1982 at gmail.com (Chuanwen Chen)
Date: Thu, 7 Jan 2016 13:16:55 +0800
Subject: support for proxy_http_version automatically detect
Message-ID: <CAKPCrs0U25DSToxw0N+=ZhjafWqjPk_bTpNx272KYrbMPEt22g@mail.gmail.com>

In our case, we needed to set proxy_http_version to 1.0 or 1.1 according to
client, because the clients for http 1.0 could not proceed chunked encoding
gzip through keep-alive connections correctly, but 'proxy_http_version'
didn't help.

Finally we had this patch, and a simple test case for it, and I checked it
under nginx-1.9.7.

Wish it helps.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160107/020e4326/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch
Type: application/octet-stream
Size: 3229 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160107/020e4326/attachment.obj>

From mdounin at mdounin.ru  Fri Jan  8 18:09:50 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 8 Jan 2016 21:09:50 +0300
Subject: support for proxy_http_version automatically detect
In-Reply-To: <CAKPCrs0U25DSToxw0N+=ZhjafWqjPk_bTpNx272KYrbMPEt22g@mail.gmail.com>
References: <CAKPCrs0U25DSToxw0N+=ZhjafWqjPk_bTpNx272KYrbMPEt22g@mail.gmail.com>
Message-ID: <20160108180949.GN74233@mdounin.ru>

Hello!

On Thu, Jan 07, 2016 at 01:16:55PM +0800, Chuanwen Chen wrote:

> In our case, we needed to set proxy_http_version to 1.0 or 1.1 according to
> client, because the clients for http 1.0 could not proceed chunked encoding
> gzip through keep-alive connections correctly, but 'proxy_http_version'
> didn't help.

I don't understand what problem you are trying to solve.

Protocols used by proxy and by clients are unrelated.  As long as 
a response returned by a backend don't have Content-Length set, 
nginx will automatically use chunked transfer encoding if client 
supports it.  And vice versa, if a client uses HTTP/1.0, nginx will 
not use chunked transfer encoding in such a case, and will close 
the connection instead.

-- 
Maxim Dounin
http://nginx.org/


From ru at nginx.com  Mon Jan 11 07:54:26 2016
From: ru at nginx.com (Ruslan Ermilov)
Date: Mon, 11 Jan 2016 07:54:26 +0000
Subject: [nginx] Year 2016.
Message-ID: <hg.9d06921918af.1452498866.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/9d06921918af
branches:  
changeset: 6333:9d06921918af
user:      Ruslan Ermilov <ru at nginx.com>
date:      Mon Jan 11 10:53:49 2016 +0300
description:
Year 2016.

diffstat:

 docs/text/LICENSE |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (12 lines):

diff -r 78b4e10b4367 -r 9d06921918af docs/text/LICENSE
--- a/docs/text/LICENSE	Thu Dec 17 16:39:15 2015 +0300
+++ b/docs/text/LICENSE	Mon Jan 11 10:53:49 2016 +0300
@@ -1,6 +1,6 @@
 /* 
- * Copyright (C) 2002-2015 Igor Sysoev
- * Copyright (C) 2011-2015 Nginx, Inc.
+ * Copyright (C) 2002-2016 Igor Sysoev
+ * Copyright (C) 2011-2016 Nginx, Inc.
  * All rights reserved.
  *
  * Redistribution and use in source and binary forms, with or without


From mdounin at mdounin.ru  Mon Jan 11 16:10:48 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 11 Jan 2016 16:10:48 +0000
Subject: [nginx] Upstream: fixed changing method on X-Accel-Redirect.
Message-ID: <hg.b31928ca3870.1452528648.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/b31928ca3870
branches:  
changeset: 6334:b31928ca3870
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Jan 11 19:08:12 2016 +0300
description:
Upstream: fixed changing method on X-Accel-Redirect.

Previously, only r->method was changed, resulting in handling of a request
as GET within nginx itself, but not in requests to proxied servers.

See http://mailman.nginx.org/pipermail/nginx/2015-December/049518.html.

diffstat:

 src/http/ngx_http_upstream.c |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (11 lines):

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -2499,6 +2499,7 @@ ngx_http_upstream_process_headers(ngx_ht
 
             if (r->method != NGX_HTTP_HEAD) {
                 r->method = NGX_HTTP_GET;
+                r->method_name = ngx_http_core_get_method;
             }
 
             ngx_http_internal_redirect(r, &uri, &args);


From mdounin at mdounin.ru  Tue Jan 12 16:03:56 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 12 Jan 2016 16:03:56 +0000
Subject: [nginx] Core: worker_cpu_affinity auto.
Message-ID: <hg.96c4297375bc.1452614636.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/96c4297375bc
branches:  
changeset: 6335:96c4297375bc
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Jan 11 19:23:17 2016 +0300
description:
Core: worker_cpu_affinity auto.

If enabled, workers are bound to available CPUs, each worker to once CPU
in order.  If there are more workers than available CPUs, remaining are
bound in a loop, starting again from the first available CPU.

The optional mask parameter defines which CPUs are available for automatic
binding.

In collaboration with Vladimir Homutov.

diffstat:

 src/core/nginx.c     |  43 +++++++++++++++++++++++++++++++++++++++++--
 src/core/ngx_cycle.h |   1 +
 2 files changed, 42 insertions(+), 2 deletions(-)

diffs (92 lines):

diff --git a/src/core/nginx.c b/src/core/nginx.c
--- a/src/core/nginx.c
+++ b/src/core/nginx.c
@@ -961,6 +961,7 @@ ngx_core_module_create_conf(ngx_cycle_t 
      *     ccf->pid = NULL;
      *     ccf->oldpid = NULL;
      *     ccf->priority = 0;
+     *     ccf->cpu_affinity_auto = 0;
      *     ccf->cpu_affinity_n = 0;
      *     ccf->cpu_affinity = NULL;
      */
@@ -1002,7 +1003,8 @@ ngx_core_module_init_conf(ngx_cycle_t *c
 
 #if (NGX_HAVE_CPU_AFFINITY)
 
-    if (ccf->cpu_affinity_n
+    if (!ccf->cpu_affinity_auto
+        && ccf->cpu_affinity_n
         && ccf->cpu_affinity_n != 1
         && ccf->cpu_affinity_n != (ngx_uint_t) ccf->worker_processes)
     {
@@ -1273,7 +1275,24 @@ ngx_set_cpu_affinity(ngx_conf_t *cf, ngx
 
     value = cf->args->elts;
 
-    for (n = 1; n < cf->args->nelts; n++) {
+    if (ngx_strcmp(value[1].data, "auto") == 0) {
+
+        if (cf->args->nelts > 3) {
+            ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
+                               "invalid number of arguments in "
+                               "\"worker_cpu_affinity\" directive");
+            return NGX_CONF_ERROR;
+        }
+
+        ccf->cpu_affinity_auto = 1;
+        mask[0] = (uint64_t) -1 >> (64 - ngx_min(64, ngx_ncpu));
+        n = 2;
+
+    } else {
+        n = 1;
+    }
+
+    for ( /* void */ ; n < cf->args->nelts; n++) {
 
         if (value[n].len > 64) {
             ngx_conf_log_error(NGX_LOG_EMERG, cf, 0,
@@ -1323,6 +1342,8 @@ ngx_set_cpu_affinity(ngx_conf_t *cf, ngx
 uint64_t
 ngx_get_cpu_affinity(ngx_uint_t n)
 {
+    uint64_t          mask;
+    ngx_uint_t        i;
     ngx_core_conf_t  *ccf;
 
     ccf = (ngx_core_conf_t *) ngx_get_conf(ngx_cycle->conf_ctx,
@@ -1332,6 +1353,24 @@ ngx_get_cpu_affinity(ngx_uint_t n)
         return 0;
     }
 
+    if (ccf->cpu_affinity_auto) {
+        mask = ccf->cpu_affinity[ccf->cpu_affinity_n - 1];
+
+        if (mask == 0) {
+            return 0;
+        }
+
+        for (i = 0; /* void */ ; i++) {
+            if ((mask & ((uint64_t) 1 << (i % 64))) && n-- == 0) {
+                break;
+            }
+
+            /* void */
+        }
+
+        return (uint64_t) 1 << (i % 64);
+    }
+
     if (ccf->cpu_affinity_n > n) {
         return ccf->cpu_affinity[n];
     }
diff --git a/src/core/ngx_cycle.h b/src/core/ngx_cycle.h
--- a/src/core/ngx_cycle.h
+++ b/src/core/ngx_cycle.h
@@ -88,6 +88,7 @@ typedef struct {
 
      int                      priority;
 
+     ngx_uint_t               cpu_affinity_auto;
      ngx_uint_t               cpu_affinity_n;
      uint64_t                *cpu_affinity;
 


From vbart at nginx.com  Tue Jan 12 16:20:24 2016
From: vbart at nginx.com (Valentin Bartenev)
Date: Tue, 12 Jan 2016 16:20:24 +0000
Subject: [nginx] Request body: removed surplus assigment, no functional c...
Message-ID: <hg.02abce4764b7.1452615624.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/02abce4764b7
branches:  
changeset: 6336:02abce4764b7
user:      Valentin Bartenev <vbart at nginx.com>
date:      Tue Jan 12 19:19:07 2016 +0300
description:
Request body: removed surplus assigment, no functional changes.

Setting rb->bufs to NULL is surplus after ngx_http_write_request_body()
has returned NGX_OK.

diffstat:

 src/http/ngx_http_request_body.c |  6 ------
 1 files changed, 0 insertions(+), 6 deletions(-)

diffs (23 lines):

diff -r 96c4297375bc -r 02abce4764b7 src/http/ngx_http_request_body.c
--- a/src/http/ngx_http_request_body.c	Mon Jan 11 19:23:17 2016 +0300
+++ b/src/http/ngx_http_request_body.c	Tue Jan 12 19:19:07 2016 +0300
@@ -172,9 +172,6 @@ ngx_http_read_client_request_body(ngx_ht
                 b->file = &rb->temp_file->file;
 
                 rb->bufs = cl;
-
-            } else {
-                rb->bufs = NULL;
             }
         }
 
@@ -466,9 +463,6 @@ ngx_http_do_read_client_request_body(ngx
             b->file = &rb->temp_file->file;
 
             rb->bufs = cl;
-
-        } else {
-            rb->bufs = NULL;
         }
     }
 


From igor at sysoev.ru  Wed Jan 13 17:04:17 2016
From: igor at sysoev.ru (Igor Sysoev)
Date: Wed, 13 Jan 2016 20:04:17 +0300
Subject: njs isuse -- ext->set is null
In-Reply-To: <CAJuLd0jqKfTz1uJTCKWNQJ99-LenE5RFXeK31GENDA9kwYsZDA@mail.gmail.com>
References: <CAJuLd0gMWnV_GWkhr1RU8S+ikV793ZqYC5KCC2mC3xx+a7dSrA@mail.gmail.com>
 <F78C95D1-FB6B-49EF-BD78-9AD3A67BC356@sysoev.ru>
 <CAJuLd0jqKfTz1uJTCKWNQJ99-LenE5RFXeK31GENDA9kwYsZDA@mail.gmail.com>
Message-ID: <68364C5B-D602-434D-B778-93ADDE236EE1@sysoev.ru>

Thank you, these and other bugs are fixed in the recent commits.


-- 
Igor Sysoev
http://nginx.com

On 03 Jan 2016, at 08:18, ??? <hongzhidao at gmail.com> wrote:

> yep, happy new year.
> 
> i found anthoer bug.
> 
> js_run "
>             function foo()
>             {
>                 return 'bug while there is empty statement such ; after return statement';
>             }
>         ";
> 
> i think in the source:
> njs/njs_parser.c/njs_parser_function_lambda: 520
> 
> /*
>      * There is no function body or the last function body statement is not
>      * "return" statement.  If function has body then the body->right node is
>      * always present and it is a NJS_TOKEN_STATEMENT link node.
>      */
>     body = parser->node;
> 
> if function has body but the last statement is empty such simicolon, the body->right node is also null.
> 
> 
> 2016-01-03 1:34 GMT+08:00 Igor Sysoev <igor at sysoev.ru>:
> On 02 Jan 2016, at 17:17, ??? <hongzhidao at gmail.com> wrote:
> 
>> js_set $bar "
>>   $r.error = "this will cause coredump file since ext->set is null";
>> ?;
> 
> Thank you, I will this after new year vacation.
> 
> 
> -- 
> Igor Sysoev
> http://nginx.com
> 
> 
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
> 
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160113/7f242ddf/attachment.html>

From thibault.koechlin at nbs-system.com  Fri Jan 15 14:01:06 2016
From: thibault.koechlin at nbs-system.com (Thibault Koechlin)
Date: Fri, 15 Jan 2016 15:01:06 +0100
Subject: Reading body during the REWRITE phase ?
Message-ID: <5698FBA2.50901@nbs-system.com>

Hi,

I have a module (naxsi) that reads the body during the REWRITE phase, in
the exact same way that ngx_form_input does :
https://github.com/calio/form-input-nginx-module.

When used with auth_request (or maybe other modules, but that's the
first time I encounter this issue within a few years of usage), there is
no request made to the upstream if the request is made using POST/PUT
and the body is bigger than client_body_buffer_size.

For the simplicity of the example, we'll assume I'm talking about
ngx_form_input (behaviour is the same, except code is way shorter).

The user reporting me the bug opened a ticket :
https://trac.nginx.org/nginx/ticket/801. It is possible to replace naxsi
with ngx_for_input and obtain the same results.

From Maxim's reply, it seems I failed to properly restore request
handlers after reading body.

What would be (if there is any) the proper way to read body within
REWRITE phase ? Is there any example/existing module that does such so I
can understand what am I doing wrong ? (In the past, I always thought
ngx_form_input was the reference one).


PS: You can find a bit more details here :
https://github.com/nbs-system/naxsi/issues/226 (including sample config
& commands to reproduce bug)


Thanks,

-- 
Thibault Koechlin


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160115/2cfad03e/attachment.bin>

From serg.brester at sebres.de  Fri Jan 15 15:04:17 2016
From: serg.brester at sebres.de (Sergey Brester)
Date: Fri, 15 Jan 2016 16:04:17 +0100
Subject: Reading body during the REWRITE phase ?
In-Reply-To: <5698FBA2.50901@nbs-system.com>
References: <5698FBA2.50901@nbs-system.com>
Message-ID: <00501af8f0f40ac41716da8577d47466@sebres.de>

 

Hi, 

It's normally no proper way to read a body within REWRITE phase (I think
you mean body of something like POST request). 

The request body will be typically read resp. upstreamed first if
location will be processed (after all of the phases, including a rewrite
also). 

But you can use that inside another location using some modules, that
may read a body (even manipulate it for example inside a filter) and
then upstream it further to another location.
For example see the nginx-upload-module [5], that read
multipart/form-data, disaggregates all parts with file-data (writes such
parts in files), and hereafter passes modified request body (even
without file-data) to another location... The similar things will do the
lua-module, etc.

You can also overwrite some default filter handlers, to read the body
formerly, but it would be not really nginx-way (you know, everything
should be be asynchronously =). 

BTW, if you use "auth_request" (and other things internally using
sub_request) you can disable passing of the body inside a location that
do auth_request (not in main location), with directives like
"scgi_pass_request_body off;", "fastcgi_pass_request_body off;", etc. 

Hope it helps reasonably...

Regards,
sebres. 

Am 15.01.2016 15:01, schrieb Thibault Koechlin: 

> Hi,
> 
> I have a module (naxsi) that reads the body during the REWRITE phase, in
> the exact same way that ngx_form_input does :
> https://github.com/calio/form-input-nginx-module [1].
> 
> When used with auth_request (or maybe other modules, but that's the
> first time I encounter this issue within a few years of usage), there is
> no request made to the upstream if the request is made using POST/PUT
> and the body is bigger than client_body_buffer_size.
> 
> For the simplicity of the example, we'll assume I'm talking about
> ngx_form_input (behaviour is the same, except code is way shorter).
> 
> The user reporting me the bug opened a ticket :
> https://trac.nginx.org/nginx/ticket/801 [2]. It is possible to replace naxsi
> with ngx_for_input and obtain the same results.
> 
> From Maxim's reply, it seems I failed to properly restore request
> handlers after reading body.
> 
> What would be (if there is any) the proper way to read body within
> REWRITE phase ? Is there any example/existing module that does such so I
> can understand what am I doing wrong ? (In the past, I always thought
> ngx_form_input was the reference one).
> 
> PS: You can find a bit more details here :
> https://github.com/nbs-system/naxsi/issues/226 [3] (including sample config
> & commands to reproduce bug)
> 
> Thanks,
> 
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel [4]
 

Links:
------
[1] https://github.com/calio/form-input-nginx-module
[2] https://trac.nginx.org/nginx/ticket/801
[3] https://github.com/nbs-system/naxsi/issues/226
[4] http://mailman.nginx.org/mailman/listinfo/nginx-devel
[5] https://github.com/vkholodkov/nginx-upload-module/tree/2.2
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160115/4eacb5a1/attachment.html>

From mdounin at mdounin.ru  Fri Jan 15 15:53:04 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 15 Jan 2016 18:53:04 +0300
Subject: Reading body during the REWRITE phase ?
In-Reply-To: <5698FBA2.50901@nbs-system.com>
References: <5698FBA2.50901@nbs-system.com>
Message-ID: <20160115155304.GP74233@mdounin.ru>

Hello!

On Fri, Jan 15, 2016 at 03:01:06PM +0100, Thibault Koechlin wrote:

> Hi,
> 
> I have a module (naxsi) that reads the body during the REWRITE phase, in
> the exact same way that ngx_form_input does :
> https://github.com/calio/form-input-nginx-module.
> 
> When used with auth_request (or maybe other modules, but that's the
> first time I encounter this issue within a few years of usage), there is
> no request made to the upstream if the request is made using POST/PUT
> and the body is bigger than client_body_buffer_size.
> 
> For the simplicity of the example, we'll assume I'm talking about
> ngx_form_input (behaviour is the same, except code is way shorter).
> 
> The user reporting me the bug opened a ticket :
> https://trac.nginx.org/nginx/ticket/801. It is possible to replace naxsi
> with ngx_for_input and obtain the same results.
> 
> From Maxim's reply, it seems I failed to properly restore request
> handlers after reading body.
> 
> What would be (if there is any) the proper way to read body within
> REWRITE phase ? Is there any example/existing module that does such so I
> can understand what am I doing wrong ? (In the past, I always thought
> ngx_form_input was the reference one).

No modules in nginx itself attempt to read body before it knows 
it'll handle the request.  So I don't think there are examples of 
doing this correctly.

The problem as seen from debug logs is that reading body will 
reset request handlers, including r->write_event_handler, which is 
used to continue processing of request phases.  That is, as a 
minimum you'll have to do something like:

    r->write_event_handler = ngx_http_core_run_phases;
    ngx_http_core_run_phases(r);

when you restart processing of phases after the request body is 
read.  As of now your code seems to call ngx_http_core_run_phases() 
without restoring the r->write_event_handler.

Similar code can be seen in ngx_http_finalize_request() when 
processing of phases is restared after a content handler returned 
NGX_DECLINED, see src/http/ngx_http_request.c:

    if (rc == NGX_DECLINED) {
        r->content_handler = NULL;
        r->write_event_handler = ngx_http_core_run_phases;
        ngx_http_core_run_phases(r);
        return;
    }

Hope this helps.

-- 
Maxim Dounin
http://nginx.org/


From agentzh at gmail.com  Fri Jan 15 21:58:56 2016
From: agentzh at gmail.com (Yichun Zhang (agentzh))
Date: Fri, 15 Jan 2016 13:58:56 -0800
Subject: Reading body during the REWRITE phase ?
In-Reply-To: <5698FBA2.50901@nbs-system.com>
References: <5698FBA2.50901@nbs-system.com>
Message-ID: <CAB4Tn6M2GG_efCVG76Us-N=TwqdGVY6_=9Rteujy4yav_NSfCg@mail.gmail.com>

Hello!

On Fri, Jan 15, 2016 at 6:01 AM, Thibault Koechlin wrote:
> When used with auth_request (or maybe other modules, but that's the
> first time I encounter this issue within a few years of usage), there is
> no request made to the upstream if the request is made using POST/PUT
> and the body is bigger than client_body_buffer_size.
>

The problem here seems to be that the subrequest initiated by
auth_request already *consumes* the request body buffer chains which
make later main request handlers like ngx_proxy fail to see the
request body.

Regards,
-agentzh


From mdounin at mdounin.ru  Mon Jan 18 10:33:38 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 18 Jan 2016 13:33:38 +0300
Subject: Reading body during the REWRITE phase ?
In-Reply-To: <CAB4Tn6M2GG_efCVG76Us-N=TwqdGVY6_=9Rteujy4yav_NSfCg@mail.gmail.com>
References: <5698FBA2.50901@nbs-system.com>
 <CAB4Tn6M2GG_efCVG76Us-N=TwqdGVY6_=9Rteujy4yav_NSfCg@mail.gmail.com>
Message-ID: <20160118103338.GV74233@mdounin.ru>

Hello!

On Fri, Jan 15, 2016 at 01:58:56PM -0800, Yichun Zhang (agentzh) wrote:

> Hello!
> 
> On Fri, Jan 15, 2016 at 6:01 AM, Thibault Koechlin wrote:
> > When used with auth_request (or maybe other modules, but that's the
> > first time I encounter this issue within a few years of usage), there is
> > no request made to the upstream if the request is made using POST/PUT
> > and the body is bigger than client_body_buffer_size.
> >
> 
> The problem here seems to be that the subrequest initiated by
> auth_request already *consumes* the request body buffer chains which
> make later main request handlers like ngx_proxy fail to see the
> request body.

No.

-- 
Maxim Dounin
http://nginx.org/


From rodin.alexander at gmail.com  Mon Jan 18 15:03:45 2016
From: rodin.alexander at gmail.com (Alexander Rodin)
Date: Mon, 18 Jan 2016 18:03:45 +0300
Subject: Architecture for asynchronous Mongo GridFS module for Nginx
Message-ID: <CAFr1zyRp3X0QbBGefZ1vvjM97+=EnqHLX-6Kz+dKzjcZN7CDYw@mail.gmail.com>

Hi all!

I'm researching possible approaches to development of asynchronous GridFS
module for Nginx (because the existing one blocks the main Nginx thread).

I've read the source codes of most modules from
https://www.nginx.com/resources/wiki/modules/ and found that all of
asynchronous modules (e.g. Postgres or Mongo) implement the database
protocol by itself working with upstream. Is it the only way? I would like
to use mongo-c-driver (https://github.com/mongodb/mongo-c-driver) but I
need an example how to embed it into Nginx event loop to make non-blocking
module.

Could you suggest any module that uses the same approach or some manual
about it?

Best regards
Alexander
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160118/4411412f/attachment.html>

From tigran.bayburtsyan at gmail.com  Mon Jan 18 15:20:23 2016
From: tigran.bayburtsyan at gmail.com (Tigran Bayburtsyan)
Date: Mon, 18 Jan 2016 19:20:23 +0400
Subject: Architecture for asynchronous Mongo GridFS module for Nginx
In-Reply-To: <CAFr1zyRp3X0QbBGefZ1vvjM97+=EnqHLX-6Kz+dKzjcZN7CDYw@mail.gmail.com>
References: <CAFr1zyRp3X0QbBGefZ1vvjM97+=EnqHLX-6Kz+dKzjcZN7CDYw@mail.gmail.com>
Message-ID: <008b01d15203$c1b047f0$4510d7d0$@gmail.com>

Hi , I?ve tried to do same thing, but new https://github.com/mongodb/mongo-c-driver  is extremely buggy and still don?t have support for async cursor reading or keep alive connection (it keeps only handler structure for making new connection on the fly). 

After a week of development and researches I got some working version but it?s didn?t work out on production, on existing database new mongo-c driver tried to make new kind of indexes which is not good for our database. 

So at the end I?ve just created new concurrent service with Go and upstream for nginx ?. And it works perfectly !!

 

I?ve tried to change Mongo C driver to work with Nginx upstream API, but I ended up with a lot of big changes, so I just gave up.

I think using separate service for GridFS is more scalable than making it inside Nginx, because you can just make upstream to 100s of GrdiFS frontend services like I did.

 

But anyway if you will do something like that I?ll defiantly contribute !!

 

Thanks 

 

From: nginx-devel [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Alexander Rodin
Sent: Monday, January 18, 2016 7:04 PM
To: nginx-devel at nginx.org
Subject: Architecture for asynchronous Mongo GridFS module for Nginx

 

Hi all!

I'm researching possible approaches to development of asynchronous GridFS module for Nginx (because the existing one blocks the main Nginx thread).

I've read the source codes of most modules from https://www.nginx.com/resources/wiki/modules/ and found that all of asynchronous modules (e.g. Postgres or Mongo) implement the database protocol by itself working with upstream. Is it the only way? I would like to use mongo-c-driver (https://github.com/mongodb/mongo-c-driver) but I need an example how to embed it into Nginx event loop to make non-blocking module.

Could you suggest any module that uses the same approach or some manual about it?

Best regards

Alexander

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160118/e2932c0c/attachment.html>

From hyphen9 at foxmail.com  Tue Jan 19 04:04:54 2016
From: hyphen9 at foxmail.com (=?utf-8?B?aHlwaGVu?=)
Date: Tue, 19 Jan 2016 12:04:54 +0800
Subject: Fixed request method must be already supported
Message-ID: <tencent_5B4032BC2DCDBE695E323B21@qq.com>

# HG changeset patch
# User goecho <hyphen9 at foxmail.com>
# Date 1453175856 -28800
#      Tue Jan 19 11:57:36 2016 +0800
# Node ID 25548a0b15afd46f3177c34a3d6af1dbe830cef0
# Parent  02abce4764b7c964e7ee6e10b1a7abf1fee95f79
Fixed request method must be already supported

diff -r 02abce4764b7 -r 25548a0b15af src/http/ngx_http_parse.c
--- a/src/http/ngx_http_parse.c Tue Jan 12 19:19:07 2016 +0300
+++ b/src/http/ngx_http_parse.c Tue Jan 19 11:57:36 2016 +0800
@@ -264,7 +264,10 @@
                     }

                     break;
-                }
+
+               default :
+                   return NGX_HTTP_PARSE_INVALID_METHOD;
+               }

                 state = sw_spaces_before_uri;
                 break;
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160119/20b242f8/attachment.html>

From mdounin at mdounin.ru  Tue Jan 19 11:19:52 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 19 Jan 2016 14:19:52 +0300
Subject: Fixed request method must be already supported
In-Reply-To: <tencent_5B4032BC2DCDBE695E323B21@qq.com>
References: <tencent_5B4032BC2DCDBE695E323B21@qq.com>
Message-ID: <20160119111952.GA74233@mdounin.ru>

Hello!

On Tue, Jan 19, 2016 at 12:04:54PM +0800, hyphen wrote:

> # HG changeset patch
> # User goecho <hyphen9 at foxmail.com>
> # Date 1453175856 -28800
> #      Tue Jan 19 11:57:36 2016 +0800
> # Node ID 25548a0b15afd46f3177c34a3d6af1dbe830cef0
> # Parent  02abce4764b7c964e7ee6e10b1a7abf1fee95f79
> Fixed request method must be already supported
> 
> diff -r 02abce4764b7 -r 25548a0b15af src/http/ngx_http_parse.c
> --- a/src/http/ngx_http_parse.c Tue Jan 12 19:19:07 2016 +0300
> +++ b/src/http/ngx_http_parse.c Tue Jan 19 11:57:36 2016 +0800
> @@ -264,7 +264,10 @@
>                      }
> 
>                      break;
> -                }
> +
> +               default :
> +                   return NGX_HTTP_PARSE_INVALID_METHOD;
> +               }
> 
>                  state = sw_spaces_before_uri;
>                  break;

Current approach is to limit methods for resources served by nginx 
itself, but allow unknown request methods to be passed to upstream 
servers with proxy_pass.  This is known to be required at least by 
some well-known software - e.g., various MS tools introduce their 
own methods like RPC_IN_DATA, and RESTful APIs sometimes do the 
same.

If you want to limit methods allowed, you can do so by 
using the "limit_except" and/or "if" directives, see here:

http://nginx.org/r/limit_except
http://nginx.org/r/if

-- 
Maxim Dounin
http://nginx.org/


From vbart at nginx.com  Tue Jan 19 17:20:25 2016
From: vbart at nginx.com (Valentin V. Bartenev)
Date: Tue, 19 Jan 2016 20:20:25 +0300
Subject: [PATCH] HTTP/2: HPACK Huffman encoding
In-Reply-To: <d672e9c2b81471098668.1450352972@ubuntu>
References: <d672e9c2b81471098668.1450352972@ubuntu>
Message-ID: <2548150.3ZiQ2bDBD5@vbart-workstation>

On Thursday 17 December 2015 03:49:32 Vlad Krasnov wrote:
> # HG changeset patch
> # User Vlad Krasnov <vlad at cloudflare.com>
> # Date 1450274269 28800
> #      Wed Dec 16 05:57:49 2015 -0800
> # Node ID d672e9c2b814710986683e050491536355c90f0d
> # Parent  def9c9c9ae05cfa7467b0ec96e76afa180c23dfb
> HTTP/2: HPACK Huffman encoding

Style.  The dot at the end of summary line is needed.

> 
> Implement HPACK Huffman encoding for HTTP/2.
> This reduces the size of headers by over 30% on average.

See the comments below and a reworked patch in attachment.


> 
> diff -r def9c9c9ae05 -r d672e9c2b814 src/http/v2/ngx_http_v2.h
> --- a/src/http/v2/ngx_http_v2.h	Sat Dec 12 10:32:58 2015 +0300
> +++ b/src/http/v2/ngx_http_v2.h	Wed Dec 16 05:57:49 2015 -0800
> @@ -51,6 +51,9 @@
>  #define NGX_HTTP_V2_PRIORITY_FLAG        0x20
>  
>  
> +#define NGX_HTTP_V2_ENCODE_RAW            0
> +#define NGX_HTTP_V2_ENCODE_HUFF           0x80
> +
>  typedef struct ngx_http_v2_connection_s   ngx_http_v2_connection_t;
>  typedef struct ngx_http_v2_node_s         ngx_http_v2_node_t;
>  typedef struct ngx_http_v2_out_frame_s    ngx_http_v2_out_frame_t;
> @@ -255,6 +258,28 @@
>  }
>  
>  
> +static ngx_inline u_char *
> +ngx_http_v2_write_int(u_char *pos, ngx_uint_t prefix, ngx_uint_t value)
> +{
> +    if (value < prefix) {
> +        *pos++ |= value;
> +        return pos;
> +    }
> +
> +    *pos++ |= prefix;
> +    value -= prefix;
> +
> +    while (value >= 128) {
> +        *pos++ = value % 128 + 128;
> +        value /= 128;
> +    }
> +
> +    *pos++ = (u_char) value;
> +
> +    return pos;
> +}
> +
> +
>  void ngx_http_v2_init(ngx_event_t *rev);
>  void ngx_http_v2_request_headers_init(void);
>  
[..]

It's better to put ngx_http_v2_string_encode() into the
"ngx_http_v2_filter_module.c", than moving all these to
the header file.

Also that will be more consistent and symmetric with the
"ngx_http_v2_huff_decode.c".



> @@ -275,7 +300,8 @@
>  
>  ngx_int_t ngx_http_v2_huff_decode(u_char *state, u_char *src, size_t len,
>      u_char **dst, ngx_uint_t last, ngx_log_t *log);
> -
> +u_char *ngx_http_v2_string_encode(u_char *src, size_t len, u_char *dst,
> +    u_char *tmp, ngx_log_t *log, ngx_flag_t lower);
>  
>  #define ngx_http_v2_prefix(bits)  ((1 << (bits)) - 1)
>  
> diff -r def9c9c9ae05 -r d672e9c2b814 src/http/v2/ngx_http_v2_filter_module.c
> --- a/src/http/v2/ngx_http_v2_filter_module.c	Sat Dec 12 10:32:58 2015 +0300
> +++ b/src/http/v2/ngx_http_v2_filter_module.c	Wed Dec 16 05:57:49 2015 -0800
> @@ -25,9 +25,6 @@
>  #define ngx_http_v2_indexed(i)      (128 + (i))
>  #define ngx_http_v2_inc_indexed(i)  (64 + (i))
>  
> -#define NGX_HTTP_V2_ENCODE_RAW            0
> -#define NGX_HTTP_V2_ENCODE_HUFF           0x80
> -
>  #define NGX_HTTP_V2_STATUS_INDEX          8
>  #define NGX_HTTP_V2_STATUS_200_INDEX      8
>  #define NGX_HTTP_V2_STATUS_204_INDEX      9
> @@ -46,8 +43,9 @@
>  #define NGX_HTTP_V2_VARY_INDEX            59
>  
>  
> -static u_char *ngx_http_v2_write_int(u_char *pos, ngx_uint_t prefix,
> -    ngx_uint_t value);
> +#define ACCEPT_ENCODING  "\x84\x84\x2d\x69\x5b\x05\x44\x3c\x86\xaa\x6f"
> +
> +

You should either add NGX_* prefix or define it as a local
static constant.



>  static ngx_http_v2_out_frame_t *ngx_http_v2_create_headers_frame(
>      ngx_http_request_t *r, u_char *pos, u_char *end);
>  
> @@ -119,8 +117,8 @@
>  static ngx_int_t
>  ngx_http_v2_header_filter(ngx_http_request_t *r)
>  {
> -    u_char                     status, *pos, *start, *p;
> -    size_t                     len;
> +    u_char                     status, *pos, *start, *p, *huff;
> +    size_t                     len, hlen, nlen;
>      ngx_str_t                  host, location;
>      ngx_uint_t                 i, port;
>      ngx_list_part_t           *part;
> @@ -343,7 +341,7 @@
>  #if (NGX_HTTP_GZIP)
>      if (r->gzip_vary) {
>          if (clcf->gzip_vary) {
> -            len += 1 + ngx_http_v2_literal_size("Accept-Encoding");
> +            len += 1 + ngx_http_v2_literal_size(ACCEPT_ENCODING);
>  
>          } else {
>              r->gzip_vary = 0;
> @@ -354,6 +352,8 @@
>      part = &r->headers_out.headers.part;
>      header = part->elts;
>  
> +    hlen = len;
> +
>      for (i = 0; /* void */; i++) {
>  
>          if (i >= part->nelts) {
> @@ -384,11 +384,14 @@
>              return NGX_ERROR;
>          }
>  
> -        len += 1 + NGX_HTTP_V2_INT_OCTETS + header[i].key.len
> +        nlen = 1 + NGX_HTTP_V2_INT_OCTETS + header[i].key.len
>                   + NGX_HTTP_V2_INT_OCTETS + header[i].value.len;
> +        len += nlen;
> +        hlen = nlen > hlen ? nlen : hlen;

I don't understand why you are comparing hlen with the sum of
header name and value including the type byte.

Headers names and values are encoded separately.


>      }
>  
>      pos = ngx_palloc(r->pool, len);
> +    huff = ngx_palloc(r->pool, hlen);
>      if (pos == NULL) {
>          return NGX_ERROR;
>      }

The "huff" can be NULL which will result in segfault.



> @@ -408,12 +411,15 @@
>          *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_SERVER_INDEX);
>  
>          if (clcf->server_tokens) {
> -            *pos++ = NGX_HTTP_V2_ENCODE_RAW | (sizeof(NGINX_VER) - 1);
> -            pos = ngx_cpymem(pos, NGINX_VER, sizeof(NGINX_VER) - 1);
> +            pos = ngx_http_v2_string_encode((u_char*)NGINX_VER,
> +                                            sizeof(NGINX_VER) - 1, pos, huff,
> +                                            r->connection->log, 0);
>  
>          } else {
> -            *pos++ = NGX_HTTP_V2_ENCODE_RAW | (sizeof("nginx") - 1);
> -            pos = ngx_cpymem(pos, "nginx", sizeof("nginx") - 1);
> +            pos = ngx_http_v2_string_encode((u_char*)"nginx",
> +                                            sizeof("nginx") - 1, pos, huff,
> +                                            r->connection->log, 0);

The "nginx" constant can also be preencoded like "Accept-Encoding".


> +

Style nit.  A surplus empty line.


>          }
>      }
>  
> @@ -453,11 +459,9 @@
>              r->headers_out.content_type.data = p;
>  
>          } else {

Why don't you try to encode "Content-Type" in other case,
when the condition is true?

The patch looks also incomplete without the encoding of
"Date" header (it's simple to implement and quite useful).


> -            *pos = NGX_HTTP_V2_ENCODE_RAW;
> -            pos = ngx_http_v2_write_int(pos, ngx_http_v2_prefix(7),
> -                                        r->headers_out.content_type.len);
> -            pos = ngx_cpymem(pos, r->headers_out.content_type.data,
> -                             r->headers_out.content_type.len);
> +            pos = ngx_http_v2_string_encode(r->headers_out.content_type.data,
> +                                            r->headers_out.content_type.len,
> +                                            pos, huff, r->connection->log, 0);
>          }
>      }
>  
> @@ -476,26 +480,27 @@
>      {
>          *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_LAST_MODIFIED_INDEX);
>  
> -        *pos++ = NGX_HTTP_V2_ENCODE_RAW
> -                 | (sizeof("Wed, 31 Dec 1986 18:00:00 GMT") - 1);
> -        pos = ngx_http_time(pos, r->headers_out.last_modified_time);
> +        nlen = sizeof("Wed, 31 Dec 1986 18:00:00 GMT") - 1;
> +        ngx_http_time(pos + 1, r->headers_out.last_modified_time);
> +
> +        pos = ngx_http_v2_string_encode(pos + 1, nlen, pos, huff,
> +                                        r->connection->log, 0);

"pos + 1" looks wrong here.

Also the safety of using *pos as source and destination requires
a comment.


>      }
>  
>      if (r->headers_out.location && r->headers_out.location->value.len) {
>          *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_LOCATION_INDEX);
>  
>          *pos = NGX_HTTP_V2_ENCODE_RAW;

This assignment after the change below is surplus and misleading.


> -        pos = ngx_http_v2_write_int(pos, ngx_http_v2_prefix(7),
> -                                    r->headers_out.location->value.len);
> -        pos = ngx_cpymem(pos, r->headers_out.location->value.data,
> -                              r->headers_out.location->value.len);
> +        pos = ngx_http_v2_string_encode(r->headers_out.location->value.data,
> +                                        r->headers_out.location->value.len,
> +                                        pos, huff, r->connection->log, 0);
>      }
>  
>  #if (NGX_HTTP_GZIP)
>      if (r->gzip_vary) {
>          *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_VARY_INDEX);
> -        *pos++ = NGX_HTTP_V2_ENCODE_RAW | (sizeof("Accept-Encoding") - 1);
> -        pos = ngx_cpymem(pos, "Accept-Encoding", sizeof("Accept-Encoding") - 1);
> +        *pos++ = NGX_HTTP_V2_ENCODE_HUFF | (sizeof(ACCEPT_ENCODING) - 1);
> +        pos = ngx_cpymem(pos, ACCEPT_ENCODING, sizeof(ACCEPT_ENCODING) - 1);
>      }
>  #endif
>  
> @@ -520,16 +525,14 @@
>  
>          *pos++ = 0;
>  
> -        *pos = NGX_HTTP_V2_ENCODE_RAW;
> -        pos = ngx_http_v2_write_int(pos, ngx_http_v2_prefix(7),
> -                                    header[i].key.len);
> -        ngx_strlow(pos, header[i].key.data, header[i].key.len);
> -        pos += header[i].key.len;
> +        pos = ngx_http_v2_string_encode(header[i].key.data,
> +                                        header[i].key.len,
> +                                        pos, huff, r->connection->log, 1);
>  
> -        *pos = NGX_HTTP_V2_ENCODE_RAW;
> -        pos = ngx_http_v2_write_int(pos, ngx_http_v2_prefix(7),
> -                                    header[i].value.len);
> -        pos = ngx_cpymem(pos, header[i].value.data, header[i].value.len);
> +        pos = ngx_http_v2_string_encode(header[i].value.data,
> +                                        header[i].value.len,
> +                                        pos, huff, r->connection->log, 0);
> +
>      }
>  
>      frame = ngx_http_v2_create_headers_frame(r, start, pos);
> @@ -556,28 +559,6 @@
>  }
>  
>  
> -static u_char *
> -ngx_http_v2_write_int(u_char *pos, ngx_uint_t prefix, ngx_uint_t value)
> -{
> -    if (value < prefix) {
> -        *pos++ |= value;
> -        return pos;
> -    }
> -
> -    *pos++ |= prefix;
> -    value -= prefix;
> -
> -    while (value >= 128) {
> -        *pos++ = value % 128 + 128;
> -        value /= 128;
> -    }
> -
> -    *pos++ = (u_char) value;
> -
> -    return pos;
> -}
> -
> -
>  static ngx_http_v2_out_frame_t *
>  ngx_http_v2_create_headers_frame(ngx_http_request_t *r, u_char *pos,
>      u_char *end)
> diff -r def9c9c9ae05 -r d672e9c2b814 src/http/v2/ngx_http_v2_huff_encode.c
> --- a/src/http/v2/ngx_http_v2_huff_encode.c	Sat Dec 12 10:32:58 2015 +0300
> +++ b/src/http/v2/ngx_http_v2_huff_encode.c	Wed Dec 16 05:57:49 2015 -0800
> @@ -1,10 +1,296 @@
>  
>  /*
>   * Copyright (C) Nginx, Inc.
> - * Copyright (C) Valentin V. Bartenev
> + * Copyright (C) Vlad Krasnov
>   */
>  
>  
>  #include <ngx_config.h>
>  #include <ngx_core.h>
>  #include <ngx_http.h>
> +#if (__GNUC__)
> +#include <byteswap.h>
> +#endif

IMHO from portability point of view it's better to check the presence
of required builtin than to relate on __GNUC__.


> +
> +
> +typedef struct {
> +    ngx_uint_t  code;
> +    ngx_uint_t  len;
> +} ngx_http_v2_huff_encode_code_t;

It's better to keep compact these tables.
There are no need in 64 bits integers here.


> +
> +
> +static ngx_http_v2_huff_encode_code_t  ngx_http_v2_huff_encode_codes[256] =
> +{
> +    {0x00001ff8, 13}, {0x007fffd8, 23}, {0x0fffffe2, 28}, {0x0fffffe3, 28},
> +    {0x0fffffe4, 28}, {0x0fffffe5, 28}, {0x0fffffe6, 28}, {0x0fffffe7, 28},
> +    {0x0fffffe8, 28}, {0x00ffffea, 24}, {0x3ffffffc, 30}, {0x0fffffe9, 28},
> +    {0x0fffffea, 28}, {0x3ffffffd, 30}, {0x0fffffeb, 28}, {0x0fffffec, 28},
> +    {0x0fffffed, 28}, {0x0fffffee, 28}, {0x0fffffef, 28}, {0x0ffffff0, 28},
> +    {0x0ffffff1, 28}, {0x0ffffff2, 28}, {0x3ffffffe, 30}, {0x0ffffff3, 28},
> +    {0x0ffffff4, 28}, {0x0ffffff5, 28}, {0x0ffffff6, 28}, {0x0ffffff7, 28},
> +    {0x0ffffff8, 28}, {0x0ffffff9, 28}, {0x0ffffffa, 28}, {0x0ffffffb, 28},
> +    {0x00000014,  6}, {0x000003f8, 10}, {0x000003f9, 10}, {0x00000ffa, 12},
> +    {0x00001ff9, 13}, {0x00000015,  6}, {0x000000f8,  8}, {0x000007fa, 11},
> +    {0x000003fa, 10}, {0x000003fb, 10}, {0x000000f9,  8}, {0x000007fb, 11},
> +    {0x000000fa,  8}, {0x00000016,  6}, {0x00000017,  6}, {0x00000018,  6},
> +    {0x00000000,  5}, {0x00000001,  5}, {0x00000002,  5}, {0x00000019,  6},
> +    {0x0000001a,  6}, {0x0000001b,  6}, {0x0000001c,  6}, {0x0000001d,  6},
> +    {0x0000001e,  6}, {0x0000001f,  6}, {0x0000005c,  7}, {0x000000fb,  8},
> +    {0x00007ffc, 15}, {0x00000020,  6}, {0x00000ffb, 12}, {0x000003fc, 10},
> +    {0x00001ffa, 13}, {0x00000021,  6}, {0x0000005d,  7}, {0x0000005e,  7},
> +    {0x0000005f,  7}, {0x00000060,  7}, {0x00000061,  7}, {0x00000062,  7},
> +    {0x00000063,  7}, {0x00000064,  7}, {0x00000065,  7}, {0x00000066,  7},
> +    {0x00000067,  7}, {0x00000068,  7}, {0x00000069,  7}, {0x0000006a,  7},
> +    {0x0000006b,  7}, {0x0000006c,  7}, {0x0000006d,  7}, {0x0000006e,  7},
> +    {0x0000006f,  7}, {0x00000070,  7}, {0x00000071,  7}, {0x00000072,  7},
> +    {0x000000fc,  8}, {0x00000073,  7}, {0x000000fd,  8}, {0x00001ffb, 13},
> +    {0x0007fff0, 19}, {0x00001ffc, 13}, {0x00003ffc, 14}, {0x00000022,  6},
> +    {0x00007ffd, 15}, {0x00000003,  5}, {0x00000023,  6}, {0x00000004,  5},
> +    {0x00000024,  6}, {0x00000005,  5}, {0x00000025,  6}, {0x00000026,  6},
> +    {0x00000027,  6}, {0x00000006,  5}, {0x00000074,  7}, {0x00000075,  7},
> +    {0x00000028,  6}, {0x00000029,  6}, {0x0000002a,  6}, {0x00000007,  5},
> +    {0x0000002b,  6}, {0x00000076,  7}, {0x0000002c,  6}, {0x00000008,  5},
> +    {0x00000009,  5}, {0x0000002d,  6}, {0x00000077,  7}, {0x00000078,  7},
> +    {0x00000079,  7}, {0x0000007a,  7}, {0x0000007b,  7}, {0x00007ffe, 15},
> +    {0x000007fc, 11}, {0x00003ffd, 14}, {0x00001ffd, 13}, {0x0ffffffc, 28},
> +    {0x000fffe6, 20}, {0x003fffd2, 22}, {0x000fffe7, 20}, {0x000fffe8, 20},
> +    {0x003fffd3, 22}, {0x003fffd4, 22}, {0x003fffd5, 22}, {0x007fffd9, 23},
> +    {0x003fffd6, 22}, {0x007fffda, 23}, {0x007fffdb, 23}, {0x007fffdc, 23},
> +    {0x007fffdd, 23}, {0x007fffde, 23}, {0x00ffffeb, 24}, {0x007fffdf, 23},
> +    {0x00ffffec, 24}, {0x00ffffed, 24}, {0x003fffd7, 22}, {0x007fffe0, 23},
> +    {0x00ffffee, 24}, {0x007fffe1, 23}, {0x007fffe2, 23}, {0x007fffe3, 23},
> +    {0x007fffe4, 23}, {0x001fffdc, 21}, {0x003fffd8, 22}, {0x007fffe5, 23},
> +    {0x003fffd9, 22}, {0x007fffe6, 23}, {0x007fffe7, 23}, {0x00ffffef, 24},
> +    {0x003fffda, 22}, {0x001fffdd, 21}, {0x000fffe9, 20}, {0x003fffdb, 22},
> +    {0x003fffdc, 22}, {0x007fffe8, 23}, {0x007fffe9, 23}, {0x001fffde, 21},
> +    {0x007fffea, 23}, {0x003fffdd, 22}, {0x003fffde, 22}, {0x00fffff0, 24},
> +    {0x001fffdf, 21}, {0x003fffdf, 22}, {0x007fffeb, 23}, {0x007fffec, 23},
> +    {0x001fffe0, 21}, {0x001fffe1, 21}, {0x003fffe0, 22}, {0x001fffe2, 21},
> +    {0x007fffed, 23}, {0x003fffe1, 22}, {0x007fffee, 23}, {0x007fffef, 23},
> +    {0x000fffea, 20}, {0x003fffe2, 22}, {0x003fffe3, 22}, {0x003fffe4, 22},
> +    {0x007ffff0, 23}, {0x003fffe5, 22}, {0x003fffe6, 22}, {0x007ffff1, 23},
> +    {0x03ffffe0, 26}, {0x03ffffe1, 26}, {0x000fffeb, 20}, {0x0007fff1, 19},
> +    {0x003fffe7, 22}, {0x007ffff2, 23}, {0x003fffe8, 22}, {0x01ffffec, 25},
> +    {0x03ffffe2, 26}, {0x03ffffe3, 26}, {0x03ffffe4, 26}, {0x07ffffde, 27},
> +    {0x07ffffdf, 27}, {0x03ffffe5, 26}, {0x00fffff1, 24}, {0x01ffffed, 25},
> +    {0x0007fff2, 19}, {0x001fffe3, 21}, {0x03ffffe6, 26}, {0x07ffffe0, 27},
> +    {0x07ffffe1, 27}, {0x03ffffe7, 26}, {0x07ffffe2, 27}, {0x00fffff2, 24},
> +    {0x001fffe4, 21}, {0x001fffe5, 21}, {0x03ffffe8, 26}, {0x03ffffe9, 26},
> +    {0x0ffffffd, 28}, {0x07ffffe3, 27}, {0x07ffffe4, 27}, {0x07ffffe5, 27},
> +    {0x000fffec, 20}, {0x00fffff3, 24}, {0x000fffed, 20}, {0x001fffe6, 21},
> +    {0x003fffe9, 22}, {0x001fffe7, 21}, {0x001fffe8, 21}, {0x007ffff3, 23},
> +    {0x003fffea, 22}, {0x003fffeb, 22}, {0x01ffffee, 25}, {0x01ffffef, 25},
> +    {0x00fffff4, 24}, {0x00fffff5, 24}, {0x03ffffea, 26}, {0x007ffff4, 23},
> +    {0x03ffffeb, 26}, {0x07ffffe6, 27}, {0x03ffffec, 26}, {0x03ffffed, 26},
> +    {0x07ffffe7, 27}, {0x07ffffe8, 27}, {0x07ffffe9, 27}, {0x07ffffea, 27},
> +    {0x07ffffeb, 27}, {0x0ffffffe, 28}, {0x07ffffec, 27}, {0x07ffffed, 27},
> +    {0x07ffffee, 27}, {0x07ffffef, 27}, {0x07fffff0, 27}, {0x03ffffee, 26}
> +};
> +
> +
> +/* Same as above, but embedes to lower case transformations */

Style nit: either add dot at the end of sentence or lowercase the first letter.

Also grammar: s/embedes/embeds/


> +static ngx_http_v2_huff_encode_code_t  ngx_http_v2_huff_encode_codes_low[256] =
> +{
> +    {0x00001ff8, 13}, {0x007fffd8, 23}, {0x0fffffe2, 28}, {0x0fffffe3, 28},
> +    {0x0fffffe4, 28}, {0x0fffffe5, 28}, {0x0fffffe6, 28}, {0x0fffffe7, 28},
> +    {0x0fffffe8, 28}, {0x00ffffea, 24}, {0x3ffffffc, 30}, {0x0fffffe9, 28},
> +    {0x0fffffea, 28}, {0x3ffffffd, 30}, {0x0fffffeb, 28}, {0x0fffffec, 28},
> +    {0x0fffffed, 28}, {0x0fffffee, 28}, {0x0fffffef, 28}, {0x0ffffff0, 28},
> +    {0x0ffffff1, 28}, {0x0ffffff2, 28}, {0x3ffffffe, 30}, {0x0ffffff3, 28},
> +    {0x0ffffff4, 28}, {0x0ffffff5, 28}, {0x0ffffff6, 28}, {0x0ffffff7, 28},
> +    {0x0ffffff8, 28}, {0x0ffffff9, 28}, {0x0ffffffa, 28}, {0x0ffffffb, 28},
> +    {0x00000014,  6}, {0x000003f8, 10}, {0x000003f9, 10}, {0x00000ffa, 12},
> +    {0x00001ff9, 13}, {0x00000015,  6}, {0x000000f8,  8}, {0x000007fa, 11},
> +    {0x000003fa, 10}, {0x000003fb, 10}, {0x000000f9,  8}, {0x000007fb, 11},
> +    {0x000000fa,  8}, {0x00000016,  6}, {0x00000017,  6}, {0x00000018,  6},
> +    {0x00000000,  5}, {0x00000001,  5}, {0x00000002,  5}, {0x00000019,  6},
> +    {0x0000001a,  6}, {0x0000001b,  6}, {0x0000001c,  6}, {0x0000001d,  6},
> +    {0x0000001e,  6}, {0x0000001f,  6}, {0x0000005c,  7}, {0x000000fb,  8},
> +    {0x00007ffc, 15}, {0x00000020,  6}, {0x00000ffb, 12}, {0x000003fc, 10},
> +    {0x00001ffa, 13}, {0x00000003,  5}, {0x00000023,  6}, {0x00000004,  5},
> +    {0x00000024,  6}, {0x00000005,  5}, {0x00000025,  6}, {0x00000026,  6},
> +    {0x00000027,  6}, {0x00000006,  5}, {0x00000074,  7}, {0x00000075,  7},
> +    {0x00000028,  6}, {0x00000029,  6}, {0x0000002a,  6}, {0x00000007,  5},
> +    {0x0000002b,  6}, {0x00000076,  7}, {0x0000002c,  6}, {0x00000008,  5},
> +    {0x00000009,  5}, {0x0000002d,  6}, {0x00000077,  7}, {0x00000078,  7},
> +    {0x00000079,  7}, {0x0000007a,  7}, {0x0000007b,  7}, {0x00001ffb, 13},
> +    {0x0007fff0, 19}, {0x00001ffc, 13}, {0x00003ffc, 14}, {0x00000022,  6},
> +    {0x00007ffd, 15}, {0x00000003,  5}, {0x00000023,  6}, {0x00000004,  5},
> +    {0x00000024,  6}, {0x00000005,  5}, {0x00000025,  6}, {0x00000026,  6},
> +    {0x00000027,  6}, {0x00000006,  5}, {0x00000074,  7}, {0x00000075,  7},
> +    {0x00000028,  6}, {0x00000029,  6}, {0x0000002a,  6}, {0x00000007,  5},
> +    {0x0000002b,  6}, {0x00000076,  7}, {0x0000002c,  6}, {0x00000008,  5},
> +    {0x00000009,  5}, {0x0000002d,  6}, {0x00000077,  7}, {0x00000078,  7},
> +    {0x00000079,  7}, {0x0000007a,  7}, {0x0000007b,  7}, {0x00007ffe, 15},
> +    {0x000007fc, 11}, {0x00003ffd, 14}, {0x00001ffd, 13}, {0x0ffffffc, 28},
> +    {0x000fffe6, 20}, {0x003fffd2, 22}, {0x000fffe7, 20}, {0x000fffe8, 20},
> +    {0x003fffd3, 22}, {0x003fffd4, 22}, {0x003fffd5, 22}, {0x007fffd9, 23},
> +    {0x003fffd6, 22}, {0x007fffda, 23}, {0x007fffdb, 23}, {0x007fffdc, 23},
> +    {0x007fffdd, 23}, {0x007fffde, 23}, {0x00ffffeb, 24}, {0x007fffdf, 23},
> +    {0x00ffffec, 24}, {0x00ffffed, 24}, {0x003fffd7, 22}, {0x007fffe0, 23},
> +    {0x00ffffee, 24}, {0x007fffe1, 23}, {0x007fffe2, 23}, {0x007fffe3, 23},
> +    {0x007fffe4, 23}, {0x001fffdc, 21}, {0x003fffd8, 22}, {0x007fffe5, 23},
> +    {0x003fffd9, 22}, {0x007fffe6, 23}, {0x007fffe7, 23}, {0x00ffffef, 24},
> +    {0x003fffda, 22}, {0x001fffdd, 21}, {0x000fffe9, 20}, {0x003fffdb, 22},
> +    {0x003fffdc, 22}, {0x007fffe8, 23}, {0x007fffe9, 23}, {0x001fffde, 21},
> +    {0x007fffea, 23}, {0x003fffdd, 22}, {0x003fffde, 22}, {0x00fffff0, 24},
> +    {0x001fffdf, 21}, {0x003fffdf, 22}, {0x007fffeb, 23}, {0x007fffec, 23},
> +    {0x001fffe0, 21}, {0x001fffe1, 21}, {0x003fffe0, 22}, {0x001fffe2, 21},
> +    {0x007fffed, 23}, {0x003fffe1, 22}, {0x007fffee, 23}, {0x007fffef, 23},
> +    {0x000fffea, 20}, {0x003fffe2, 22}, {0x003fffe3, 22}, {0x003fffe4, 22},
> +    {0x007ffff0, 23}, {0x003fffe5, 22}, {0x003fffe6, 22}, {0x007ffff1, 23},
> +    {0x03ffffe0, 26}, {0x03ffffe1, 26}, {0x000fffeb, 20}, {0x0007fff1, 19},
> +    {0x003fffe7, 22}, {0x007ffff2, 23}, {0x003fffe8, 22}, {0x01ffffec, 25},
> +    {0x03ffffe2, 26}, {0x03ffffe3, 26}, {0x03ffffe4, 26}, {0x07ffffde, 27},
> +    {0x07ffffdf, 27}, {0x03ffffe5, 26}, {0x00fffff1, 24}, {0x01ffffed, 25},
> +    {0x0007fff2, 19}, {0x001fffe3, 21}, {0x03ffffe6, 26}, {0x07ffffe0, 27},
> +    {0x07ffffe1, 27}, {0x03ffffe7, 26}, {0x07ffffe2, 27}, {0x00fffff2, 24},
> +    {0x001fffe4, 21}, {0x001fffe5, 21}, {0x03ffffe8, 26}, {0x03ffffe9, 26},
> +    {0x0ffffffd, 28}, {0x07ffffe3, 27}, {0x07ffffe4, 27}, {0x07ffffe5, 27},
> +    {0x000fffec, 20}, {0x00fffff3, 24}, {0x000fffed, 20}, {0x001fffe6, 21},
> +    {0x003fffe9, 22}, {0x001fffe7, 21}, {0x001fffe8, 21}, {0x007ffff3, 23},
> +    {0x003fffea, 22}, {0x003fffeb, 22}, {0x01ffffee, 25}, {0x01ffffef, 25},
> +    {0x00fffff4, 24}, {0x00fffff5, 24}, {0x03ffffea, 26}, {0x007ffff4, 23},
> +    {0x03ffffeb, 26}, {0x07ffffe6, 27}, {0x03ffffec, 26}, {0x03ffffed, 26},
> +    {0x07ffffe7, 27}, {0x07ffffe8, 27}, {0x07ffffe9, 27}, {0x07ffffea, 27},
> +    {0x07ffffeb, 27}, {0x0ffffffe, 28}, {0x07ffffec, 27}, {0x07ffffed, 27},
> +    {0x07ffffee, 27}, {0x07ffffef, 27}, {0x07fffff0, 27}, {0x03ffffee, 26}
> +};
> +
> +
> +#if (NGX_HAVE_LITTLE_ENDIAN)
> +#if (__GNUC__)
> +
> +static void
> +put64(u_char *dst, uint64_t val)

Please use ngx_* prefix.


> +{
> +    *(uint64_t*)dst = __bswap_64(val);
> +}
> +
> +#else /* !__GNUC__ */
> +
> +static void
> +put64(u_char *dst, uint64_t val)
> +{
> +    dst[0] = val >> 56;
> +    dst[1] = val >> 48;
> +    dst[2] = val >> 40;
> +    dst[3] = val >> 32;
> +    dst[4] = val >> 24;
> +    dst[5] = val >> 16;
> +    dst[6] = val >> 8;
> +    dst[7] = val;
> +}
> +#endif /* __GNUC__ */
> +
> +#else /* !NGX_HAVE_LITTLE_ENDIAN */
> +
> +static void
> +put64(u_char *dst, uint64_t val)
> +{
> +    *(uint64_t*)dst = val;
> +}
> +
> +#endif

In nginx is common practice to use macros for such simple expressions.


> +
> +
> +static ngx_int_t
> +ngx_http_v2_huff_encode(u_char *src, size_t len, u_char *dst, ngx_log_t *log,
> +    ngx_flag_t lower)
> +{
> +    ngx_uint_t                      inp, outp;
> +    ngx_int_t                       pending;
> +    uint64_t                        buf;

IMHO it's better to accumulate up to 32-bits on 32 bits platforms.


> +    ngx_http_v2_huff_encode_code_t  next;
> +    ngx_http_v2_huff_encode_code_t *table;

Style nit.

In general, the order of variable declarations need to be arranged by
the length of their types.

Alignment rule here: two spaces between the longest type and asterisk
or (if no asterisk) the first letter of the name.

I've also changed the names of local variables to be more inline with
what usually are used in nginx for similar tasks.


> +
> +    if (!lower) {
> +        table = ngx_http_v2_huff_encode_codes;
> +
> +    } else {
> +        table = ngx_http_v2_huff_encode_codes_low;
> +    }

It's a good case for ternary operator.


> +
> +    inp = 0;
> +    outp = 0;
> +    pending = 0;
> +    buf = 0;
> +
> +    while (inp < len) {

Effectively it's "for (inp = 0; inp < len; inp++)".


> +        next = table[src[inp]];
> +        /* Accumulate 64 bits */
> +        if ((next.len + pending) < 64) {

Effectively it's "next.len < 64 - pending"


> +            buf ^= (uint64_t)next.code << (64 - pending - next.len);
> +            pending += next.len;
> +
> +        } else {
> +        /* If compressed result is longer than source, no point in using it */
> +            if ((outp + 8) >= len) {
> +                return NGX_ERROR;
> +            }
> +
> +            buf ^= (uint64_t)next.code >> (next.len - (64 - pending));
> +            put64(&dst[outp], buf);
> +            outp += 8;
> +            pending = (next.len - (64 - pending));

Up to this point you have 4 equal cases of "64 - pending".


> +
> +            if (pending) {
> +                buf = (uint64_t)next.code << (64 - pending);
> +
> +            } else {
> +                buf = 0;
> +            }

Another good case for ternary.


> +        }
> +
> +        inp++;
> +    }
> +
> +    buf ^= 0xffffffffffffffffUL >> pending;
> +
> +    while (pending > 0) {
> +        dst[outp] = buf >> 56;
> +        buf <<= 8;
> +        pending -= 8;
> +        outp++;
> +
> +        if (outp >= len) {
> +            return NGX_ERROR;
> +        }

It's better to move the check out of the cycle.


> +    }
> +
> +    ngx_log_debug2(NGX_LOG_DEBUG_HTTP, log, 0,
> +                   "http2 huffman encoding successful; "
> +                   "input len: %d, output len %d", len, outp);

IMHO the debug is surplus here.

This message is the only consumer for the log pointer.


> +
> +    return outp;
> +}
> +
> +
> +u_char *
> +ngx_http_v2_string_encode(u_char *src, size_t len, u_char *dst, u_char *tmp,
> +    ngx_log_t *log, ngx_flag_t lower)
> +{
> +    ngx_int_t hlen;

Style nit: two spaces between type and name.


> +    
   ^^^^
Style nit: spaces at the beginning of the empty line.


> +    hlen = ngx_http_v2_huff_encode(src, len, tmp, log, lower);
> +
> +    if (hlen != NGX_ERROR) {
> +        *dst = NGX_HTTP_V2_ENCODE_HUFF;
> +        dst = ngx_http_v2_write_int(dst, ngx_http_v2_prefix(7), hlen);
> +        dst = ngx_cpymem(dst, tmp, hlen);
> +
> +    } else {
> +        *dst = NGX_HTTP_V2_ENCODE_RAW;
> +        dst = ngx_http_v2_write_int(dst, ngx_http_v2_prefix(7), len);
> +
> +        if (lower) {
> +            ngx_strlow(dst, src, len);
> +            dst += len;
> +
> +        } else {
> +            dst = ngx_cpymem(dst, src, len);
> +        }
> +    }
> +
> +    return dst;
> +}
> 

See the patch in attachment where I reworked the compression
function to be more 32-bits friendly and fixed all the mentioned
problems.

  wbr, Valentin V. Bartenev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: http2_huff_encode.patch
Type: text/x-patch
Size: 25116 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160119/b70ace6a/attachment.bin>

From tigran.bayburtsyan at gmail.com  Wed Jan 20 08:05:51 2016
From: tigran.bayburtsyan at gmail.com (Tigran Bayburtsyan)
Date: Wed, 20 Jan 2016 12:05:51 +0400
Subject: Nginx cache filename
Message-ID: <003601d15359$61e1d580$25a58080$@gmail.com>

Hi All

I'm doing centralized service for clearing our servers Nginx cache for
specific files, but I can't find the algorithm how Nginx making cache file
based on request and cache_key.

The only thing in documentation is
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_path 

How can I get cache file path for this static file
http://example.com/static/test.png ?

 

Thanks

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160120/ad3f6951/attachment.html>

From daniel.biazus at azion.com  Wed Jan 20 19:11:34 2016
From: daniel.biazus at azion.com (Daniel Biazus)
Date: Wed, 20 Jan 2016 17:11:34 -0200
Subject: Nginx cache filename
In-Reply-To: <003601d15359$61e1d580$25a58080$@gmail.com>
References: <003601d15359$61e1d580$25a58080$@gmail.com>
Message-ID: <CAL9LgEMyUk=WVMXbHFrk5gCoxCN9W=yP2rUWrYAwTwhu2qSgaw@mail.gmail.com>

You may use the md5 function in order to get the cache file on filesystem,
Ex:

If you use the cache key "$host$uri$is_args$args" and the cache base path
is /var/cache/nginx/ and cache has 2 levels (levels=1:2)

Considering the URL "http://example.com/static/test.png"

The cache file name will be
/var/cache/nginx/5/43/6e281642dfbf0b9ebf4cfb30142ab435

Based on:

echo -n 'example.com/static/test.png' | md5sum
6e281642dfbf0b9ebf4cfb30142ab435

First level = 5
Second level = 43
Filename = 6e281642dfbf0b9ebf4cfb30142ab435


Hope it helps

On Wed, Jan 20, 2016 at 6:05 AM, Tigran Bayburtsyan <
tigran.bayburtsyan at gmail.com> wrote:

> Hi All
>
> I?m doing centralized service for clearing our servers Nginx cache for
> specific files, but I can?t find the algorithm how Nginx making cache file
> based on request and cache_key.
>
> The only thing in documentation is
> http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_path
>
> How can I get cache file path for this static file
> http://example.com/static/test.png ?
>
>
>
> Thanks
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>



-- 

Daniel Biazus
infrastructure Engineering
Azion Technologies
Porto Alegre, Brasil +55 51 3012 3005 | +55 51 82279032
Miami, USA +1 305 704 8816

Quaisquer informa??es contidas neste e-mail e anexos podem ser
confidenciais e privilegiadas, protegidas por sigilo legal. Qualquer forma
de utiliza??o deste documento depende de autoriza??o do emissor, sujeito as
penalidades cab?veis.

Any information in this e-mail and attachments may be confidential and
privileged, protected by legal confidentiality. The use of this document
require authorization by the issuer, subject to penalties.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160120/e8a2ae5a/attachment.html>

From vlad at cloudflare.com  Wed Jan 20 22:01:15 2016
From: vlad at cloudflare.com (Vlad Krasnov)
Date: Wed, 20 Jan 2016 14:01:15 -0800
Subject: [PATCH] HTTP/2: HPACK Huffman encoding
In-Reply-To: <2548150.3ZiQ2bDBD5@vbart-workstation>
References: <d672e9c2b81471098668.1450352972@ubuntu>
 <2548150.3ZiQ2bDBD5@vbart-workstation>
Message-ID: <284F77E2-2D4B-4BE5-A7F8-374C10EE36B4@cloudflare.com>

LGTM

Cheers,
Vlad

> On 19 Jan 2016, at 09:20, Valentin V. Bartenev <vbart at nginx.com> wrote:
> 
> On Thursday 17 December 2015 03:49:32 Vlad Krasnov wrote:
>> # HG changeset patch
>> # User Vlad Krasnov <vlad at cloudflare.com <mailto:vlad at cloudflare.com>>
>> # Date 1450274269 28800
>> #      Wed Dec 16 05:57:49 2015 -0800
>> # Node ID d672e9c2b814710986683e050491536355c90f0d
>> # Parent  def9c9c9ae05cfa7467b0ec96e76afa180c23dfb
>> HTTP/2: HPACK Huffman encoding
> 
> Style.  The dot at the end of summary line is needed.
> 
>> 
>> Implement HPACK Huffman encoding for HTTP/2.
>> This reduces the size of headers by over 30% on average.
> 
> See the comments below and a reworked patch in attachment.
> 
> 
>> 
>> diff -r def9c9c9ae05 -r d672e9c2b814 src/http/v2/ngx_http_v2.h
>> --- a/src/http/v2/ngx_http_v2.h	Sat Dec 12 10:32:58 2015 +0300
>> +++ b/src/http/v2/ngx_http_v2.h	Wed Dec 16 05:57:49 2015 -0800
>> @@ -51,6 +51,9 @@
>> #define NGX_HTTP_V2_PRIORITY_FLAG        0x20
>> 
>> 
>> +#define NGX_HTTP_V2_ENCODE_RAW            0
>> +#define NGX_HTTP_V2_ENCODE_HUFF           0x80
>> +
>> typedef struct ngx_http_v2_connection_s   ngx_http_v2_connection_t;
>> typedef struct ngx_http_v2_node_s         ngx_http_v2_node_t;
>> typedef struct ngx_http_v2_out_frame_s    ngx_http_v2_out_frame_t;
>> @@ -255,6 +258,28 @@
>> }
>> 
>> 
>> +static ngx_inline u_char *
>> +ngx_http_v2_write_int(u_char *pos, ngx_uint_t prefix, ngx_uint_t value)
>> +{
>> +    if (value < prefix) {
>> +        *pos++ |= value;
>> +        return pos;
>> +    }
>> +
>> +    *pos++ |= prefix;
>> +    value -= prefix;
>> +
>> +    while (value >= 128) {
>> +        *pos++ = value % 128 + 128;
>> +        value /= 128;
>> +    }
>> +
>> +    *pos++ = (u_char) value;
>> +
>> +    return pos;
>> +}
>> +
>> +
>> void ngx_http_v2_init(ngx_event_t *rev);
>> void ngx_http_v2_request_headers_init(void);
>> 
> [..]
> 
> It's better to put ngx_http_v2_string_encode() into the
> "ngx_http_v2_filter_module.c", than moving all these to
> the header file.
> 
> Also that will be more consistent and symmetric with the
> "ngx_http_v2_huff_decode.c".
> 
> 
> 
>> @@ -275,7 +300,8 @@
>> 
>> ngx_int_t ngx_http_v2_huff_decode(u_char *state, u_char *src, size_t len,
>>     u_char **dst, ngx_uint_t last, ngx_log_t *log);
>> -
>> +u_char *ngx_http_v2_string_encode(u_char *src, size_t len, u_char *dst,
>> +    u_char *tmp, ngx_log_t *log, ngx_flag_t lower);
>> 
>> #define ngx_http_v2_prefix(bits)  ((1 << (bits)) - 1)
>> 
>> diff -r def9c9c9ae05 -r d672e9c2b814 src/http/v2/ngx_http_v2_filter_module.c
>> --- a/src/http/v2/ngx_http_v2_filter_module.c	Sat Dec 12 10:32:58 2015 +0300
>> +++ b/src/http/v2/ngx_http_v2_filter_module.c	Wed Dec 16 05:57:49 2015 -0800
>> @@ -25,9 +25,6 @@
>> #define ngx_http_v2_indexed(i)      (128 + (i))
>> #define ngx_http_v2_inc_indexed(i)  (64 + (i))
>> 
>> -#define NGX_HTTP_V2_ENCODE_RAW            0
>> -#define NGX_HTTP_V2_ENCODE_HUFF           0x80
>> -
>> #define NGX_HTTP_V2_STATUS_INDEX          8
>> #define NGX_HTTP_V2_STATUS_200_INDEX      8
>> #define NGX_HTTP_V2_STATUS_204_INDEX      9
>> @@ -46,8 +43,9 @@
>> #define NGX_HTTP_V2_VARY_INDEX            59
>> 
>> 
>> -static u_char *ngx_http_v2_write_int(u_char *pos, ngx_uint_t prefix,
>> -    ngx_uint_t value);
>> +#define ACCEPT_ENCODING  "\x84\x84\x2d\x69\x5b\x05\x44\x3c\x86\xaa\x6f"
>> +
>> +
> 
> You should either add NGX_* prefix or define it as a local
> static constant.
> 
> 
> 
>> static ngx_http_v2_out_frame_t *ngx_http_v2_create_headers_frame(
>>     ngx_http_request_t *r, u_char *pos, u_char *end);
>> 
>> @@ -119,8 +117,8 @@
>> static ngx_int_t
>> ngx_http_v2_header_filter(ngx_http_request_t *r)
>> {
>> -    u_char                     status, *pos, *start, *p;
>> -    size_t                     len;
>> +    u_char                     status, *pos, *start, *p, *huff;
>> +    size_t                     len, hlen, nlen;
>>     ngx_str_t                  host, location;
>>     ngx_uint_t                 i, port;
>>     ngx_list_part_t           *part;
>> @@ -343,7 +341,7 @@
>> #if (NGX_HTTP_GZIP)
>>     if (r->gzip_vary) {
>>         if (clcf->gzip_vary) {
>> -            len += 1 + ngx_http_v2_literal_size("Accept-Encoding");
>> +            len += 1 + ngx_http_v2_literal_size(ACCEPT_ENCODING);
>> 
>>         } else {
>>             r->gzip_vary = 0;
>> @@ -354,6 +352,8 @@
>>     part = &r->headers_out.headers.part;
>>     header = part->elts;
>> 
>> +    hlen = len;
>> +
>>     for (i = 0; /* void */; i++) {
>> 
>>         if (i >= part->nelts) {
>> @@ -384,11 +384,14 @@
>>             return NGX_ERROR;
>>         }
>> 
>> -        len += 1 + NGX_HTTP_V2_INT_OCTETS + header[i].key.len
>> +        nlen = 1 + NGX_HTTP_V2_INT_OCTETS + header[i].key.len
>>                  + NGX_HTTP_V2_INT_OCTETS + header[i].value.len;
>> +        len += nlen;
>> +        hlen = nlen > hlen ? nlen : hlen;
> 
> I don't understand why you are comparing hlen with the sum of
> header name and value including the type byte.
> 
> Headers names and values are encoded separately.
> 
> 
>>     }
>> 
>>     pos = ngx_palloc(r->pool, len);
>> +    huff = ngx_palloc(r->pool, hlen);
>>     if (pos == NULL) {
>>         return NGX_ERROR;
>>     }
> 
> The "huff" can be NULL which will result in segfault.
> 
> 
> 
>> @@ -408,12 +411,15 @@
>>         *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_SERVER_INDEX);
>> 
>>         if (clcf->server_tokens) {
>> -            *pos++ = NGX_HTTP_V2_ENCODE_RAW | (sizeof(NGINX_VER) - 1);
>> -            pos = ngx_cpymem(pos, NGINX_VER, sizeof(NGINX_VER) - 1);
>> +            pos = ngx_http_v2_string_encode((u_char*)NGINX_VER,
>> +                                            sizeof(NGINX_VER) - 1, pos, huff,
>> +                                            r->connection->log, 0);
>> 
>>         } else {
>> -            *pos++ = NGX_HTTP_V2_ENCODE_RAW | (sizeof("nginx") - 1);
>> -            pos = ngx_cpymem(pos, "nginx", sizeof("nginx") - 1);
>> +            pos = ngx_http_v2_string_encode((u_char*)"nginx",
>> +                                            sizeof("nginx") - 1, pos, huff,
>> +                                            r->connection->log, 0);
> 
> The "nginx" constant can also be preencoded like "Accept-Encoding".
> 
> 
>> +
> 
> Style nit.  A surplus empty line.
> 
> 
>>         }
>>     }
>> 
>> @@ -453,11 +459,9 @@
>>             r->headers_out.content_type.data = p;
>> 
>>         } else {
> 
> Why don't you try to encode "Content-Type" in other case,
> when the condition is true?
> 
> The patch looks also incomplete without the encoding of
> "Date" header (it's simple to implement and quite useful).
> 
> 
>> -            *pos = NGX_HTTP_V2_ENCODE_RAW;
>> -            pos = ngx_http_v2_write_int(pos, ngx_http_v2_prefix(7),
>> -                                        r->headers_out.content_type.len);
>> -            pos = ngx_cpymem(pos, r->headers_out.content_type.data,
>> -                             r->headers_out.content_type.len);
>> +            pos = ngx_http_v2_string_encode(r->headers_out.content_type.data,
>> +                                            r->headers_out.content_type.len,
>> +                                            pos, huff, r->connection->log, 0);
>>         }
>>     }
>> 
>> @@ -476,26 +480,27 @@
>>     {
>>         *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_LAST_MODIFIED_INDEX);
>> 
>> -        *pos++ = NGX_HTTP_V2_ENCODE_RAW
>> -                 | (sizeof("Wed, 31 Dec 1986 18:00:00 GMT") - 1);
>> -        pos = ngx_http_time(pos, r->headers_out.last_modified_time);
>> +        nlen = sizeof("Wed, 31 Dec 1986 18:00:00 GMT") - 1;
>> +        ngx_http_time(pos + 1, r->headers_out.last_modified_time);
>> +
>> +        pos = ngx_http_v2_string_encode(pos + 1, nlen, pos, huff,
>> +                                        r->connection->log, 0);
> 
> "pos + 1" looks wrong here.
> 
> Also the safety of using *pos as source and destination requires
> a comment.
> 
> 
>>     }
>> 
>>     if (r->headers_out.location && r->headers_out.location->value.len) {
>>         *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_LOCATION_INDEX);
>> 
>>         *pos = NGX_HTTP_V2_ENCODE_RAW;
> 
> This assignment after the change below is surplus and misleading.
> 
> 
>> -        pos = ngx_http_v2_write_int(pos, ngx_http_v2_prefix(7),
>> -                                    r->headers_out.location->value.len);
>> -        pos = ngx_cpymem(pos, r->headers_out.location->value.data,
>> -                              r->headers_out.location->value.len);
>> +        pos = ngx_http_v2_string_encode(r->headers_out.location->value.data,
>> +                                        r->headers_out.location->value.len,
>> +                                        pos, huff, r->connection->log, 0);
>>     }
>> 
>> #if (NGX_HTTP_GZIP)
>>     if (r->gzip_vary) {
>>         *pos++ = ngx_http_v2_inc_indexed(NGX_HTTP_V2_VARY_INDEX);
>> -        *pos++ = NGX_HTTP_V2_ENCODE_RAW | (sizeof("Accept-Encoding") - 1);
>> -        pos = ngx_cpymem(pos, "Accept-Encoding", sizeof("Accept-Encoding") - 1);
>> +        *pos++ = NGX_HTTP_V2_ENCODE_HUFF | (sizeof(ACCEPT_ENCODING) - 1);
>> +        pos = ngx_cpymem(pos, ACCEPT_ENCODING, sizeof(ACCEPT_ENCODING) - 1);
>>     }
>> #endif
>> 
>> @@ -520,16 +525,14 @@
>> 
>>         *pos++ = 0;
>> 
>> -        *pos = NGX_HTTP_V2_ENCODE_RAW;
>> -        pos = ngx_http_v2_write_int(pos, ngx_http_v2_prefix(7),
>> -                                    header[i].key.len);
>> -        ngx_strlow(pos, header[i].key.data, header[i].key.len);
>> -        pos += header[i].key.len;
>> +        pos = ngx_http_v2_string_encode(header[i].key.data,
>> +                                        header[i].key.len,
>> +                                        pos, huff, r->connection->log, 1);
>> 
>> -        *pos = NGX_HTTP_V2_ENCODE_RAW;
>> -        pos = ngx_http_v2_write_int(pos, ngx_http_v2_prefix(7),
>> -                                    header[i].value.len);
>> -        pos = ngx_cpymem(pos, header[i].value.data, header[i].value.len);
>> +        pos = ngx_http_v2_string_encode(header[i].value.data,
>> +                                        header[i].value.len,
>> +                                        pos, huff, r->connection->log, 0);
>> +
>>     }
>> 
>>     frame = ngx_http_v2_create_headers_frame(r, start, pos);
>> @@ -556,28 +559,6 @@
>> }
>> 
>> 
>> -static u_char *
>> -ngx_http_v2_write_int(u_char *pos, ngx_uint_t prefix, ngx_uint_t value)
>> -{
>> -    if (value < prefix) {
>> -        *pos++ |= value;
>> -        return pos;
>> -    }
>> -
>> -    *pos++ |= prefix;
>> -    value -= prefix;
>> -
>> -    while (value >= 128) {
>> -        *pos++ = value % 128 + 128;
>> -        value /= 128;
>> -    }
>> -
>> -    *pos++ = (u_char) value;
>> -
>> -    return pos;
>> -}
>> -
>> -
>> static ngx_http_v2_out_frame_t *
>> ngx_http_v2_create_headers_frame(ngx_http_request_t *r, u_char *pos,
>>     u_char *end)
>> diff -r def9c9c9ae05 -r d672e9c2b814 src/http/v2/ngx_http_v2_huff_encode.c
>> --- a/src/http/v2/ngx_http_v2_huff_encode.c	Sat Dec 12 10:32:58 2015 +0300
>> +++ b/src/http/v2/ngx_http_v2_huff_encode.c	Wed Dec 16 05:57:49 2015 -0800
>> @@ -1,10 +1,296 @@
>> 
>> /*
>>  * Copyright (C) Nginx, Inc.
>> - * Copyright (C) Valentin V. Bartenev
>> + * Copyright (C) Vlad Krasnov
>>  */
>> 
>> 
>> #include <ngx_config.h>
>> #include <ngx_core.h>
>> #include <ngx_http.h>
>> +#if (__GNUC__)
>> +#include <byteswap.h>
>> +#endif
> 
> IMHO from portability point of view it's better to check the presence
> of required builtin than to relate on __GNUC__.
> 
> 
>> +
>> +
>> +typedef struct {
>> +    ngx_uint_t  code;
>> +    ngx_uint_t  len;
>> +} ngx_http_v2_huff_encode_code_t;
> 
> It's better to keep compact these tables.
> There are no need in 64 bits integers here.
> 
> 
>> +
>> +
>> +static ngx_http_v2_huff_encode_code_t  ngx_http_v2_huff_encode_codes[256] =
>> +{
>> +    {0x00001ff8, 13}, {0x007fffd8, 23}, {0x0fffffe2, 28}, {0x0fffffe3, 28},
>> +    {0x0fffffe4, 28}, {0x0fffffe5, 28}, {0x0fffffe6, 28}, {0x0fffffe7, 28},
>> +    {0x0fffffe8, 28}, {0x00ffffea, 24}, {0x3ffffffc, 30}, {0x0fffffe9, 28},
>> +    {0x0fffffea, 28}, {0x3ffffffd, 30}, {0x0fffffeb, 28}, {0x0fffffec, 28},
>> +    {0x0fffffed, 28}, {0x0fffffee, 28}, {0x0fffffef, 28}, {0x0ffffff0, 28},
>> +    {0x0ffffff1, 28}, {0x0ffffff2, 28}, {0x3ffffffe, 30}, {0x0ffffff3, 28},
>> +    {0x0ffffff4, 28}, {0x0ffffff5, 28}, {0x0ffffff6, 28}, {0x0ffffff7, 28},
>> +    {0x0ffffff8, 28}, {0x0ffffff9, 28}, {0x0ffffffa, 28}, {0x0ffffffb, 28},
>> +    {0x00000014,  6}, {0x000003f8, 10}, {0x000003f9, 10}, {0x00000ffa, 12},
>> +    {0x00001ff9, 13}, {0x00000015,  6}, {0x000000f8,  8}, {0x000007fa, 11},
>> +    {0x000003fa, 10}, {0x000003fb, 10}, {0x000000f9,  8}, {0x000007fb, 11},
>> +    {0x000000fa,  8}, {0x00000016,  6}, {0x00000017,  6}, {0x00000018,  6},
>> +    {0x00000000,  5}, {0x00000001,  5}, {0x00000002,  5}, {0x00000019,  6},
>> +    {0x0000001a,  6}, {0x0000001b,  6}, {0x0000001c,  6}, {0x0000001d,  6},
>> +    {0x0000001e,  6}, {0x0000001f,  6}, {0x0000005c,  7}, {0x000000fb,  8},
>> +    {0x00007ffc, 15}, {0x00000020,  6}, {0x00000ffb, 12}, {0x000003fc, 10},
>> +    {0x00001ffa, 13}, {0x00000021,  6}, {0x0000005d,  7}, {0x0000005e,  7},
>> +    {0x0000005f,  7}, {0x00000060,  7}, {0x00000061,  7}, {0x00000062,  7},
>> +    {0x00000063,  7}, {0x00000064,  7}, {0x00000065,  7}, {0x00000066,  7},
>> +    {0x00000067,  7}, {0x00000068,  7}, {0x00000069,  7}, {0x0000006a,  7},
>> +    {0x0000006b,  7}, {0x0000006c,  7}, {0x0000006d,  7}, {0x0000006e,  7},
>> +    {0x0000006f,  7}, {0x00000070,  7}, {0x00000071,  7}, {0x00000072,  7},
>> +    {0x000000fc,  8}, {0x00000073,  7}, {0x000000fd,  8}, {0x00001ffb, 13},
>> +    {0x0007fff0, 19}, {0x00001ffc, 13}, {0x00003ffc, 14}, {0x00000022,  6},
>> +    {0x00007ffd, 15}, {0x00000003,  5}, {0x00000023,  6}, {0x00000004,  5},
>> +    {0x00000024,  6}, {0x00000005,  5}, {0x00000025,  6}, {0x00000026,  6},
>> +    {0x00000027,  6}, {0x00000006,  5}, {0x00000074,  7}, {0x00000075,  7},
>> +    {0x00000028,  6}, {0x00000029,  6}, {0x0000002a,  6}, {0x00000007,  5},
>> +    {0x0000002b,  6}, {0x00000076,  7}, {0x0000002c,  6}, {0x00000008,  5},
>> +    {0x00000009,  5}, {0x0000002d,  6}, {0x00000077,  7}, {0x00000078,  7},
>> +    {0x00000079,  7}, {0x0000007a,  7}, {0x0000007b,  7}, {0x00007ffe, 15},
>> +    {0x000007fc, 11}, {0x00003ffd, 14}, {0x00001ffd, 13}, {0x0ffffffc, 28},
>> +    {0x000fffe6, 20}, {0x003fffd2, 22}, {0x000fffe7, 20}, {0x000fffe8, 20},
>> +    {0x003fffd3, 22}, {0x003fffd4, 22}, {0x003fffd5, 22}, {0x007fffd9, 23},
>> +    {0x003fffd6, 22}, {0x007fffda, 23}, {0x007fffdb, 23}, {0x007fffdc, 23},
>> +    {0x007fffdd, 23}, {0x007fffde, 23}, {0x00ffffeb, 24}, {0x007fffdf, 23},
>> +    {0x00ffffec, 24}, {0x00ffffed, 24}, {0x003fffd7, 22}, {0x007fffe0, 23},
>> +    {0x00ffffee, 24}, {0x007fffe1, 23}, {0x007fffe2, 23}, {0x007fffe3, 23},
>> +    {0x007fffe4, 23}, {0x001fffdc, 21}, {0x003fffd8, 22}, {0x007fffe5, 23},
>> +    {0x003fffd9, 22}, {0x007fffe6, 23}, {0x007fffe7, 23}, {0x00ffffef, 24},
>> +    {0x003fffda, 22}, {0x001fffdd, 21}, {0x000fffe9, 20}, {0x003fffdb, 22},
>> +    {0x003fffdc, 22}, {0x007fffe8, 23}, {0x007fffe9, 23}, {0x001fffde, 21},
>> +    {0x007fffea, 23}, {0x003fffdd, 22}, {0x003fffde, 22}, {0x00fffff0, 24},
>> +    {0x001fffdf, 21}, {0x003fffdf, 22}, {0x007fffeb, 23}, {0x007fffec, 23},
>> +    {0x001fffe0, 21}, {0x001fffe1, 21}, {0x003fffe0, 22}, {0x001fffe2, 21},
>> +    {0x007fffed, 23}, {0x003fffe1, 22}, {0x007fffee, 23}, {0x007fffef, 23},
>> +    {0x000fffea, 20}, {0x003fffe2, 22}, {0x003fffe3, 22}, {0x003fffe4, 22},
>> +    {0x007ffff0, 23}, {0x003fffe5, 22}, {0x003fffe6, 22}, {0x007ffff1, 23},
>> +    {0x03ffffe0, 26}, {0x03ffffe1, 26}, {0x000fffeb, 20}, {0x0007fff1, 19},
>> +    {0x003fffe7, 22}, {0x007ffff2, 23}, {0x003fffe8, 22}, {0x01ffffec, 25},
>> +    {0x03ffffe2, 26}, {0x03ffffe3, 26}, {0x03ffffe4, 26}, {0x07ffffde, 27},
>> +    {0x07ffffdf, 27}, {0x03ffffe5, 26}, {0x00fffff1, 24}, {0x01ffffed, 25},
>> +    {0x0007fff2, 19}, {0x001fffe3, 21}, {0x03ffffe6, 26}, {0x07ffffe0, 27},
>> +    {0x07ffffe1, 27}, {0x03ffffe7, 26}, {0x07ffffe2, 27}, {0x00fffff2, 24},
>> +    {0x001fffe4, 21}, {0x001fffe5, 21}, {0x03ffffe8, 26}, {0x03ffffe9, 26},
>> +    {0x0ffffffd, 28}, {0x07ffffe3, 27}, {0x07ffffe4, 27}, {0x07ffffe5, 27},
>> +    {0x000fffec, 20}, {0x00fffff3, 24}, {0x000fffed, 20}, {0x001fffe6, 21},
>> +    {0x003fffe9, 22}, {0x001fffe7, 21}, {0x001fffe8, 21}, {0x007ffff3, 23},
>> +    {0x003fffea, 22}, {0x003fffeb, 22}, {0x01ffffee, 25}, {0x01ffffef, 25},
>> +    {0x00fffff4, 24}, {0x00fffff5, 24}, {0x03ffffea, 26}, {0x007ffff4, 23},
>> +    {0x03ffffeb, 26}, {0x07ffffe6, 27}, {0x03ffffec, 26}, {0x03ffffed, 26},
>> +    {0x07ffffe7, 27}, {0x07ffffe8, 27}, {0x07ffffe9, 27}, {0x07ffffea, 27},
>> +    {0x07ffffeb, 27}, {0x0ffffffe, 28}, {0x07ffffec, 27}, {0x07ffffed, 27},
>> +    {0x07ffffee, 27}, {0x07ffffef, 27}, {0x07fffff0, 27}, {0x03ffffee, 26}
>> +};
>> +
>> +
>> +/* Same as above, but embedes to lower case transformations */
> 
> Style nit: either add dot at the end of sentence or lowercase the first letter.
> 
> Also grammar: s/embedes/embeds/
> 
> 
>> +static ngx_http_v2_huff_encode_code_t  ngx_http_v2_huff_encode_codes_low[256] =
>> +{
>> +    {0x00001ff8, 13}, {0x007fffd8, 23}, {0x0fffffe2, 28}, {0x0fffffe3, 28},
>> +    {0x0fffffe4, 28}, {0x0fffffe5, 28}, {0x0fffffe6, 28}, {0x0fffffe7, 28},
>> +    {0x0fffffe8, 28}, {0x00ffffea, 24}, {0x3ffffffc, 30}, {0x0fffffe9, 28},
>> +    {0x0fffffea, 28}, {0x3ffffffd, 30}, {0x0fffffeb, 28}, {0x0fffffec, 28},
>> +    {0x0fffffed, 28}, {0x0fffffee, 28}, {0x0fffffef, 28}, {0x0ffffff0, 28},
>> +    {0x0ffffff1, 28}, {0x0ffffff2, 28}, {0x3ffffffe, 30}, {0x0ffffff3, 28},
>> +    {0x0ffffff4, 28}, {0x0ffffff5, 28}, {0x0ffffff6, 28}, {0x0ffffff7, 28},
>> +    {0x0ffffff8, 28}, {0x0ffffff9, 28}, {0x0ffffffa, 28}, {0x0ffffffb, 28},
>> +    {0x00000014,  6}, {0x000003f8, 10}, {0x000003f9, 10}, {0x00000ffa, 12},
>> +    {0x00001ff9, 13}, {0x00000015,  6}, {0x000000f8,  8}, {0x000007fa, 11},
>> +    {0x000003fa, 10}, {0x000003fb, 10}, {0x000000f9,  8}, {0x000007fb, 11},
>> +    {0x000000fa,  8}, {0x00000016,  6}, {0x00000017,  6}, {0x00000018,  6},
>> +    {0x00000000,  5}, {0x00000001,  5}, {0x00000002,  5}, {0x00000019,  6},
>> +    {0x0000001a,  6}, {0x0000001b,  6}, {0x0000001c,  6}, {0x0000001d,  6},
>> +    {0x0000001e,  6}, {0x0000001f,  6}, {0x0000005c,  7}, {0x000000fb,  8},
>> +    {0x00007ffc, 15}, {0x00000020,  6}, {0x00000ffb, 12}, {0x000003fc, 10},
>> +    {0x00001ffa, 13}, {0x00000003,  5}, {0x00000023,  6}, {0x00000004,  5},
>> +    {0x00000024,  6}, {0x00000005,  5}, {0x00000025,  6}, {0x00000026,  6},
>> +    {0x00000027,  6}, {0x00000006,  5}, {0x00000074,  7}, {0x00000075,  7},
>> +    {0x00000028,  6}, {0x00000029,  6}, {0x0000002a,  6}, {0x00000007,  5},
>> +    {0x0000002b,  6}, {0x00000076,  7}, {0x0000002c,  6}, {0x00000008,  5},
>> +    {0x00000009,  5}, {0x0000002d,  6}, {0x00000077,  7}, {0x00000078,  7},
>> +    {0x00000079,  7}, {0x0000007a,  7}, {0x0000007b,  7}, {0x00001ffb, 13},
>> +    {0x0007fff0, 19}, {0x00001ffc, 13}, {0x00003ffc, 14}, {0x00000022,  6},
>> +    {0x00007ffd, 15}, {0x00000003,  5}, {0x00000023,  6}, {0x00000004,  5},
>> +    {0x00000024,  6}, {0x00000005,  5}, {0x00000025,  6}, {0x00000026,  6},
>> +    {0x00000027,  6}, {0x00000006,  5}, {0x00000074,  7}, {0x00000075,  7},
>> +    {0x00000028,  6}, {0x00000029,  6}, {0x0000002a,  6}, {0x00000007,  5},
>> +    {0x0000002b,  6}, {0x00000076,  7}, {0x0000002c,  6}, {0x00000008,  5},
>> +    {0x00000009,  5}, {0x0000002d,  6}, {0x00000077,  7}, {0x00000078,  7},
>> +    {0x00000079,  7}, {0x0000007a,  7}, {0x0000007b,  7}, {0x00007ffe, 15},
>> +    {0x000007fc, 11}, {0x00003ffd, 14}, {0x00001ffd, 13}, {0x0ffffffc, 28},
>> +    {0x000fffe6, 20}, {0x003fffd2, 22}, {0x000fffe7, 20}, {0x000fffe8, 20},
>> +    {0x003fffd3, 22}, {0x003fffd4, 22}, {0x003fffd5, 22}, {0x007fffd9, 23},
>> +    {0x003fffd6, 22}, {0x007fffda, 23}, {0x007fffdb, 23}, {0x007fffdc, 23},
>> +    {0x007fffdd, 23}, {0x007fffde, 23}, {0x00ffffeb, 24}, {0x007fffdf, 23},
>> +    {0x00ffffec, 24}, {0x00ffffed, 24}, {0x003fffd7, 22}, {0x007fffe0, 23},
>> +    {0x00ffffee, 24}, {0x007fffe1, 23}, {0x007fffe2, 23}, {0x007fffe3, 23},
>> +    {0x007fffe4, 23}, {0x001fffdc, 21}, {0x003fffd8, 22}, {0x007fffe5, 23},
>> +    {0x003fffd9, 22}, {0x007fffe6, 23}, {0x007fffe7, 23}, {0x00ffffef, 24},
>> +    {0x003fffda, 22}, {0x001fffdd, 21}, {0x000fffe9, 20}, {0x003fffdb, 22},
>> +    {0x003fffdc, 22}, {0x007fffe8, 23}, {0x007fffe9, 23}, {0x001fffde, 21},
>> +    {0x007fffea, 23}, {0x003fffdd, 22}, {0x003fffde, 22}, {0x00fffff0, 24},
>> +    {0x001fffdf, 21}, {0x003fffdf, 22}, {0x007fffeb, 23}, {0x007fffec, 23},
>> +    {0x001fffe0, 21}, {0x001fffe1, 21}, {0x003fffe0, 22}, {0x001fffe2, 21},
>> +    {0x007fffed, 23}, {0x003fffe1, 22}, {0x007fffee, 23}, {0x007fffef, 23},
>> +    {0x000fffea, 20}, {0x003fffe2, 22}, {0x003fffe3, 22}, {0x003fffe4, 22},
>> +    {0x007ffff0, 23}, {0x003fffe5, 22}, {0x003fffe6, 22}, {0x007ffff1, 23},
>> +    {0x03ffffe0, 26}, {0x03ffffe1, 26}, {0x000fffeb, 20}, {0x0007fff1, 19},
>> +    {0x003fffe7, 22}, {0x007ffff2, 23}, {0x003fffe8, 22}, {0x01ffffec, 25},
>> +    {0x03ffffe2, 26}, {0x03ffffe3, 26}, {0x03ffffe4, 26}, {0x07ffffde, 27},
>> +    {0x07ffffdf, 27}, {0x03ffffe5, 26}, {0x00fffff1, 24}, {0x01ffffed, 25},
>> +    {0x0007fff2, 19}, {0x001fffe3, 21}, {0x03ffffe6, 26}, {0x07ffffe0, 27},
>> +    {0x07ffffe1, 27}, {0x03ffffe7, 26}, {0x07ffffe2, 27}, {0x00fffff2, 24},
>> +    {0x001fffe4, 21}, {0x001fffe5, 21}, {0x03ffffe8, 26}, {0x03ffffe9, 26},
>> +    {0x0ffffffd, 28}, {0x07ffffe3, 27}, {0x07ffffe4, 27}, {0x07ffffe5, 27},
>> +    {0x000fffec, 20}, {0x00fffff3, 24}, {0x000fffed, 20}, {0x001fffe6, 21},
>> +    {0x003fffe9, 22}, {0x001fffe7, 21}, {0x001fffe8, 21}, {0x007ffff3, 23},
>> +    {0x003fffea, 22}, {0x003fffeb, 22}, {0x01ffffee, 25}, {0x01ffffef, 25},
>> +    {0x00fffff4, 24}, {0x00fffff5, 24}, {0x03ffffea, 26}, {0x007ffff4, 23},
>> +    {0x03ffffeb, 26}, {0x07ffffe6, 27}, {0x03ffffec, 26}, {0x03ffffed, 26},
>> +    {0x07ffffe7, 27}, {0x07ffffe8, 27}, {0x07ffffe9, 27}, {0x07ffffea, 27},
>> +    {0x07ffffeb, 27}, {0x0ffffffe, 28}, {0x07ffffec, 27}, {0x07ffffed, 27},
>> +    {0x07ffffee, 27}, {0x07ffffef, 27}, {0x07fffff0, 27}, {0x03ffffee, 26}
>> +};
>> +
>> +
>> +#if (NGX_HAVE_LITTLE_ENDIAN)
>> +#if (__GNUC__)
>> +
>> +static void
>> +put64(u_char *dst, uint64_t val)
> 
> Please use ngx_* prefix.
> 
> 
>> +{
>> +    *(uint64_t*)dst = __bswap_64(val);
>> +}
>> +
>> +#else /* !__GNUC__ */
>> +
>> +static void
>> +put64(u_char *dst, uint64_t val)
>> +{
>> +    dst[0] = val >> 56;
>> +    dst[1] = val >> 48;
>> +    dst[2] = val >> 40;
>> +    dst[3] = val >> 32;
>> +    dst[4] = val >> 24;
>> +    dst[5] = val >> 16;
>> +    dst[6] = val >> 8;
>> +    dst[7] = val;
>> +}
>> +#endif /* __GNUC__ */
>> +
>> +#else /* !NGX_HAVE_LITTLE_ENDIAN */
>> +
>> +static void
>> +put64(u_char *dst, uint64_t val)
>> +{
>> +    *(uint64_t*)dst = val;
>> +}
>> +
>> +#endif
> 
> In nginx is common practice to use macros for such simple expressions.
> 
> 
>> +
>> +
>> +static ngx_int_t
>> +ngx_http_v2_huff_encode(u_char *src, size_t len, u_char *dst, ngx_log_t *log,
>> +    ngx_flag_t lower)
>> +{
>> +    ngx_uint_t                      inp, outp;
>> +    ngx_int_t                       pending;
>> +    uint64_t                        buf;
> 
> IMHO it's better to accumulate up to 32-bits on 32 bits platforms.
> 
> 
>> +    ngx_http_v2_huff_encode_code_t  next;
>> +    ngx_http_v2_huff_encode_code_t *table;
> 
> Style nit.
> 
> In general, the order of variable declarations need to be arranged by
> the length of their types.
> 
> Alignment rule here: two spaces between the longest type and asterisk
> or (if no asterisk) the first letter of the name.
> 
> I've also changed the names of local variables to be more inline with
> what usually are used in nginx for similar tasks.
> 
> 
>> +
>> +    if (!lower) {
>> +        table = ngx_http_v2_huff_encode_codes;
>> +
>> +    } else {
>> +        table = ngx_http_v2_huff_encode_codes_low;
>> +    }
> 
> It's a good case for ternary operator.
> 
> 
>> +
>> +    inp = 0;
>> +    outp = 0;
>> +    pending = 0;
>> +    buf = 0;
>> +
>> +    while (inp < len) {
> 
> Effectively it's "for (inp = 0; inp < len; inp++)".
> 
> 
>> +        next = table[src[inp]];
>> +        /* Accumulate 64 bits */
>> +        if ((next.len + pending) < 64) {
> 
> Effectively it's "next.len < 64 - pending"
> 
> 
>> +            buf ^= (uint64_t)next.code << (64 - pending - next.len);
>> +            pending += next.len;
>> +
>> +        } else {
>> +        /* If compressed result is longer than source, no point in using it */
>> +            if ((outp + 8) >= len) {
>> +                return NGX_ERROR;
>> +            }
>> +
>> +            buf ^= (uint64_t)next.code >> (next.len - (64 - pending));
>> +            put64(&dst[outp], buf);
>> +            outp += 8;
>> +            pending = (next.len - (64 - pending));
> 
> Up to this point you have 4 equal cases of "64 - pending".
> 
> 
>> +
>> +            if (pending) {
>> +                buf = (uint64_t)next.code << (64 - pending);
>> +
>> +            } else {
>> +                buf = 0;
>> +            }
> 
> Another good case for ternary.
> 
> 
>> +        }
>> +
>> +        inp++;
>> +    }
>> +
>> +    buf ^= 0xffffffffffffffffUL >> pending;
>> +
>> +    while (pending > 0) {
>> +        dst[outp] = buf >> 56;
>> +        buf <<= 8;
>> +        pending -= 8;
>> +        outp++;
>> +
>> +        if (outp >= len) {
>> +            return NGX_ERROR;
>> +        }
> 
> It's better to move the check out of the cycle.
> 
> 
>> +    }
>> +
>> +    ngx_log_debug2(NGX_LOG_DEBUG_HTTP, log, 0,
>> +                   "http2 huffman encoding successful; "
>> +                   "input len: %d, output len %d", len, outp);
> 
> IMHO the debug is surplus here.
> 
> This message is the only consumer for the log pointer.
> 
> 
>> +
>> +    return outp;
>> +}
>> +
>> +
>> +u_char *
>> +ngx_http_v2_string_encode(u_char *src, size_t len, u_char *dst, u_char *tmp,
>> +    ngx_log_t *log, ngx_flag_t lower)
>> +{
>> +    ngx_int_t hlen;
> 
> Style nit: two spaces between type and name.
> 
> 
>> +    
>   ^^^^
> Style nit: spaces at the beginning of the empty line.
> 
> 
>> +    hlen = ngx_http_v2_huff_encode(src, len, tmp, log, lower);
>> +
>> +    if (hlen != NGX_ERROR) {
>> +        *dst = NGX_HTTP_V2_ENCODE_HUFF;
>> +        dst = ngx_http_v2_write_int(dst, ngx_http_v2_prefix(7), hlen);
>> +        dst = ngx_cpymem(dst, tmp, hlen);
>> +
>> +    } else {
>> +        *dst = NGX_HTTP_V2_ENCODE_RAW;
>> +        dst = ngx_http_v2_write_int(dst, ngx_http_v2_prefix(7), len);
>> +
>> +        if (lower) {
>> +            ngx_strlow(dst, src, len);
>> +            dst += len;
>> +
>> +        } else {
>> +            dst = ngx_cpymem(dst, src, len);
>> +        }
>> +    }
>> +
>> +    return dst;
>> +}
>> 
> 
> See the patch in attachment where I reworked the compression
> function to be more 32-bits friendly and fixed all the mentioned
> problems.
> 
>  wbr, Valentin V. Bartenev
> <http2_huff_encode.patch>_______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org <mailto:nginx-devel at nginx.org>
> http://mailman.nginx.org/mailman/listinfo/nginx-devel <http://mailman.nginx.org/mailman/listinfo/nginx-devel>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160120/f7f1f036/attachment.html>

From vbart at nginx.com  Fri Jan 22 10:53:01 2016
From: vbart at nginx.com (Valentin V. Bartenev)
Date: Fri, 22 Jan 2016 13:53:01 +0300
Subject: [PATCH] HTTP/2: HPACK Huffman encoding
In-Reply-To: <284F77E2-2D4B-4BE5-A7F8-374C10EE36B4@cloudflare.com>
References: <d672e9c2b81471098668.1450352972@ubuntu>
 <2548150.3ZiQ2bDBD5@vbart-workstation>
 <284F77E2-2D4B-4BE5-A7F8-374C10EE36B4@cloudflare.com>
Message-ID: <1506160.nyu9cTvipg@vbart-workstation>

On Wednesday 20 January 2016 14:01:15 Vlad Krasnov wrote:
> LGTM
> 
> Cheers,
> Vlad
> 
[..]

Thank you.  The patch will be committed after internal review.

  wbr, Valentin V. Bartenev


From aleksandr.vin at gmail.com  Fri Jan 22 12:51:05 2016
From: aleksandr.vin at gmail.com (Aleksandr Vinokurov)
Date: Fri, 22 Jan 2016 15:51:05 +0300
Subject: How to ask 'nginx -t' to remove pid file it created?
Message-ID: <BC57F461-0AFD-4BC4-B26C-CFC9E356C61B@gmail.com>

Hi all,

I?ve met with a rather unpleasant behaviour of nginx validation process (the one triggered by '-t? command line option): it creates a pid file at the path from the configuration it validate, and leave this file alone.

It sounds rather inoffensively until you step in a RHEL7 with SELinux and wanted to use Ansible role for installing nginx there.

If you call ?nginx -t -c /etc/nginx/nginx.conf?, then it creates a pid file with wrong SELinux context and the presence of this pid file breaks the next call to 'systemctl start nginx?. Which while failing to start nginx service, removes the pid file. So next calls to ?systemctl start nginx? will succeed.

Here is the proof log:

[root at SRV2-ELOG-VM58 ~]# ls -Z /run/nginx.pid
ls: cannot access /run/nginx.pid: No such file or directory
[root at SRV2-ELOG-VM58 ~]# systemctl start nginx
[root at SRV2-ELOG-VM58 ~]# ls -Z /run/nginx.pid
-rw-r--r--. root root system_u:object_r:httpd_var_run_t:s0 /run/nginx.pid
[root at SRV2-ELOG-VM58 ~]# systemctl stop nginx
[root at SRV2-ELOG-VM58 ~]# ls -Z /run/nginx.pid
ls: cannot access /run/nginx.pid: No such file or directory
[root at SRV2-ELOG-VM58 ~]# /usr/sbin/nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root at SRV2-ELOG-VM58 ~]# ls -Z /run/nginx.pid
-rw-r--r--. root root unconfined_u:object_r:var_run_t:s0 /run/nginx.pid
[root at SRV2-ELOG-VM58 ~]# systemctl start nginx
Job for nginx.service failed. See 'systemctl status nginx.service' and 'journalctl -xn' for details.
[root at SRV2-ELOG-VM58 ~]# ls -Z /run/nginx.pid
ls: cannot access /run/nginx.pid: No such file or directory
[root at SRV2-ELOG-VM58 ~]# systemctl start nginx
[root at SRV2-ELOG-VM58 ~]# ls -Z /run/nginx.pid
-rw-r--r--. root root system_u:object_r:httpd_var_run_t:s0 /run/nginx.pid
[root at SRV2-ELOG-VM58 ~]#

Is there a way to ask nginx to remove the pid file it created during the validation process or not to create it at all?

Some additional info is here: https://github.com/geerlingguy/ansible-role-nginx/issues/41

With all the best,
Alex Vinokurov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160122/502a40d4/attachment.html>

From mdounin at mdounin.ru  Fri Jan 22 15:12:47 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 22 Jan 2016 18:12:47 +0300
Subject: How to ask 'nginx -t' to remove pid file it created?
In-Reply-To: <BC57F461-0AFD-4BC4-B26C-CFC9E356C61B@gmail.com>
References: <BC57F461-0AFD-4BC4-B26C-CFC9E356C61B@gmail.com>
Message-ID: <20160122151246.GH74233@mdounin.ru>

Hello!

On Fri, Jan 22, 2016 at 03:51:05PM +0300, Aleksandr Vinokurov wrote:

> Hi all,
> 
> I?ve met with a rather unpleasant behaviour of nginx validation 
> process (the one triggered by '-t? command line option): it 
> creates a pid file at the path from the configuration it 
> validate, and leave this file alone.
> 
> It sounds rather inoffensively until you step in a RHEL7 with 
> SELinux and wanted to use Ansible role for installing nginx 
> there.
> 
> If you call ?nginx -t -c /etc/nginx/nginx.conf?, then it creates 
> a pid file with wrong SELinux context and the presence of this 
> pid file breaks the next call to 'systemctl start nginx?. Which 
> while failing to start nginx service, removes the pid file. So 
> next calls to ?systemctl start nginx? will succeed.

[...]

> Is there a way to ask nginx to remove the pid file it created 
> during the validation process or not to create it at all?

The file is created to make sure nginx is able to open it for 
writing.  It can't be removed though, as there may be a running 
nginx instance.  Note well that the same behaviour also applies to 
various other files - e.g., nginx will try to open logs for 
writing as well.

Most obvious solution I can see is to avoid calling "nginx -t" 
with wrong SELinux context.

-- 
Maxim Dounin
http://nginx.org/


From alessandro at cloudflare.com  Fri Jan 22 17:37:47 2016
From: alessandro at cloudflare.com (Alessandro Ghedini)
Date: Fri, 22 Jan 2016 17:37:47 +0000
Subject: [PATCH 0 of 2] Support for OCSP stapling verification from upstream
Message-ID: <20160122173747.GA19316@mandy.local>

Hello,

this patchset adds support for requesting and verifying OCSP stapled
responses from an HTTP upstream.

In order to avoid code duplication, the first patch refactors the existing
OCSP verification code so that it can be reused for this new functionality.

The diff is a bit messy, so please advise if there's a better way to
accomplish the same and make reviewing the patch easier.

The second patch actually adds the OCSP stapling verification via a new
option "proxy_ssl_stapling_verify".

Note that older OpenSSL versions (pre-1.0.2) had a bug [0] that caused
OCSP verification to fail for valid responses. I developed a work-around
so I could properly test my code, but it's a bit ugly so it's probably
best to not merge it. I can share it if anyone is interested though.

Cheers

[0] https://rt.openssl.org/Ticket/Display.html?id=3668&user=guest&pass=guest


From alessandro at cloudflare.com  Fri Jan 22 17:37:56 2016
From: alessandro at cloudflare.com (Alessandro Ghedini)
Date: Fri, 22 Jan 2016 17:37:56 +0000
Subject: [PATCH 1 of 2] OCSP stapling: refactor OCSP response verification
In-Reply-To: <patchbomb.1453483534@mandy.local>
References: <patchbomb.1453483534@mandy.local>
Message-ID: <20160122173756.GB19316@mandy.local>

# HG changeset patch
# User Alessandro Ghedini <alessandro at cloudflare.com>
# Date 1453481195 0
#      Fri Jan 22 16:46:35 2016 +0000
# Node ID a8c4f65236ad90138863d5295ca059a3d37da37e
# Parent  02abce4764b7c964e7ee6e10b1a7abf1fee95f79
OCSP stapling: refactor OCSP response verification

Move the OCSP response verification and the certificate issuer code to
stand-alone functions, so that it can be reused by other code.

Signed-off-by: Alessandro Ghedini <alessandro at cloudflare.com>

diff -r 02abce4764b7 -r a8c4f65236ad src/event/ngx_event_openssl_stapling.c
--- a/src/event/ngx_event_openssl_stapling.c	Tue Jan 12 19:19:07 2016 +0300
+++ b/src/event/ngx_event_openssl_stapling.c	Fri Jan 22 16:46:35 2016 +0000
@@ -85,10 +85,15 @@
 
 static ngx_int_t ngx_ssl_stapling_file(ngx_conf_t *cf, ngx_ssl_t *ssl,
     ngx_str_t *file);
+static ngx_int_t ngx_ssl_stapling_find_issuer(ngx_ssl_stapling_t *staple,
+    ngx_log_t *log, SSL_CTX *ssl_ctx, X509 *cert, STACK_OF(X509) *chain);
 static ngx_int_t ngx_ssl_stapling_issuer(ngx_conf_t *cf, ngx_ssl_t *ssl);
 static ngx_int_t ngx_ssl_stapling_responder(ngx_conf_t *cf, ngx_ssl_t *ssl,
     ngx_str_t *responder);
 
+static ngx_int_t ngx_ssl_stapling_verify_response(ngx_log_t *log,
+    ngx_str_t *resp, ngx_ssl_stapling_t *staple, STACK_OF(X509) *chain,
+    ngx_int_t flags, time_t *valid);
 static int ngx_ssl_certificate_status_callback(ngx_ssl_conn_t *ssl_conn,
     void *data);
 static void ngx_ssl_stapling_update(ngx_ssl_stapling_t *staple);
@@ -259,12 +264,87 @@
 
 
 static ngx_int_t
+ngx_ssl_stapling_find_issuer(ngx_ssl_stapling_t *staple, ngx_log_t *log,
+    SSL_CTX *ssl_ctx, X509 *cert, STACK_OF(X509) *chain)
+{
+    int                  i, n, rc;
+    X509                *issuer;
+    X509_STORE          *store;
+    X509_STORE_CTX      *store_ctx;
+
+    n = sk_X509_num(chain);
+
+    ngx_log_debug1(NGX_LOG_DEBUG_EVENT, log, 0,
+                   "SSL get issuer: %d extra certs", n);
+
+    for (i = 0; i < n; i++) {
+        issuer = sk_X509_value(chain, i);
+        if (X509_check_issued(issuer, cert) == X509_V_OK) {
+            CRYPTO_add(&issuer->references, 1, CRYPTO_LOCK_X509);
+
+            ngx_log_debug1(NGX_LOG_DEBUG_EVENT, log, 0,
+                           "SSL get issuer: found %p in extra certs", issuer);
+
+            staple->cert = cert;
+            staple->issuer = issuer;
+
+            return NGX_OK;
+        }
+    }
+
+    store = SSL_CTX_get_cert_store(ssl_ctx);
+    if (store == NULL) {
+        ngx_ssl_error(NGX_LOG_EMERG, log, 0,
+                      "SSL_CTX_get_cert_store() failed");
+        return NGX_ERROR;
+    }
+
+    store_ctx = X509_STORE_CTX_new();
+    if (store_ctx == NULL) {
+        ngx_ssl_error(NGX_LOG_EMERG, log, 0,
+                      "X509_STORE_CTX_new() failed");
+        return NGX_ERROR;
+    }
+
+    if (X509_STORE_CTX_init(store_ctx, store, NULL, NULL) == 0) {
+        ngx_ssl_error(NGX_LOG_EMERG, log, 0,
+                      "X509_STORE_CTX_init() failed");
+        X509_STORE_CTX_free(store_ctx);
+        return NGX_ERROR;
+    }
+
+    rc = X509_STORE_CTX_get1_issuer(&issuer, store_ctx, cert);
+
+    if (rc == -1) {
+        ngx_ssl_error(NGX_LOG_EMERG, log, 0,
+                      "X509_STORE_CTX_get1_issuer() failed");
+        X509_STORE_CTX_free(store_ctx);
+        return NGX_ERROR;
+    }
+
+    if (rc == 0) {
+        ngx_log_error(NGX_LOG_WARN, log, 0,
+                      "\"ssl_stapling\" ignored, issuer certificate not found");
+        X509_STORE_CTX_free(store_ctx);
+        return NGX_DECLINED;
+    }
+
+    X509_STORE_CTX_free(store_ctx);
+
+    ngx_log_debug1(NGX_LOG_DEBUG_EVENT, log, 0,
+                   "SSL get issuer: found %p in cert store", issuer);
+
+    staple->cert = cert;
+    staple->issuer = issuer;
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
 ngx_ssl_stapling_issuer(ngx_conf_t *cf, ngx_ssl_t *ssl)
 {
-    int                  i, n, rc;
-    X509                *cert, *issuer;
-    X509_STORE          *store;
-    X509_STORE_CTX      *store_ctx;
+    X509                *cert;
     STACK_OF(X509)      *chain;
     ngx_ssl_stapling_t  *staple;
 
@@ -277,72 +357,8 @@
     chain = ssl->ctx->extra_certs;
 #endif
 
-    n = sk_X509_num(chain);
-
-    ngx_log_debug1(NGX_LOG_DEBUG_EVENT, ssl->log, 0,
-                   "SSL get issuer: %d extra certs", n);
-
-    for (i = 0; i < n; i++) {
-        issuer = sk_X509_value(chain, i);
-        if (X509_check_issued(issuer, cert) == X509_V_OK) {
-            CRYPTO_add(&issuer->references, 1, CRYPTO_LOCK_X509);
-
-            ngx_log_debug1(NGX_LOG_DEBUG_EVENT, ssl->log, 0,
-                           "SSL get issuer: found %p in extra certs", issuer);
-
-            staple->cert = cert;
-            staple->issuer = issuer;
-
-            return NGX_OK;
-        }
-    }
-
-    store = SSL_CTX_get_cert_store(ssl->ctx);
-    if (store == NULL) {
-        ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0,
-                      "SSL_CTX_get_cert_store() failed");
-        return NGX_ERROR;
-    }
-
-    store_ctx = X509_STORE_CTX_new();
-    if (store_ctx == NULL) {
-        ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0,
-                      "X509_STORE_CTX_new() failed");
-        return NGX_ERROR;
-    }
-
-    if (X509_STORE_CTX_init(store_ctx, store, NULL, NULL) == 0) {
-        ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0,
-                      "X509_STORE_CTX_init() failed");
-        X509_STORE_CTX_free(store_ctx);
-        return NGX_ERROR;
-    }
-
-    rc = X509_STORE_CTX_get1_issuer(&issuer, store_ctx, cert);
-
-    if (rc == -1) {
-        ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0,
-                      "X509_STORE_CTX_get1_issuer() failed");
-        X509_STORE_CTX_free(store_ctx);
-        return NGX_ERROR;
-    }
-
-    if (rc == 0) {
-        ngx_log_error(NGX_LOG_WARN, ssl->log, 0,
-                      "\"ssl_stapling\" ignored, issuer certificate not found");
-        X509_STORE_CTX_free(store_ctx);
-        return NGX_DECLINED;
-    }
-
-    X509_STORE_CTX_free(store_ctx);
-
-    ngx_log_debug1(NGX_LOG_DEBUG_EVENT, ssl->log, 0,
-                   "SSL get issuer: found %p in cert store", issuer);
-
-    staple->cert = cert;
-    staple->issuer = issuer;
-
-    return NGX_OK;
+    return ngx_ssl_stapling_find_issuer(staple, ssl->log, ssl->ctx,
+                                        cert, chain);
 }
 
 
@@ -529,8 +545,10 @@
 }
 
 
-static void
-ngx_ssl_stapling_ocsp_handler(ngx_ssl_ocsp_ctx_t *ctx)
+static ngx_int_t
+ngx_ssl_stapling_verify_response(ngx_log_t *log, ngx_str_t *resp,
+    ngx_ssl_stapling_t *staple, STACK_OF(X509) *chain, ngx_int_t flags,
+    time_t *valid)
 {
 #if OPENSSL_VERSION_NUMBER >= 0x0090707fL
     const
@@ -538,34 +556,24 @@
     u_char                *p;
     int                    n;
     size_t                 len;
-    time_t                 now, valid;
-    ngx_str_t              response;
     X509_STORE            *store;
-    STACK_OF(X509)        *chain;
     OCSP_CERTID           *id;
     OCSP_RESPONSE         *ocsp;
     OCSP_BASICRESP        *basic;
-    ngx_ssl_stapling_t    *staple;
     ASN1_GENERALIZEDTIME  *thisupdate, *nextupdate;
 
-    staple = ctx->data;
-    now = ngx_time();
     ocsp = NULL;
     basic = NULL;
     id = NULL;
 
-    if (ctx->code != 200) {
-        goto error;
-    }
-
     /* check the response */
 
-    len = ctx->response->last - ctx->response->pos;
-    p = ctx->response->pos;
+    len = resp->len;
+    p = resp->data;
 
     ocsp = d2i_OCSP_RESPONSE(NULL, &p, len);
     if (ocsp == NULL) {
-        ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0,
+        ngx_ssl_error(NGX_LOG_ERR, log, 0,
                       "d2i_OCSP_RESPONSE() failed");
         goto error;
     }
@@ -573,7 +581,7 @@
     n = OCSP_response_status(ocsp);
 
     if (n != OCSP_RESPONSE_STATUS_SUCCESSFUL) {
-        ngx_log_error(NGX_LOG_ERR, ctx->log, 0,
+        ngx_log_error(NGX_LOG_ERR, log, 0,
                       "OCSP response not successful (%d: %s)",
                       n, OCSP_response_status_str(n));
         goto error;
@@ -581,36 +589,27 @@
 
     basic = OCSP_response_get1_basic(ocsp);
     if (basic == NULL) {
-        ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0,
+        ngx_ssl_error(NGX_LOG_ERR, log, 0,
                       "OCSP_response_get1_basic() failed");
         goto error;
     }
 
     store = SSL_CTX_get_cert_store(staple->ssl_ctx);
     if (store == NULL) {
-        ngx_ssl_error(NGX_LOG_CRIT, ctx->log, 0,
+        ngx_ssl_error(NGX_LOG_CRIT, log, 0,
                       "SSL_CTX_get_cert_store() failed");
         goto error;
     }
 
-#if OPENSSL_VERSION_NUMBER >= 0x10001000L
-    SSL_CTX_get_extra_chain_certs(staple->ssl_ctx, &chain);
-#else
-    chain = staple->ssl_ctx->extra_certs;
-#endif
-
-    if (OCSP_basic_verify(basic, chain, store,
-                          staple->verify ? OCSP_TRUSTOTHER : OCSP_NOVERIFY)
-        != 1)
-    {
-        ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0,
+    if (OCSP_basic_verify(basic, chain, store, flags) != 1) {
+        ngx_ssl_error(NGX_LOG_ERR, log, 0,
                       "OCSP_basic_verify() failed");
         goto error;
     }
 
-    id = OCSP_cert_to_id(NULL, ctx->cert, ctx->issuer);
+    id = OCSP_cert_to_id(NULL, staple->cert, staple->issuer);
     if (id == NULL) {
-        ngx_ssl_error(NGX_LOG_CRIT, ctx->log, 0,
+        ngx_ssl_error(NGX_LOG_CRIT, log, 0,
                       "OCSP_cert_to_id() failed");
         goto error;
     }
@@ -619,34 +618,34 @@
                               &thisupdate, &nextupdate)
         != 1)
     {
-        ngx_log_error(NGX_LOG_ERR, ctx->log, 0,
+        ngx_log_error(NGX_LOG_ERR, log, 0,
                       "certificate status not found in the OCSP response");
         goto error;
     }
 
     if (n != V_OCSP_CERTSTATUS_GOOD) {
-        ngx_log_error(NGX_LOG_ERR, ctx->log, 0,
+        ngx_log_error(NGX_LOG_ERR, log, 0,
                       "certificate status \"%s\" in the OCSP response",
                       OCSP_cert_status_str(n));
         goto error;
     }
 
     if (OCSP_check_validity(thisupdate, nextupdate, 300, -1) != 1) {
-        ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0,
+        ngx_ssl_error(NGX_LOG_ERR, log, 0,
                       "OCSP_check_validity() failed");
         goto error;
     }
 
     if (nextupdate) {
-        valid = ngx_ssl_stapling_time(nextupdate);
-        if (valid == (time_t) NGX_ERROR) {
-            ngx_log_error(NGX_LOG_ERR, ctx->log, 0,
+        *valid = ngx_ssl_stapling_time(nextupdate);
+        if (*valid == (time_t) NGX_ERROR) {
+            ngx_log_error(NGX_LOG_ERR, log, 0,
                           "invalid nextUpdate time in certificate status");
             goto error;
         }
 
     } else {
-        valid = NGX_MAX_TIME_T_VALUE;
+        *valid = NGX_MAX_TIME_T_VALUE;
     }
 
     OCSP_CERTID_free(id);
@@ -657,9 +656,54 @@
     basic = NULL;
     ocsp = NULL;
 
+    return NGX_OK;
+
+error:
+
+    if (id) {
+        OCSP_CERTID_free(id);
+    }
+
+    if (basic) {
+        OCSP_BASICRESP_free(basic);
+    }
+
+    if (ocsp) {
+        OCSP_RESPONSE_free(ocsp);
+    }
+
+    return NGX_ERROR;
+}
+
+
+static void
+ngx_ssl_stapling_ocsp_handler(ngx_ssl_ocsp_ctx_t *ctx)
+{
+    time_t                 now, valid;
+    ngx_int_t              rc, flags;
+    ngx_str_t              response;
+    STACK_OF(X509)        *chain;
+    ngx_ssl_stapling_t    *staple;
+
+    now = ngx_time();
+
+    staple = ctx->data;
+
+    if (ctx->code != 200) {
+        goto error;
+    }
+
+#if OPENSSL_VERSION_NUMBER >= 0x10001000L
+    SSL_CTX_get_extra_chain_certs(staple->ssl_ctx, &chain);
+#else
+    chain = staple->ssl_ctx->extra_certs;
+#endif
+
+    flags = staple->verify ? OCSP_TRUSTOTHER : OCSP_NOVERIFY;
+
     /* copy the response to memory not in ctx->pool */
 
-    response.len = len;
+    response.len = ctx->response->last - ctx->response->pos;
     response.data = ngx_alloc(response.len, ctx->log);
 
     if (response.data == NULL) {
@@ -668,6 +712,14 @@
 
     ngx_memcpy(response.data, ctx->response->pos, response.len);
 
+    /* check the response */
+
+    rc = ngx_ssl_stapling_verify_response(ctx->log, &response, staple, chain,
+                                          flags, &valid);
+    if (rc != NGX_OK) {
+        goto error;
+    }
+
     ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ctx->log, 0,
                    "ssl ocsp response, %s, %uz",
                    OCSP_cert_status_str(n), response.len);
@@ -695,16 +747,8 @@
     staple->loading = 0;
     staple->refresh = now + 300;
 
-    if (id) {
-        OCSP_CERTID_free(id);
-    }
-
-    if (basic) {
-        OCSP_BASICRESP_free(basic);
-    }
-
-    if (ocsp) {
-        OCSP_RESPONSE_free(ocsp);
+    if (response.data != NULL) {
+        ngx_free(response.data);
     }
 
     ngx_ssl_ocsp_done(ctx);


From alessandro at cloudflare.com  Fri Jan 22 17:38:06 2016
From: alessandro at cloudflare.com (Alessandro Ghedini)
Date: Fri, 22 Jan 2016 17:38:06 +0000
Subject: [PATCH 2 of 2] Proxy: add support for OCSP stapling verification from
 upstream
In-Reply-To: <patchbomb.1453483534@mandy.local>
References: <patchbomb.1453483534@mandy.local>
Message-ID: <20160122173806.GC19316@mandy.local>

# HG changeset patch
# User Alessandro Ghedini <alessandro at cloudflare.com>
# Date 1453481233 0
#      Fri Jan 22 16:47:13 2016 +0000
# Node ID c6668c14a2d168307bcfade0cc2e01c92c31312a
# Parent  a8c4f65236ad90138863d5295ca059a3d37da37e
Proxy: add support for OCSP stapling verification from upstream

This patch adds the "proxy_ssl_stapling_verify" option that controls OCSP
stapling verification from an upstream server.

The option allows three values:

 - "off" (default): disable OCSP stapling completely.
 - "on": request OCSP stapling from upstream and verify response if
         provided.
 - "full": same as "on", but fail also when no response is received.

Signed-off-by: Alessandro Ghedini <alessandro at cloudflare.com>

diff -r a8c4f65236ad -r c6668c14a2d1 src/event/ngx_event_openssl.h
--- a/src/event/ngx_event_openssl.h	Fri Jan 22 16:46:35 2016 +0000
+++ b/src/event/ngx_event_openssl.h	Fri Jan 22 16:47:13 2016 +0000
@@ -121,6 +121,11 @@
 #define NGX_SSL_BUFSIZE  16384
 
 
+#define NGX_SSL_STAPLING_VERIFY_OFF   0
+#define NGX_SSL_STAPLING_VERIFY_ON    1
+#define NGX_SSL_STAPLING_VERIFY_FULL  2
+
+
 ngx_int_t ngx_ssl_init(ngx_log_t *log);
 ngx_int_t ngx_ssl_create(ngx_ssl_t *ssl, ngx_uint_t protocols, void *data);
 ngx_int_t ngx_ssl_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl,
@@ -132,6 +137,7 @@
 ngx_int_t ngx_ssl_crl(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *crl);
 ngx_int_t ngx_ssl_stapling(ngx_conf_t *cf, ngx_ssl_t *ssl,
     ngx_str_t *file, ngx_str_t *responder, ngx_uint_t verify);
+ngx_int_t ngx_ssl_stapling_verify(ngx_connection_t *c, ngx_int_t strict);
 ngx_int_t ngx_ssl_stapling_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl,
     ngx_resolver_t *resolver, ngx_msec_t resolver_timeout);
 RSA *ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl_conn, int is_export,
diff -r a8c4f65236ad -r c6668c14a2d1 src/event/ngx_event_openssl_stapling.c
--- a/src/event/ngx_event_openssl_stapling.c	Fri Jan 22 16:46:35 2016 +0000
+++ b/src/event/ngx_event_openssl_stapling.c	Fri Jan 22 16:47:13 2016 +0000
@@ -194,6 +194,48 @@
 }
 
 
+ngx_int_t
+ngx_ssl_stapling_verify(ngx_connection_t *c, ngx_int_t strict)
+{
+    int                   rc;
+    long                  len;
+    time_t                valid;
+    ngx_str_t             ocsp_resp;
+    STACK_OF(X509)       *chain;
+    ngx_ssl_stapling_t    staple;
+
+    staple.ssl_ctx = c->ssl->connection->ctx;
+
+    len = SSL_get_tlsext_status_ocsp_resp(c->ssl->connection, &ocsp_resp.data);
+    if (len < 0) {
+        if (strict) {
+            ngx_log_error(NGX_LOG_ERR, c->log, 0, "missing OCSP response");
+            return NGX_ERROR;
+        }
+
+        return NGX_OK;
+    }
+
+    ocsp_resp.len = len;
+
+    chain = SSL_get_peer_cert_chain(c->ssl->connection);
+
+    rc = ngx_ssl_stapling_find_issuer(&staple, c->log, c->ssl->connection->ctx,
+                                      sk_X509_value(chain, 0), chain);
+    if (rc != NGX_OK) {
+        return NGX_ERROR;
+    }
+
+    rc = ngx_ssl_stapling_verify_response(c->log, &ocsp_resp, &staple,
+                                          chain, 0, &valid);
+    if (rc != NGX_OK) {
+        return NGX_ERROR;
+    }
+
+    return NGX_OK;
+}
+
+
 static ngx_int_t
 ngx_ssl_stapling_file(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file)
 {
@@ -1860,6 +1902,12 @@
 }
 
 ngx_int_t
+ngx_ssl_stapling_verify(ngx_connection_t *c, ngx_int_t strict)
+{
+    return NGX_OK;
+}
+
+ngx_int_t
 ngx_ssl_stapling_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl,
     ngx_resolver_t *resolver, ngx_msec_t resolver_timeout)
 {
diff -r a8c4f65236ad -r c6668c14a2d1 src/http/modules/ngx_http_proxy_module.c
--- a/src/http/modules/ngx_http_proxy_module.c	Fri Jan 22 16:46:35 2016 +0000
+++ b/src/http/modules/ngx_http_proxy_module.c	Fri Jan 22 16:47:13 2016 +0000
@@ -236,6 +236,14 @@
     { ngx_null_string, 0 }
 };
 
+
+static ngx_conf_enum_t  ngx_http_proxy_ssl_stapling_verify[] = {
+    { ngx_string("off"), NGX_SSL_STAPLING_VERIFY_OFF },
+    { ngx_string("on"), NGX_SSL_STAPLING_VERIFY_ON },
+    { ngx_string("full"), NGX_SSL_STAPLING_VERIFY_FULL },
+    { ngx_null_string, 0 }
+};
+
 #endif
 
 
@@ -698,6 +706,13 @@
       0,
       NULL },
 
+    { ngx_string("proxy_ssl_stapling_verify"),
+      NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG,
+      ngx_conf_set_enum_slot,
+      NGX_HTTP_LOC_CONF_OFFSET,
+      offsetof(ngx_http_proxy_loc_conf_t, upstream.ssl_stapling_verify),
+      &ngx_http_proxy_ssl_stapling_verify },
+
 #endif
 
       ngx_null_command
@@ -2865,6 +2880,7 @@
     conf->upstream.ssl_session_reuse = NGX_CONF_UNSET;
     conf->upstream.ssl_server_name = NGX_CONF_UNSET;
     conf->upstream.ssl_verify = NGX_CONF_UNSET;
+    conf->upstream.ssl_stapling_verify = NGX_CONF_UNSET;
     conf->ssl_verify_depth = NGX_CONF_UNSET_UINT;
     conf->ssl_passwords = NGX_CONF_UNSET_PTR;
 #endif
@@ -3187,6 +3203,8 @@
                               prev->upstream.ssl_server_name, 0);
     ngx_conf_merge_value(conf->upstream.ssl_verify,
                               prev->upstream.ssl_verify, 0);
+    ngx_conf_merge_value(conf->upstream.ssl_stapling_verify,
+                              prev->upstream.ssl_stapling_verify, 0);
     ngx_conf_merge_uint_value(conf->ssl_verify_depth,
                               prev->ssl_verify_depth, 1);
     ngx_conf_merge_str_value(conf->ssl_trusted_certificate,
diff -r a8c4f65236ad -r c6668c14a2d1 src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Fri Jan 22 16:46:35 2016 +0000
+++ b/src/http/ngx_http_upstream.c	Fri Jan 22 16:47:13 2016 +0000
@@ -1496,6 +1496,12 @@
         }
     }
 
+#if (!defined OPENSSL_NO_OCSP && defined TLSEXT_STATUSTYPE_ocsp)
+    if (u->conf->ssl_stapling_verify) {
+        SSL_set_tlsext_status_type(c->ssl->connection, TLSEXT_STATUSTYPE_ocsp);
+    }
+#endif
+
     if (u->conf->ssl_session_reuse) {
         if (u->peer.set_session(&u->peer, u->peer.data) != NGX_OK) {
             ngx_http_upstream_finalize_request(r, u,
@@ -1548,6 +1554,7 @@
 ngx_http_upstream_ssl_handshake(ngx_connection_t *c)
 {
     long                  rc;
+    ngx_int_t             ocsp_strict;
     ngx_http_request_t   *r;
     ngx_http_upstream_t  *u;
 
@@ -1576,6 +1583,16 @@
             }
         }
 
+        if (u->conf->ssl_stapling_verify) {
+            ocsp_strict =
+                u->conf->ssl_stapling_verify == NGX_SSL_STAPLING_VERIFY_FULL;
+
+            rc = ngx_ssl_stapling_verify(c, ocsp_strict);
+            if (rc != NGX_OK) {
+                goto failed;
+            }
+        }
+
         if (u->conf->ssl_session_reuse) {
             u->peer.save_session(&u->peer, u->peer.data);
         }
diff -r a8c4f65236ad -r c6668c14a2d1 src/http/ngx_http_upstream.h
--- a/src/http/ngx_http_upstream.h	Fri Jan 22 16:46:35 2016 +0000
+++ b/src/http/ngx_http_upstream.h	Fri Jan 22 16:47:13 2016 +0000
@@ -217,6 +217,7 @@
     ngx_http_complex_value_t        *ssl_name;
     ngx_flag_t                       ssl_server_name;
     ngx_flag_t                       ssl_verify;
+    ngx_int_t                        ssl_stapling_verify;
 #endif
 
     ngx_str_t                        module;


From mdounin at mdounin.ru  Fri Jan 22 17:49:26 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 22 Jan 2016 20:49:26 +0300
Subject: [PATCH 2 of 2] Proxy: add support for OCSP stapling verification
 from upstream
In-Reply-To: <20160122173806.GC19316@mandy.local>
References: <patchbomb.1453483534@mandy.local>
 <20160122173806.GC19316@mandy.local>
Message-ID: <20160122174926.GK74233@mdounin.ru>

Hello!

On Fri, Jan 22, 2016 at 05:38:06PM +0000, Alessandro Ghedini wrote:

> # HG changeset patch
> # User Alessandro Ghedini <alessandro at cloudflare.com>
> # Date 1453481233 0
> #      Fri Jan 22 16:47:13 2016 +0000
> # Node ID c6668c14a2d168307bcfade0cc2e01c92c31312a
> # Parent  a8c4f65236ad90138863d5295ca059a3d37da37e
> Proxy: add support for OCSP stapling verification from upstream
> 
> This patch adds the "proxy_ssl_stapling_verify" option that controls OCSP
> stapling verification from an upstream server.
> 
> The option allows three values:
> 
>  - "off" (default): disable OCSP stapling completely.
>  - "on": request OCSP stapling from upstream and verify response if
>          provided.
>  - "full": same as "on", but fail also when no response is received.

The "on" seems to be no different from "off" and hardly make 
sense, as an attacker can easily avoid returning stapled OCSP 
response.

The "full" in turn doesn't seem to be correct feature, as stapled 
OCSP response may be legitimately absent for multiple reasons.

[...]

-- 
Maxim Dounin
http://nginx.org/


From alessandro at cloudflare.com  Fri Jan 22 18:02:14 2016
From: alessandro at cloudflare.com (Alessandro Ghedini)
Date: Fri, 22 Jan 2016 18:02:14 +0000
Subject: [PATCH 2 of 2] Proxy: add support for OCSP stapling verification
 from upstream
In-Reply-To: <20160122174926.GK74233@mdounin.ru>
References: <patchbomb.1453483534@mandy.local>
 <20160122173806.GC19316@mandy.local>
 <20160122174926.GK74233@mdounin.ru>
Message-ID: <20160122180214.GA19389@mandy.local>

On Fri, Jan 22, 2016 at 08:49:26pm +0300, Maxim Dounin wrote:
> Hello!
> 
> On Fri, Jan 22, 2016 at 05:38:06PM +0000, Alessandro Ghedini wrote:
> 
> > # HG changeset patch
> > # User Alessandro Ghedini <alessandro at cloudflare.com>
> > # Date 1453481233 0
> > #      Fri Jan 22 16:47:13 2016 +0000
> > # Node ID c6668c14a2d168307bcfade0cc2e01c92c31312a
> > # Parent  a8c4f65236ad90138863d5295ca059a3d37da37e
> > Proxy: add support for OCSP stapling verification from upstream
> > 
> > This patch adds the "proxy_ssl_stapling_verify" option that controls OCSP
> > stapling verification from an upstream server.
> > 
> > The option allows three values:
> > 
> >  - "off" (default): disable OCSP stapling completely.
> >  - "on": request OCSP stapling from upstream and verify response if
> >          provided.
> >  - "full": same as "on", but fail also when no response is received.
> 
> The "on" seems to be no different from "off" and hardly make 
> sense, as an attacker can easily avoid returning stapled OCSP 
> response.

Yes, of course. This is what browsers currently do, and is IMO better than
doing nothing. Once Must-Staple (aka "TLS Feature" x509 extension) starts
to be used in the wild this can be updated.

> The "full" in turn doesn't seem to be correct feature, as stapled 
> OCSP response may be legitimately absent for multiple reasons.

If you control the upstream servers than I don't see any reason why you
couldn't just enable OCSP stapling unconditionally and enforce this on
the downstream with the "full" option. Maybe I'm missing something?

Cheers


From mdounin at mdounin.ru  Fri Jan 22 18:46:01 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 22 Jan 2016 21:46:01 +0300
Subject: [PATCH 2 of 2] Proxy: add support for OCSP stapling verification
 from upstream
In-Reply-To: <20160122180214.GA19389@mandy.local>
References: <patchbomb.1453483534@mandy.local>
 <20160122173806.GC19316@mandy.local>
 <20160122174926.GK74233@mdounin.ru>
 <20160122180214.GA19389@mandy.local>
Message-ID: <20160122184601.GL74233@mdounin.ru>

Hello!

On Fri, Jan 22, 2016 at 06:02:14PM +0000, Alessandro Ghedini wrote:

> On Fri, Jan 22, 2016 at 08:49:26pm +0300, Maxim Dounin wrote:
> > Hello!
> > 
> > On Fri, Jan 22, 2016 at 05:38:06PM +0000, Alessandro Ghedini wrote:
> > 
> > > # HG changeset patch
> > > # User Alessandro Ghedini <alessandro at cloudflare.com>
> > > # Date 1453481233 0
> > > #      Fri Jan 22 16:47:13 2016 +0000
> > > # Node ID c6668c14a2d168307bcfade0cc2e01c92c31312a
> > > # Parent  a8c4f65236ad90138863d5295ca059a3d37da37e
> > > Proxy: add support for OCSP stapling verification from upstream
> > > 
> > > This patch adds the "proxy_ssl_stapling_verify" option that controls OCSP
> > > stapling verification from an upstream server.
> > > 
> > > The option allows three values:
> > > 
> > >  - "off" (default): disable OCSP stapling completely.
> > >  - "on": request OCSP stapling from upstream and verify response if
> > >          provided.
> > >  - "full": same as "on", but fail also when no response is received.
> > 
> > The "on" seems to be no different from "off" and hardly make 
> > sense, as an attacker can easily avoid returning stapled OCSP 
> > response.
> 
> Yes, of course. This is what browsers currently do, and is IMO better than
> doing nothing. Once Must-Staple (aka "TLS Feature" x509 extension) starts
> to be used in the wild this can be updated.

What browsers do is quite different: they use OCSP, and use OCSP 
Stapling as an optimization.  And even if they can't obtain a 
response, they show this visually to users.

What your code does is no diffent from doing nothing, as it 
doesn't try to do anything if there is no OCSP response provided.  
So the code can only result in false alerts due to configuration 
errors, but won't be able to stop any attacks.

> > The "full" in turn doesn't seem to be correct feature, as stapled 
> > OCSP response may be legitimately absent for multiple reasons.
> 
> If you control the upstream servers than I don't see any reason why you
> couldn't just enable OCSP stapling unconditionally and enforce this on
> the downstream with the "full" option. Maybe I'm missing something?

Much like any other arbitrary requirement, this one of course can 
be enforced as well.  The question is how this is different from 
other arbitrary requirements we don't provide options for.

Note well that nginx itself, even when configured with OCSP 
stapling enabled, does _not_ guarantee that OCSP responses will be 
stapled in all SSL connections - as obtaining an OCSP response 
takes time, and OpenSSL does not provide a non-blocking interface 
for certificate status callbacks.

-- 
Maxim Dounin
http://nginx.org/


From alessandro at cloudflare.com  Fri Jan 22 21:47:09 2016
From: alessandro at cloudflare.com (Alessandro Ghedini)
Date: Fri, 22 Jan 2016 21:47:09 +0000
Subject: [PATCH 2 of 2] Proxy: add support for OCSP stapling verification
 from upstream
In-Reply-To: <20160122184601.GL74233@mdounin.ru>
References: <patchbomb.1453483534@mandy.local>
 <20160122173806.GC19316@mandy.local>
 <20160122174926.GK74233@mdounin.ru>
 <20160122180214.GA19389@mandy.local>
 <20160122184601.GL74233@mdounin.ru>
Message-ID: <20160122214709.GA575@mandy.cfops.it>

On Fri, Jan 22, 2016 at 09:46:01pm +0300, Maxim Dounin wrote:
> Hello!
> 
> On Fri, Jan 22, 2016 at 06:02:14PM +0000, Alessandro Ghedini wrote:
> 
> > On Fri, Jan 22, 2016 at 08:49:26pm +0300, Maxim Dounin wrote:
> > > Hello!
> > > 
> > > On Fri, Jan 22, 2016 at 05:38:06PM +0000, Alessandro Ghedini wrote:
> > > 
> > > > # HG changeset patch
> > > > # User Alessandro Ghedini <alessandro at cloudflare.com>
> > > > # Date 1453481233 0
> > > > #      Fri Jan 22 16:47:13 2016 +0000
> > > > # Node ID c6668c14a2d168307bcfade0cc2e01c92c31312a
> > > > # Parent  a8c4f65236ad90138863d5295ca059a3d37da37e
> > > > Proxy: add support for OCSP stapling verification from upstream
> > > > 
> > > > This patch adds the "proxy_ssl_stapling_verify" option that controls OCSP
> > > > stapling verification from an upstream server.
> > > > 
> > > > The option allows three values:
> > > > 
> > > >  - "off" (default): disable OCSP stapling completely.
> > > >  - "on": request OCSP stapling from upstream and verify response if
> > > >          provided.
> > > >  - "full": same as "on", but fail also when no response is received.
> > > 
> > > The "on" seems to be no different from "off" and hardly make 
> > > sense, as an attacker can easily avoid returning stapled OCSP 
> > > response.
> > 
> > Yes, of course. This is what browsers currently do, and is IMO better than
> > doing nothing. Once Must-Staple (aka "TLS Feature" x509 extension) starts
> > to be used in the wild this can be updated.
> 
> What browsers do is quite different: they use OCSP, and use OCSP 
> Stapling as an optimization.  And even if they can't obtain a 
> response, they show this visually to users.

Firefox shows an error message only if the OCSP responder actually returns a
negative response. If the connection to the responder fails Firefox just
ignores the problem ("soft-fail") and continues like nothing happened, You can
enable hard-fail mode through about:config but it's not on by default.

Chrome/Chromium doesn't even implement full OCSP anymore IIRC (it used to do
it, in soft-fail mode by default as well) and replaced it with its own CRLSet
mechanism (basically it tries OCSP stapling first, and then falls back to
checking the CRL sets).

So, no, in the default configuration most browsers behave like my patch does,
except that they may try a full OCSP request before giving up or try another
method that is even more error-prone (e.g. if the CRL/CRLset is out-of-date).

But if this is a really critical issue for you, I'd be okay with dropping the
soft-fail mode if that meant getting this functionality merged.

> > > The "full" in turn doesn't seem to be correct feature, as stapled 
> > > OCSP response may be legitimately absent for multiple reasons.
> > 
> > If you control the upstream servers than I don't see any reason why you
> > couldn't just enable OCSP stapling unconditionally and enforce this on
> > the downstream with the "full" option. Maybe I'm missing something?
> 
> Much like any other arbitrary requirement, this one of course can 
> be enforced as well.  The question is how this is different from 
> other arbitrary requirements we don't provide options for.

nginx's proxy module already supports checking CRLs, which are an even bigger
pain to deal with, and full OCSP has so many problems that it's not really a
viable option in practice (see above). As far as certificate revocation goes
that's it, there aren't any more "arbitrary requirements" as far as I know. so
it seems to me that upstreadm OCSP stapling checking would be a fairly nice and
useful improvement over the current status and while my patches aren't exactly
simple they are not that compilcated either.

One possible improvement over my current implementation is, if the user
configures both stapling and CRL checking, nginx could try stapling first and
if thet doesn't return any answer fallback to CRL. But the CRL won't return a
proper answer all the time either (e.g. even if it's only slightly out of date)
so "soft-fail" mode would still be effectively present.

Hope this helps.

Cheers


From mdounin at mdounin.ru  Mon Jan 25 14:59:30 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 25 Jan 2016 17:59:30 +0300
Subject: [PATCH 2 of 2] Proxy: add support for OCSP stapling verification
 from upstream
In-Reply-To: <20160122214709.GA575@mandy.cfops.it>
References: <patchbomb.1453483534@mandy.local>
 <20160122173806.GC19316@mandy.local>
 <20160122174926.GK74233@mdounin.ru>
 <20160122180214.GA19389@mandy.local>
 <20160122184601.GL74233@mdounin.ru>
 <20160122214709.GA575@mandy.cfops.it>
Message-ID: <20160125145929.GC9449@mdounin.ru>

Hello!

On Fri, Jan 22, 2016 at 09:47:09PM +0000, Alessandro Ghedini wrote:

> On Fri, Jan 22, 2016 at 09:46:01pm +0300, Maxim Dounin wrote:
> > Hello!
> > 
> > On Fri, Jan 22, 2016 at 06:02:14PM +0000, Alessandro Ghedini wrote:
> > 
> > > On Fri, Jan 22, 2016 at 08:49:26pm +0300, Maxim Dounin wrote:
> > > > Hello!
> > > > 
> > > > On Fri, Jan 22, 2016 at 05:38:06PM +0000, Alessandro Ghedini wrote:
> > > > 
> > > > > # HG changeset patch
> > > > > # User Alessandro Ghedini <alessandro at cloudflare.com>
> > > > > # Date 1453481233 0
> > > > > #      Fri Jan 22 16:47:13 2016 +0000
> > > > > # Node ID c6668c14a2d168307bcfade0cc2e01c92c31312a
> > > > > # Parent  a8c4f65236ad90138863d5295ca059a3d37da37e
> > > > > Proxy: add support for OCSP stapling verification from upstream
> > > > > 
> > > > > This patch adds the "proxy_ssl_stapling_verify" option that controls OCSP
> > > > > stapling verification from an upstream server.
> > > > > 
> > > > > The option allows three values:
> > > > > 
> > > > >  - "off" (default): disable OCSP stapling completely.
> > > > >  - "on": request OCSP stapling from upstream and verify response if
> > > > >          provided.
> > > > >  - "full": same as "on", but fail also when no response is received.
> > > > 
> > > > The "on" seems to be no different from "off" and hardly make 
> > > > sense, as an attacker can easily avoid returning stapled OCSP 
> > > > response.
> > > 
> > > Yes, of course. This is what browsers currently do, and is IMO better than
> > > doing nothing. Once Must-Staple (aka "TLS Feature" x509 extension) starts
> > > to be used in the wild this can be updated.
> > 
> > What browsers do is quite different: they use OCSP, and use OCSP 
> > Stapling as an optimization.  And even if they can't obtain a 
> > response, they show this visually to users.
> 
> Firefox shows an error message only if the OCSP responder actually returns a
> negative response. If the connection to the responder fails Firefox just
> ignores the problem ("soft-fail") and continues like nothing happened, You can
> enable hard-fail mode through about:config but it's not on by default.
> 
> Chrome/Chromium doesn't even implement full OCSP anymore IIRC (it used to do
> it, in soft-fail mode by default as well) and replaced it with its own CRLSet
> mechanism (basically it tries OCSP stapling first, and then falls back to
> checking the CRL sets).
> 
> So, no, in the default configuration most browsers behave like my patch does,
> except that they may try a full OCSP request before giving up or try another
> method that is even more error-prone (e.g. if the CRL/CRLset is out-of-date).

As said above, browsers use OCSP, not OCSP stapling.  And "soft    
fail" makes much more sense for OCSP than for OCSP stapling.  When
using OCSP, only an attacker in a privileged network position can
stop OCSP requests.  But if a certificate was compromised (and 
revoked accordingly), OCSP response will show that the certificate 
was revoked.  But this is not the case with OCSP stapling: an 
attacker can easily avoid returning anything stapled.

Moreover, even with OCSP "soft fail" and "hard fail" are not the 
only options available and used in practice.  E.g., Opera do not 
show the lock icon till an OCSP response is received.

[...]

> > > > The "full" in turn doesn't seem to be correct feature, as stapled 
> > > > OCSP response may be legitimately absent for multiple reasons.
> > > 
> > > If you control the upstream servers than I don't see any reason why you
> > > couldn't just enable OCSP stapling unconditionally and enforce this on
> > > the downstream with the "full" option. Maybe I'm missing something?
> > 
> > Much like any other arbitrary requirement, this one of course can 
> > be enforced as well.  The question is how this is different from 
> > other arbitrary requirements we don't provide options for.
> 
> nginx's proxy module already supports checking CRLs, which are an even bigger
> pain to deal with, and full OCSP has so many problems that it's not really a
> viable option in practice (see above). As far as certificate revocation goes
> that's it, there aren't any more "arbitrary requirements" as far as I know. so
> it seems to me that upstreadm OCSP stapling checking would be a fairly nice and
> useful improvement over the current status and while my patches aren't exactly
> simple they are not that compilcated either.

You are essentially trying to push "must staple" extension into 
nginx configuration.  And I'm not fan of both the "must staple" 
and what you are trying to do.

OCSP stapling was designed as an optimization for OCSP.  That is, 
if OCSP stapling is used, it saves an OCSP lookup.  But 
introducing "must staple" changes things a lot: now servers are 
required to provide OCSP responses even if they can't do so for 
some reason.  You can't start answering requests till you've 
loaded an OCSP response to staple it, and you essentially never know 
if will be able to start server or not.

I tend to think that "must staple" introduces much more 
complexity than it solves.  And the same applies to the 
configuration directive introduced by your patch.

-- 
Maxim Dounin
http://nginx.org/


From alessandro at cloudflare.com  Mon Jan 25 15:41:25 2016
From: alessandro at cloudflare.com (Alessandro Ghedini)
Date: Mon, 25 Jan 2016 15:41:25 +0000
Subject: [PATCH 2 of 2] Proxy: add support for OCSP stapling verification
 from upstream
In-Reply-To: <20160125145929.GC9449@mdounin.ru>
References: <patchbomb.1453483534@mandy.local>
 <20160122173806.GC19316@mandy.local>
 <20160122174926.GK74233@mdounin.ru>
 <20160122180214.GA19389@mandy.local>
 <20160122184601.GL74233@mdounin.ru>
 <20160122214709.GA575@mandy.cfops.it>
 <20160125145929.GC9449@mdounin.ru>
Message-ID: <20160125154125.GA18601@mandy.local>

> > > > > The "full" in turn doesn't seem to be correct feature, as stapled 
> > > > > OCSP response may be legitimately absent for multiple reasons.
> > > > 
> > > > If you control the upstream servers than I don't see any reason why you
> > > > couldn't just enable OCSP stapling unconditionally and enforce this on
> > > > the downstream with the "full" option. Maybe I'm missing something?
> > > 
> > > Much like any other arbitrary requirement, this one of course can 
> > > be enforced as well.  The question is how this is different from 
> > > other arbitrary requirements we don't provide options for.
> > 
> > nginx's proxy module already supports checking CRLs, which are an even bigger
> > pain to deal with, and full OCSP has so many problems that it's not really a
> > viable option in practice (see above). As far as certificate revocation goes
> > that's it, there aren't any more "arbitrary requirements" as far as I know. so
> > it seems to me that upstreadm OCSP stapling checking would be a fairly nice and
> > useful improvement over the current status and while my patches aren't exactly
> > simple they are not that compilcated either.
> 
> You are essentially trying to push "must staple" extension into 
> nginx configuration.  And I'm not fan of both the "must staple" 
> and what you are trying to do.
> 
> OCSP stapling was designed as an optimization for OCSP.  That is, 
> if OCSP stapling is used, it saves an OCSP lookup.  But 
> introducing "must staple" changes things a lot: now servers are 
> required to provide OCSP responses even if they can't do so for 
> some reason.  You can't start answering requests till you've 
> loaded an OCSP response to staple it, and you essentially never know 
> if will be able to start server or not.
> 
> I tend to think that "must staple" introduces much more 
> complexity than it solves.  And the same applies to the 
> configuration directive introduced by your patch.

Would it make a difference if I added full (not just stapling) OCSP support to
NGINX's proxy module using stapling only as an optimization as you say, or are
you against this regrdless?

That should address your concerns I think, and the code to support OCSP is
already in place anyway. Of course it would be disabled by default, so the
decision of whether enabling it is worth the trouble would be left to the
users.

Cheers


From mdounin at mdounin.ru  Mon Jan 25 19:42:50 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 25 Jan 2016 19:42:50 +0000
Subject: [nginx] Version bump.
Message-ID: <hg.5a6dd26567ee.1453750970.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/5a6dd26567ee
branches:  stable-1.8
changeset: 6337:5a6dd26567ee
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Jan 25 19:32:58 2016 +0300
description:
Version bump.

diffstat:

 src/core/nginx.h |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff --git a/src/core/nginx.h b/src/core/nginx.h
--- a/src/core/nginx.h
+++ b/src/core/nginx.h
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1008000
-#define NGINX_VERSION      "1.8.0"
+#define nginx_version      1008001
+#define NGINX_VERSION      "1.8.1"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #ifdef NGX_BUILD


From mdounin at mdounin.ru  Mon Jan 25 19:42:53 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 25 Jan 2016 19:42:53 +0000
Subject: [nginx] Merge proxy_protocol setting of listen directives.
Message-ID: <hg.f32d073b83e1.1453750973.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/f32d073b83e1
branches:  stable-1.8
changeset: 6338:f32d073b83e1
user:      Roman Arutyunyan <arut at nginx.com>
date:      Fri Apr 24 10:54:06 2015 +0300
description:
Merge proxy_protocol setting of listen directives.

It's now enough to specify proxy_protocol option in one listen directive to
enable it in all servers listening on the same address/port.  Previously,
the setting from the first directive was always used.

diffstat:

 src/http/ngx_http.c |  5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diffs (29 lines):

diff --git a/src/http/ngx_http.c b/src/http/ngx_http.c
--- a/src/http/ngx_http.c
+++ b/src/http/ngx_http.c
@@ -1220,7 +1220,7 @@ ngx_http_add_addresses(ngx_conf_t *cf, n
 {
     u_char                *p;
     size_t                 len, off;
-    ngx_uint_t             i, default_server;
+    ngx_uint_t             i, default_server, proxy_protocol;
     struct sockaddr       *sa;
     ngx_http_conf_addr_t  *addr;
 #if (NGX_HAVE_UNIX_DOMAIN)
@@ -1281,6 +1281,8 @@ ngx_http_add_addresses(ngx_conf_t *cf, n
         /* preserve default_server bit during listen options overwriting */
         default_server = addr[i].opt.default_server;
 
+        proxy_protocol = lsopt->proxy_protocol || addr[i].opt.proxy_protocol;
+
 #if (NGX_HTTP_SSL)
         ssl = lsopt->ssl || addr[i].opt.ssl;
 #endif
@@ -1314,6 +1316,7 @@ ngx_http_add_addresses(ngx_conf_t *cf, n
         }
 
         addr[i].opt.default_server = default_server;
+        addr[i].opt.proxy_protocol = proxy_protocol;
 #if (NGX_HTTP_SSL)
         addr[i].opt.ssl = ssl;
 #endif


From mdounin at mdounin.ru  Mon Jan 25 19:42:55 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 25 Jan 2016 19:42:55 +0000
Subject: [nginx] Events: made a failure to create a notification channel ...
Message-ID: <hg.67422a0a8ed2.1453750975.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/67422a0a8ed2
branches:  stable-1.8
changeset: 6339:67422a0a8ed2
user:      Ruslan Ermilov <ru at nginx.com>
date:      Wed May 06 17:04:00 2015 +0300
description:
Events: made a failure to create a notification channel non-fatal.

This may happen if eventfd() returns ENOSYS, notably seen on CentOS 5.4.
Such a failure will now just disable the notification mechanism and let
the callers cope with it, instead of failing to start worker processes.
If thread pools are not configured, this can safely be ignored.

diffstat:

 src/event/modules/ngx_epoll_module.c |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff --git a/src/event/modules/ngx_epoll_module.c b/src/event/modules/ngx_epoll_module.c
--- a/src/event/modules/ngx_epoll_module.c
+++ b/src/event/modules/ngx_epoll_module.c
@@ -329,7 +329,7 @@ ngx_epoll_init(ngx_cycle_t *cycle, ngx_m
 
 #if (NGX_HAVE_EVENTFD)
         if (ngx_epoll_notify_init(cycle->log) != NGX_OK) {
-            return NGX_ERROR;
+            ngx_epoll_module_ctx.actions.notify = NULL;
         }
 #endif
 


From mdounin at mdounin.ru  Mon Jan 25 19:42:57 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 25 Jan 2016 19:42:57 +0000
Subject: [nginx] Fixed segfault with try_files introduced by c985d90a8d1f.
Message-ID: <hg.312472807db8.1453750977.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/312472807db8
branches:  stable-1.8
changeset: 6340:312472807db8
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Sun Aug 16 10:51:16 2015 +0300
description:
Fixed segfault with try_files introduced by c985d90a8d1f.

If alias was used in a location given by a regular expression,
nginx used to do wrong thing in try_files if a location name (i.e.,
regular expression) was an exact prefix of URI.  The following
configuration triggered a segmentation fault on a request to "/mail":

    location ~ /mail {
        alias /path/to/directory;
        try_files $uri =404;
    }

Reported by Per Hansson.

diffstat:

 src/http/ngx_http_core_module.c |  4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diffs (14 lines):

diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c
+++ b/src/http/ngx_http_core_module.c
@@ -1272,7 +1272,9 @@ ngx_http_core_try_files_phase(ngx_http_r
 
             *e.pos = '\0';
 
-            if (alias && ngx_strncmp(name, clcf->name.data, alias) == 0) {
+            if (alias && alias != NGX_MAX_SIZE_T_VALUE
+                && ngx_strncmp(name, clcf->name.data, alias) == 0)
+            {
                 ngx_memmove(name, name + alias, len - alias);
                 path.len -= alias;
             }


From mdounin at mdounin.ru  Mon Jan 25 19:43:00 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 25 Jan 2016 19:43:00 +0000
Subject: [nginx] Fixed wrong URI after try_files in nested location (tick...
Message-ID: <hg.2797b4347a2a.1453750980.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/2797b4347a2a
branches:  stable-1.8
changeset: 6341:2797b4347a2a
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Sun Aug 16 10:51:34 2015 +0300
description:
Fixed wrong URI after try_files in nested location (ticket #97).

The following configuration with alias, nested location and try_files
resulted in wrong file being used.  Request "/foo/test.gif" tried to
use "/tmp//foo/test.gif" instead of "/tmp/test.gif":

    location /foo/ {
        alias /tmp/;
        location ~ gif {
            try_files $uri =405;
        }
    }

Additionally, rev. c985d90a8d1f introduced a regression if
the "/tmp//foo/test.gif" file was found (ticket #768).  Resulting URI
was set to "gif?/foo/test.gif", as the code used clcf->name of current
location ("location ~ gif") instead of parent one ("location /foo/").

Fix is to use r->uri instead of clcf->name in all cases in the
ngx_http_core_try_files_phase() function.  It is expected to be
already matched and identical to the clcf->name of the right
location.

diffstat:

 src/http/ngx_http_core_module.c |  8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

diffs (32 lines):

diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c
--- a/src/http/ngx_http_core_module.c
+++ b/src/http/ngx_http_core_module.c
@@ -1273,7 +1273,7 @@ ngx_http_core_try_files_phase(ngx_http_r
             *e.pos = '\0';
 
             if (alias && alias != NGX_MAX_SIZE_T_VALUE
-                && ngx_strncmp(name, clcf->name.data, alias) == 0)
+                && ngx_strncmp(name, r->uri.data, alias) == 0)
             {
                 ngx_memmove(name, name + alias, len - alias);
                 path.len -= alias;
@@ -1357,6 +1357,8 @@ ngx_http_core_try_files_phase(ngx_http_r
             }
 
         } else {
+            name = r->uri.data;
+
             r->uri.len = alias + path.len;
             r->uri.data = ngx_pnalloc(r->pool, r->uri.len);
             if (r->uri.data == NULL) {
@@ -1364,8 +1366,8 @@ ngx_http_core_try_files_phase(ngx_http_r
                 return NGX_OK;
             }
 
-            p = ngx_copy(r->uri.data, clcf->name.data, alias);
-            ngx_memcpy(p, name, path.len);
+            p = ngx_copy(r->uri.data, name, alias);
+            ngx_memcpy(p, path.data, path.len);
         }
 
         ngx_http_set_exten(r);


From mdounin at mdounin.ru  Mon Jan 25 19:43:02 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 25 Jan 2016 19:43:02 +0000
Subject: [nginx] Upstream: fixed cache send error handling.
Message-ID: <hg.50169ef2f3fe.1453750982.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/50169ef2f3fe
branches:  stable-1.8
changeset: 6342:50169ef2f3fe
user:      Roman Arutyunyan <arut at nginx.com>
date:      Thu Sep 03 15:09:21 2015 +0300
description:
Upstream: fixed cache send error handling.

The value of NGX_ERROR, returned from filter handlers, was treated as a generic
upstream error and changed to NGX_HTTP_INTERNAL_SERVER_ERROR before calling
ngx_http_finalize_request().  This resulted in "header already sent" alert
if header was already sent in filter handlers.

The problem appeared in 54e9b83d00f0 (1.7.5).

diffstat:

 src/http/ngx_http_upstream.c |  25 ++++++++++++++-----------
 1 files changed, 14 insertions(+), 11 deletions(-)

diffs (47 lines):

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -530,15 +530,24 @@ ngx_http_upstream_init_request(ngx_http_
 
         r->write_event_handler = ngx_http_request_empty_handler;
 
-        if (rc == NGX_DONE) {
-            return;
-        }
-
         if (rc == NGX_ERROR) {
             ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
             return;
         }
 
+        if (rc == NGX_OK) {
+            rc = ngx_http_upstream_cache_send(r, u);
+
+            if (rc == NGX_DONE) {
+                return;
+            }
+
+            if (rc == NGX_HTTP_UPSTREAM_INVALID_HEADER) {
+                rc = NGX_DECLINED;
+                r->cached = 0;
+            }
+        }
+
         if (rc != NGX_DECLINED) {
             ngx_http_finalize_request(r, rc);
             return;
@@ -833,13 +842,7 @@ ngx_http_upstream_cache(ngx_http_request
 
     case NGX_OK:
 
-        rc = ngx_http_upstream_cache_send(r, u);
-
-        if (rc != NGX_HTTP_UPSTREAM_INVALID_HEADER) {
-            return rc;
-        }
-
-        break;
+        return NGX_OK;
 
     case NGX_HTTP_CACHE_STALE:
 


From mdounin at mdounin.ru  Mon Jan 25 19:43:05 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 25 Jan 2016 19:43:05 +0000
Subject: [nginx] SSL: preserve default server context in connection (tick...
Message-ID: <hg.60ae75969588.1453750985.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/60ae75969588
branches:  stable-1.8
changeset: 6343:60ae75969588
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Oct 19 21:22:38 2015 +0300
description:
SSL: preserve default server context in connection (ticket #235).

This context is needed for shared sessions cache to work in configurations
with multiple virtual servers sharing the same port.  Unfortunately, OpenSSL
does not provide an API to access the session context, thus storing it
separately.

In collaboration with Vladimir Homutov.

diffstat:

 src/event/ngx_event_openssl.c |  21 ++++++++-------------
 src/event/ngx_event_openssl.h |   1 +
 2 files changed, 9 insertions(+), 13 deletions(-)

diffs (84 lines):

diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c
+++ b/src/event/ngx_event_openssl.c
@@ -1038,6 +1038,8 @@ ngx_ssl_create_connection(ngx_ssl_t *ssl
     sc->buffer = ((flags & NGX_SSL_BUFFER) != 0);
     sc->buffer_size = ssl->buffer_size;
 
+    sc->session_ctx = ssl->ctx;
+
     sc->connection = SSL_new(ssl->ctx);
 
     if (sc->connection == NULL) {
@@ -2303,7 +2305,7 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_
 
     c = ngx_ssl_get_connection(ssl_conn);
 
-    ssl_ctx = SSL_get_SSL_CTX(ssl_conn);
+    ssl_ctx = c->ssl->session_ctx;
     shm_zone = SSL_CTX_get_ex_data(ssl_ctx, ngx_ssl_session_cache_index);
 
     cache = shm_zone->data;
@@ -2441,21 +2443,17 @@ ngx_ssl_get_cached_session(ngx_ssl_conn_
     ngx_ssl_sess_id_t        *sess_id;
     ngx_ssl_session_cache_t  *cache;
     u_char                    buf[NGX_SSL_MAX_SESSION_SIZE];
-#if (NGX_DEBUG)
     ngx_connection_t         *c;
-#endif
 
     hash = ngx_crc32_short(id, (size_t) len);
     *copy = 0;
 
-#if (NGX_DEBUG)
     c = ngx_ssl_get_connection(ssl_conn);
 
     ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0,
                    "ssl get session: %08XD:%d", hash, len);
-#endif
-
-    shm_zone = SSL_CTX_get_ex_data(SSL_get_SSL_CTX(ssl_conn),
+
+    shm_zone = SSL_CTX_get_ex_data(c->ssl->session_ctx,
                                    ngx_ssl_session_cache_index);
 
     cache = shm_zone->data;
@@ -2834,13 +2832,14 @@ ngx_ssl_session_ticket_key_callback(ngx_
     SSL_CTX                       *ssl_ctx;
     ngx_uint_t                     i;
     ngx_array_t                   *keys;
+    ngx_connection_t              *c;
     ngx_ssl_session_ticket_key_t  *key;
 #if (NGX_DEBUG)
     u_char                         buf[32];
-    ngx_connection_t              *c;
 #endif
 
-    ssl_ctx = SSL_get_SSL_CTX(ssl_conn);
+    c = ngx_ssl_get_connection(ssl_conn);
+    ssl_ctx = c->ssl->session_ctx;
 
     keys = SSL_CTX_get_ex_data(ssl_ctx, ngx_ssl_session_ticket_keys_index);
     if (keys == NULL) {
@@ -2849,10 +2848,6 @@ ngx_ssl_session_ticket_key_callback(ngx_
 
     key = keys->elts;
 
-#if (NGX_DEBUG)
-    c = ngx_ssl_get_connection(ssl_conn);
-#endif
-
     if (enc == 1) {
         /* encrypt session ticket */
 
diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h
--- a/src/event/ngx_event_openssl.h
+++ b/src/event/ngx_event_openssl.h
@@ -46,6 +46,7 @@ typedef struct {
 
 typedef struct {
     ngx_ssl_conn_t             *connection;
+    SSL_CTX                    *session_ctx;
 
     ngx_int_t                   last;
     ngx_buf_t                  *buf;


From mdounin at mdounin.ru  Mon Jan 25 19:43:07 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 25 Jan 2016 19:43:07 +0000
Subject: [nginx] Fixed ngx_parse_time() out of bounds access (ticket #821).
Message-ID: <hg.a8ecb0a2193f.1453750987.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/a8ecb0a2193f
branches:  stable-1.8
changeset: 6344:a8ecb0a2193f
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Fri Oct 30 21:43:30 2015 +0300
description:
Fixed ngx_parse_time() out of bounds access (ticket #821).

The code failed to ensure that "s" is within the buffer passed for
parsing when checking for "ms", and this resulted in unexpected errors when
parsing non-null-terminated strings with trailing "m".  The bug manifested
itself when the expires directive was used with variables.

Found by Roman Arutyunyan.

diffstat:

 src/core/ngx_parse.c |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff --git a/src/core/ngx_parse.c b/src/core/ngx_parse.c
--- a/src/core/ngx_parse.c
+++ b/src/core/ngx_parse.c
@@ -188,7 +188,7 @@ ngx_parse_time(ngx_str_t *line, ngx_uint
             break;
 
         case 'm':
-            if (*p == 's') {
+            if (p < last && *p == 's') {
                 if (is_sec || step >= st_msec) {
                     return NGX_ERROR;
                 }


From mdounin at mdounin.ru  Mon Jan 25 19:43:10 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 25 Jan 2016 19:43:10 +0000
Subject: [nginx] SSL: only select SPDY using NPN if "spdy" is enabled.
Message-ID: <hg.5ae5142d39a3.1453750990.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/5ae5142d39a3
branches:  stable-1.8
changeset: 6345:5ae5142d39a3
user:      Valentin Bartenev <vbart at nginx.com>
date:      Thu Nov 05 15:01:09 2015 +0300
description:
SSL: only select SPDY using NPN if "spdy" is enabled.

OpenSSL doesn't check if the negotiated protocol has been announced.
As a result, the client might force using SPDY even if it wasn't
enabled in configuration.

diffstat:

 src/http/ngx_http_request.c |  28 ++++++++++++++++++----------
 1 files changed, 18 insertions(+), 10 deletions(-)

diffs (46 lines):

diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c
--- a/src/http/ngx_http_request.c
+++ b/src/http/ngx_http_request.c
@@ -770,24 +770,32 @@ ngx_http_ssl_handshake_handler(ngx_conne
         {
         unsigned int             len;
         const unsigned char     *data;
+        ngx_http_connection_t   *hc;
         static const ngx_str_t   spdy = ngx_string(NGX_SPDY_NPN_NEGOTIATED);
 
+        hc = c->data;
+
+        if (hc->addr_conf->spdy) {
+
 #ifdef TLSEXT_TYPE_application_layer_protocol_negotiation
-        SSL_get0_alpn_selected(c->ssl->connection, &data, &len);
+            SSL_get0_alpn_selected(c->ssl->connection, &data, &len);
 
 #ifdef TLSEXT_TYPE_next_proto_neg
-        if (len == 0) {
+            if (len == 0) {
+                SSL_get0_next_proto_negotiated(c->ssl->connection, &data, &len);
+            }
+#endif
+
+#else /* TLSEXT_TYPE_next_proto_neg */
             SSL_get0_next_proto_negotiated(c->ssl->connection, &data, &len);
-        }
 #endif
 
-#else /* TLSEXT_TYPE_next_proto_neg */
-        SSL_get0_next_proto_negotiated(c->ssl->connection, &data, &len);
-#endif
-
-        if (len == spdy.len && ngx_strncmp(data, spdy.data, spdy.len) == 0) {
-            ngx_http_spdy_init(c->read);
-            return;
+            if (len == spdy.len
+                && ngx_strncmp(data, spdy.data, spdy.len) == 0)
+            {
+                ngx_http_spdy_init(c->read);
+                return;
+            }
         }
         }
 #endif


From mdounin at mdounin.ru  Mon Jan 25 19:43:12 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 25 Jan 2016 19:43:12 +0000
Subject: [nginx] Updated OpenSSL and PCRE used for win32 builds.
Message-ID: <hg.e9a4531a2a5d.1453750992.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/e9a4531a2a5d
branches:  stable-1.8
changeset: 6346:e9a4531a2a5d
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Mon Jan 25 21:58:21 2016 +0300
description:
Updated OpenSSL and PCRE used for win32 builds.

diffstat:

 misc/GNUmakefile |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (15 lines):

diff --git a/misc/GNUmakefile b/misc/GNUmakefile
--- a/misc/GNUmakefile
+++ b/misc/GNUmakefile
@@ -5,9 +5,9 @@ NGINX =		nginx-$(VER)
 TEMP =		tmp
 
 OBJS =		objs.msvc8
-OPENSSL =	openssl-1.0.1m
+OPENSSL =	openssl-1.0.1q
 ZLIB =		zlib-1.2.8
-PCRE =		pcre-8.35
+PCRE =		pcre-8.38
 
 
 release: export


From mdounin at mdounin.ru  Tue Jan 26 16:27:14 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:14 +0000
Subject: [nginx] Resolver: fixed possible segmentation fault on DNS forma...
Message-ID: <hg.81d44cd4044e.1453825634.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/81d44cd4044e
branches:  
changeset: 6347:81d44cd4044e
user:      Roman Arutyunyan <arut at nginx.com>
date:      Tue Jan 26 16:46:18 2016 +0300
description:
Resolver: fixed possible segmentation fault on DNS format error.

diffstat:

 src/core/ngx_resolver.c |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -1312,7 +1312,7 @@ ngx_resolver_process_response(ngx_resolv
         times = 0;
 
         for (q = ngx_queue_head(&r->name_resend_queue);
-             q != ngx_queue_sentinel(&r->name_resend_queue) || times++ < 100;
+             q != ngx_queue_sentinel(&r->name_resend_queue) && times++ < 100;
              q = ngx_queue_next(q))
         {
             rn = ngx_queue_data(q, ngx_resolver_node_t, queue);


From mdounin at mdounin.ru  Tue Jan 26 16:27:17 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:17 +0000
Subject: [nginx] Resolver: fixed crashes in timeout handler.
Message-ID: <hg.7316c57e4fe7.1453825637.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/7316c57e4fe7
branches:  
changeset: 6348:7316c57e4fe7
user:      Ruslan Ermilov <ru at nginx.com>
date:      Tue Jan 26 16:46:31 2016 +0300
description:
Resolver: fixed crashes in timeout handler.

If one or more requests were waiting for a response, then after
getting a CNAME response, the timeout event on the first request
remained active, pointing to the wrong node with an empty
rn->waiting list, and that could cause either null pointer
dereference or use-after-free memory access if this timeout
expired.

If several requests were waiting for a response, and the first
request terminated (e.g., due to client closing a connection),
other requests were left without a timeout and could potentially
wait indefinitely.

This is fixed by introducing per-request independent timeouts.
This change also reverts 954867a2f0a6 and 5004210e8c78.

diffstat:

 src/core/ngx_resolver.c |  60 +++++++++++++++++++++++++++++++-----------------
 src/core/ngx_resolver.h |  13 ++++-----
 2 files changed, 45 insertions(+), 28 deletions(-)

diffs (147 lines):

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -423,7 +423,7 @@ ngx_resolve_name_done(ngx_resolver_ctx_t
 
     /* lock name mutex */
 
-    if (ctx->state == NGX_AGAIN) {
+    if (ctx->state == NGX_AGAIN || ctx->state == NGX_RESOLVE_TIMEDOUT) {
 
         hash = ngx_crc32_short(ctx->name.data, ctx->name.len);
 
@@ -581,6 +581,20 @@ ngx_resolve_name_locked(ngx_resolver_t *
 
         if (rn->waiting) {
 
+            if (ctx->event == NULL) {
+                ctx->event = ngx_resolver_calloc(r, sizeof(ngx_event_t));
+                if (ctx->event == NULL) {
+                    return NGX_ERROR;
+                }
+
+                ctx->event->handler = ngx_resolver_timeout_handler;
+                ctx->event->data = ctx;
+                ctx->event->log = r->log;
+                ctx->ident = -1;
+
+                ngx_add_timer(ctx->event, ctx->timeout);
+            }
+
             ctx->next = rn->waiting;
             rn->waiting = ctx;
             ctx->state = NGX_AGAIN;
@@ -674,9 +688,9 @@ ngx_resolve_name_locked(ngx_resolver_t *
         }
 
         ctx->event->handler = ngx_resolver_timeout_handler;
-        ctx->event->data = rn;
+        ctx->event->data = ctx;
         ctx->event->log = r->log;
-        rn->ident = -1;
+        ctx->ident = -1;
 
         ngx_add_timer(ctx->event, ctx->timeout);
     }
@@ -804,6 +818,18 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx
 
         if (rn->waiting) {
 
+            ctx->event = ngx_resolver_calloc(r, sizeof(ngx_event_t));
+            if (ctx->event == NULL) {
+                return NGX_ERROR;
+            }
+
+            ctx->event->handler = ngx_resolver_timeout_handler;
+            ctx->event->data = ctx;
+            ctx->event->log = r->log;
+            ctx->ident = -1;
+
+            ngx_add_timer(ctx->event, ctx->timeout);
+
             ctx->next = rn->waiting;
             rn->waiting = ctx;
             ctx->state = NGX_AGAIN;
@@ -867,9 +893,9 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx
     }
 
     ctx->event->handler = ngx_resolver_timeout_handler;
-    ctx->event->data = rn;
+    ctx->event->data = ctx;
     ctx->event->log = r->log;
-    rn->ident = -1;
+    ctx->ident = -1;
 
     ngx_add_timer(ctx->event, ctx->timeout);
 
@@ -959,7 +985,7 @@ ngx_resolve_addr_done(ngx_resolver_ctx_t
 
     /* lock addr mutex */
 
-    if (ctx->state == NGX_AGAIN) {
+    if (ctx->state == NGX_AGAIN || ctx->state == NGX_RESOLVE_TIMEDOUT) {
 
         switch (ctx->addr.sockaddr->sa_family) {
 
@@ -2815,21 +2841,13 @@ done:
 static void
 ngx_resolver_timeout_handler(ngx_event_t *ev)
 {
-    ngx_resolver_ctx_t   *ctx, *next;
-    ngx_resolver_node_t  *rn;
-
-    rn = ev->data;
-    ctx = rn->waiting;
-    rn->waiting = NULL;
-
-    do {
-        ctx->state = NGX_RESOLVE_TIMEDOUT;
-        next = ctx->next;
-
-        ctx->handler(ctx);
-
-        ctx = next;
-    } while (ctx);
+    ngx_resolver_ctx_t  *ctx;
+
+    ctx = ev->data;
+
+    ctx->state = NGX_RESOLVE_TIMEDOUT;
+
+    ctx->handler(ctx);
 }
 
 
diff --git a/src/core/ngx_resolver.h b/src/core/ngx_resolver.h
--- a/src/core/ngx_resolver.h
+++ b/src/core/ngx_resolver.h
@@ -51,16 +51,12 @@ typedef void (*ngx_resolver_handler_pt)(
 
 
 typedef struct {
+    ngx_rbtree_node_t         node;
+    ngx_queue_t               queue;
+
     /* PTR: resolved name, A: name to resolve */
     u_char                   *name;
 
-    ngx_queue_t               queue;
-
-    /* event ident must be after 3 pointers as in ngx_connection_t */
-    ngx_int_t                 ident;
-
-    ngx_rbtree_node_t         node;
-
 #if (NGX_HAVE_INET6)
     /* PTR: IPv6 address to resolve (IPv4 address is in rbtree node key) */
     struct in6_addr           addr6;
@@ -147,6 +143,9 @@ struct ngx_resolver_ctx_s {
     ngx_resolver_t           *resolver;
     ngx_udp_connection_t     *udp_connection;
 
+    /* event ident must be after 3 pointers as in ngx_connection_t */
+    ngx_int_t                 ident;
+
     ngx_int_t                 state;
     ngx_str_t                 name;
 


From mdounin at mdounin.ru  Tue Jan 26 16:27:19 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:19 +0000
Subject: [nginx] Resolver: fixed CNAME processing for several requests.
Message-ID: <hg.978e79b95c9f.1453825639.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/978e79b95c9f
branches:  
changeset: 6349:978e79b95c9f
user:      Ruslan Ermilov <ru at nginx.com>
date:      Tue Jan 26 16:46:38 2016 +0300
description:
Resolver: fixed CNAME processing for several requests.

When several requests were waiting for a response, then after getting
a CNAME response only the last request was properly processed, while
others were left waiting.

diffstat:

 src/core/ngx_resolver.c |  21 +++++++++++++++------
 1 files changed, 15 insertions(+), 6 deletions(-)

diffs (66 lines):

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -478,7 +478,7 @@ ngx_resolve_name_locked(ngx_resolver_t *
     ngx_int_t             rc;
     ngx_uint_t            naddrs;
     ngx_addr_t           *addrs;
-    ngx_resolver_ctx_t   *next;
+    ngx_resolver_ctx_t   *next, *last;
     ngx_resolver_node_t  *rn;
 
     ngx_strlow(ctx->name.data, ctx->name.data, ctx->name.len);
@@ -489,6 +489,9 @@ ngx_resolve_name_locked(ngx_resolver_t *
 
     if (rn) {
 
+        /* ctx can be a list after NGX_RESOLVE_CNAME */
+        for (last = ctx; last->next; last = last->next);
+
         if (rn->valid >= ngx_time()) {
 
             ngx_log_debug0(NGX_LOG_DEBUG_CORE, r->log, 0, "resolve cached");
@@ -516,7 +519,7 @@ ngx_resolve_name_locked(ngx_resolver_t *
                     }
                 }
 
-                ctx->next = rn->waiting;
+                last->next = rn->waiting;
                 rn->waiting = NULL;
 
                 /* unlock name mutex */
@@ -562,7 +565,7 @@ ngx_resolve_name_locked(ngx_resolver_t *
                 return ngx_resolve_name_locked(r, ctx);
             }
 
-            ctx->next = rn->waiting;
+            last->next = rn->waiting;
             rn->waiting = NULL;
 
             /* unlock name mutex */
@@ -595,7 +598,7 @@ ngx_resolve_name_locked(ngx_resolver_t *
                 ngx_add_timer(ctx->event, ctx->timeout);
             }
 
-            ctx->next = rn->waiting;
+            last->next = rn->waiting;
             rn->waiting = ctx;
             ctx->state = NGX_AGAIN;
 
@@ -666,8 +669,14 @@ ngx_resolve_name_locked(ngx_resolver_t *
         ngx_resolver_free(r, rn->name);
         ngx_resolver_free(r, rn);
 
-        ctx->state = NGX_RESOLVE_NXDOMAIN;
-        ctx->handler(ctx);
+        do {
+            ctx->state = NGX_RESOLVE_NXDOMAIN;
+            next = ctx->next;
+
+            ctx->handler(ctx);
+
+            ctx = next;
+        } while (ctx);
 
         return NGX_OK;
     }


From mdounin at mdounin.ru  Tue Jan 26 16:27:22 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:22 +0000
Subject: [nginx] Resolver: changed the ngx_resolver_create_*_query() argu...
Message-ID: <hg.a5767988c022.1453825642.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/a5767988c022
branches:  
changeset: 6350:a5767988c022
user:      Roman Arutyunyan <arut at nginx.com>
date:      Tue Jan 26 16:46:48 2016 +0300
description:
Resolver: changed the ngx_resolver_create_*_query() arguments.

No functional changes.

This is needed by the following change.

diffstat:

 src/core/ngx_resolver.c |  57 +++++++++++++++++++++++-------------------------
 1 files changed, 27 insertions(+), 30 deletions(-)

diffs (167 lines):

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -64,10 +64,10 @@ static void ngx_resolver_expire(ngx_reso
     ngx_queue_t *queue);
 static ngx_int_t ngx_resolver_send_query(ngx_resolver_t *r,
     ngx_resolver_node_t *rn);
-static ngx_int_t ngx_resolver_create_name_query(ngx_resolver_node_t *rn,
-    ngx_resolver_ctx_t *ctx);
-static ngx_int_t ngx_resolver_create_addr_query(ngx_resolver_node_t *rn,
-    ngx_resolver_ctx_t *ctx);
+static ngx_int_t ngx_resolver_create_name_query(ngx_resolver_t *r,
+    ngx_resolver_node_t *rn, ngx_str_t *name);
+static ngx_int_t ngx_resolver_create_addr_query(ngx_resolver_t *r,
+    ngx_resolver_node_t *rn, ngx_addr_t *addr);
 static void ngx_resolver_resend_handler(ngx_event_t *ev);
 static time_t ngx_resolver_resend(ngx_resolver_t *r, ngx_rbtree_t *tree,
     ngx_queue_t *queue);
@@ -656,7 +656,7 @@ ngx_resolve_name_locked(ngx_resolver_t *
         ngx_rbtree_insert(&r->name_rbtree, &rn->node);
     }
 
-    rc = ngx_resolver_create_name_query(rn, ctx);
+    rc = ngx_resolver_create_name_query(r, rn, &ctx->name);
 
     if (rc == NGX_ERROR) {
         goto failed;
@@ -883,7 +883,7 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx
         ngx_rbtree_insert(tree, &rn->node);
     }
 
-    if (ngx_resolver_create_addr_query(rn, ctx) != NGX_OK) {
+    if (ngx_resolver_create_addr_query(r, rn, &ctx->addr) != NGX_OK) {
         goto failed;
     }
 
@@ -2531,27 +2531,23 @@ ngx_resolver_rbtree_insert_addr6_value(n
 
 
 static ngx_int_t
-ngx_resolver_create_name_query(ngx_resolver_node_t *rn, ngx_resolver_ctx_t *ctx)
+ngx_resolver_create_name_query(ngx_resolver_t *r, ngx_resolver_node_t *rn,
+    ngx_str_t *name)
 {
     u_char              *p, *s;
     size_t               len, nlen;
     ngx_uint_t           ident;
-#if (NGX_HAVE_INET6)
-    ngx_resolver_t      *r;
-#endif
     ngx_resolver_qs_t   *qs;
     ngx_resolver_hdr_t  *query;
 
-    nlen = ctx->name.len ? (1 + ctx->name.len + 1) : 1;
+    nlen = name->len ? (1 + name->len + 1) : 1;
 
     len = sizeof(ngx_resolver_hdr_t) + nlen + sizeof(ngx_resolver_qs_t);
 
 #if (NGX_HAVE_INET6)
-    r = ctx->resolver;
-
-    p = ngx_resolver_alloc(ctx->resolver, r->ipv6 ? len * 2 : len);
+    p = ngx_resolver_alloc(r, r->ipv6 ? len * 2 : len);
 #else
-    p = ngx_resolver_alloc(ctx->resolver, len);
+    p = ngx_resolver_alloc(r, len);
 #endif
     if (p == NULL) {
         return NGX_ERROR;
@@ -2570,8 +2566,8 @@ ngx_resolver_create_name_query(ngx_resol
 
     ident = ngx_random();
 
-    ngx_log_debug2(NGX_LOG_DEBUG_CORE, ctx->resolver->log, 0,
-                   "resolve: \"%V\" A %i", &ctx->name, ident & 0xffff);
+    ngx_log_debug2(NGX_LOG_DEBUG_CORE, r->log, 0,
+                   "resolve: \"%V\" A %i", name, ident & 0xffff);
 
     query->ident_hi = (u_char) ((ident >> 8) & 0xff);
     query->ident_lo = (u_char) (ident & 0xff);
@@ -2601,11 +2597,11 @@ ngx_resolver_create_name_query(ngx_resol
     p--;
     *p-- = '\0';
 
-    if (ctx->name.len == 0)  {
+    if (name->len == 0)  {
         return NGX_DECLINED;
     }
 
-    for (s = ctx->name.data + ctx->name.len - 1; s >= ctx->name.data; s--) {
+    for (s = name->data + name->len - 1; s >= name->data; s--) {
         if (*s != '.') {
             *p = *s;
             len++;
@@ -2641,8 +2637,8 @@ ngx_resolver_create_name_query(ngx_resol
 
     ident = ngx_random();
 
-    ngx_log_debug2(NGX_LOG_DEBUG_CORE, ctx->resolver->log, 0,
-                   "resolve: \"%V\" AAAA %i", &ctx->name, ident & 0xffff);
+    ngx_log_debug2(NGX_LOG_DEBUG_CORE, r->log, 0,
+                   "resolve: \"%V\" AAAA %i", name, ident & 0xffff);
 
     query->ident_hi = (u_char) ((ident >> 8) & 0xff);
     query->ident_lo = (u_char) (ident & 0xff);
@@ -2659,11 +2655,12 @@ ngx_resolver_create_name_query(ngx_resol
 
 
 static ngx_int_t
-ngx_resolver_create_addr_query(ngx_resolver_node_t *rn, ngx_resolver_ctx_t *ctx)
+ngx_resolver_create_addr_query(ngx_resolver_t *r, ngx_resolver_node_t *rn,
+    ngx_addr_t *addr)
 {
     u_char               *p, *d;
     size_t                len;
-    in_addr_t             addr;
+    in_addr_t             inaddr;
     ngx_int_t             n;
     ngx_uint_t            ident;
     ngx_resolver_hdr_t   *query;
@@ -2672,7 +2669,7 @@ ngx_resolver_create_addr_query(ngx_resol
     struct sockaddr_in6  *sin6;
 #endif
 
-    switch (ctx->addr.sockaddr->sa_family) {
+    switch (addr->sockaddr->sa_family) {
 
 #if (NGX_HAVE_INET6)
     case AF_INET6:
@@ -2689,7 +2686,7 @@ ngx_resolver_create_addr_query(ngx_resol
               + sizeof(ngx_resolver_qs_t);
     }
 
-    p = ngx_resolver_alloc(ctx->resolver, len);
+    p = ngx_resolver_alloc(r, len);
     if (p == NULL) {
         return NGX_ERROR;
     }
@@ -2713,11 +2710,11 @@ ngx_resolver_create_addr_query(ngx_resol
 
     p += sizeof(ngx_resolver_hdr_t);
 
-    switch (ctx->addr.sockaddr->sa_family) {
+    switch (addr->sockaddr->sa_family) {
 
 #if (NGX_HAVE_INET6)
     case AF_INET6:
-        sin6 = (struct sockaddr_in6 *) ctx->addr.sockaddr;
+        sin6 = (struct sockaddr_in6 *) addr->sockaddr;
 
         for (n = 15; n >= 0; n--) {
             p = ngx_sprintf(p, "\1%xd\1%xd",
@@ -2732,11 +2729,11 @@ ngx_resolver_create_addr_query(ngx_resol
 
     default: /* AF_INET */
 
-        sin = (struct sockaddr_in *) ctx->addr.sockaddr;
-        addr = ntohl(sin->sin_addr.s_addr);
+        sin = (struct sockaddr_in *) addr->sockaddr;
+        inaddr = ntohl(sin->sin_addr.s_addr);
 
         for (n = 0; n < 32; n += 8) {
-            d = ngx_sprintf(&p[1], "%ud", (addr >> n) & 0xff);
+            d = ngx_sprintf(&p[1], "%ud", (inaddr >> n) & 0xff);
             *p = (u_char) (d - &p[1]);
             p = d;
         }


From mdounin at mdounin.ru  Tue Jan 26 16:27:24 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:24 +0000
Subject: [nginx] Resolver: fixed use-after-free memory accesses with CNAME.
Message-ID: <hg.497d0cff8ace.1453825644.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/497d0cff8ace
branches:  
changeset: 6351:497d0cff8ace
user:      Roman Arutyunyan <arut at nginx.com>
date:      Tue Jan 26 16:46:59 2016 +0300
description:
Resolver: fixed use-after-free memory accesses with CNAME.

When several requests were waiting for a response, then after getting
a CNAME response only the last request's context had the name updated.
Contexts of other requests had the wrong name.  This name was used by
ngx_resolve_name_done() to find the node to remove the request context
from.  When the name was wrong, the request could not be properly
cancelled, its context was freed but stayed linked to the node's waiting
list.  This happened e.g. when the first request was aborted or timed
out before the resolving completed.  When it completed, this triggered
a use-after-free memory access by calling ctx->handler of already freed
request context.  The bug manifests itself by
"could not cancel <name> resolving" alerts in error_log.

When a request was responded with a CNAME, the request context kept
the pointer to the original node's rn->u.cname.  If the original node
expired before the resolving timed out or completed with an error,
this would trigger a use-after-free memory access via ctx->name in
ctx->handler().

The fix is to keep ctx->name unmodified.  The name from context
is no longer used by ngx_resolve_name_done().  Instead, we now keep
the pointer to resolver node to which this request is linked.
Keeping the original name intact also improves logging.

diffstat:

 src/core/ngx_resolver.c |  80 ++++++++++++++++++++++--------------------------
 src/core/ngx_resolver.h |   2 +
 2 files changed, 39 insertions(+), 43 deletions(-)

diffs (219 lines):

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -59,7 +59,7 @@ ngx_int_t ngx_udp_connect(ngx_udp_connec
 static void ngx_resolver_cleanup(void *data);
 static void ngx_resolver_cleanup_tree(ngx_resolver_t *r, ngx_rbtree_t *tree);
 static ngx_int_t ngx_resolve_name_locked(ngx_resolver_t *r,
-    ngx_resolver_ctx_t *ctx);
+    ngx_resolver_ctx_t *ctx, ngx_str_t *name);
 static void ngx_resolver_expire(ngx_resolver_t *r, ngx_rbtree_t *tree,
     ngx_queue_t *queue);
 static ngx_int_t ngx_resolver_send_query(ngx_resolver_t *r,
@@ -376,7 +376,7 @@ ngx_resolve_name(ngx_resolver_ctx_t *ctx
 
     /* lock name mutex */
 
-    rc = ngx_resolve_name_locked(r, ctx);
+    rc = ngx_resolve_name_locked(r, ctx, &ctx->name);
 
     if (rc == NGX_OK) {
         return NGX_OK;
@@ -403,7 +403,6 @@ ngx_resolve_name(ngx_resolver_ctx_t *ctx
 void
 ngx_resolve_name_done(ngx_resolver_ctx_t *ctx)
 {
-    uint32_t              hash;
     ngx_resolver_t       *r;
     ngx_resolver_ctx_t   *w, **p;
     ngx_resolver_node_t  *rn;
@@ -425,9 +424,7 @@ ngx_resolve_name_done(ngx_resolver_ctx_t
 
     if (ctx->state == NGX_AGAIN || ctx->state == NGX_RESOLVE_TIMEDOUT) {
 
-        hash = ngx_crc32_short(ctx->name.data, ctx->name.len);
-
-        rn = ngx_resolver_lookup_name(r, &ctx->name, hash);
+        rn = ctx->node;
 
         if (rn) {
             p = &rn->waiting;
@@ -472,20 +469,22 @@ done:
 
 
 static ngx_int_t
-ngx_resolve_name_locked(ngx_resolver_t *r, ngx_resolver_ctx_t *ctx)
+ngx_resolve_name_locked(ngx_resolver_t *r, ngx_resolver_ctx_t *ctx,
+    ngx_str_t *name)
 {
     uint32_t              hash;
     ngx_int_t             rc;
+    ngx_str_t             cname;
     ngx_uint_t            naddrs;
     ngx_addr_t           *addrs;
     ngx_resolver_ctx_t   *next, *last;
     ngx_resolver_node_t  *rn;
 
-    ngx_strlow(ctx->name.data, ctx->name.data, ctx->name.len);
-
-    hash = ngx_crc32_short(ctx->name.data, ctx->name.len);
-
-    rn = ngx_resolver_lookup_name(r, &ctx->name, hash);
+    ngx_strlow(name->data, name->data, name->len);
+
+    hash = ngx_crc32_short(name->data, name->len);
+
+    rn = ngx_resolver_lookup_name(r, name, hash);
 
     if (rn) {
 
@@ -559,10 +558,10 @@ ngx_resolve_name_locked(ngx_resolver_t *
 
             if (ctx->recursion++ < NGX_RESOLVER_MAX_RECURSION) {
 
-                ctx->name.len = rn->cnlen;
-                ctx->name.data = rn->u.cname;
-
-                return ngx_resolve_name_locked(r, ctx);
+                cname.len = rn->cnlen;
+                cname.data = rn->u.cname;
+
+                return ngx_resolve_name_locked(r, ctx, &cname);
             }
 
             last->next = rn->waiting;
@@ -602,6 +601,11 @@ ngx_resolve_name_locked(ngx_resolver_t *
             rn->waiting = ctx;
             ctx->state = NGX_AGAIN;
 
+            do {
+                ctx->node = rn;
+                ctx = ctx->next;
+            } while (ctx);
+
             return NGX_AGAIN;
         }
 
@@ -640,14 +644,14 @@ ngx_resolve_name_locked(ngx_resolver_t *
             return NGX_ERROR;
         }
 
-        rn->name = ngx_resolver_dup(r, ctx->name.data, ctx->name.len);
+        rn->name = ngx_resolver_dup(r, name->data, name->len);
         if (rn->name == NULL) {
             ngx_resolver_free(r, rn);
             return NGX_ERROR;
         }
 
         rn->node.key = hash;
-        rn->nlen = (u_short) ctx->name.len;
+        rn->nlen = (u_short) name->len;
         rn->query = NULL;
 #if (NGX_HAVE_INET6)
         rn->query6 = NULL;
@@ -656,7 +660,7 @@ ngx_resolve_name_locked(ngx_resolver_t *
         ngx_rbtree_insert(&r->name_rbtree, &rn->node);
     }
 
-    rc = ngx_resolver_create_name_query(r, rn, &ctx->name);
+    rc = ngx_resolver_create_name_query(r, rn, name);
 
     if (rc == NGX_ERROR) {
         goto failed;
@@ -720,6 +724,11 @@ ngx_resolve_name_locked(ngx_resolver_t *
 
     ctx->state = NGX_AGAIN;
 
+    do {
+        ctx->node = rn;
+        ctx = ctx->next;
+    } while (ctx);
+
     return NGX_AGAIN;
 
 failed:
@@ -842,6 +851,7 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx
             ctx->next = rn->waiting;
             rn->waiting = ctx;
             ctx->state = NGX_AGAIN;
+            ctx->node = rn;
 
             /* unlock addr mutex */
 
@@ -927,6 +937,7 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx
     /* unlock addr mutex */
 
     ctx->state = NGX_AGAIN;
+    ctx->node = rn;
 
     return NGX_OK;
 
@@ -957,17 +968,11 @@ failed:
 void
 ngx_resolve_addr_done(ngx_resolver_ctx_t *ctx)
 {
-    in_addr_t             addr;
     ngx_queue_t          *expire_queue;
     ngx_rbtree_t         *tree;
     ngx_resolver_t       *r;
     ngx_resolver_ctx_t   *w, **p;
-    struct sockaddr_in   *sin;
     ngx_resolver_node_t  *rn;
-#if (NGX_HAVE_INET6)
-    uint32_t              hash;
-    struct sockaddr_in6  *sin6;
-#endif
 
     r = ctx->resolver;
 
@@ -996,21 +1001,7 @@ ngx_resolve_addr_done(ngx_resolver_ctx_t
 
     if (ctx->state == NGX_AGAIN || ctx->state == NGX_RESOLVE_TIMEDOUT) {
 
-        switch (ctx->addr.sockaddr->sa_family) {
-
-#if (NGX_HAVE_INET6)
-        case AF_INET6:
-            sin6 = (struct sockaddr_in6 *) ctx->addr.sockaddr;
-            hash = ngx_crc32_short(sin6->sin6_addr.s6_addr, 16);
-            rn = ngx_resolver_lookup_addr6(r, &sin6->sin6_addr, hash);
-            break;
-#endif
-
-        default: /* AF_INET */
-            sin = (struct sockaddr_in *) ctx->addr.sockaddr;
-            addr = ntohl(sin->sin_addr.s_addr);
-            rn = ngx_resolver_lookup_addr(r, addr);
-        }
+        rn = ctx->node;
 
         if (rn) {
             p = &rn->waiting;
@@ -2014,9 +2005,12 @@ ngx_resolver_process_a(ngx_resolver_t *r
         rn->waiting = NULL;
 
         if (ctx) {
-            ctx->name = name;
-
-            (void) ngx_resolve_name_locked(r, ctx);
+
+            for (next = ctx; next; next = next->next) {
+                next->node = NULL;
+            }
+
+            (void) ngx_resolve_name_locked(r, ctx, &name);
         }
 
         ngx_resolver_free(r, rn->query);
diff --git a/src/core/ngx_resolver.h b/src/core/ngx_resolver.h
--- a/src/core/ngx_resolver.h
+++ b/src/core/ngx_resolver.h
@@ -161,6 +161,8 @@ struct ngx_resolver_ctx_s {
     ngx_uint_t                quick;  /* unsigned  quick:1; */
     ngx_uint_t                recursion;
     ngx_event_t              *event;
+
+    ngx_resolver_node_t      *node;
 };
 
 


From mdounin at mdounin.ru  Tue Jan 26 16:27:26 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:26 +0000
Subject: [nginx] Resolver: limited CNAME recursion.
Message-ID: <hg.ff9b32c0e141.1453825646.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/ff9b32c0e141
branches:  
changeset: 6352:ff9b32c0e141
user:      Ruslan Ermilov <ru at nginx.com>
date:      Tue Jan 26 16:47:14 2016 +0300
description:
Resolver: limited CNAME recursion.

Previously, the recursion was only limited for cached responses.

diffstat:

 src/core/ngx_resolver.c |  40 ++++++++++++++++++++++++++++------------
 1 files changed, 28 insertions(+), 12 deletions(-)

diffs (56 lines):

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -2001,24 +2001,40 @@ ngx_resolver_process_a(ngx_resolver_t *r
 
         ngx_queue_insert_head(&r->name_expire_queue, &rn->queue);
 
-        ctx = rn->waiting;
-        rn->waiting = NULL;
-
-        if (ctx) {
-
-            for (next = ctx; next; next = next->next) {
-                next->node = NULL;
-            }
-
-            (void) ngx_resolve_name_locked(r, ctx, &name);
-        }
-
         ngx_resolver_free(r, rn->query);
         rn->query = NULL;
 #if (NGX_HAVE_INET6)
         rn->query6 = NULL;
 #endif
 
+        ctx = rn->waiting;
+        rn->waiting = NULL;
+
+        if (ctx) {
+
+            if (ctx->recursion++ >= NGX_RESOLVER_MAX_RECURSION) {
+
+                /* unlock name mutex */
+
+                do {
+                    ctx->state = NGX_RESOLVE_NXDOMAIN;
+                    next = ctx->next;
+
+                    ctx->handler(ctx);
+
+                    ctx = next;
+                } while (ctx);
+
+                return;
+            }
+
+            for (next = ctx; next; next = next->next) {
+                next->node = NULL;
+            }
+
+            (void) ngx_resolve_name_locked(r, ctx, &name);
+        }
+
         /* unlock name mutex */
 
         return;


From mdounin at mdounin.ru  Tue Jan 26 16:27:29 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:29 +0000
Subject: [nginx] nginx-1.9.10-RELEASE
Message-ID: <hg.be00ca08e41a.1453825649.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/be00ca08e41a
branches:  
changeset: 6353:be00ca08e41a
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Jan 26 17:27:40 2016 +0300
description:
nginx-1.9.10-RELEASE

diffstat:

 docs/xml/nginx/changes.xml |  103 +++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 103 insertions(+), 0 deletions(-)

diffs (113 lines):

diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml
--- a/docs/xml/nginx/changes.xml
+++ b/docs/xml/nginx/changes.xml
@@ -5,6 +5,109 @@
 <change_log title="nginx">
 
 
+<changes ver="1.9.10" date="26.01.2016">
+
+<change type="security">
+<para lang="ru">
+??? ????????????? ????????? resolver
+?? ????? ????????? ??????? DNS-???????
+????? ??????????? ????????????? ????????????? ??????,
+??? ????????? ??????????,
+???????? ??????????? ??????????? UDP-?????? ?? DNS-???????,
+??????? segmentation fault ? ??????? ???????? (CVE-2016-0742).
+</para>
+<para lang="en">
+invalid pointer dereference might occur
+during DNS server response processing
+if the "resolver" directive was used,
+allowing an attacker who is able to forge UDP packets from the DNS server
+to cause segmentation fault in a worker process (CVE-2016-0742).
+</para>
+</change>
+
+<change type="security">
+<para lang="ru">
+??? ????????????? ????????? resolver
+?? ????? ????????? CNAME-???????
+????? ????????? ????????? ? ????? ????????????? ??????,
+??? ????????? ??????????,
+???????? ??????????? ???????????? ?????????????? ???????????? ???? ? ??????,
+??????? segmentation fault ? ??????? ????????,
+? ????? ???????????? ????? ????? ?????? ??????????? (CVE-2016-0746).
+</para>
+<para lang="en">
+use-after-free condition might occur
+during CNAME response processing
+if the "resolver" directive was used,
+allowing an attacker who is able to trigger name resolution
+to cause segmentation fault in a worker process,
+or might have potential other impact (CVE-2016-0746).
+</para>
+</change>
+
+<change type="security">
+<para lang="ru">
+??? ????????????? ????????? resolver
+?? ????? ????????? CNAME-???????
+?? ?? ???? ??????? ??????????? ???????????
+?? ???????????? ?????????? ??????? ? ???????,
+??? ????????? ??????????,
+???????? ??????????? ???????????? ?????????????? ???????????? ???? ? ??????,
+??????? ?????????? ??????????? ???????? ???????? ?????????? (CVE-2016-0747).
+</para>
+<para lang="en">
+CNAME resolution was insufficiently limited
+if the "resolver" directive was used,
+allowing an attacker who is able to trigger arbitrary name resolution
+to cause excessive resource consumption in worker processes (CVE-2016-0747).
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+???????? auto ????????? worker_cpu_affinity.
+</para>
+<para lang="en">
+the "auto" parameter of the "worker_cpu_affinity" directive.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+???????? proxy_protocol ????????? listen ?? ???????
+? IPv6 listen-????????.
+</para>
+<para lang="en">
+the "proxy_protocol" parameter of the "listen" directive did not work
+with IPv6 listen sockets.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ????????????? ????????? keepalive
+?????????? ? ???????? ????? ???????????? ???????????.
+</para>
+<para lang="en">
+connections to upstream servers might be cached incorrectly
+when using the "keepalive" directive.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+????? ??????????????? ??????? ? ??????? X-Accel-Redirect
+??? ????????????? ????????????? HTTP-????? ????????????? ???????.
+</para>
+<para lang="en">
+proxying used the HTTP method of the original request
+after an "X-Accel-Redirect" redirection.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.9.9" date="09.12.2015">
 
 <change type="bugfix">

From mdounin at mdounin.ru  Tue Jan 26 16:27:31 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:31 +0000
Subject: [nginx] release-1.9.10 tag
Message-ID: <hg.75c71083f4cc.1453825651.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/75c71083f4cc
branches:  
changeset: 6354:75c71083f4cc
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Jan 26 17:27:41 2016 +0300
description:
release-1.9.10 tag

diffstat:

 .hgtags |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (8 lines):

diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -392,3 +392,4 @@ b78018cfaa2f0ec20494fccb16252daa87c48a31
 54117529e40b988590ea2d38aae909b0b191663f release-1.9.7
 1bdc497c81607d854e3edf8b9a3be324c3d136b6 release-1.9.8
 ef107f3ddc237a3007e2769ec04adde0dcf627fa release-1.9.9
+be00ca08e41a69e585b6aff70a725ed6c9e1a876 release-1.9.10


From mdounin at mdounin.ru  Tue Jan 26 16:27:34 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:34 +0000
Subject: [nginx] Resolver: fixed possible segmentation fault on DNS forma...
Message-ID: <hg.c36482d0a79f.1453825654.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/c36482d0a79f
branches:  stable-1.8
changeset: 6355:c36482d0a79f
user:      Roman Arutyunyan <arut at nginx.com>
date:      Tue Jan 26 16:46:18 2016 +0300
description:
Resolver: fixed possible segmentation fault on DNS format error.

diffstat:

 src/core/ngx_resolver.c |  2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diffs (12 lines):

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -1292,7 +1292,7 @@ ngx_resolver_process_response(ngx_resolv
         times = 0;
 
         for (q = ngx_queue_head(&r->name_resend_queue);
-             q != ngx_queue_sentinel(&r->name_resend_queue) || times++ < 100;
+             q != ngx_queue_sentinel(&r->name_resend_queue) && times++ < 100;
              q = ngx_queue_next(q))
         {
             rn = ngx_queue_data(q, ngx_resolver_node_t, queue);


From mdounin at mdounin.ru  Tue Jan 26 16:27:36 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:36 +0000
Subject: [nginx] Resolver: fixed crashes in timeout handler.
Message-ID: <hg.f63dd04c1580.1453825656.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/f63dd04c1580
branches:  stable-1.8
changeset: 6356:f63dd04c1580
user:      Ruslan Ermilov <ru at nginx.com>
date:      Tue Jan 26 16:46:31 2016 +0300
description:
Resolver: fixed crashes in timeout handler.

If one or more requests were waiting for a response, then after
getting a CNAME response, the timeout event on the first request
remained active, pointing to the wrong node with an empty
rn->waiting list, and that could cause either null pointer
dereference or use-after-free memory access if this timeout
expired.

If several requests were waiting for a response, and the first
request terminated (e.g., due to client closing a connection),
other requests were left without a timeout and could potentially
wait indefinitely.

This is fixed by introducing per-request independent timeouts.
This change also reverts 954867a2f0a6 and 5004210e8c78.

diffstat:

 src/core/ngx_resolver.c |  60 +++++++++++++++++++++++++++++++-----------------
 src/core/ngx_resolver.h |  13 ++++-----
 2 files changed, 45 insertions(+), 28 deletions(-)

diffs (147 lines):

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -422,7 +422,7 @@ ngx_resolve_name_done(ngx_resolver_ctx_t
 
     /* lock name mutex */
 
-    if (ctx->state == NGX_AGAIN) {
+    if (ctx->state == NGX_AGAIN || ctx->state == NGX_RESOLVE_TIMEDOUT) {
 
         hash = ngx_crc32_short(ctx->name.data, ctx->name.len);
 
@@ -576,6 +576,20 @@ ngx_resolve_name_locked(ngx_resolver_t *
 
         if (rn->waiting) {
 
+            if (ctx->event == NULL) {
+                ctx->event = ngx_resolver_calloc(r, sizeof(ngx_event_t));
+                if (ctx->event == NULL) {
+                    return NGX_ERROR;
+                }
+
+                ctx->event->handler = ngx_resolver_timeout_handler;
+                ctx->event->data = ctx;
+                ctx->event->log = r->log;
+                ctx->ident = -1;
+
+                ngx_add_timer(ctx->event, ctx->timeout);
+            }
+
             ctx->next = rn->waiting;
             rn->waiting = ctx;
             ctx->state = NGX_AGAIN;
@@ -669,9 +683,9 @@ ngx_resolve_name_locked(ngx_resolver_t *
         }
 
         ctx->event->handler = ngx_resolver_timeout_handler;
-        ctx->event->data = rn;
+        ctx->event->data = ctx;
         ctx->event->log = r->log;
-        rn->ident = -1;
+        ctx->ident = -1;
 
         ngx_add_timer(ctx->event, ctx->timeout);
     }
@@ -799,6 +813,18 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx
 
         if (rn->waiting) {
 
+            ctx->event = ngx_resolver_calloc(r, sizeof(ngx_event_t));
+            if (ctx->event == NULL) {
+                return NGX_ERROR;
+            }
+
+            ctx->event->handler = ngx_resolver_timeout_handler;
+            ctx->event->data = ctx;
+            ctx->event->log = r->log;
+            ctx->ident = -1;
+
+            ngx_add_timer(ctx->event, ctx->timeout);
+
             ctx->next = rn->waiting;
             rn->waiting = ctx;
             ctx->state = NGX_AGAIN;
@@ -862,9 +888,9 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx
     }
 
     ctx->event->handler = ngx_resolver_timeout_handler;
-    ctx->event->data = rn;
+    ctx->event->data = ctx;
     ctx->event->log = r->log;
-    rn->ident = -1;
+    ctx->ident = -1;
 
     ngx_add_timer(ctx->event, ctx->timeout);
 
@@ -954,7 +980,7 @@ ngx_resolve_addr_done(ngx_resolver_ctx_t
 
     /* lock addr mutex */
 
-    if (ctx->state == NGX_AGAIN) {
+    if (ctx->state == NGX_AGAIN || ctx->state == NGX_RESOLVE_TIMEDOUT) {
 
         switch (ctx->addr.sockaddr->sa_family) {
 
@@ -2795,21 +2821,13 @@ done:
 static void
 ngx_resolver_timeout_handler(ngx_event_t *ev)
 {
-    ngx_resolver_ctx_t   *ctx, *next;
-    ngx_resolver_node_t  *rn;
-
-    rn = ev->data;
-    ctx = rn->waiting;
-    rn->waiting = NULL;
-
-    do {
-        ctx->state = NGX_RESOLVE_TIMEDOUT;
-        next = ctx->next;
-
-        ctx->handler(ctx);
-
-        ctx = next;
-    } while (ctx);
+    ngx_resolver_ctx_t  *ctx;
+
+    ctx = ev->data;
+
+    ctx->state = NGX_RESOLVE_TIMEDOUT;
+
+    ctx->handler(ctx);
 }
 
 
diff --git a/src/core/ngx_resolver.h b/src/core/ngx_resolver.h
--- a/src/core/ngx_resolver.h
+++ b/src/core/ngx_resolver.h
@@ -51,16 +51,12 @@ typedef void (*ngx_resolver_handler_pt)(
 
 
 typedef struct {
+    ngx_rbtree_node_t         node;
+    ngx_queue_t               queue;
+
     /* PTR: resolved name, A: name to resolve */
     u_char                   *name;
 
-    ngx_queue_t               queue;
-
-    /* event ident must be after 3 pointers as in ngx_connection_t */
-    ngx_int_t                 ident;
-
-    ngx_rbtree_node_t         node;
-
 #if (NGX_HAVE_INET6)
     /* PTR: IPv6 address to resolve (IPv4 address is in rbtree node key) */
     struct in6_addr           addr6;
@@ -147,6 +143,9 @@ struct ngx_resolver_ctx_s {
     ngx_resolver_t           *resolver;
     ngx_udp_connection_t     *udp_connection;
 
+    /* event ident must be after 3 pointers as in ngx_connection_t */
+    ngx_int_t                 ident;
+
     ngx_int_t                 state;
     ngx_str_t                 name;
 


From mdounin at mdounin.ru  Tue Jan 26 16:27:39 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:39 +0000
Subject: [nginx] Resolver: fixed CNAME processing for several requests.
Message-ID: <hg.838946300825.1453825659.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/838946300825
branches:  stable-1.8
changeset: 6357:838946300825
user:      Ruslan Ermilov <ru at nginx.com>
date:      Tue Jan 26 16:46:38 2016 +0300
description:
Resolver: fixed CNAME processing for several requests.

When several requests were waiting for a response, then after getting
a CNAME response only the last request was properly processed, while
others were left waiting.

diffstat:

 src/core/ngx_resolver.c |  21 +++++++++++++++------
 1 files changed, 15 insertions(+), 6 deletions(-)

diffs (66 lines):

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -473,7 +473,7 @@ ngx_resolve_name_locked(ngx_resolver_t *
     ngx_int_t             rc;
     ngx_uint_t            naddrs;
     ngx_addr_t           *addrs;
-    ngx_resolver_ctx_t   *next;
+    ngx_resolver_ctx_t   *next, *last;
     ngx_resolver_node_t  *rn;
 
     ngx_strlow(ctx->name.data, ctx->name.data, ctx->name.len);
@@ -484,6 +484,9 @@ ngx_resolve_name_locked(ngx_resolver_t *
 
     if (rn) {
 
+        /* ctx can be a list after NGX_RESOLVE_CNAME */
+        for (last = ctx; last->next; last = last->next);
+
         if (rn->valid >= ngx_time()) {
 
             ngx_log_debug0(NGX_LOG_DEBUG_CORE, r->log, 0, "resolve cached");
@@ -511,7 +514,7 @@ ngx_resolve_name_locked(ngx_resolver_t *
                     }
                 }
 
-                ctx->next = rn->waiting;
+                last->next = rn->waiting;
                 rn->waiting = NULL;
 
                 /* unlock name mutex */
@@ -557,7 +560,7 @@ ngx_resolve_name_locked(ngx_resolver_t *
                 return ngx_resolve_name_locked(r, ctx);
             }
 
-            ctx->next = rn->waiting;
+            last->next = rn->waiting;
             rn->waiting = NULL;
 
             /* unlock name mutex */
@@ -590,7 +593,7 @@ ngx_resolve_name_locked(ngx_resolver_t *
                 ngx_add_timer(ctx->event, ctx->timeout);
             }
 
-            ctx->next = rn->waiting;
+            last->next = rn->waiting;
             rn->waiting = ctx;
             ctx->state = NGX_AGAIN;
 
@@ -661,8 +664,14 @@ ngx_resolve_name_locked(ngx_resolver_t *
         ngx_resolver_free(r, rn->name);
         ngx_resolver_free(r, rn);
 
-        ctx->state = NGX_RESOLVE_NXDOMAIN;
-        ctx->handler(ctx);
+        do {
+            ctx->state = NGX_RESOLVE_NXDOMAIN;
+            next = ctx->next;
+
+            ctx->handler(ctx);
+
+            ctx = next;
+        } while (ctx);
 
         return NGX_OK;
     }


From mdounin at mdounin.ru  Tue Jan 26 16:27:41 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:41 +0000
Subject: [nginx] Resolver: changed the ngx_resolver_create_*_query() argu...
Message-ID: <hg.5557bf31e25d.1453825661.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/5557bf31e25d
branches:  stable-1.8
changeset: 6358:5557bf31e25d
user:      Roman Arutyunyan <arut at nginx.com>
date:      Tue Jan 26 16:46:48 2016 +0300
description:
Resolver: changed the ngx_resolver_create_*_query() arguments.

No functional changes.

This is needed by the following change.

diffstat:

 src/core/ngx_resolver.c |  57 +++++++++++++++++++++++-------------------------
 1 files changed, 27 insertions(+), 30 deletions(-)

diffs (167 lines):

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -64,10 +64,10 @@ static void ngx_resolver_expire(ngx_reso
     ngx_queue_t *queue);
 static ngx_int_t ngx_resolver_send_query(ngx_resolver_t *r,
     ngx_resolver_node_t *rn);
-static ngx_int_t ngx_resolver_create_name_query(ngx_resolver_node_t *rn,
-    ngx_resolver_ctx_t *ctx);
-static ngx_int_t ngx_resolver_create_addr_query(ngx_resolver_node_t *rn,
-    ngx_resolver_ctx_t *ctx);
+static ngx_int_t ngx_resolver_create_name_query(ngx_resolver_t *r,
+    ngx_resolver_node_t *rn, ngx_str_t *name);
+static ngx_int_t ngx_resolver_create_addr_query(ngx_resolver_t *r,
+    ngx_resolver_node_t *rn, ngx_addr_t *addr);
 static void ngx_resolver_resend_handler(ngx_event_t *ev);
 static time_t ngx_resolver_resend(ngx_resolver_t *r, ngx_rbtree_t *tree,
     ngx_queue_t *queue);
@@ -651,7 +651,7 @@ ngx_resolve_name_locked(ngx_resolver_t *
         ngx_rbtree_insert(&r->name_rbtree, &rn->node);
     }
 
-    rc = ngx_resolver_create_name_query(rn, ctx);
+    rc = ngx_resolver_create_name_query(r, rn, &ctx->name);
 
     if (rc == NGX_ERROR) {
         goto failed;
@@ -878,7 +878,7 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx
         ngx_rbtree_insert(tree, &rn->node);
     }
 
-    if (ngx_resolver_create_addr_query(rn, ctx) != NGX_OK) {
+    if (ngx_resolver_create_addr_query(r, rn, &ctx->addr) != NGX_OK) {
         goto failed;
     }
 
@@ -2511,27 +2511,23 @@ ngx_resolver_rbtree_insert_addr6_value(n
 
 
 static ngx_int_t
-ngx_resolver_create_name_query(ngx_resolver_node_t *rn, ngx_resolver_ctx_t *ctx)
+ngx_resolver_create_name_query(ngx_resolver_t *r, ngx_resolver_node_t *rn,
+    ngx_str_t *name)
 {
     u_char              *p, *s;
     size_t               len, nlen;
     ngx_uint_t           ident;
-#if (NGX_HAVE_INET6)
-    ngx_resolver_t      *r;
-#endif
     ngx_resolver_qs_t   *qs;
     ngx_resolver_hdr_t  *query;
 
-    nlen = ctx->name.len ? (1 + ctx->name.len + 1) : 1;
+    nlen = name->len ? (1 + name->len + 1) : 1;
 
     len = sizeof(ngx_resolver_hdr_t) + nlen + sizeof(ngx_resolver_qs_t);
 
 #if (NGX_HAVE_INET6)
-    r = ctx->resolver;
-
-    p = ngx_resolver_alloc(ctx->resolver, r->ipv6 ? len * 2 : len);
+    p = ngx_resolver_alloc(r, r->ipv6 ? len * 2 : len);
 #else
-    p = ngx_resolver_alloc(ctx->resolver, len);
+    p = ngx_resolver_alloc(r, len);
 #endif
     if (p == NULL) {
         return NGX_ERROR;
@@ -2550,8 +2546,8 @@ ngx_resolver_create_name_query(ngx_resol
 
     ident = ngx_random();
 
-    ngx_log_debug2(NGX_LOG_DEBUG_CORE, ctx->resolver->log, 0,
-                   "resolve: \"%V\" A %i", &ctx->name, ident & 0xffff);
+    ngx_log_debug2(NGX_LOG_DEBUG_CORE, r->log, 0,
+                   "resolve: \"%V\" A %i", name, ident & 0xffff);
 
     query->ident_hi = (u_char) ((ident >> 8) & 0xff);
     query->ident_lo = (u_char) (ident & 0xff);
@@ -2581,11 +2577,11 @@ ngx_resolver_create_name_query(ngx_resol
     p--;
     *p-- = '\0';
 
-    if (ctx->name.len == 0)  {
+    if (name->len == 0)  {
         return NGX_DECLINED;
     }
 
-    for (s = ctx->name.data + ctx->name.len - 1; s >= ctx->name.data; s--) {
+    for (s = name->data + name->len - 1; s >= name->data; s--) {
         if (*s != '.') {
             *p = *s;
             len++;
@@ -2621,8 +2617,8 @@ ngx_resolver_create_name_query(ngx_resol
 
     ident = ngx_random();
 
-    ngx_log_debug2(NGX_LOG_DEBUG_CORE, ctx->resolver->log, 0,
-                   "resolve: \"%V\" AAAA %i", &ctx->name, ident & 0xffff);
+    ngx_log_debug2(NGX_LOG_DEBUG_CORE, r->log, 0,
+                   "resolve: \"%V\" AAAA %i", name, ident & 0xffff);
 
     query->ident_hi = (u_char) ((ident >> 8) & 0xff);
     query->ident_lo = (u_char) (ident & 0xff);
@@ -2639,11 +2635,12 @@ ngx_resolver_create_name_query(ngx_resol
 
 
 static ngx_int_t
-ngx_resolver_create_addr_query(ngx_resolver_node_t *rn, ngx_resolver_ctx_t *ctx)
+ngx_resolver_create_addr_query(ngx_resolver_t *r, ngx_resolver_node_t *rn,
+    ngx_addr_t *addr)
 {
     u_char               *p, *d;
     size_t                len;
-    in_addr_t             addr;
+    in_addr_t             inaddr;
     ngx_int_t             n;
     ngx_uint_t            ident;
     ngx_resolver_hdr_t   *query;
@@ -2652,7 +2649,7 @@ ngx_resolver_create_addr_query(ngx_resol
     struct sockaddr_in6  *sin6;
 #endif
 
-    switch (ctx->addr.sockaddr->sa_family) {
+    switch (addr->sockaddr->sa_family) {
 
 #if (NGX_HAVE_INET6)
     case AF_INET6:
@@ -2669,7 +2666,7 @@ ngx_resolver_create_addr_query(ngx_resol
               + sizeof(ngx_resolver_qs_t);
     }
 
-    p = ngx_resolver_alloc(ctx->resolver, len);
+    p = ngx_resolver_alloc(r, len);
     if (p == NULL) {
         return NGX_ERROR;
     }
@@ -2693,11 +2690,11 @@ ngx_resolver_create_addr_query(ngx_resol
 
     p += sizeof(ngx_resolver_hdr_t);
 
-    switch (ctx->addr.sockaddr->sa_family) {
+    switch (addr->sockaddr->sa_family) {
 
 #if (NGX_HAVE_INET6)
     case AF_INET6:
-        sin6 = (struct sockaddr_in6 *) ctx->addr.sockaddr;
+        sin6 = (struct sockaddr_in6 *) addr->sockaddr;
 
         for (n = 15; n >= 0; n--) {
             p = ngx_sprintf(p, "\1%xd\1%xd",
@@ -2712,11 +2709,11 @@ ngx_resolver_create_addr_query(ngx_resol
 
     default: /* AF_INET */
 
-        sin = (struct sockaddr_in *) ctx->addr.sockaddr;
-        addr = ntohl(sin->sin_addr.s_addr);
+        sin = (struct sockaddr_in *) addr->sockaddr;
+        inaddr = ntohl(sin->sin_addr.s_addr);
 
         for (n = 0; n < 32; n += 8) {
-            d = ngx_sprintf(&p[1], "%ud", (addr >> n) & 0xff);
+            d = ngx_sprintf(&p[1], "%ud", (inaddr >> n) & 0xff);
             *p = (u_char) (d - &p[1]);
             p = d;
         }


From mdounin at mdounin.ru  Tue Jan 26 16:27:44 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:44 +0000
Subject: [nginx] Resolver: fixed use-after-free memory accesses with CNAME.
Message-ID: <hg.dac6eda40475.1453825664.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/dac6eda40475
branches:  stable-1.8
changeset: 6359:dac6eda40475
user:      Roman Arutyunyan <arut at nginx.com>
date:      Tue Jan 26 16:46:59 2016 +0300
description:
Resolver: fixed use-after-free memory accesses with CNAME.

When several requests were waiting for a response, then after getting
a CNAME response only the last request's context had the name updated.
Contexts of other requests had the wrong name.  This name was used by
ngx_resolve_name_done() to find the node to remove the request context
from.  When the name was wrong, the request could not be properly
cancelled, its context was freed but stayed linked to the node's waiting
list.  This happened e.g. when the first request was aborted or timed
out before the resolving completed.  When it completed, this triggered
a use-after-free memory access by calling ctx->handler of already freed
request context.  The bug manifests itself by
"could not cancel <name> resolving" alerts in error_log.

When a request was responded with a CNAME, the request context kept
the pointer to the original node's rn->u.cname.  If the original node
expired before the resolving timed out or completed with an error,
this would trigger a use-after-free memory access via ctx->name in
ctx->handler().

The fix is to keep ctx->name unmodified.  The name from context
is no longer used by ngx_resolve_name_done().  Instead, we now keep
the pointer to resolver node to which this request is linked.
Keeping the original name intact also improves logging.

diffstat:

 src/core/ngx_resolver.c |  80 ++++++++++++++++++++++--------------------------
 src/core/ngx_resolver.h |   2 +
 2 files changed, 39 insertions(+), 43 deletions(-)

diffs (219 lines):

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -59,7 +59,7 @@ ngx_int_t ngx_udp_connect(ngx_udp_connec
 static void ngx_resolver_cleanup(void *data);
 static void ngx_resolver_cleanup_tree(ngx_resolver_t *r, ngx_rbtree_t *tree);
 static ngx_int_t ngx_resolve_name_locked(ngx_resolver_t *r,
-    ngx_resolver_ctx_t *ctx);
+    ngx_resolver_ctx_t *ctx, ngx_str_t *name);
 static void ngx_resolver_expire(ngx_resolver_t *r, ngx_rbtree_t *tree,
     ngx_queue_t *queue);
 static ngx_int_t ngx_resolver_send_query(ngx_resolver_t *r,
@@ -375,7 +375,7 @@ ngx_resolve_name(ngx_resolver_ctx_t *ctx
 
     /* lock name mutex */
 
-    rc = ngx_resolve_name_locked(r, ctx);
+    rc = ngx_resolve_name_locked(r, ctx, &ctx->name);
 
     if (rc == NGX_OK) {
         return NGX_OK;
@@ -402,7 +402,6 @@ ngx_resolve_name(ngx_resolver_ctx_t *ctx
 void
 ngx_resolve_name_done(ngx_resolver_ctx_t *ctx)
 {
-    uint32_t              hash;
     ngx_resolver_t       *r;
     ngx_resolver_ctx_t   *w, **p;
     ngx_resolver_node_t  *rn;
@@ -424,9 +423,7 @@ ngx_resolve_name_done(ngx_resolver_ctx_t
 
     if (ctx->state == NGX_AGAIN || ctx->state == NGX_RESOLVE_TIMEDOUT) {
 
-        hash = ngx_crc32_short(ctx->name.data, ctx->name.len);
-
-        rn = ngx_resolver_lookup_name(r, &ctx->name, hash);
+        rn = ctx->node;
 
         if (rn) {
             p = &rn->waiting;
@@ -467,20 +464,22 @@ done:
 
 
 static ngx_int_t
-ngx_resolve_name_locked(ngx_resolver_t *r, ngx_resolver_ctx_t *ctx)
+ngx_resolve_name_locked(ngx_resolver_t *r, ngx_resolver_ctx_t *ctx,
+    ngx_str_t *name)
 {
     uint32_t              hash;
     ngx_int_t             rc;
+    ngx_str_t             cname;
     ngx_uint_t            naddrs;
     ngx_addr_t           *addrs;
     ngx_resolver_ctx_t   *next, *last;
     ngx_resolver_node_t  *rn;
 
-    ngx_strlow(ctx->name.data, ctx->name.data, ctx->name.len);
-
-    hash = ngx_crc32_short(ctx->name.data, ctx->name.len);
-
-    rn = ngx_resolver_lookup_name(r, &ctx->name, hash);
+    ngx_strlow(name->data, name->data, name->len);
+
+    hash = ngx_crc32_short(name->data, name->len);
+
+    rn = ngx_resolver_lookup_name(r, name, hash);
 
     if (rn) {
 
@@ -554,10 +553,10 @@ ngx_resolve_name_locked(ngx_resolver_t *
 
             if (ctx->recursion++ < NGX_RESOLVER_MAX_RECURSION) {
 
-                ctx->name.len = rn->cnlen;
-                ctx->name.data = rn->u.cname;
-
-                return ngx_resolve_name_locked(r, ctx);
+                cname.len = rn->cnlen;
+                cname.data = rn->u.cname;
+
+                return ngx_resolve_name_locked(r, ctx, &cname);
             }
 
             last->next = rn->waiting;
@@ -597,6 +596,11 @@ ngx_resolve_name_locked(ngx_resolver_t *
             rn->waiting = ctx;
             ctx->state = NGX_AGAIN;
 
+            do {
+                ctx->node = rn;
+                ctx = ctx->next;
+            } while (ctx);
+
             return NGX_AGAIN;
         }
 
@@ -635,14 +639,14 @@ ngx_resolve_name_locked(ngx_resolver_t *
             return NGX_ERROR;
         }
 
-        rn->name = ngx_resolver_dup(r, ctx->name.data, ctx->name.len);
+        rn->name = ngx_resolver_dup(r, name->data, name->len);
         if (rn->name == NULL) {
             ngx_resolver_free(r, rn);
             return NGX_ERROR;
         }
 
         rn->node.key = hash;
-        rn->nlen = (u_short) ctx->name.len;
+        rn->nlen = (u_short) name->len;
         rn->query = NULL;
 #if (NGX_HAVE_INET6)
         rn->query6 = NULL;
@@ -651,7 +655,7 @@ ngx_resolve_name_locked(ngx_resolver_t *
         ngx_rbtree_insert(&r->name_rbtree, &rn->node);
     }
 
-    rc = ngx_resolver_create_name_query(r, rn, &ctx->name);
+    rc = ngx_resolver_create_name_query(r, rn, name);
 
     if (rc == NGX_ERROR) {
         goto failed;
@@ -715,6 +719,11 @@ ngx_resolve_name_locked(ngx_resolver_t *
 
     ctx->state = NGX_AGAIN;
 
+    do {
+        ctx->node = rn;
+        ctx = ctx->next;
+    } while (ctx);
+
     return NGX_AGAIN;
 
 failed:
@@ -837,6 +846,7 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx
             ctx->next = rn->waiting;
             rn->waiting = ctx;
             ctx->state = NGX_AGAIN;
+            ctx->node = rn;
 
             /* unlock addr mutex */
 
@@ -922,6 +932,7 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx
     /* unlock addr mutex */
 
     ctx->state = NGX_AGAIN;
+    ctx->node = rn;
 
     return NGX_OK;
 
@@ -952,17 +963,11 @@ failed:
 void
 ngx_resolve_addr_done(ngx_resolver_ctx_t *ctx)
 {
-    in_addr_t             addr;
     ngx_queue_t          *expire_queue;
     ngx_rbtree_t         *tree;
     ngx_resolver_t       *r;
     ngx_resolver_ctx_t   *w, **p;
-    struct sockaddr_in   *sin;
     ngx_resolver_node_t  *rn;
-#if (NGX_HAVE_INET6)
-    uint32_t              hash;
-    struct sockaddr_in6  *sin6;
-#endif
 
     r = ctx->resolver;
 
@@ -991,21 +996,7 @@ ngx_resolve_addr_done(ngx_resolver_ctx_t
 
     if (ctx->state == NGX_AGAIN || ctx->state == NGX_RESOLVE_TIMEDOUT) {
 
-        switch (ctx->addr.sockaddr->sa_family) {
-
-#if (NGX_HAVE_INET6)
-        case AF_INET6:
-            sin6 = (struct sockaddr_in6 *) ctx->addr.sockaddr;
-            hash = ngx_crc32_short(sin6->sin6_addr.s6_addr, 16);
-            rn = ngx_resolver_lookup_addr6(r, &sin6->sin6_addr, hash);
-            break;
-#endif
-
-        default: /* AF_INET */
-            sin = (struct sockaddr_in *) ctx->addr.sockaddr;
-            addr = ntohl(sin->sin_addr.s_addr);
-            rn = ngx_resolver_lookup_addr(r, addr);
-        }
+        rn = ctx->node;
 
         if (rn) {
             p = &rn->waiting;
@@ -1994,9 +1985,12 @@ ngx_resolver_process_a(ngx_resolver_t *r
         rn->waiting = NULL;
 
         if (ctx) {
-            ctx->name = name;
-
-            (void) ngx_resolve_name_locked(r, ctx);
+
+            for (next = ctx; next; next = next->next) {
+                next->node = NULL;
+            }
+
+            (void) ngx_resolve_name_locked(r, ctx, &name);
         }
 
         ngx_resolver_free(r, rn->query);
diff --git a/src/core/ngx_resolver.h b/src/core/ngx_resolver.h
--- a/src/core/ngx_resolver.h
+++ b/src/core/ngx_resolver.h
@@ -161,6 +161,8 @@ struct ngx_resolver_ctx_s {
     ngx_uint_t                quick;  /* unsigned  quick:1; */
     ngx_uint_t                recursion;
     ngx_event_t              *event;
+
+    ngx_resolver_node_t      *node;
 };
 
 


From mdounin at mdounin.ru  Tue Jan 26 16:27:46 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:46 +0000
Subject: [nginx] Resolver: limited CNAME recursion.
Message-ID: <hg.93d70d87914c.1453825666.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/93d70d87914c
branches:  stable-1.8
changeset: 6360:93d70d87914c
user:      Ruslan Ermilov <ru at nginx.com>
date:      Tue Jan 26 16:47:14 2016 +0300
description:
Resolver: limited CNAME recursion.

Previously, the recursion was only limited for cached responses.

diffstat:

 src/core/ngx_resolver.c |  40 ++++++++++++++++++++++++++++------------
 1 files changed, 28 insertions(+), 12 deletions(-)

diffs (56 lines):

diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c
+++ b/src/core/ngx_resolver.c
@@ -1981,24 +1981,40 @@ ngx_resolver_process_a(ngx_resolver_t *r
 
         ngx_queue_insert_head(&r->name_expire_queue, &rn->queue);
 
-        ctx = rn->waiting;
-        rn->waiting = NULL;
-
-        if (ctx) {
-
-            for (next = ctx; next; next = next->next) {
-                next->node = NULL;
-            }
-
-            (void) ngx_resolve_name_locked(r, ctx, &name);
-        }
-
         ngx_resolver_free(r, rn->query);
         rn->query = NULL;
 #if (NGX_HAVE_INET6)
         rn->query6 = NULL;
 #endif
 
+        ctx = rn->waiting;
+        rn->waiting = NULL;
+
+        if (ctx) {
+
+            if (ctx->recursion++ >= NGX_RESOLVER_MAX_RECURSION) {
+
+                /* unlock name mutex */
+
+                do {
+                    ctx->state = NGX_RESOLVE_NXDOMAIN;
+                    next = ctx->next;
+
+                    ctx->handler(ctx);
+
+                    ctx = next;
+                } while (ctx);
+
+                return;
+            }
+
+            for (next = ctx; next; next = next->next) {
+                next->node = NULL;
+            }
+
+            (void) ngx_resolve_name_locked(r, ctx, &name);
+        }
+
         /* unlock name mutex */
 
         return;


From mdounin at mdounin.ru  Tue Jan 26 16:27:48 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:48 +0000
Subject: [nginx] nginx-1.8.1-RELEASE
Message-ID: <hg.5959efb40b07.1453825668.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/5959efb40b07
branches:  stable-1.8
changeset: 6361:5959efb40b07
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Jan 26 17:39:30 2016 +0300
description:
nginx-1.8.1-RELEASE

diffstat:

 docs/xml/nginx/changes.xml |  159 +++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 159 insertions(+), 0 deletions(-)

diffs (169 lines):

diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml
--- a/docs/xml/nginx/changes.xml
+++ b/docs/xml/nginx/changes.xml
@@ -5,6 +5,165 @@
 <change_log title="nginx">
 
 
+<changes ver="1.8.1" date="26.01.2016">
+
+<change type="security">
+<para lang="ru">
+??? ????????????? ????????? resolver
+?? ????? ????????? ??????? DNS-???????
+????? ??????????? ????????????? ????????????? ??????,
+??? ????????? ??????????,
+???????? ??????????? ??????????? UDP-?????? ?? DNS-???????,
+??????? segmentation fault ? ??????? ???????? (CVE-2016-0742).
+</para>
+<para lang="en">
+invalid pointer dereference might occur
+during DNS server response processing
+if the "resolver" directive was used,
+allowing an attacker who is able to forge UDP packets from the DNS server
+to cause segmentation fault in a worker process (CVE-2016-0742).
+</para>
+</change>
+
+<change type="security">
+<para lang="ru">
+??? ????????????? ????????? resolver
+?? ????? ????????? CNAME-???????
+????? ????????? ????????? ? ????? ????????????? ??????,
+??? ????????? ??????????,
+???????? ??????????? ???????????? ?????????????? ???????????? ???? ? ??????,
+??????? segmentation fault ? ??????? ????????,
+? ????? ???????????? ????? ????? ?????? ??????????? (CVE-2016-0746).
+</para>
+<para lang="en">
+use-after-free condition might occur
+during CNAME response processing
+if the "resolver" directive was used,
+allowing an attacker who is able to trigger name resolution
+to cause segmentation fault in a worker process,
+or might have potential other impact (CVE-2016-0746).
+</para>
+</change>
+
+<change type="security">
+<para lang="ru">
+??? ????????????? ????????? resolver
+?? ????? ????????? CNAME-???????
+?? ?? ???? ??????? ??????????? ???????????
+?? ???????????? ?????????? ??????? ? ???????,
+??? ????????? ??????????,
+???????? ??????????? ???????????? ?????????????? ???????????? ???? ? ??????,
+??????? ?????????? ??????????? ???????? ???????? ?????????? (CVE-2016-0747).
+</para>
+<para lang="en">
+CNAME resolution was insufficiently limited
+if the "resolver" directive was used,
+allowing an attacker who is able to trigger arbitrary name resolution
+to cause excessive resource consumption in worker processes (CVE-2016-0747).
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+???????? proxy_protocol ????????? listen ?? ???????,
+???? ?? ??? ?????? ? ?????? ????????? listen ??? ??????? listen-??????.
+</para>
+<para lang="en">
+the "proxy_protocol" parameter of the "listen" directive did not work
+if not specified in the first "listen" directive for a listen socket.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+nginx ??? ?? ??????????? ?? ????????? ?????? ??????? Linux;
+?????? ????????? ? 1.7.11.
+</para>
+<para lang="en">
+nginx might fail to start on some old Linux variants;
+the bug had appeared in 1.7.11.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ?????????? ????????????? ???????? try_files ? alias
+?????? location'?, ????????? ?????????? ??????????,
+? ??????? ???????? ??? ????????? segmentation fault;
+?????? ????????? ? 1.7.1.
+</para>
+<para lang="en">
+a segmentation fault might occur in a worker process
+if the "try_files" and "alias" directives were used
+inside a location given by a regular expression;
+the bug had appeared in 1.7.1.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+????????? try_files ?????? ?????????? location'?, ????????? ??????????
+??????????, ???????? ???????????, ???? ?? ??????? location'? ??????????????
+????????? alias.
+</para>
+<para lang="en">
+the "try_files" directive inside a nested location
+given by a regular expression worked incorrectly
+if the "alias" directive was used in the outer location.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ????????????? ????
+? ????? ????? ?????????? ????????? "header already sent";
+?????? ????????? ? 1.7.5.
+</para>
+<para lang="en">
+"header already sent" alerts might appear in logs
+when using cache;
+the bug had appeared in 1.7.5.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+??? ????????????? ????????? ???????? ssl_session_cache
+? ?????? ??????????? ????????
+? ??????? ???????? ??? ????????? segmentation fault.
+</para>
+<para lang="en">
+a segmentation fault might occur in a worker process
+if different ssl_session_cache settings were used
+in different virtual servers.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+????????? expires ????? ?? ??????????? ??? ????????????? ??????????.
+</para>
+<para lang="en">
+the "expires" directive might not work when using variables.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+???? nginx ??? ?????? ? ??????? ngx_http_spdy_module,
+???????? SPDY ??? ???? ??????????? ????????,
+???? ???? ?? ??? ?????? ???????? spdy ????????? listen.
+</para>
+<para lang="en">
+if nginx was built with the ngx_http_spdy_module
+it was possible to use the SPDY protocol
+even if the "spdy" parameter of the "listen" directive was not specified.
+</para>
+</change>
+
+</changes>
+
+
 <changes ver="1.8.0" date="21.04.2015">
 
 <change>

From mdounin at mdounin.ru  Tue Jan 26 16:27:51 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 16:27:51 +0000
Subject: [nginx] release-1.8.1 tag
Message-ID: <hg.28bb9683dd9a.1453825671.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/28bb9683dd9a
branches:  stable-1.8
changeset: 6362:28bb9683dd9a
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Tue Jan 26 17:39:31 2016 +0300
description:
release-1.8.1 tag

diffstat:

 .hgtags |  1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diffs (8 lines):

diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -383,3 +383,4 @@ 860cfbcc4606ee36d898a9cd0c5ae8858db984d6
 2b3b737b5456c05cd63d3d834f4fb4d3776953d0 release-1.7.11
 3ef00a71f56420a9c3e9cec311c9a2109a015d67 release-1.7.12
 01d52c2b460da263fd1557ab5f8d0af5e404b47a release-1.8.0
+5959efb40b07ea30693c92ddbdec82a78577e671 release-1.8.1


From yatiohi at ideopolis.gr  Tue Jan 26 19:11:39 2016
From: yatiohi at ideopolis.gr (Christos Trochalakis)
Date: Tue, 26 Jan 2016 21:11:39 +0200
Subject: [nginx-announce] nginx security advisory (CVE-2016-0742,
 CVE-2016-0746, CVE-2016-0747)
In-Reply-To: <20160126163217.GX9449@mdounin.ru>
References: <20160126163217.GX9449@mdounin.ru>
Message-ID: <20160126191139.GA3646@luke.ws.skroutz.gr>

On Tue, Jan 26, 2016 at 07:32:17PM +0300, Maxim Dounin wrote:
>Hello!
>
>Several problems in nginx resolver were identified, which might allow
>an attacker to cause worker process crash, or might have potential
>other impact
>
>The problems are fixed in nginx 1.9.10, 1.8.1.
>

Hello all,

I am one of debian's nginx maintainers, I have just uploaded
nginx-1.9.10 for unstable, so we are ready on that front. But debian
stable is also affected (1.6.x series) and we will need to prepare a
patch. Is it possible to ask for a single combined patch (or even better
an 1.6.x release)?

I know that you have a policy of providing security support for mainline
and stable (1.9, 1.8), but since there are a lot of nginx users using
debian stable, we'd be glad if we could cooperate and make an exception
whenever possible.

Thanks again,
Christos


From mdounin at mdounin.ru  Tue Jan 26 19:48:36 2016
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 26 Jan 2016 22:48:36 +0300
Subject: [nginx-announce] nginx security advisory (CVE-2016-0742,
 CVE-2016-0746, CVE-2016-0747)
In-Reply-To: <20160126191139.GA3646@luke.ws.skroutz.gr>
References: <20160126163217.GX9449@mdounin.ru>
 <20160126191139.GA3646@luke.ws.skroutz.gr>
Message-ID: <20160126194836.GC9449@mdounin.ru>

Hello!

On Tue, Jan 26, 2016 at 09:11:39PM +0200, Christos Trochalakis wrote:

> On Tue, Jan 26, 2016 at 07:32:17PM +0300, Maxim Dounin wrote:
> >Hello!
> >
> >Several problems in nginx resolver were identified, which might allow
> >an attacker to cause worker process crash, or might have potential
> >other impact
> >
> >The problems are fixed in nginx 1.9.10, 1.8.1.
> >
> 
> Hello all,
> 
> I am one of debian's nginx maintainers, I have just uploaded
> nginx-1.9.10 for unstable, so we are ready on that front. But debian
> stable is also affected (1.6.x series) and we will need to prepare a
> patch. Is it possible to ask for a single combined patch (or even better
> an 1.6.x release)?
> 
> I know that you have a policy of providing security support for mainline
> and stable (1.9, 1.8), but since there are a lot of nginx users using
> debian stable, we'd be glad if we could cooperate and make an exception
> whenever possible.

Just merging src/core/ngx_resolver.[ch] from 1.8.1 should 
be optimal solution for 1.6.x.

-- 
Maxim Dounin
http://nginx.org/


From ahutchings at nginx.com  Tue Jan 26 19:49:21 2016
From: ahutchings at nginx.com (Andrew Hutchings)
Date: Tue, 26 Jan 2016 19:49:21 +0000
Subject: [nginx-announce] nginx security advisory (CVE-2016-0742,
 CVE-2016-0746, CVE-2016-0747)
In-Reply-To: <20160126191139.GA3646@luke.ws.skroutz.gr>
References: <20160126163217.GX9449@mdounin.ru>
 <20160126191139.GA3646@luke.ws.skroutz.gr>
Message-ID: <56A7CDC1.7040405@nginx.com>

Hi Christos,

On 26/01/16 19:11, Christos Trochalakis wrote:
> On Tue, Jan 26, 2016 at 07:32:17PM +0300, Maxim Dounin wrote:
>> Hello!
>>
>> Several problems in nginx resolver were identified, which might allow
>> an attacker to cause worker process crash, or might have potential
>> other impact
>>
>> The problems are fixed in nginx 1.9.10, 1.8.1.
>>
>
> I am one of debian's nginx maintainers, I have just uploaded
> nginx-1.9.10 for unstable, so we are ready on that front. But debian
> stable is also affected (1.6.x series) and we will need to prepare a
> patch. Is it possible to ask for a single combined patch (or even better
> an 1.6.x release)?

It should be possible to get a combined patch straight out of the 
mercurial repository just by doing a range on the diff. Alternatively 
Fedora has already backported the patches to 1.6 which can be found 
here: http://koji.fedoraproject.org/koji/buildinfo?buildID=713981

Hope this helps

Kind Regards
-- 
Andrew Hutchings (LinuxJedi)
Technical Product Manager, NGINX Inc.


From arut at nginx.com  Thu Jan 28 12:43:22 2016
From: arut at nginx.com (Roman Arutyunyan)
Date: Thu, 28 Jan 2016 12:43:22 +0000
Subject: [nginx] Version bump.
Message-ID: <hg.7fbeb0651e38.1453985002.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/7fbeb0651e38
branches:  
changeset: 6363:7fbeb0651e38
user:      Roman Arutyunyan <arut at nginx.com>
date:      Thu Jan 28 15:25:58 2016 +0300
description:
Version bump.

diffstat:

 src/core/nginx.h |  4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diffs (14 lines):

diff -r 75c71083f4cc -r 7fbeb0651e38 src/core/nginx.h
--- a/src/core/nginx.h	Tue Jan 26 17:27:41 2016 +0300
+++ b/src/core/nginx.h	Thu Jan 28 15:25:58 2016 +0300
@@ -9,8 +9,8 @@
 #define _NGINX_H_INCLUDED_
 
 
-#define nginx_version      1009010
-#define NGINX_VERSION      "1.9.10"
+#define nginx_version      1009011
+#define NGINX_VERSION      "1.9.11"
 #define NGINX_VER          "nginx/" NGINX_VERSION
 
 #ifdef NGX_BUILD


From arut at nginx.com  Thu Jan 28 12:43:24 2016
From: arut at nginx.com (Roman Arutyunyan)
Date: Thu, 28 Jan 2016 12:43:24 +0000
Subject: [nginx] Resolver: removed unused field from ngx_resolver_ctx_t.
Message-ID: <hg.69977457e1a6.1453985004.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/69977457e1a6
branches:  
changeset: 6364:69977457e1a6
user:      Roman Arutyunyan <arut at nginx.com>
date:      Thu Jan 28 15:28:19 2016 +0300
description:
Resolver: removed unused field from ngx_resolver_ctx_t.

diffstat:

 src/core/ngx_resolver.h |  4 +---
 1 files changed, 1 insertions(+), 3 deletions(-)

diffs (21 lines):

diff -r 7fbeb0651e38 -r 69977457e1a6 src/core/ngx_resolver.h
--- a/src/core/ngx_resolver.h	Thu Jan 28 15:25:58 2016 +0300
+++ b/src/core/ngx_resolver.h	Thu Jan 28 15:28:19 2016 +0300
@@ -141,7 +141,7 @@ typedef struct {
 struct ngx_resolver_ctx_s {
     ngx_resolver_ctx_t       *next;
     ngx_resolver_t           *resolver;
-    ngx_udp_connection_t     *udp_connection;
+    ngx_resolver_node_t      *node;
 
     /* event ident must be after 3 pointers as in ngx_connection_t */
     ngx_int_t                 ident;
@@ -161,8 +161,6 @@ struct ngx_resolver_ctx_s {
     ngx_uint_t                quick;  /* unsigned  quick:1; */
     ngx_uint_t                recursion;
     ngx_event_t              *event;
-
-    ngx_resolver_node_t      *node;
 };
 
 


From arut at nginx.com  Thu Jan 28 12:43:27 2016
From: arut at nginx.com (Roman Arutyunyan)
Date: Thu, 28 Jan 2016 12:43:27 +0000
Subject: [nginx] Resolver: renamed UDP-specific structures, fields and va...
Message-ID: <hg.d35b4d590b2d.1453985007.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/d35b4d590b2d
branches:  
changeset: 6365:d35b4d590b2d
user:      Roman Arutyunyan <arut at nginx.com>
date:      Thu Jan 28 15:28:20 2016 +0300
description:
Resolver: renamed UDP-specific structures, fields and variables.

They will be used for TCP connections as well.

diffstat:

 src/core/ngx_resolver.c |  138 ++++++++++++++++++++++++------------------------
 src/core/ngx_resolver.h |    6 +-
 2 files changed, 72 insertions(+), 72 deletions(-)

diffs (truncated from 321 to 300 lines):

diff -r 69977457e1a6 -r d35b4d590b2d src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c	Thu Jan 28 15:28:19 2016 +0300
+++ b/src/core/ngx_resolver.c	Thu Jan 28 15:28:20 2016 +0300
@@ -53,7 +53,7 @@ typedef struct {
         ((u_char *) (n) - offsetof(ngx_resolver_node_t, node))
 
 
-ngx_int_t ngx_udp_connect(ngx_udp_connection_t *uc);
+ngx_int_t ngx_udp_connect(ngx_resolver_connection_t *rec);
 
 
 static void ngx_resolver_cleanup(void *data);
@@ -110,12 +110,12 @@ static ngx_resolver_node_t *ngx_resolver
 ngx_resolver_t *
 ngx_resolver_create(ngx_conf_t *cf, ngx_str_t *names, ngx_uint_t n)
 {
-    ngx_str_t              s;
-    ngx_url_t              u;
-    ngx_uint_t             i, j;
-    ngx_resolver_t        *r;
-    ngx_pool_cleanup_t    *cln;
-    ngx_udp_connection_t  *uc;
+    ngx_str_t                   s;
+    ngx_url_t                   u;
+    ngx_uint_t                  i, j;
+    ngx_resolver_t             *r;
+    ngx_pool_cleanup_t         *cln;
+    ngx_resolver_connection_t  *rec;
 
     cln = ngx_pool_cleanup_add(cf->pool, 0);
     if (cln == NULL) {
@@ -172,8 +172,8 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_
     r->log_level = NGX_LOG_ERR;
 
     if (n) {
-        if (ngx_array_init(&r->udp_connections, cf->pool, n,
-                           sizeof(ngx_udp_connection_t))
+        if (ngx_array_init(&r->connections, cf->pool, n,
+                           sizeof(ngx_resolver_connection_t))
             != NGX_OK)
         {
             return NULL;
@@ -230,17 +230,17 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_
             return NULL;
         }
 
-        uc = ngx_array_push_n(&r->udp_connections, u.naddrs);
-        if (uc == NULL) {
+        rec = ngx_array_push_n(&r->connections, u.naddrs);
+        if (rec == NULL) {
             return NULL;
         }
 
-        ngx_memzero(uc, u.naddrs * sizeof(ngx_udp_connection_t));
+        ngx_memzero(rec, u.naddrs * sizeof(ngx_resolver_connection_t));
 
         for (j = 0; j < u.naddrs; j++) {
-            uc[j].sockaddr = u.addrs[j].sockaddr;
-            uc[j].socklen = u.addrs[j].socklen;
-            uc[j].server = u.addrs[j].name;
+            rec[j].sockaddr = u.addrs[j].sockaddr;
+            rec[j].socklen = u.addrs[j].socklen;
+            rec[j].server = u.addrs[j].name;
         }
     }
 
@@ -253,8 +253,8 @@ ngx_resolver_cleanup(void *data)
 {
     ngx_resolver_t  *r = data;
 
-    ngx_uint_t             i;
-    ngx_udp_connection_t  *uc;
+    ngx_uint_t                  i;
+    ngx_resolver_connection_t  *rec;
 
     if (r) {
         ngx_log_debug0(NGX_LOG_DEBUG_CORE, ngx_cycle->log, 0,
@@ -273,11 +273,11 @@ ngx_resolver_cleanup(void *data)
         }
 
 
-        uc = r->udp_connections.elts;
-
-        for (i = 0; i < r->udp_connections.nelts; i++) {
-            if (uc[i].connection) {
-                ngx_close_connection(uc[i].connection);
+        rec = r->connections.elts;
+
+        for (i = 0; i < r->connections.nelts; i++) {
+            if (rec[i].udp) {
+                ngx_close_connection(rec[i].udp);
             }
         }
 
@@ -340,7 +340,7 @@ ngx_resolve_start(ngx_resolver_t *r, ngx
         }
     }
 
-    if (r->udp_connections.nelts == 0) {
+    if (r->connections.nelts == 0) {
         return NGX_NO_RESOLVER;
     }
 
@@ -1094,55 +1094,55 @@ ngx_resolver_expire(ngx_resolver_t *r, n
 static ngx_int_t
 ngx_resolver_send_query(ngx_resolver_t *r, ngx_resolver_node_t *rn)
 {
-    ssize_t                n;
-    ngx_udp_connection_t  *uc;
-
-    uc = r->udp_connections.elts;
-
-    uc = &uc[r->last_connection++];
-    if (r->last_connection == r->udp_connections.nelts) {
+    ssize_t                     n;
+    ngx_resolver_connection_t  *rec;
+
+    rec = r->connections.elts;
+
+    rec = &rec[r->last_connection++];
+    if (r->last_connection == r->connections.nelts) {
         r->last_connection = 0;
     }
 
-    if (uc->connection == NULL) {
-
-        uc->log = *r->log;
-        uc->log.handler = ngx_resolver_log_error;
-        uc->log.data = uc;
-        uc->log.action = "resolving";
-
-        if (ngx_udp_connect(uc) != NGX_OK) {
+    if (rec->udp == NULL) {
+
+        rec->log = *r->log;
+        rec->log.handler = ngx_resolver_log_error;
+        rec->log.data = rec;
+        rec->log.action = "resolving";
+
+        if (ngx_udp_connect(rec) != NGX_OK) {
             return NGX_ERROR;
         }
 
-        uc->connection->data = r;
-        uc->connection->read->handler = ngx_resolver_read_response;
-        uc->connection->read->resolver = 1;
+        rec->udp->data = r;
+        rec->udp->read->handler = ngx_resolver_read_response;
+        rec->udp->read->resolver = 1;
     }
 
     if (rn->naddrs == (u_short) -1) {
-        n = ngx_send(uc->connection, rn->query, rn->qlen);
+        n = ngx_send(rec->udp, rn->query, rn->qlen);
 
         if (n == -1) {
             return NGX_ERROR;
         }
 
         if ((size_t) n != (size_t) rn->qlen) {
-            ngx_log_error(NGX_LOG_CRIT, &uc->log, 0, "send() incomplete");
+            ngx_log_error(NGX_LOG_CRIT, &rec->log, 0, "send() incomplete");
             return NGX_ERROR;
         }
     }
 
 #if (NGX_HAVE_INET6)
     if (rn->query6 && rn->naddrs6 == (u_short) -1) {
-        n = ngx_send(uc->connection, rn->query6, rn->qlen);
+        n = ngx_send(rec->udp, rn->query6, rn->qlen);
 
         if (n == -1) {
             return NGX_ERROR;
         }
 
         if ((size_t) n != (size_t) rn->qlen) {
-            ngx_log_error(NGX_LOG_CRIT, &uc->log, 0, "send() incomplete");
+            ngx_log_error(NGX_LOG_CRIT, &rec->log, 0, "send() incomplete");
             return NGX_ERROR;
         }
     }
@@ -3074,8 +3074,8 @@ ngx_resolver_strerror(ngx_int_t err)
 static u_char *
 ngx_resolver_log_error(ngx_log_t *log, u_char *buf, size_t len)
 {
-    u_char                *p;
-    ngx_udp_connection_t  *uc;
+    u_char                     *p;
+    ngx_resolver_connection_t  *rec;
 
     p = buf;
 
@@ -3084,10 +3084,10 @@ ngx_resolver_log_error(ngx_log_t *log, u
         len -= p - buf;
     }
 
-    uc = log->data;
-
-    if (uc) {
-        p = ngx_snprintf(p, len, ", resolver: %V", &uc->server);
+    rec = log->data;
+
+    if (rec) {
+        p = ngx_snprintf(p, len, ", resolver: %V", &rec->server);
     }
 
     return p;
@@ -3095,7 +3095,7 @@ ngx_resolver_log_error(ngx_log_t *log, u
 
 
 ngx_int_t
-ngx_udp_connect(ngx_udp_connection_t *uc)
+ngx_udp_connect(ngx_resolver_connection_t *rec)
 {
     int                rc;
     ngx_int_t          event;
@@ -3103,21 +3103,21 @@ ngx_udp_connect(ngx_udp_connection_t *uc
     ngx_socket_t       s;
     ngx_connection_t  *c;
 
-    s = ngx_socket(uc->sockaddr->sa_family, SOCK_DGRAM, 0);
-
-    ngx_log_debug1(NGX_LOG_DEBUG_EVENT, &uc->log, 0, "UDP socket %d", s);
+    s = ngx_socket(rec->sockaddr->sa_family, SOCK_DGRAM, 0);
+
+    ngx_log_debug1(NGX_LOG_DEBUG_EVENT, &rec->log, 0, "UDP socket %d", s);
 
     if (s == (ngx_socket_t) -1) {
-        ngx_log_error(NGX_LOG_ALERT, &uc->log, ngx_socket_errno,
+        ngx_log_error(NGX_LOG_ALERT, &rec->log, ngx_socket_errno,
                       ngx_socket_n " failed");
         return NGX_ERROR;
     }
 
-    c = ngx_get_connection(s, &uc->log);
+    c = ngx_get_connection(s, &rec->log);
 
     if (c == NULL) {
         if (ngx_close_socket(s) == -1) {
-            ngx_log_error(NGX_LOG_ALERT, &uc->log, ngx_socket_errno,
+            ngx_log_error(NGX_LOG_ALERT, &rec->log, ngx_socket_errno,
                           ngx_close_socket_n "failed");
         }
 
@@ -3125,7 +3125,7 @@ ngx_udp_connect(ngx_udp_connection_t *uc
     }
 
     if (ngx_nonblocking(s) == -1) {
-        ngx_log_error(NGX_LOG_ALERT, &uc->log, ngx_socket_errno,
+        ngx_log_error(NGX_LOG_ALERT, &rec->log, ngx_socket_errno,
                       ngx_nonblocking_n " failed");
 
         goto failed;
@@ -3134,22 +3134,22 @@ ngx_udp_connect(ngx_udp_connection_t *uc
     rev = c->read;
     wev = c->write;
 
-    rev->log = &uc->log;
-    wev->log = &uc->log;
-
-    uc->connection = c;
+    rev->log = &rec->log;
+    wev->log = &rec->log;
+
+    rec->udp = c;
 
     c->number = ngx_atomic_fetch_add(ngx_connection_counter, 1);
 
-    ngx_log_debug3(NGX_LOG_DEBUG_EVENT, &uc->log, 0,
-                   "connect to %V, fd:%d #%uA", &uc->server, s, c->number);
-
-    rc = connect(s, uc->sockaddr, uc->socklen);
+    ngx_log_debug3(NGX_LOG_DEBUG_EVENT, &rec->log, 0,
+                   "connect to %V, fd:%d #%uA", &rec->server, s, c->number);
+
+    rc = connect(s, rec->sockaddr, rec->socklen);
 
     /* TODO: iocp */
 
     if (rc == -1) {
-        ngx_log_error(NGX_LOG_CRIT, &uc->log, ngx_socket_errno,
+        ngx_log_error(NGX_LOG_CRIT, &rec->log, ngx_socket_errno,
                       "connect() failed");
 
         goto failed;
@@ -3172,7 +3172,7 @@ ngx_udp_connect(ngx_udp_connection_t *uc
 failed:
 
     ngx_close_connection(c);
-    uc->connection = NULL;
+    rec->udp = NULL;
 
     return NGX_ERROR;
 }
diff -r 69977457e1a6 -r d35b4d590b2d src/core/ngx_resolver.h
--- a/src/core/ngx_resolver.h	Thu Jan 28 15:28:19 2016 +0300
+++ b/src/core/ngx_resolver.h	Thu Jan 28 15:28:20 2016 +0300
@@ -37,12 +37,12 @@
 
 


From arut at nginx.com  Thu Jan 28 12:43:29 2016
From: arut at nginx.com (Roman Arutyunyan)
Date: Thu, 28 Jan 2016 12:43:29 +0000
Subject: [nginx] Resolver: per-request DNS server balancer.
Message-ID: <hg.2e5c027f2a98.1453985009.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/2e5c027f2a98
branches:  
changeset: 6366:2e5c027f2a98
user:      Roman Arutyunyan <arut at nginx.com>
date:      Thu Jan 28 15:28:20 2016 +0300
description:
Resolver: per-request DNS server balancer.

Previously, a global server balancer was used to assign the next DNS server to
send a query to.  That could lead to a non-uniform distribution of servers per
request.  A request could be assigned to the same dead server several times in a
row and wait longer for a valid server or even time out without being processed.

Now each query is sent to all servers sequentially in a circle until a
response is received or timeout expires.  Initial server for each request is
still globally balanced.

diffstat:

 src/core/ngx_resolver.c |  20 +++++++++++++++-----
 src/core/ngx_resolver.h |   2 ++
 2 files changed, 17 insertions(+), 5 deletions(-)

diffs (63 lines):

diff -r d35b4d590b2d -r 2e5c027f2a98 src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c	Thu Jan 28 15:28:20 2016 +0300
+++ b/src/core/ngx_resolver.c	Thu Jan 28 15:28:20 2016 +0300
@@ -685,6 +685,11 @@ ngx_resolve_name_locked(ngx_resolver_t *
         return NGX_OK;
     }
 
+    rn->last_connection = r->last_connection++;
+    if (r->last_connection == r->connections.nelts) {
+        r->last_connection = 0;
+    }
+
     rn->naddrs = (u_short) -1;
 #if (NGX_HAVE_INET6)
     rn->naddrs6 = r->ipv6 ? (u_short) -1 : 0;
@@ -897,6 +902,11 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx
         goto failed;
     }
 
+    rn->last_connection = r->last_connection++;
+    if (r->last_connection == r->connections.nelts) {
+        r->last_connection = 0;
+    }
+
     rn->naddrs = (u_short) -1;
 #if (NGX_HAVE_INET6)
     rn->naddrs6 = (u_short) -1;
@@ -1098,11 +1108,7 @@ ngx_resolver_send_query(ngx_resolver_t *
     ngx_resolver_connection_t  *rec;
 
     rec = r->connections.elts;
-
-    rec = &rec[r->last_connection++];
-    if (r->last_connection == r->connections.nelts) {
-        r->last_connection = 0;
-    }
+    rec = &rec[rn->last_connection];
 
     if (rec->udp == NULL) {
 
@@ -1244,6 +1250,10 @@ ngx_resolver_resend(ngx_resolver_t *r, n
 
         if (rn->waiting) {
 
+            if (++rn->last_connection == r->connections.nelts) {
+                rn->last_connection = 0;
+            }
+
             (void) ngx_resolver_send_query(r, rn);
 
             rn->expire = now + r->resend_timeout;
diff -r d35b4d590b2d -r 2e5c027f2a98 src/core/ngx_resolver.h
--- a/src/core/ngx_resolver.h	Thu Jan 28 15:28:20 2016 +0300
+++ b/src/core/ngx_resolver.h	Thu Jan 28 15:28:20 2016 +0300
@@ -93,6 +93,8 @@ typedef struct {
     time_t                    valid;
     uint32_t                  ttl;
 
+    ngx_uint_t                last_connection;
+
     ngx_resolver_ctx_t       *waiting;
 } ngx_resolver_node_t;
 


From arut at nginx.com  Thu Jan 28 12:43:32 2016
From: arut at nginx.com (Roman Arutyunyan)
Date: Thu, 28 Jan 2016 12:43:32 +0000
Subject: [nginx] Resolver: TCP support.
Message-ID: <hg.5a16d40c63de.1453985012.6026610855610030274@dev.nginx.com>

details:   http://hg.nginx.org/nginx/rev/5a16d40c63de
branches:  
changeset: 6367:5a16d40c63de
user:      Roman Arutyunyan <arut at nginx.com>
date:      Thu Jan 28 15:28:20 2016 +0300
description:
Resolver: TCP support.

Resend DNS query over TCP once UDP response came truncated.

diffstat:

 src/core/ngx_resolver.c |  593 +++++++++++++++++++++++++++++++++++++++++++----
 src/core/ngx_resolver.h |   17 +-
 2 files changed, 554 insertions(+), 56 deletions(-)

diffs (truncated from 834 to 300 lines):

diff -r 2e5c027f2a98 -r 5a16d40c63de src/core/ngx_resolver.c
--- a/src/core/ngx_resolver.c	Thu Jan 28 15:28:20 2016 +0300
+++ b/src/core/ngx_resolver.c	Thu Jan 28 15:28:20 2016 +0300
@@ -12,6 +12,9 @@
 
 #define NGX_RESOLVER_UDP_SIZE   4096
 
+#define NGX_RESOLVER_TCP_RSIZE  (2 + 65535)
+#define NGX_RESOLVER_TCP_WSIZE  8192
+
 
 typedef struct {
     u_char  ident_hi;
@@ -54,6 +57,7 @@ typedef struct {
 
 
 ngx_int_t ngx_udp_connect(ngx_resolver_connection_t *rec);
+ngx_int_t ngx_tcp_connect(ngx_resolver_connection_t *rec);
 
 
 static void ngx_resolver_cleanup(void *data);
@@ -64,6 +68,10 @@ static void ngx_resolver_expire(ngx_reso
     ngx_queue_t *queue);
 static ngx_int_t ngx_resolver_send_query(ngx_resolver_t *r,
     ngx_resolver_node_t *rn);
+static ngx_int_t ngx_resolver_send_udp_query(ngx_resolver_t *r,
+    ngx_resolver_connection_t *rec, u_char *query, u_short qlen);
+static ngx_int_t ngx_resolver_send_tcp_query(ngx_resolver_t *r,
+    ngx_resolver_connection_t *rec, u_char *query, u_short qlen);
 static ngx_int_t ngx_resolver_create_name_query(ngx_resolver_t *r,
     ngx_resolver_node_t *rn, ngx_str_t *name);
 static ngx_int_t ngx_resolver_create_addr_query(ngx_resolver_t *r,
@@ -72,12 +80,14 @@ static void ngx_resolver_resend_handler(
 static time_t ngx_resolver_resend(ngx_resolver_t *r, ngx_rbtree_t *tree,
     ngx_queue_t *queue);
 static ngx_uint_t ngx_resolver_resend_empty(ngx_resolver_t *r);
-static void ngx_resolver_read_response(ngx_event_t *rev);
+static void ngx_resolver_udp_read(ngx_event_t *rev);
+static void ngx_resolver_tcp_write(ngx_event_t *wev);
+static void ngx_resolver_tcp_read(ngx_event_t *rev);
 static void ngx_resolver_process_response(ngx_resolver_t *r, u_char *buf,
-    size_t n);
+    size_t n, ngx_uint_t tcp);
 static void ngx_resolver_process_a(ngx_resolver_t *r, u_char *buf, size_t n,
     ngx_uint_t ident, ngx_uint_t code, ngx_uint_t qtype,
-    ngx_uint_t nan, ngx_uint_t ans);
+    ngx_uint_t nan, ngx_uint_t trunc, ngx_uint_t ans);
 static void ngx_resolver_process_ptr(ngx_resolver_t *r, u_char *buf, size_t n,
     ngx_uint_t ident, ngx_uint_t code, ngx_uint_t nan);
 static ngx_resolver_node_t *ngx_resolver_lookup_name(ngx_resolver_t *r,
@@ -165,6 +175,7 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_
     r->ident = -1;
 
     r->resend_timeout = 5;
+    r->tcp_timeout = 5;
     r->expire = 30;
     r->valid = 0;
 
@@ -241,6 +252,7 @@ ngx_resolver_create(ngx_conf_t *cf, ngx_
             rec[j].sockaddr = u.addrs[j].sockaddr;
             rec[j].socklen = u.addrs[j].socklen;
             rec[j].server = u.addrs[j].name;
+            rec[j].resolver = r;
         }
     }
 
@@ -279,6 +291,10 @@ ngx_resolver_cleanup(void *data)
             if (rec[i].udp) {
                 ngx_close_connection(rec[i].udp);
             }
+
+            if (rec[i].tcp) {
+                ngx_close_connection(rec[i].tcp);
+            }
         }
 
         ngx_free(r);
@@ -691,8 +707,10 @@ ngx_resolve_name_locked(ngx_resolver_t *
     }
 
     rn->naddrs = (u_short) -1;
+    rn->tcp = 0;
 #if (NGX_HAVE_INET6)
     rn->naddrs6 = r->ipv6 ? (u_short) -1 : 0;
+    rn->tcp6 = 0;
 #endif
 
     if (ngx_resolver_send_query(r, rn) != NGX_OK) {
@@ -908,8 +926,10 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx
     }
 
     rn->naddrs = (u_short) -1;
+    rn->tcp = 0;
 #if (NGX_HAVE_INET6)
     rn->naddrs6 = (u_short) -1;
+    rn->tcp6 = 0;
 #endif
 
     if (ngx_resolver_send_query(r, rn) != NGX_OK) {
@@ -1104,55 +1124,156 @@ ngx_resolver_expire(ngx_resolver_t *r, n
 static ngx_int_t
 ngx_resolver_send_query(ngx_resolver_t *r, ngx_resolver_node_t *rn)
 {
-    ssize_t                     n;
+    ngx_int_t                   rc;
     ngx_resolver_connection_t  *rec;
 
     rec = r->connections.elts;
     rec = &rec[rn->last_connection];
 
-    if (rec->udp == NULL) {
-
+    if (rec->log.handler == NULL) {
         rec->log = *r->log;
         rec->log.handler = ngx_resolver_log_error;
         rec->log.data = rec;
         rec->log.action = "resolving";
-
+    }
+
+    if (rn->naddrs == (u_short) -1) {
+        rc = rn->tcp ? ngx_resolver_send_tcp_query(r, rec, rn->query, rn->qlen)
+                     : ngx_resolver_send_udp_query(r, rec, rn->query, rn->qlen);
+
+        if (rc != NGX_OK) {
+            return rc;
+        }
+    }
+
+#if (NGX_HAVE_INET6)
+
+    if (rn->query6 && rn->naddrs6 == (u_short) -1) {
+        rc = rn->tcp6
+                    ? ngx_resolver_send_tcp_query(r, rec, rn->query6, rn->qlen)
+                    : ngx_resolver_send_udp_query(r, rec, rn->query6, rn->qlen);
+
+        if (rc != NGX_OK) {
+            return rc;
+        }
+    }
+
+#endif
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_resolver_send_udp_query(ngx_resolver_t *r, ngx_resolver_connection_t  *rec,
+    u_char *query, u_short qlen)
+{
+    ssize_t  n;
+
+    if (rec->udp == NULL) {
         if (ngx_udp_connect(rec) != NGX_OK) {
             return NGX_ERROR;
         }
 
-        rec->udp->data = r;
-        rec->udp->read->handler = ngx_resolver_read_response;
+        rec->udp->data = rec;
+        rec->udp->read->handler = ngx_resolver_udp_read;
         rec->udp->read->resolver = 1;
     }
 
-    if (rn->naddrs == (u_short) -1) {
-        n = ngx_send(rec->udp, rn->query, rn->qlen);
-
-        if (n == -1) {
+    n = ngx_send(rec->udp, query, qlen);
+
+    if (n == -1) {
+        return NGX_ERROR;
+    }
+
+    if ((size_t) n != (size_t) qlen) {
+        ngx_log_error(NGX_LOG_CRIT, &rec->log, 0, "send() incomplete");
+        return NGX_ERROR;
+    }
+
+    return NGX_OK;
+}
+
+
+static ngx_int_t
+ngx_resolver_send_tcp_query(ngx_resolver_t *r, ngx_resolver_connection_t *rec,
+    u_char *query, u_short qlen)
+{
+    ngx_buf_t  *b;
+    ngx_int_t   rc;
+
+    rc = NGX_OK;
+
+    if (rec->tcp == NULL) {
+        b = rec->read_buf;
+
+        if (b == NULL) {
+            b = ngx_resolver_calloc(r, sizeof(ngx_buf_t));
+            if (b == NULL) {
+                return NGX_ERROR;
+            }
+
+            b->start = ngx_resolver_alloc(r, NGX_RESOLVER_TCP_RSIZE);
+            if (b->start == NULL) {
+                return NGX_ERROR;
+            }
+
+            b->end = b->start + NGX_RESOLVER_TCP_RSIZE;
+
+            rec->read_buf = b;
+        }
+
+        b->pos = b->start;
+        b->last = b->start;
+
+        b = rec->write_buf;
+
+        if (b == NULL) {
+            b = ngx_resolver_calloc(r, sizeof(ngx_buf_t));
+            if (b == NULL) {
+                return NGX_ERROR;
+            }
+
+            b->start = ngx_resolver_alloc(r, NGX_RESOLVER_TCP_WSIZE);
+            if (b->start == NULL) {
+                return NGX_ERROR;
+            }
+
+            b->end = b->start + NGX_RESOLVER_TCP_WSIZE;
+
+            rec->write_buf = b;
+        }
+
+        b->pos = b->start;
+        b->last = b->start;
+
+        rc = ngx_tcp_connect(rec);
+        if (rc == NGX_ERROR) {
             return NGX_ERROR;
         }
 
-        if ((size_t) n != (size_t) rn->qlen) {
-            ngx_log_error(NGX_LOG_CRIT, &rec->log, 0, "send() incomplete");
-            return NGX_ERROR;
-        }
+        rec->tcp->data = rec;
+        rec->tcp->write->handler = ngx_resolver_tcp_write;
+        rec->tcp->read->handler = ngx_resolver_tcp_read;
+        rec->tcp->read->resolver = 1;
+
+        ngx_add_timer(rec->tcp->write, (ngx_msec_t) (r->tcp_timeout * 1000));
     }
 
-#if (NGX_HAVE_INET6)
-    if (rn->query6 && rn->naddrs6 == (u_short) -1) {
-        n = ngx_send(rec->udp, rn->query6, rn->qlen);
-
-        if (n == -1) {
-            return NGX_ERROR;
-        }
-
-        if ((size_t) n != (size_t) rn->qlen) {
-            ngx_log_error(NGX_LOG_CRIT, &rec->log, 0, "send() incomplete");
-            return NGX_ERROR;
-        }
+    b = rec->write_buf;
+
+    if (b->end - b->last <  2 + qlen) {
+        ngx_log_error(NGX_LOG_CRIT, &rec->log, 0, "buffer overflow");
+        return NGX_ERROR;
     }
-#endif
+
+    *b->last++ = (u_char) (qlen >> 8);
+    *b->last++ = (u_char) qlen;
+    b->last = ngx_cpymem(b->last, query, qlen);
+
+    if (rc == NGX_OK) {
+        ngx_resolver_tcp_write(rec->tcp->write);
+    }
 
     return NGX_OK;
 }
@@ -1282,13 +1403,15 @@ ngx_resolver_resend_empty(ngx_resolver_t
 
 
 static void
-ngx_resolver_read_response(ngx_event_t *rev)
+ngx_resolver_udp_read(ngx_event_t *rev)
 {
-    ssize_t            n;
-    ngx_connection_t  *c;
-    u_char             buf[NGX_RESOLVER_UDP_SIZE];
+    ssize_t                     n;
+    ngx_connection_t           *c;
+    ngx_resolver_connection_t  *rec;
+    u_char                      buf[NGX_RESOLVER_UDP_SIZE];
 
     c = rev->data;
+    rec = c->data;