From mdounin at mdounin.ru Wed Oct 1 00:32:09 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 1 Oct 2014 04:32:09 +0400 Subject: [BUG] New memory invalid read regression in resolver since nginx 1.7.5 In-Reply-To: References: Message-ID: <20141001003209.GQ69200@mdounin.ru> Hello! On Tue, Sep 30, 2014 at 03:10:42PM -0700, Yichun Zhang (agentzh) wrote: > Hello! > > I've noticed that the code re-factoring in the resolver in nginx 1.7.5 > introduces a new regression that can cause memory invalid reads when > --with-debug is used to build the nginx. The issue still exists in > nginx 1.7.6. [...] > ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0, > "event timer del: %d: %M", > ngx_event_ident(ev->data), ev->timer.key); > > while ev->data here is the resolver node that has already been freed > up earlier in ngx_resolver_free_node. Yes, thanks, it's known (though mostly harmless) issue. Ruslan is looking into it. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Wed Oct 1 00:36:19 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 1 Oct 2014 04:36:19 +0400 Subject: [BUG] New memory invalid read regression in resolver since nginx 1.7.5 In-Reply-To: References: Message-ID: <20141001003619.GR69200@mdounin.ru> Hello! On Tue, Sep 30, 2014 at 03:51:40PM -0700, Yichun Zhang (agentzh) wrote: > Hello! > > On Tue, Sep 30, 2014 at 3:10 PM, Yichun Zhang (agentzh) wrote: > > ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0, > > "event timer del: %d: %M", > > ngx_event_ident(ev->data), ev->timer.key); > > > > There is another issue here: it seems that ngx_event_ident() assumes > ev->data is an ngx_connection_t pointer: > > #define ngx_event_ident(p) ((ngx_connection_t *) (p))->fd > > But in fact, ev->data can point to any thing. Maybe it makes more > sense to just output the pointer value of ev->data here? > > How about the following patch? With such a change timer idents will become much less readable for connection-related timers (that is, most of them), so this is a last resort. -- Maxim Dounin http://nginx.org/ From thierry.magnien at sfr.com Wed Oct 1 07:39:15 2014 From: thierry.magnien at sfr.com (MAGNIEN, Thierry) Date: Wed, 1 Oct 2014 07:39:15 +0000 Subject: Variable visibility between 3rd party modules In-Reply-To: References: <5D103CE839D50E4CBC62C9FD7B83287CC9C492D8@EXCN015.encara.local.ads> Message-ID: <5D103CE839D50E4CBC62C9FD7B83287CC9C4A4E9@EXCN015.encara.local.ads> Hi, That's what I was looking for. Just tried it and it works, thanks! Thierry ------------- De?: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] De la part de bsfranks at gmail.com Envoy??: mardi 30 septembre 2014 17:48 ??: nginx-devel at nginx.org Objet?: Re: Variable visibility between 3rd party modules Hi Thierry, On Tue, Sep 30, 2014 at 2:41 AM, MAGNIEN, Thierry wrote: Hi, In my own module, do I have a simple way to retrieve a variable which has been set by another module ? For example, GeoIP module sets the geoip_org variable. Is there a set of C functions that allow my module to retrieve this value ? Thanks a lot, Thierry Check out the ngx_http_get_variable() function.? Hope this helps! Ben? From agentzh at gmail.com Wed Oct 1 19:36:32 2014 From: agentzh at gmail.com (Yichun Zhang (agentzh)) Date: Wed, 1 Oct 2014 12:36:32 -0700 Subject: [BUG] New memory invalid read regression in resolver since nginx 1.7.5 In-Reply-To: <20141001003619.GR69200@mdounin.ru> References: <20141001003619.GR69200@mdounin.ru> Message-ID: Hello! On Tue, Sep 30, 2014 at 5:36 PM, Maxim Dounin wrote: > > With such a change timer idents will become much less readable > for connection-related timers (that is, most of them), so this > is a last resort. > Yes, I know. Maybe let the caller explicitly tell ngx_event_t whether ev->data is an ngx_connect_t pointer and just in this case, output the fd in it? Regards, -agentzh From piotr at cloudflare.com Thu Oct 2 00:20:06 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 01 Oct 2014 17:20:06 -0700 Subject: [PATCH] Upstream: fix $upstream_cache_last_modified variable Message-ID: # HG changeset patch # User Piotr Sikora # Date 1412203733 25200 # Wed Oct 01 15:48:53 2014 -0700 # Node ID dbf6364ab44fcc6b5d1fbbe64c162135b18e9209 # Parent a215d9021f137b9e2d4f69c37c7f992a2bef12c6 Upstream: fix $upstream_cache_last_modified variable. Due to the u->headers_in.last_modified_time not being correctly initialized, this variable was evaluated to "Thu, 01 Jan 1970 00:00:00 GMT" for responses cached without the "Last-Modified" header which resulted in subsequent proxy requests being sent with "If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT" header. Signed-off-by: Piotr Sikora diff -r a215d9021f13 -r dbf6364ab44f src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Tue Sep 30 17:20:33 2014 +0400 +++ b/src/http/ngx_http_upstream.c Wed Oct 01 15:48:53 2014 -0700 @@ -441,6 +441,7 @@ ngx_http_upstream_create(ngx_http_reques #endif u->headers_in.content_length_n = -1; + u->headers_in.last_modified_time = -1; return NGX_OK; } @@ -885,6 +886,7 @@ ngx_http_upstream_cache_send(ngx_http_re ngx_memzero(&u->headers_in, sizeof(ngx_http_upstream_headers_in_t)); u->headers_in.content_length_n = -1; + u->headers_in.last_modified_time = -1; if (ngx_list_init(&u->headers_in.headers, r->pool, 8, sizeof(ngx_table_elt_t)) @@ -1582,6 +1584,7 @@ ngx_http_upstream_reinit(ngx_http_reques ngx_memzero(&u->headers_in, sizeof(ngx_http_upstream_headers_in_t)); u->headers_in.content_length_n = -1; + u->headers_in.last_modified_time = -1; if (ngx_list_init(&u->headers_in.headers, r->pool, 8, sizeof(ngx_table_elt_t)) From vbart at nginx.com Thu Oct 2 18:37:06 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Thu, 02 Oct 2014 18:37:06 +0000 Subject: [nginx] Version bump. Message-ID: details: http://hg.nginx.org/nginx/rev/26d28506282a branches: changeset: 5867:26d28506282a user: Valentin Bartenev date: Thu Oct 02 22:36:27 2014 +0400 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r a215d9021f13 -r 26d28506282a src/core/nginx.h --- a/src/core/nginx.h Tue Sep 30 17:20:33 2014 +0400 +++ b/src/core/nginx.h Thu Oct 02 22:36:27 2014 +0400 @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1007006 -#define NGINX_VERSION "1.7.6" +#define nginx_version 1007007 +#define NGINX_VERSION "1.7.7" #define NGINX_VER "nginx/" NGINX_VERSION #ifdef NGX_BUILD From vbart at nginx.com Thu Oct 2 18:37:09 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Thu, 02 Oct 2014 18:37:09 +0000 Subject: [nginx] Fixed counting of sent bytes in the send chain functions... Message-ID: details: http://hg.nginx.org/nginx/rev/6bbad2e73245 branches: changeset: 5868:6bbad2e73245 user: Valentin Bartenev date: Wed Aug 27 20:51:01 2014 +0400 description: Fixed counting of sent bytes in the send chain functions on EINTR. Previously, a value of the "send" variable wasn't properly adjusted in a rare case when syscall was interrupted by a signal. As a result, these functions could send less data than the limit allows. diffstat: src/os/unix/ngx_darwin_sendfile_chain.c | 1 + src/os/unix/ngx_freebsd_sendfile_chain.c | 1 + src/os/unix/ngx_linux_sendfile_chain.c | 1 + src/os/unix/ngx_solaris_sendfilev_chain.c | 1 + src/os/unix/ngx_writev_chain.c | 1 + 5 files changed, 5 insertions(+), 0 deletions(-) diffs (55 lines): diff -r 26d28506282a -r 6bbad2e73245 src/os/unix/ngx_darwin_sendfile_chain.c --- a/src/os/unix/ngx_darwin_sendfile_chain.c Thu Oct 02 22:36:27 2014 +0400 +++ b/src/os/unix/ngx_darwin_sendfile_chain.c Wed Aug 27 20:51:01 2014 +0400 @@ -308,6 +308,7 @@ ngx_darwin_sendfile_chain(ngx_connection in = ngx_handle_sent_chain(in, sent); if (eintr) { + send = prev_send + sent; continue; } diff -r 26d28506282a -r 6bbad2e73245 src/os/unix/ngx_freebsd_sendfile_chain.c --- a/src/os/unix/ngx_freebsd_sendfile_chain.c Thu Oct 02 22:36:27 2014 +0400 +++ b/src/os/unix/ngx_freebsd_sendfile_chain.c Wed Aug 27 20:51:01 2014 +0400 @@ -378,6 +378,7 @@ ngx_freebsd_sendfile_chain(ngx_connectio } if (eintr) { + send = prev_send + sent; continue; } diff -r 26d28506282a -r 6bbad2e73245 src/os/unix/ngx_linux_sendfile_chain.c --- a/src/os/unix/ngx_linux_sendfile_chain.c Thu Oct 02 22:36:27 2014 +0400 +++ b/src/os/unix/ngx_linux_sendfile_chain.c Wed Aug 27 20:51:01 2014 +0400 @@ -316,6 +316,7 @@ ngx_linux_sendfile_chain(ngx_connection_ in = ngx_handle_sent_chain(in, sent); if (eintr) { + send = prev_send; continue; } diff -r 26d28506282a -r 6bbad2e73245 src/os/unix/ngx_solaris_sendfilev_chain.c --- a/src/os/unix/ngx_solaris_sendfilev_chain.c Thu Oct 02 22:36:27 2014 +0400 +++ b/src/os/unix/ngx_solaris_sendfilev_chain.c Wed Aug 27 20:51:01 2014 +0400 @@ -200,6 +200,7 @@ ngx_solaris_sendfilev_chain(ngx_connecti in = ngx_handle_sent_chain(in, sent); if (eintr) { + send = prev_send + sent; continue; } diff -r 26d28506282a -r 6bbad2e73245 src/os/unix/ngx_writev_chain.c --- a/src/os/unix/ngx_writev_chain.c Thu Oct 02 22:36:27 2014 +0400 +++ b/src/os/unix/ngx_writev_chain.c Wed Aug 27 20:51:01 2014 +0400 @@ -134,6 +134,7 @@ ngx_writev_chain(ngx_connection_t *c, ng in = ngx_handle_sent_chain(in, sent); if (eintr) { + send = prev_send; continue; } From agentzh at gmail.com Thu Oct 2 19:32:57 2014 From: agentzh at gmail.com (Yichun Zhang (agentzh)) Date: Thu, 2 Oct 2014 12:32:57 -0700 Subject: [PATCH] Fix a buffer overflow issue in ngx_hash_t when exceeding limits Message-ID: Hello! Valgrind memcheck caught a buffer overflow issue in ngx_hash_t when exceeding the pre-configured limits on my side: ==7417== Invalid write of size 2 ==7417== at 0x40600D: ngx_hash_init (ngx_hash.c:324) ==7417== by 0x45BBFD: ngx_http_proxy_merge_loc_conf (ngx_http_proxy_module.c:3214) ==7417== by 0x422C80: ngx_http_block (ngx_http.c:597) ==7417== by 0x412CE0: ngx_conf_parse (ngx_conf_file.c:391) ==7417== by 0x410971: ngx_init_cycle (ngx_cycle.c:264) ==7417== by 0x403FF0: main (nginx.c:333) ==7417== Address 0x4e66300 is 0 bytes after a block of size 1,024 alloc'd ==7417== at 0x4A0645D: malloc (vg_replace_malloc.c:291) ==7417== by 0x41C46A: ngx_alloc (ngx_alloc.c:22) ==7417== by 0x405ED5: ngx_hash_init (ngx_hash.c:271) ==7417== by 0x45BBFD: ngx_http_proxy_merge_loc_conf (ngx_http_proxy_module.c:3214) ==7417== by 0x422C80: ngx_http_block (ngx_http.c:597) ==7417== by 0x412CE0: ngx_conf_parse (ngx_conf_file.c:391) ==7417== by 0x410971: ngx_init_cycle (ngx_cycle.c:264) ==7417== by 0x403FF0: main (nginx.c:333) ==7417== To reproduce it just use the default official nginx 1.7.5 to run the following nginx snippet: proxy_set_header X-Forwarded-For "1.2.3.4"; proxy_set_header X-Forwarded-For $http_x_forwarded_for; Basically the buffer is iterated through by its size plus one. The following patch fixes this: # HG changeset patch # User Yichun Zhang # Date 1412276417 25200 # Thu Oct 02 12:00:17 2014 -0700 # Node ID 4032b992f23b054c1a2cfb0be879330d2c6708e5 # Parent 1ff0f68d9376e3d184d65814a6372856bf65cfcd Hash: buffer overflow might happen when exceeding the pre-configured limits. diff -r 1ff0f68d9376 -r 4032b992f23b src/core/ngx_hash.c --- a/src/core/ngx_hash.c Tue Sep 30 15:50:28 2014 -0700 +++ b/src/core/ngx_hash.c Thu Oct 02 12:00:17 2014 -0700 @@ -312,6 +312,8 @@ ngx_hash_init(ngx_hash_init_t *hinit, ng continue; } + size--; + ngx_log_error(NGX_LOG_WARN, hinit->pool->log, 0, "could not build optimal %s, you should increase " "either %s_max_size: %i or %s_bucket_size: %i; " -------------- next part -------------- A non-text attachment was scrubbed... Name: hash_overflow.patch Type: text/x-patch Size: 770 bytes Desc: not available URL: From piotr at cloudflare.com Thu Oct 2 21:22:53 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Thu, 02 Oct 2014 14:22:53 -0700 Subject: [PATCH] Upstream: add $upstream_cache_age variable Message-ID: <488aa8f86041dbe99a8f.1412284973@piotrs-macbook-pro.local> # HG changeset patch # User Piotr Sikora # Date 1412284916 25200 # Thu Oct 02 14:21:56 2014 -0700 # Node ID 488aa8f86041dbe99a8f8ebbb3df533b9ce3e19e # Parent a215d9021f137b9e2d4f69c37c7f992a2bef12c6 Upstream: add $upstream_cache_age variable. This variable represents the amount of time that passed since the response was fetched (or successfully revalidated) from the upstream server. Signed-off-by: Piotr Sikora diff -r a215d9021f13 -r 488aa8f86041 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Tue Sep 30 17:20:33 2014 +0400 +++ b/src/http/ngx_http_upstream.c Thu Oct 02 14:21:56 2014 -0700 @@ -21,6 +21,8 @@ static ngx_int_t ngx_http_upstream_cache ngx_http_variable_value_t *v, uintptr_t data); static ngx_int_t ngx_http_upstream_cache_etag(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); +static ngx_int_t ngx_http_upstream_cache_age(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data); #endif static void ngx_http_upstream_init_request(ngx_http_request_t *r); @@ -373,6 +375,10 @@ static ngx_http_variable_t ngx_http_ups ngx_http_upstream_cache_etag, 0, NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_NOHASH, 0 }, + { ngx_string("upstream_cache_age"), NULL, + ngx_http_upstream_cache_age, 0, + NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_NOHASH, 0 }, + #endif { ngx_null_string, NULL, NULL, 0, 0, 0 } @@ -4851,6 +4857,32 @@ ngx_http_upstream_cache_etag(ngx_http_re return NGX_OK; } + +static ngx_int_t +ngx_http_upstream_cache_age(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data) +{ + if (r->upstream == NULL + || r->upstream->cache_status <= NGX_HTTP_CACHE_EXPIRED + || r->upstream->cache_status == NGX_HTTP_CACHE_REVALIDATED) + { + v->not_found = 1; + return NGX_OK; + } + + v->data = ngx_pnalloc(r->pool, NGX_TIME_T_LEN); + if (v->data == NULL) { + return NGX_ERROR; + } + + v->len = ngx_sprintf(v->data, "%T", ngx_time() - r->cache->date) - v->data; + v->valid = 1; + v->no_cacheable = 0; + v->not_found = 0; + + return NGX_OK; +} + #endif From arut at nginx.com Fri Oct 3 13:29:09 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Fri, 3 Oct 2014 17:29:09 +0400 Subject: [PATCH] Upstream: add $upstream_cache_age variable In-Reply-To: <488aa8f86041dbe99a8f.1412284973@piotrs-macbook-pro.local> References: <488aa8f86041dbe99a8f.1412284973@piotrs-macbook-pro.local> Message-ID: <8CF786D5-0834-4D2A-B7CB-BBA96A16293A@nginx.com> On 03 Oct 2014, at 01:22, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1412284916 25200 > # Thu Oct 02 14:21:56 2014 -0700 > # Node ID 488aa8f86041dbe99a8f8ebbb3df533b9ce3e19e > # Parent a215d9021f137b9e2d4f69c37c7f992a2bef12c6 > Upstream: add $upstream_cache_age variable. > > This variable represents the amount of time that passed since the response > was fetched (or successfully revalidated) from the upstream server. > > Signed-off-by: Piotr Sikora > > diff -r a215d9021f13 -r 488aa8f86041 src/http/ngx_http_upstream.c > --- a/src/http/ngx_http_upstream.c Tue Sep 30 17:20:33 2014 +0400 > +++ b/src/http/ngx_http_upstream.c Thu Oct 02 14:21:56 2014 -0700 > @@ -21,6 +21,8 @@ static ngx_int_t ngx_http_upstream_cache > ngx_http_variable_value_t *v, uintptr_t data); > static ngx_int_t ngx_http_upstream_cache_etag(ngx_http_request_t *r, > ngx_http_variable_value_t *v, uintptr_t data); > +static ngx_int_t ngx_http_upstream_cache_age(ngx_http_request_t *r, > + ngx_http_variable_value_t *v, uintptr_t data); > #endif > > static void ngx_http_upstream_init_request(ngx_http_request_t *r); > @@ -373,6 +375,10 @@ static ngx_http_variable_t ngx_http_ups > ngx_http_upstream_cache_etag, 0, > NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_NOHASH, 0 }, > > + { ngx_string("upstream_cache_age"), NULL, > + ngx_http_upstream_cache_age, 0, > + NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_NOHASH, 0 }, > + > #endif > > { ngx_null_string, NULL, NULL, 0, 0, 0 } > @@ -4851,6 +4857,32 @@ ngx_http_upstream_cache_etag(ngx_http_re > return NGX_OK; > } > > + > +static ngx_int_t > +ngx_http_upstream_cache_age(ngx_http_request_t *r, > + ngx_http_variable_value_t *v, uintptr_t data) > +{ > + if (r->upstream == NULL > + || r->upstream->cache_status <= NGX_HTTP_CACHE_EXPIRED > + || r->upstream->cache_status == NGX_HTTP_CACHE_REVALIDATED) > + { > + v->not_found = 1; > + return NGX_OK; > + } > + Probably it would be more clear to list the valid 3 statuses here. One extra operation though. if (r->upstream == NULL) { v->not_found = 1; return NGX_OK; } if (r->upstream->cache_status != NGX_HTTP_CACHE_STALE && r->upstream->cache_status != NGX_HTTP_CACHE_UPDATING && r->upstream->cache_status != NGX_HTTP_CACHE_HIT) { v->not_found = 1; return NGX_OK; } > + v->data = ngx_pnalloc(r->pool, NGX_TIME_T_LEN); > + if (v->data == NULL) { > + return NGX_ERROR; > + } > + > + v->len = ngx_sprintf(v->data, "%T", ngx_time() - r->cache->date) - v->data; > + v->valid = 1; > + v->no_cacheable = 0; > + v->not_found = 0; > + > + return NGX_OK; > +} > + > #endif > > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > From piotr at cloudflare.com Fri Oct 3 17:24:29 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Fri, 03 Oct 2014 10:24:29 -0700 Subject: [PATCH] Upstream: add $upstream_cache_age variable In-Reply-To: <8CF786D5-0834-4D2A-B7CB-BBA96A16293A@nginx.com> References: <8CF786D5-0834-4D2A-B7CB-BBA96A16293A@nginx.com> Message-ID: # HG changeset patch # User Piotr Sikora # Date 1412356980 25200 # Fri Oct 03 10:23:00 2014 -0700 # Node ID f343adae412afce435a18384b0aad052405a393b # Parent 6bbad2e732458bf53771e80c63a654b3d7f61963 Upstream: add $upstream_cache_age variable. This variable represents the amount of time that passed since the response was fetched (or successfully revalidated) from the upstream server. Signed-off-by: Piotr Sikora diff -r 6bbad2e73245 -r f343adae412a src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Wed Aug 27 20:51:01 2014 +0400 +++ b/src/http/ngx_http_upstream.c Fri Oct 03 10:23:00 2014 -0700 @@ -21,6 +21,8 @@ static ngx_int_t ngx_http_upstream_cache ngx_http_variable_value_t *v, uintptr_t data); static ngx_int_t ngx_http_upstream_cache_etag(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); +static ngx_int_t ngx_http_upstream_cache_age(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data); #endif static void ngx_http_upstream_init_request(ngx_http_request_t *r); @@ -373,6 +375,10 @@ static ngx_http_variable_t ngx_http_ups ngx_http_upstream_cache_etag, 0, NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_NOHASH, 0 }, + { ngx_string("upstream_cache_age"), NULL, + ngx_http_upstream_cache_age, 0, + NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_NOHASH, 0 }, + #endif { ngx_null_string, NULL, NULL, 0, 0, 0 } @@ -4851,6 +4857,37 @@ ngx_http_upstream_cache_etag(ngx_http_re return NGX_OK; } + +static ngx_int_t +ngx_http_upstream_cache_age(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data) +{ + if (r->upstream == NULL) { + v->not_found = 1; + return NGX_OK; + } + + if (r->upstream->cache_status != NGX_HTTP_CACHE_STALE + && r->upstream->cache_status != NGX_HTTP_CACHE_UPDATING + && r->upstream->cache_status != NGX_HTTP_CACHE_HIT) + { + v->not_found = 1; + return NGX_OK; + } + + v->data = ngx_pnalloc(r->pool, NGX_TIME_T_LEN); + if (v->data == NULL) { + return NGX_ERROR; + } + + v->len = ngx_sprintf(v->data, "%T", ngx_time() - r->cache->date) - v->data; + v->valid = 1; + v->no_cacheable = 0; + v->not_found = 0; + + return NGX_OK; +} + #endif From kou at cozmixng.org Sat Oct 4 10:51:56 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Sat, 04 Oct 2014 19:51:56 +0900 (JST) Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux Message-ID: <20141004.195156.801414609668336877.kou@cozmixng.org> Hi, I'm trying cross compiling nginx for Windows on Debian GNU/Linux. The current nginx doesn't support it. I created a patch to do it. With the attached patch, I can cross compile nginx for Windows on Debian GNU/Linux. Please review the attached patch. The current status (without patch) ---------------------------------- Compiler: % x86_64-w64-mingw32-gcc --version x86_64-w64-mingw32-gcc (GCC) 4.9.1 Copyright (C) 2014 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Configure command line: % auto/configure \ --prefix=/tmp/local \ --crossbuild=win32 \ --with-cc=x86_64-w64-mingw32-gcc \ --with-cpp=x86_64-w64-mingw32-c++ \ --without-http_rewrite_module \ --without-http_gzip_module Build result: % make make -f objs/Makefile make[1]: Entering directory '/home/kou/work/c/nginx.win32' x86_64-w64-mingw32-gcc -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I src/core -I src/event -I src/event/modules -I src/os/win32 -I objs \ -o objs/src/core/nginx.o \ src/core/nginx.c In file included from /usr/share/mingw-w64/include/stdio.h:80:0, from src/os/win32/ngx_win32_config.h:45, from src/core/ngx_config.h:38, from src/core/nginx.c:8: /usr/share/mingw-w64/include/_mingw_off_t.h:26:9: error: unknown type name ?off32_t? typedef off32_t off_t; ^ In file included from src/os/win32/ngx_win32_config.h:45:0, from src/core/ngx_config.h:38, from src/core/nginx.c:8: /usr/share/mingw-w64/include/stdio.h:463:28: error: unknown type name ?_off_t? int fseeko(FILE* stream, _off_t offset, int whence); ^ /usr/share/mingw-w64/include/stdio.h:474:3: error: unknown type name ?_off_t? _off_t ftello(FILE * stream); ^ In file included from src/core/ngx_config.h:38:0, from src/core/nginx.c:8: src/os/win32/ngx_win32_config.h:140:29: error: conflicting types for ?intptr_t? typedef int intptr_t; ^ In file included from /usr/share/mingw-w64/include/windows.h:9:0, from src/os/win32/ngx_win32_config.h:28, from src/core/ngx_config.h:38, from src/core/nginx.c:8: /usr/share/mingw-w64/include/_mingw.h:399:35: note: previous declaration of ?intptr_t? was here __MINGW_EXTENSION typedef __int64 intptr_t; ^ In file included from src/core/ngx_config.h:38:0, from src/core/nginx.c:8: src/os/win32/ngx_win32_config.h:141:29: error: conflicting types for ?uintptr_t? typedef u_int uintptr_t; ^ In file included from /usr/share/mingw-w64/include/windows.h:9:0, from src/os/win32/ngx_win32_config.h:28, from src/core/ngx_config.h:38, from src/core/nginx.c:8: /usr/share/mingw-w64/include/_mingw.h:412:44: note: previous declaration of ?uintptr_t? was here __MINGW_EXTENSION typedef unsigned __int64 uintptr_t; ^ In file included from src/core/ngx_config.h:38:0, from src/core/nginx.c:8: src/os/win32/ngx_win32_config.h:146:29: error: conflicting types for ?off_t? typedef __int64 off_t; ^ In file included from /usr/share/mingw-w64/include/stdio.h:80:0, from src/os/win32/ngx_win32_config.h:45, from src/core/ngx_config.h:38, from src/core/nginx.c:8: /usr/share/mingw-w64/include/_mingw_off_t.h:26:17: note: previous declaration of ?off_t? was here typedef off32_t off_t; ^ In file included from src/core/ngx_config.h:38:0, from src/core/nginx.c:8: src/os/win32/ngx_win32_config.h:167:29: error: conflicting types for ?ssize_t? typedef int ssize_t; ^ In file included from /usr/share/mingw-w64/include/windows.h:9:0, from src/os/win32/ngx_win32_config.h:28, from src/core/ngx_config.h:38, from src/core/nginx.c:8: /usr/share/mingw-w64/include/_mingw.h:387:35: note: previous declaration of ?ssize_t? was here __MINGW_EXTENSION typedef __int64 ssize_t; ^ objs/Makefile:323: recipe for target 'objs/src/core/nginx.o' failed make[1]: *** [objs/src/core/nginx.o] Error 1 make[1]: Leaving directory '/home/kou/work/c/nginx.win32' Makefile:8: recipe for target 'build' failed make: *** [build] Error 2 How to solve ------------ I use the system type definitions instead of custom type definitions for off_t, intptr_t and ssize_t to solve the errors. "#define _FILE_OFFSET_BITS 64" is for defining off_t as a 64bit type. (nginx requires off_t as a 64bit type.) See the attached patch for details. Note ---- The main change is the change for src/os/win32/ngx_win32_config.h. The changes of the followings are just trivial build errors: * src/event/modules/ngx_iocp_module.c * src/os/win32/ngx_wsasend_chain.c The change of src/event/modules/ngx_iocp_module.c is for fixing the following warning: x86_64-w64-mingw32-gcc -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I src/core -I src/event -I src/event/modules -I src/os/win32 -I objs \ -o objs/src/event/modules/ngx_iocp_module.o \ src/event/modules/ngx_iocp_module.c src/event/modules/ngx_iocp_module.c: In function 'ngx_iocp_process_events': src/event/modules/ngx_iocp_module.c:250:10: error: passing argument 3 of 'GetQueuedCompletionStatus' from incompatible pointer type [-Werror] rc = GetQueuedCompletionStatus(iocp, &bytes, (LPDWORD) &key, ^ In file included from /usr/share/mingw-w64/include/winbase.h:21:0, from /usr/share/mingw-w64/include/windows.h:70, from src/os/win32/ngx_win32_config.h:28, from src/core/ngx_config.h:38, from src/event/modules/ngx_iocp_module.c:8: /usr/share/mingw-w64/include/ioapiset.h:20:29: note: expected 'PULONG_PTR' but argument is of type 'DWORD *' WINBASEAPI WINBOOL WINAPI GetQueuedCompletionStatus (HANDLE CompletionPort, LPDWORD lpNumberOfBytesTransferred, PULONG_PTR lpCompletionKey, LPOVERLAPPED *lpOverlapped, DWORD dwMilliseconds); ^ cc1: all warnings being treated as errors GetQueuedCompletionStatus() document on MSDN says the following signature: http://msdn.microsoft.com/en-us/library/windows/desktop/aa364986.aspx BOOL WINAPI GetQueuedCompletionStatus( _In_ HANDLE CompletionPort, _Out_ LPDWORD lpNumberOfBytes, _Out_ PULONG_PTR lpCompletionKey, _Out_ LPOVERLAPPED *lpOverlapped, _In_ DWORD dwMilliseconds ); In the latest specification, the type of the third argument (lpCompletionKey) is PULONG_PTR not LPDWORD. So the attached patch casts to PULONG_PTR instead of LPDWORD. The change of src/os/win32/ngx_wsasend_chain.c is for fixing the following warnings: x86_64-w64-mingw32-gcc -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I src/core -I src/event -I src/event/modules -I src/os/win32 -I objs \ -o objs/src/os/win32/ngx_wsasend_chain.o \ src/os/win32/ngx_wsasend_chain.c src/os/win32/ngx_wsasend_chain.c: In function 'ngx_wsasend_chain': src/os/win32/ngx_wsasend_chain.c:37:29: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare] if (limit == 0 || limit > NGX_MAX_UINT32_VALUE - ngx_pagesize) { ^ src/os/win32/ngx_wsasend_chain.c: In function 'ngx_overlapped_wsasend_chain': src/os/win32/ngx_wsasend_chain.c:159:33: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare] if (limit == 0 || limit > NGX_MAX_UINT32_VALUE - ngx_pagesize) { ^ cc1: all warnings being treated as errors objs/Makefile:673: recipe for target 'objs/src/os/win32/ngx_wsasend_chain.o' failed I don't test the patch with Visual Studio. Sorry. I hope that someone try the patch with Visual Studio. Thanks, -- kou -------------- next part -------------- A non-text attachment was scrubbed... Name: nginx-support-mingw.diff Type: text/x-patch Size: 3952 bytes Desc: not available URL: From arut at nginx.com Mon Oct 6 10:10:18 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Mon, 6 Oct 2014 14:10:18 +0400 Subject: [PATCH] Upstream: add $upstream_cache_age variable In-Reply-To: References: <8CF786D5-0834-4D2A-B7CB-BBA96A16293A@nginx.com> Message-ID: <68F0B9EB-6B1E-4EFC-878B-75B59CB07872@nginx.com> On 03 Oct 2014, at 21:24, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1412356980 25200 > # Fri Oct 03 10:23:00 2014 -0700 > # Node ID f343adae412afce435a18384b0aad052405a393b > # Parent 6bbad2e732458bf53771e80c63a654b3d7f61963 > Upstream: add $upstream_cache_age variable. > > This variable represents the amount of time that passed since the response > was fetched (or successfully revalidated) from the upstream server. > > Signed-off-by: Piotr Sikora > > diff -r 6bbad2e73245 -r f343adae412a src/http/ngx_http_upstream.c > --- a/src/http/ngx_http_upstream.c Wed Aug 27 20:51:01 2014 +0400 > +++ b/src/http/ngx_http_upstream.c Fri Oct 03 10:23:00 2014 -0700 > @@ -21,6 +21,8 @@ static ngx_int_t ngx_http_upstream_cache > ngx_http_variable_value_t *v, uintptr_t data); > static ngx_int_t ngx_http_upstream_cache_etag(ngx_http_request_t *r, > ngx_http_variable_value_t *v, uintptr_t data); > +static ngx_int_t ngx_http_upstream_cache_age(ngx_http_request_t *r, > + ngx_http_variable_value_t *v, uintptr_t data); > #endif > > static void ngx_http_upstream_init_request(ngx_http_request_t *r); > @@ -373,6 +375,10 @@ static ngx_http_variable_t ngx_http_ups > ngx_http_upstream_cache_etag, 0, > NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_NOHASH, 0 }, > > + { ngx_string("upstream_cache_age"), NULL, > + ngx_http_upstream_cache_age, 0, > + NGX_HTTP_VAR_NOCACHEABLE|NGX_HTTP_VAR_NOHASH, 0 }, > + > #endif > > { ngx_null_string, NULL, NULL, 0, 0, 0 } > @@ -4851,6 +4857,37 @@ ngx_http_upstream_cache_etag(ngx_http_re > return NGX_OK; > } > > + > +static ngx_int_t > +ngx_http_upstream_cache_age(ngx_http_request_t *r, > + ngx_http_variable_value_t *v, uintptr_t data) > +{ > + if (r->upstream == NULL) { > + v->not_found = 1; > + return NGX_OK; > + } > + > + if (r->upstream->cache_status != NGX_HTTP_CACHE_STALE > + && r->upstream->cache_status != NGX_HTTP_CACHE_UPDATING > + && r->upstream->cache_status != NGX_HTTP_CACHE_HIT) > + { > + v->not_found = 1; > + return NGX_OK; > + } > + > + v->data = ngx_pnalloc(r->pool, NGX_TIME_T_LEN); > + if (v->data == NULL) { > + return NGX_ERROR; > + } > + > + v->len = ngx_sprintf(v->data, "%T", ngx_time() - r->cache->date) - v->data; > + v->valid = 1; > + v->no_cacheable = 0; > + v->not_found = 0; > + > + return NGX_OK; > +} > + > #endif > > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > Another issue i?ve found reading the RFC. If the reason for $upstream_cache_age is adding HTTP Age response header then it makes sense to improve age calculation algorithm. According to RFC 7234 / 4.2.3. "Calculating Age? the local cache age should be added to the Age value received from backend (?age_value"). response_delay = response_time - request_time; corrected_age_value = age_value + response_delay; From mdounin at mdounin.ru Mon Oct 6 11:26:03 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 6 Oct 2014 15:26:03 +0400 Subject: [PATCH] Upstream: add $upstream_cache_age variable In-Reply-To: <68F0B9EB-6B1E-4EFC-878B-75B59CB07872@nginx.com> References: <8CF786D5-0834-4D2A-B7CB-BBA96A16293A@nginx.com> <68F0B9EB-6B1E-4EFC-878B-75B59CB07872@nginx.com> Message-ID: <20141006112603.GI69200@mdounin.ru> Hello! On Mon, Oct 06, 2014 at 02:10:18PM +0400, Roman Arutyunyan wrote: [...] > Another issue i?ve found reading the RFC. > > If the reason for $upstream_cache_age is adding HTTP Age response header then it makes > sense to improve age calculation algorithm. > > According to RFC 7234 / 4.2.3. "Calculating Age? the local cache age should be added > to the Age value received from backend (?age_value"). > > response_delay = response_time - request_time; > corrected_age_value = age_value + response_delay; I agree, I don't think that the "date" value as stored in the cache can/should be used for Age header. And I doubt it should be exposed as a variable at all, even if calculated properly. -- Maxim Dounin http://nginx.org/ From richard at fussenegger.info Mon Oct 6 12:45:41 2014 From: richard at fussenegger.info (Richard Fussenegger, BSc) Date: Mon, 06 Oct 2014 14:45:41 +0200 Subject: Session Ticket Rotation In-Reply-To: <20140922123809.GE80160@mdounin.ru> References: <54200A7F.3030408@fussenegger.info> <20140922123809.GE80160@mdounin.ru> Message-ID: <54328EF5.4050608@fussenegger.info> On 9/22/2014 2:38 PM, Maxim Dounin wrote: > Hello! > > On Mon, Sep 22, 2014 at 01:39:43PM +0200, Richard Fussenegger, BSc wrote: > >> I'd like to implement built-in session ticket rotation. I know that it this >> was discussed before but it was never implemented. Right now a custom >> external ticket key system is supported. Admins with single installations >> and not enough knowledge about the topic are left with keys that are valid >> for the complete lifetime nginx is running. > That's not really true: ticket keys are regenerated on each > configuration reload. Maxim, just to clarify, will nginx really use a new key (either via OpenSSL or key files) upon *reload* or only on *restart*? In other words, this should do, right? https://github.com/Fleshgrinder/nginx-sysvinit-script/blob/master/nginx#L116 Richard From mdounin at mdounin.ru Mon Oct 6 12:53:29 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 6 Oct 2014 16:53:29 +0400 Subject: Session Ticket Rotation In-Reply-To: <54328EF5.4050608@fussenegger.info> References: <54200A7F.3030408@fussenegger.info> <20140922123809.GE80160@mdounin.ru> <54328EF5.4050608@fussenegger.info> Message-ID: <20141006125329.GJ69200@mdounin.ru> Hello! On Mon, Oct 06, 2014 at 02:45:41PM +0200, Richard Fussenegger, BSc wrote: > On 9/22/2014 2:38 PM, Maxim Dounin wrote: > >Hello! > > > >On Mon, Sep 22, 2014 at 01:39:43PM +0200, Richard Fussenegger, BSc wrote: > > > >>I'd like to implement built-in session ticket rotation. I know that it this > >>was discussed before but it was never implemented. Right now a custom > >>external ticket key system is supported. Admins with single installations > >>and not enough knowledge about the topic are left with keys that are valid > >>for the complete lifetime nginx is running. > >That's not really true: ticket keys are regenerated on each > >configuration reload. > Maxim, just to clarify, will nginx really use a new key (either via OpenSSL > or key files) upon *reload* or only on *restart*? > > In other words, this should do, right? > https://github.com/Fleshgrinder/nginx-sysvinit-script/blob/master/nginx#L116 Yes, configuration reload is enough. -- Maxim Dounin http://nginx.org/ From manuel at merchise.org Mon Oct 6 14:34:04 2014 From: manuel at merchise.org (=?UTF-8?B?TWFudWVsIFbDoXpxdWV6?=) Date: Mon, 06 Oct 2014 10:34:04 -0400 Subject: On ETag removal by the gzip module. Message-ID: <5432A85C.9050509@merchise.org> Hi all, I'm not a nginx developer; just an user. Recently I deployed OpenERP and use nginx as a front end, mainly to: 1. Deal with slow clients 2. Add gziping support to our app. Some context: Many of the application users access it using a modem connection (up to 56kbps). So both caching and gzipping are key to performance. OpenERP builds a single CSS and a single JS from installed modules' sources. And creates a strong Etag for both responses, and uses a "Cache-Control: must-revalidate, max-age=0" But then nginx removes the Etag and caching is hindered. I've read the following thread (from several months ago): http://mailman.nginx.org/pipermail/nginx-devel/2013-November/004498.html There are several things I don't fully understand: Compression is a means to improve performance, not to change semantics of the content. I don't see why compression would affect the entity's content/semantic. Both browsers and caches should store the content as if compression would not have been applied, or apply a disk-saving compression themselves, but that would be out of HTTP reach. Am I right? Or more precisely where am I wrong? Though I have read the RFC, I hardly consider myself an expert on it... In order to avoid the Etag removal I'm using just compressing the AJAX responses (application/json). But up-front then the up-front 1.5MB JS file won't be compressed and clients may have to wait longer than needed. After reading the thread above I changed the Etag to be a weak one, but it still gets removed. Do I need to do anything else? Summary: Why does gzipping interferes with etaging? Best regards, Manuel. From albertcasademont at gmail.com Mon Oct 6 15:56:09 2014 From: albertcasademont at gmail.com (Albert Casademont Filella) Date: Mon, 6 Oct 2014 17:56:09 +0200 Subject: On ETag removal by the gzip module. In-Reply-To: <5432A85C.9050509@merchise.org> References: <5432A85C.9050509@merchise.org> Message-ID: Hi Manuel, The weak etag preservation was added to nginx 1.7.3 so I guess your problem is that you're still on the 1.6.* branch! Hope that helps :) Albert On Mon, Oct 6, 2014 at 4:34 PM, Manuel V?zquez wrote: > Hi all, > > I'm not a nginx developer; just an user. > > Recently I deployed OpenERP and use nginx as a front end, mainly to: > > 1. Deal with slow clients > 2. Add gziping support to our app. > > Some context: > > Many of the application users access it using a modem connection (up to > 56kbps). So both caching and gzipping are key to performance. > > OpenERP builds a single CSS and a single JS from installed modules' > sources. And creates a strong Etag for both responses, and uses a > "Cache-Control: must-revalidate, max-age=0" > > But then nginx removes the Etag and caching is hindered. > > I've read the following thread (from several months ago): > http://mailman.nginx.org/pipermail/nginx-devel/2013-November/004498.html > > There are several things I don't fully understand: > > Compression is a means to improve performance, not to change semantics of > the content. I don't see why compression would affect the entity's > content/semantic. Both browsers and caches should store the content as if > compression would not have been applied, or apply a disk-saving compression > themselves, but that would be out of HTTP reach. Am I right? Or more > precisely where am I wrong? > > Though I have read the RFC, I hardly consider myself an expert on it... > > In order to avoid the Etag removal I'm using just compressing the AJAX > responses (application/json). But up-front then the up-front 1.5MB JS file > won't be compressed and clients may have to wait longer than needed. > > After reading the thread above I changed the Etag to be a weak one, but it > still gets removed. Do I need to do anything else? > > Summary: > > Why does gzipping interferes with etaging? > > > Best regards, > Manuel. > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From manuel at merchise.org Mon Oct 6 16:15:18 2014 From: manuel at merchise.org (=?UTF-8?B?TWFudWVsIFbDoXpxdWV6?=) Date: Mon, 06 Oct 2014 12:15:18 -0400 Subject: On ETag removal by the gzip module. In-Reply-To: References: <5432A85C.9050509@merchise.org> Message-ID: <5432C016.9050408@merchise.org> Le 06/10/14 11:56, Albert Casademont Filella a ?crit : > > The weak etag preservation was added to nginx 1.7.3 so I guess your > problem is that you're still on the 1.6.* branch! Hope that helps :) Actually 1.4.6... $ aptitude show nginx-light Paquet : nginx-light ?tat: install? Automatiquement install?: non Version : 1.4.6-1ubuntu3.1 I'll upgrade I think... On the other matter, I'm rereading the relevant parts of the RFC to see if the Etag vs Gzipping issue could be resolved... It's clear that whenever the transmitted data changes, etag must change: A "strong entity tag" MAY be shared by two entities of a resource only if they are equivalent by octet equality. But also an identity-preserving Content-Enconding says just that: The Content-Encoding entity-header field is used as a modifier to the media-type. When present, its value indicates what additional content codings have been applied to the entity-body, and thus what decoding mechanisms must be applied in order to obtain the media-type referenced by the Content-Type header field. Content-Encoding is primarily used to allow a document to be compressed *without losing** ** the identity of its underlying media type*. So I think there's room for improvement. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yatiohi at ideopolis.gr Tue Oct 7 10:22:56 2014 From: yatiohi at ideopolis.gr (Christos Trochalakis) Date: Tue, 7 Oct 2014 13:22:56 +0300 Subject: PHP fastcgi sninppet for Debian jessie Message-ID: <20141007102256.GA12894@luke.ws.skroutz.gr> Hello all, We are thinking about shipping a php-fastcgi snippet with the upcoming jessie debian stable release. I wanted to bring that to your attention to avoid shipping a broken config file that will be difficult to revert. the snippet: http://anonscm.debian.org/cgit/collab-maint/nginx.git/tree/debian/conf/snippets/fastcgi-php.conf?h=php-fastcgi&id=87f23062 the default site config that references it: http://anonscm.debian.org/cgit/collab-maint/nginx.git/tree/debian/conf/sites-available/default?h=php-fastcgi&id=87f23062 Any comments are wellcome! Thank you, chris From peter at lekensteyn.nl Tue Oct 7 11:33:26 2014 From: peter at lekensteyn.nl (Peter Wu) Date: Tue, 07 Oct 2014 13:33:26 +0200 Subject: PHP fastcgi sninppet for Debian jessie In-Reply-To: <20141007102256.GA12894@luke.ws.skroutz.gr> References: <20141007102256.GA12894@luke.ws.skroutz.gr> Message-ID: <2223073.mHXbGrTN87@al> On Tuesday 07 October 2014 13:22:56 Christos Trochalakis wrote: > Hello all, > > We are thinking about shipping a php-fastcgi snippet with the upcoming jessie > debian stable release. I wanted to bring that to your attention to avoid > shipping a broken config file that will be difficult to revert. > > the snippet: > http://anonscm.debian.org/cgit/collab-maint/nginx.git/tree/debian/conf/snippets/fastcgi-php.conf?h=php-fastcgi&id=87f23062 > > the default site config that references it: > http://anonscm.debian.org/cgit/collab-maint/nginx.git/tree/debian/conf/sites-available/default?h=php-fastcgi&id=87f23062 > > Any comments are welcome! It looks functional (though the PATH_INFO config looks like a workaround for a bug, couldn't that bug/feature be fixed instead?). Why do you have a separate fastcgi.conf while fastcgi_params already exists? Actually, it seems that Igor Sysoev added this file in December 2009. Igor, why is the file duplicated if the only difference is in SCRIPT_FILENAME? Christos, the configuration example allows for execution of xyz.php files in the document root. Another case is the use of frameworks which have a single controller, such as FuelPHP and Laravel. In such cases, this nginx configuration is sufficient (using the old config): location ~ ^/($|api/|user/|...) { try_files $uri @php_router; } location @php_router { include fastcgi_params; fastcgi_param SCRIPT_FILENAME /path/to/php/controller.php; fastcgi_pass unix:/var/run/php5-fpm.app.sock; } If you do not set fastcgi_param in the location section, then you can also mention this in the server section (or the http section if location does not set fastcgi_param either). (fastcgi_param is *not* inherited but overidden by new directives in a lower block). -- Kind regards, Peter https://lekensteyn.nl From francis at daoine.org Tue Oct 7 13:31:29 2014 From: francis at daoine.org (Francis Daly) Date: Tue, 7 Oct 2014 14:31:29 +0100 Subject: PHP fastcgi sninppet for Debian jessie In-Reply-To: <20141007102256.GA12894@luke.ws.skroutz.gr> Message-ID: <20141007133129.GY3771@daoine.org> On Tue, Oct 07, 2014 at 12:00:00PM +0000, nginx-devel-request at nginx.org wrote: Hi there, > We are thinking about shipping a php-fastcgi snippet with the upcoming jessie > debian stable release. I wanted to bring that to your attention to avoid > shipping a broken config file that will be difficult to revert. I think that any snippet will (potentially) be "right" for certain circumstances, and will (almost certainly) be "wrong" for other circumstances. The hard part is (usually) defining what the one use case that you are designing for is. > the snippet: > http://anonscm.debian.org/cgit/collab-maint/nginx.git/tree/debian/conf/snippets/fastcgi-php.conf?h=php-fastcgi&id=87f23062 fastcgi_split_path_info ^(.+\.php)(/.+)$; is probably pointless if you have already matched "location ~ \.php$", as your example config indicates. try_files $fastcgi_script_name =404; seems unnecessary to me, unless your fastcgi server is configured to keep guessing what you might have meant, instead of just doing what you said and failing when you said something wrong. (I do not know how the jessie-default php fastcgi server is configured by default.) And it is almost certainly broken if your fastcgi server is on a separate machine. With either of those lines removed, then the set $path_info $fastcgi_path_info; fastcgi_param PATH_INFO $path_info; dance is unnecessary. I think that fastcgi_index index.php; is probably also pointless within "location ~ \.php$". include fastcgi.conf; is good; but the user should be aware how their fastcgi server handles repeated fastcgi_param values, if they are going to add their own that might clash with anything in that file -- if the fastcgi server only processes the first value, then they must add their own before this include; if it processes the last value, then they must add their own after this include. > the default site config that references it: > http://anonscm.debian.org/cgit/collab-maint/nginx.git/tree/debian/conf/sites-available/default?h=php-fastcgi&id=87f23062 listen 80 default_server; listen [::]:80 default_server ipv6only=on; Using both lines may or may not be necessary depending on the configuration of the kernel involved. And they are possibly the no-configuration default. The "root" and "index" lines are probably also the compile-time defaults -- useful as an example "change this if you need to" indication, if the user does not want to refer to the documentation directly. The "server_name" directive does not do what the preceding comment says it does. (Unless I'm missing something.) Everything within the "location /" block is the default action anyway (except that this version, I think, hides things from log files). Probably simpler to erase them. (And maybe to remove the block entirely, but there are likely reasons not to do that.) The rest is commented fragments. Does /usr/share/nginx/html/50x.html exist on the default jessie? It is /usr/share/nginx/www/50x.html on a wheezy machine I have here. > Any comments are wellcome! I hope this helps. You can go a long way with a mostly-empty nginx.conf. Note, though, that while I read this list, I'm not a nginx developer. Cheers, f -- Francis Daly francis at daoine.org From bondarev at amazon.com Tue Oct 7 17:23:57 2014 From: bondarev at amazon.com (Bondarev, Daniil) Date: Tue, 7 Oct 2014 17:23:57 +0000 Subject: Prevent buffer overrun on NGX_HTTP_REQUEST_HEADER_TOO_LARGE Message-ID: <665DDF9CF644F74F91FFF1C33140FB5424F32C6C@ex10-mbx-36009.ant.amazon.com> Hi, There is an issue in nginx, when it's returning NGX_HTTP_REQUEST_HEADER_TOO_LARGE in ngx_http_process_request_headers: When large header buffer is full and the last header size is 1 or 2 bytes more than NGX_MAX_ERROR_STR - 300, nginx will write 1 or 2 '.' symbols out of large header buffer, causing unpredictable behavior. To reproduce it you can send request with total headers size of 'large client buffers size' and the last header size of 1749. Valgrind will catch this issue: ==10776== Invalid write of size 1 ==10776== at 0x426E84: ngx_http_process_request_headers (ngx_http_request.c:1230) ... The following patch fixes this issue: # HG changeset patch # User Daniil Bondarev # Date 1412401143 25200 # Node ID 2bbb5284ca7ff658ad50254fe0c5bec14247ba75 # Parent 6bbad2e732458bf53771e80c63a654b3d7f61963 Prevent buffer overrun on NGX_HTTP_REQUEST_HEADER_TOO_LARGE When large header buffer is full and the last header size is 1 or 2 bytes more than NGX_MAX_ERROR_STR - 300, nginx will write 1 or 2 '.' symbols out of large header buffer, causing unpredictable behavior. The fix is, instead of modifying a buffer, just cut the header and print '...' in log line if header is too large. diff -r 6bbad2e73245 -r 2bbb5284ca7f src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c Wed Aug 27 20:51:01 2014 +0400 +++ b/src/http/ngx_http_request.c Fri Oct 03 22:39:03 2014 -0700 @@ -1171,6 +1171,7 @@ { u_char *p; size_t len; + size_t print_len; ssize_t n; ngx_int_t rc, rv; ngx_table_elt_t *h; @@ -1225,14 +1226,13 @@ len = r->header_in->end - p; - if (len > NGX_MAX_ERROR_STR - 300) { - len = NGX_MAX_ERROR_STR - 300; - p[len++] = '.'; p[len++] = '.'; p[len++] = '.'; - } + /* Since log line size is limited to NGX_MAX_ERROR_STR, + * nginx has to limit header size it will print into log. */ + print_len = ngx_min(len, NGX_MAX_ERROR_STR - 300); ngx_log_error(NGX_LOG_INFO, c->log, 0, - "client sent too long header line: \"%*s\"", - len, r->header_name_start); + "client sent too long header line: \"%*s%s\"", + print_len, p, len != print_len ? "..." : ""); From mdounin at mdounin.ru Tue Oct 7 17:54:46 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 7 Oct 2014 21:54:46 +0400 Subject: Prevent buffer overrun on NGX_HTTP_REQUEST_HEADER_TOO_LARGE In-Reply-To: <665DDF9CF644F74F91FFF1C33140FB5424F32C6C@ex10-mbx-36009.ant.amazon.com> References: <665DDF9CF644F74F91FFF1C33140FB5424F32C6C@ex10-mbx-36009.ant.amazon.com> Message-ID: <20141007175446.GU69200@mdounin.ru> Hello! On Tue, Oct 07, 2014 at 05:23:57PM +0000, Bondarev, Daniil wrote: > Hi, > > There is an issue in nginx, when it's returning NGX_HTTP_REQUEST_HEADER_TOO_LARGE > in ngx_http_process_request_headers: > When large header buffer is full and the last header size is 1 or 2 bytes more > than NGX_MAX_ERROR_STR - 300, nginx will write 1 or 2 '.' symbols out of large > header buffer, causing unpredictable behavior. > > To reproduce it you can send request with total headers size of 'large client buffers size' and > the last header size of 1749. Valgrind will catch this issue: > > ==10776== Invalid write of size 1 > ==10776== at 0x426E84: ngx_http_process_request_headers (ngx_http_request.c:1230) > ... > > The following patch fixes this issue: > > # HG changeset patch > # User Daniil Bondarev > # Date 1412401143 25200 > # Node ID 2bbb5284ca7ff658ad50254fe0c5bec14247ba75 > # Parent 6bbad2e732458bf53771e80c63a654b3d7f61963 > Prevent buffer overrun on NGX_HTTP_REQUEST_HEADER_TOO_LARGE > > When large header buffer is full and the last header size is 1 or 2 bytes more > than NGX_MAX_ERROR_STR - 300, nginx will write 1 or 2 '.' symbols out of large > header buffer, causing unpredictable behavior. > > The fix is, instead of modifying a buffer, just cut the header and print '...' > in log line if header is too large. > > diff -r 6bbad2e73245 -r 2bbb5284ca7f src/http/ngx_http_request.c > --- a/src/http/ngx_http_request.c Wed Aug 27 20:51:01 2014 +0400 > +++ b/src/http/ngx_http_request.c Fri Oct 03 22:39:03 2014 -0700 > @@ -1171,6 +1171,7 @@ > { > u_char *p; > size_t len; > + size_t print_len; > ssize_t n; > ngx_int_t rc, rv; > ngx_table_elt_t *h; > @@ -1225,14 +1226,13 @@ > > len = r->header_in->end - p; > > - if (len > NGX_MAX_ERROR_STR - 300) { > - len = NGX_MAX_ERROR_STR - 300; > - p[len++] = '.'; p[len++] = '.'; p[len++] = '.'; > - } > + /* Since log line size is limited to NGX_MAX_ERROR_STR, > + * nginx has to limit header size it will print into log. */ > + print_len = ngx_min(len, NGX_MAX_ERROR_STR - 300); > > ngx_log_error(NGX_LOG_INFO, c->log, 0, > - "client sent too long header line: \"%*s\"", > - len, r->header_name_start); > + "client sent too long header line: \"%*s%s\"", > + print_len, p, len != print_len ? "..." : ""); Thanks for the report. What do you think about something as simple as: --- a/src/http/ngx_http_request.c +++ b/src/http/ngx_http_request.c @@ -1226,7 +1226,7 @@ ngx_http_process_request_headers(ngx_eve len = r->header_in->end - p; if (len > NGX_MAX_ERROR_STR - 300) { - len = NGX_MAX_ERROR_STR - 300; + len = NGX_MAX_ERROR_STR - 300 - 3; p[len++] = '.'; p[len++] = '.'; p[len++] = '.'; } ? -- Maxim Dounin http://nginx.org/ From bondarev at amazon.com Tue Oct 7 18:10:04 2014 From: bondarev at amazon.com (Bondarev, Daniil) Date: Tue, 7 Oct 2014 18:10:04 +0000 Subject: Prevent buffer overrun on NGX_HTTP_REQUEST_HEADER_TOO_LARGE In-Reply-To: <20141007175446.GU69200@mdounin.ru> References: <665DDF9CF644F74F91FFF1C33140FB5424F32C6C@ex10-mbx-36009.ant.amazon.com>, <20141007175446.GU69200@mdounin.ru> Message-ID: <665DDF9CF644F74F91FFF1C33140FB5424F32DAC@ex10-mbx-36009.ant.amazon.com> Yep, I've thought about this, but prefer not to modify buffer at all, since it feels more error-prone. F.E: SB might decide to change number of dots, or reuse last header from this buffer, etc. Do you feel strongly against printing "..." just at log line? ________________________________________ From: nginx-devel-bounces at nginx.org [nginx-devel-bounces at nginx.org] on behalf of Maxim Dounin [mdounin at mdounin.ru] Sent: Tuesday, October 07, 2014 10:54 AM To: nginx-devel at nginx.org Subject: Re: Prevent buffer overrun on NGX_HTTP_REQUEST_HEADER_TOO_LARGE Hello! On Tue, Oct 07, 2014 at 05:23:57PM +0000, Bondarev, Daniil wrote: > Hi, > > There is an issue in nginx, when it's returning NGX_HTTP_REQUEST_HEADER_TOO_LARGE > in ngx_http_process_request_headers: > When large header buffer is full and the last header size is 1 or 2 bytes more > than NGX_MAX_ERROR_STR - 300, nginx will write 1 or 2 '.' symbols out of large > header buffer, causing unpredictable behavior. > > To reproduce it you can send request with total headers size of 'large client buffers size' and > the last header size of 1749. Valgrind will catch this issue: > > ==10776== Invalid write of size 1 > ==10776== at 0x426E84: ngx_http_process_request_headers (ngx_http_request.c:1230) > ... > > The following patch fixes this issue: > > # HG changeset patch > # User Daniil Bondarev > # Date 1412401143 25200 > # Node ID 2bbb5284ca7ff658ad50254fe0c5bec14247ba75 > # Parent 6bbad2e732458bf53771e80c63a654b3d7f61963 > Prevent buffer overrun on NGX_HTTP_REQUEST_HEADER_TOO_LARGE > > When large header buffer is full and the last header size is 1 or 2 bytes more > than NGX_MAX_ERROR_STR - 300, nginx will write 1 or 2 '.' symbols out of large > header buffer, causing unpredictable behavior. > > The fix is, instead of modifying a buffer, just cut the header and print '...' > in log line if header is too large. > > diff -r 6bbad2e73245 -r 2bbb5284ca7f src/http/ngx_http_request.c > --- a/src/http/ngx_http_request.c Wed Aug 27 20:51:01 2014 +0400 > +++ b/src/http/ngx_http_request.c Fri Oct 03 22:39:03 2014 -0700 > @@ -1171,6 +1171,7 @@ > { > u_char *p; > size_t len; > + size_t print_len; > ssize_t n; > ngx_int_t rc, rv; > ngx_table_elt_t *h; > @@ -1225,14 +1226,13 @@ > > len = r->header_in->end - p; > > - if (len > NGX_MAX_ERROR_STR - 300) { > - len = NGX_MAX_ERROR_STR - 300; > - p[len++] = '.'; p[len++] = '.'; p[len++] = '.'; > - } > + /* Since log line size is limited to NGX_MAX_ERROR_STR, > + * nginx has to limit header size it will print into log. */ > + print_len = ngx_min(len, NGX_MAX_ERROR_STR - 300); > > ngx_log_error(NGX_LOG_INFO, c->log, 0, > - "client sent too long header line: \"%*s\"", > - len, r->header_name_start); > + "client sent too long header line: \"%*s%s\"", > + print_len, p, len != print_len ? "..." : ""); Thanks for the report. What do you think about something as simple as: --- a/src/http/ngx_http_request.c +++ b/src/http/ngx_http_request.c @@ -1226,7 +1226,7 @@ ngx_http_process_request_headers(ngx_eve len = r->header_in->end - p; if (len > NGX_MAX_ERROR_STR - 300) { - len = NGX_MAX_ERROR_STR - 300; + len = NGX_MAX_ERROR_STR - 300 - 3; p[len++] = '.'; p[len++] = '.'; p[len++] = '.'; } ? -- Maxim Dounin http://nginx.org/ _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Tue Oct 7 18:57:44 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 7 Oct 2014 22:57:44 +0400 Subject: Prevent buffer overrun on NGX_HTTP_REQUEST_HEADER_TOO_LARGE In-Reply-To: <665DDF9CF644F74F91FFF1C33140FB5424F32DAC@ex10-mbx-36009.ant.amazon.com> References: <665DDF9CF644F74F91FFF1C33140FB5424F32C6C@ex10-mbx-36009.ant.amazon.com> <20141007175446.GU69200@mdounin.ru> <665DDF9CF644F74F91FFF1C33140FB5424F32DAC@ex10-mbx-36009.ant.amazon.com> Message-ID: <20141007185744.GV69200@mdounin.ru> Hello! On Tue, Oct 07, 2014 at 06:10:04PM +0000, Bondarev, Daniil wrote: > Yep, I've thought about this, but prefer not to modify buffer at all, since it feels more error-prone. > F.E: SB might decide to change number of dots, or reuse last header from this buffer, etc. Changing the number of dots is highly unlikely and it will be hard to do it incorrectly, as the "3" in the patch directly corresponds to the number. Reuse of the header is highly unlikely too, as it is the fatal error and the header is known to be incomplete. > Do you feel strongly against printing "..." just at log line? Resulting code is way longer than it should be, so I would rather prefer simplier variant. On the other hand, looking into this more closely, I tend to think that ellipsis should be always added to make it clear that the header logged is incomplete. Here is a patch: --- a/src/http/ngx_http_request.c +++ b/src/http/ngx_http_request.c @@ -1227,12 +1227,11 @@ ngx_http_process_request_headers(ngx_eve if (len > NGX_MAX_ERROR_STR - 300) { len = NGX_MAX_ERROR_STR - 300; - p[len++] = '.'; p[len++] = '.'; p[len++] = '.'; } ngx_log_error(NGX_LOG_INFO, c->log, 0, - "client sent too long header line: \"%*s\"", - len, r->header_name_start); + "client sent too long header line: \"%*s...\"", + len, r->header_name_start); ngx_http_finalize_request(r, NGX_HTTP_REQUEST_HEADER_TOO_LARGE); -- Maxim Dounin http://nginx.org/ From bondarev at amazon.com Tue Oct 7 19:11:24 2014 From: bondarev at amazon.com (Bondarev, Daniil) Date: Tue, 7 Oct 2014 19:11:24 +0000 Subject: Prevent buffer overrun on NGX_HTTP_REQUEST_HEADER_TOO_LARGE In-Reply-To: <20141007185744.GV69200@mdounin.ru> References: <665DDF9CF644F74F91FFF1C33140FB5424F32C6C@ex10-mbx-36009.ant.amazon.com> <20141007175446.GU69200@mdounin.ru> <665DDF9CF644F74F91FFF1C33140FB5424F32DAC@ex10-mbx-36009.ant.amazon.com>, <20141007185744.GV69200@mdounin.ru> Message-ID: <665DDF9CF644F74F91FFF1C33140FB5424F32FB1@ex10-mbx-36009.ant.amazon.com> Hey Maxim, > On the other hand, looking into this more closely, I tend to think > that ellipsis should be always added to make it clear that the > header logged is incomplete. Agree, good point! Patch looks good to me, only note - you can reduce amount of lines by ngx_min, if you wish: diff -r 6bbad2e73245 src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c Wed Aug 27 20:51:01 2014 +0400 +++ b/src/http/ngx_http_request.c Tue Oct 07 12:06:36 2014 -0700 @@ -1223,15 +1223,11 @@ return; } - len = r->header_in->end - p; - - if (len > NGX_MAX_ERROR_STR - 300) { - len = NGX_MAX_ERROR_STR - 300; - p[len++] = '.'; p[len++] = '.'; p[len++] = '.'; - } + len = ngx_min(r->header_in->end - p, + NGX_MAX_ERROR_STR - 300); ngx_log_error(NGX_LOG_INFO, c->log, 0, - "client sent too long header line: \"%*s\"", + "client sent too long header line: \"%*s...\"", len, r->header_name_start); ngx_http_finalize_request(r, Thanks! Daniil From yingqi.lu at intel.com Tue Oct 7 19:32:08 2014 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Tue, 7 Oct 2014 19:32:08 +0000 Subject: [Patch] SO_REUSEPORT support from master process In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AA37A52F5@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AA37A52F5@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA37D0C4E@ORSMSX113.amr.corp.intel.com> Dear All, It has been quiet for a while on this patch. I am checking to see if there is any questions/feedbacks/concerns we need to address? Please let me know. Thanks very much for your help! Yingqi -----Original Message----- From: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Lu, Yingqi Sent: Wednesday, August 27, 2014 10:33 AM To: nginx-devel at nginx.org Subject: RE: [Patch] SO_REUSEPORT support from master process Dear All, I am resending this patch with plain text instead of HTML format. I will also post the patch at the end of this email. Hope this will be easier for all of you to review. Please let me know if you have trouble viewing the message or the patch itself. This is our first time submitting the patch here. Your feedback and suggestions are highly appreciated. The "SO_REUSEPORT support for listen sockets support" patches submitted by Sepherosa Ziehau are posted and discussed in [1] and [2]. Last update on the threads was 09/05/2013 and the patch is not included in the current Nginx code. Reading from the discussion, my understanding is that his patch makes a dedicated listen socket for each of the child process. In order to make sure at any given time there is always a listen socket available, the patch makes the first worker process different/special than the rest. Here, I am proposing a simpler way to enable the SO_REUSEPORT support. It is just to create and configure certain number of listen sockets in the master process with SO_REUSEPORT enabled. All the children processes can inherit. In this case, we do not need to worry about ensuring 1 available listen socket at the run time. The number of the listen sockets to be created is calculated based on the number of active CPU threads. With big system that has more CPU threads (where we have the scalability issue), there are more duplicated listen sockets created to improve the throughput and scalability. With system that has only 8 or less CPU threads, there will be only 1 listen socket. This makes sure duplicated listen sockets only being created when necessary. In case that SO_REUSEPORT is not supported by the OS, it will fall back to the default/original behavior (this is tested on Linux kernel 3.8.8 where SO_REUSEPORT is not supported). This prototype patch has been tested on an Intel modern dual socket platform with a three tier open source web server workload (PHP+Nginx/memcached/MySQL). The web server has 2 IP network interfaces configured for testing. The Linux kernel used for testing is 3.13.9. Data show: Case 1: with single listen statement (Listen 80) specified in the configuration file, there is 46.3% throughout increase. Case 2: with dual listen statements (for example, Listen 192.168.1.1:80 and Listen 192.168.1.2:80), there is 10% throughput increase. Both testing cases keep everything the same except the patch itself to get above result. The reason that Case1 has bigger performance gains is that Case1 by default only has 1 listen socket while Case2 by default already has 2. Please review it and let me know your questions and comments. Thanks very much for your time reviewing the patch. Thanks, Yingqi Lu [1] http://forum.nginx.org/read.php?29,241283,241283 [2] http://forum.nginx.org/read.php?29,241470,241470 # HG changeset patch # User Yingqi Lu # Date 1408145210 25200 # Fri Aug 15 16:26:50 2014 -0700 # Node ID d9c7259d275dbcae8a0d001ee9703b13312b3263 # Parent 6edcb183e62d610808addebbd18249abb7224a0a These are the patch files for SO_REUSEPORT support. diff -r 6edcb183e62d -r d9c7259d275d ngx_connection.c --- a/ngx_connection.c Fri Aug 15 16:25:32 2014 -0700 +++ b/ngx_connection.c Fri Aug 15 16:26:50 2014 -0700 @@ -304,7 +304,7 @@ ngx_int_t ngx_open_listening_sockets(ngx_cycle_t *cycle) { - int reuseaddr; + int reuseaddr, reuseport; ngx_uint_t i, tries, failed; ngx_err_t err; ngx_log_t *log; @@ -312,6 +312,7 @@ ngx_listening_t *ls; reuseaddr = 1; + reuseport = 1; #if (NGX_SUPPRESS_WARN) failed = 0; #endif @@ -370,6 +371,24 @@ return NGX_ERROR; } + if (so_reuseport_enabled) + { + if (setsockopt(s, SOL_SOCKET, SO_REUSEPORT, + (const void *) &reuseport, sizeof(int)) + == -1) { + ngx_log_error(NGX_LOG_EMERG, log, ngx_socket_errno, + "setsockopt(SO_REUSEPORT) %V failed", + &ls[i].addr_text); + if (ngx_close_socket(s) == -1) { + ngx_log_error(NGX_LOG_EMERG, log, ngx_socket_errno, + ngx_close_socket_n " %V failed", + &ls[i].addr_text); + } + + return NGX_ERROR; + } + } + #if (NGX_HAVE_INET6 && defined IPV6_V6ONLY) if (ls[i].sockaddr->sa_family == AF_INET6) { diff -r 6edcb183e62d -r d9c7259d275d ngx_cycle.c --- a/ngx_cycle.c Fri Aug 15 16:25:32 2014 -0700 +++ b/ngx_cycle.c Fri Aug 15 16:26:50 2014 -0700 @@ -25,7 +25,7 @@ ngx_uint_t ngx_test_config; ngx_uint_t ngx_quiet_mode; - +ngx_uint_t so_reuseport_enabled; #if (NGX_THREADS) ngx_tls_key_t ngx_core_tls_key; #endif @@ -55,6 +55,34 @@ ngx_core_module_t *module; char hostname[NGX_MAXHOSTNAMELEN]; + ngx_uint_t j, num_cores, num_dup_sockets, orig_nelts; + ngx_socket_t temp_s; + int one = 1; + so_reuseport_enabled = 0; + temp_s = ngx_socket(AF_INET, SOCK_STREAM, 0); #ifndef SO_REUSEPORT +#define SO_REUSEPORT 15 #endif + if (setsockopt(temp_s, SOL_SOCKET, SO_REUSEPORT, + (const void *) &one, sizeof(int)) == 0) { + so_reuseport_enabled = 1; + } + ngx_close_socket(temp_s); + + if (so_reuseport_enabled) { +#ifdef _SC_NPROCESSORS_ONLN + num_cores = sysconf(_SC_NPROCESSORS_ONLN); #else + num_cores = 1; +#endif + if (num_cores > 8) { + num_dup_sockets = num_cores/8; + } else { + num_dup_sockets = 1; + } + } else { + num_dup_sockets = 1; + } ngx_timezone_update(); /* force localtime update with a new timezone */ @@ -114,7 +142,7 @@ } - n = old_cycle->paths.nelts ? old_cycle->paths.nelts : 10; + n = old_cycle->paths.nelts ? old_cycle->paths.nelts : 10 * + num_dup_sockets; cycle->paths.elts = ngx_pcalloc(pool, n * sizeof(ngx_path_t *)); if (cycle->paths.elts == NULL) { @@ -164,7 +192,7 @@ return NULL; } - n = old_cycle->listening.nelts ? old_cycle->listening.nelts : 10; + n = old_cycle->listening.nelts ? old_cycle->listening.nelts : 10 * + num_dup_sockets; cycle->listening.elts = ngx_pcalloc(pool, n * sizeof(ngx_listening_t)); if (cycle->listening.elts == NULL) { @@ -231,7 +259,7 @@ ngx_memzero(&conf, sizeof(ngx_conf_t)); /* STUB: init array ? */ - conf.args = ngx_array_create(pool, 10, sizeof(ngx_str_t)); + conf.args = ngx_array_create(pool, (10 * num_dup_sockets), + sizeof(ngx_str_t)); if (conf.args == NULL) { ngx_destroy_pool(pool); return NULL; @@ -575,7 +603,15 @@ #endif } } + orig_nelts = cycle->listening.nelts; + cycle->listening.nelts = cycle->listening.nelts * num_dup_sockets; + ls = cycle->listening.elts; + for (i = 0; i < num_dup_sockets; i++) { + for(j = 0; j < orig_nelts; j++) { + ls[j + i * orig_nelts] = ls[j]; + } + } if (ngx_open_listening_sockets(cycle) != NGX_OK) { goto failed; } @@ -747,7 +783,7 @@ exit(1); } - n = 10; + n = 10 * num_dup_sockets; ngx_old_cycles.elts = ngx_pcalloc(ngx_temp_pool, n * sizeof(ngx_cycle_t *)); if (ngx_old_cycles.elts == NULL) { diff -r 6edcb183e62d -r d9c7259d275d ngx_cycle.h --- a/ngx_cycle.h Fri Aug 15 16:25:32 2014 -0700 +++ b/ngx_cycle.h Fri Aug 15 16:26:50 2014 -0700 @@ -136,6 +136,7 @@ extern ngx_module_t ngx_core_module; extern ngx_uint_t ngx_test_config; extern ngx_uint_t ngx_quiet_mode; +extern ngx_uint_t so_reuseport_enabled; #if (NGX_THREADS) extern ngx_tls_key_t ngx_core_tls_key; #endif 1. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel From yatiohi at ideopolis.gr Wed Oct 8 08:09:10 2014 From: yatiohi at ideopolis.gr (Christos Trochalakis) Date: Wed, 8 Oct 2014 11:09:10 +0300 Subject: PHP fastcgi sninppet for Debian jessie In-Reply-To: <20141007133129.GY3771@daoine.org> References: <20141007102256.GA12894@luke.ws.skroutz.gr> <20141007133129.GY3771@daoine.org> Message-ID: <20141008080910.GA18868@luke.ws.skroutz.gr> On Tue, Oct 07, 2014 at 02:31:29PM +0100, Francis Daly wrote: >On Tue, Oct 07, 2014 at 12:00:00PM +0000, nginx-devel-request at nginx.org wrote: >> the snippet: >> http://anonscm.debian.org/cgit/collab-maint/nginx.git/tree/debian/conf/snippets/fastcgi-php.conf?h=php-fastcgi&id=87f23062 > > fastcgi_split_path_info ^(.+\.php)(/.+)$; > >is probably pointless if you have already matched "location ~ \.php$", >as your example config indicates. > > try_files $fastcgi_script_name =404; > >seems unnecessary to me, unless your fastcgi server is configured to keep >guessing what you might have meant, instead of just doing what you said >and failing when you said something wrong. (I do not know how the >jessie-default php fastcgi server is configured by default.) > >And it is almost certainly broken if your fastcgi server is on a separate >machine. > >With either of those lines removed, then the > > set $path_info $fastcgi_path_info; > fastcgi_param PATH_INFO $path_info; > >dance is unnecessary. > >I think that > > fastcgi_index index.php; > >is probably also pointless within "location ~ \.php$". > Yes, it's pointless on a "location ~ \.php$" block, but this snippet can be used in other locations as well, so I tried to be more generic. It might be used in a named @php location and used as the last argument to a try_files directive for example). > include fastcgi.conf; > >is good; but the user should be aware how their fastcgi server handles >repeated fastcgi_param values, if they are going to add their own that >might clash with anything in that file -- if the fastcgi server only >processes the first value, then they must add their own before this >include; if it processes the last value, then they must add their own >after this include. Good point, we should add a comment about that to be clear. > >> the default site config that references it: >> http://anonscm.debian.org/cgit/collab-maint/nginx.git/tree/debian/conf/sites-available/default?h=php-fastcgi&id=87f23062 > > listen 80 default_server; > listen [::]:80 default_server ipv6only=on; > >Using both lines may or may not be necessary depending on the >configuration of the kernel involved. And they are possibly the >no-configuration default. > >The "root" and "index" lines are probably also the compile-time defaults >-- useful as an example "change this if you need to" indication, if the >user does not want to refer to the documentation directly. > >The "server_name" directive does not do what the preceding comment says >it does. (Unless I'm missing something.) > >Everything within the "location /" block is the default action anyway >(except that this version, I think, hides things from log files). Probably >simpler to erase them. (And maybe to remove the block entirely, but >there are likely reasons not to do that.) > >The rest is commented fragments. Does /usr/share/nginx/html/50x.html >exist on the default jessie? It is /usr/share/nginx/www/50x.html on a >wheezy machine I have here. > Yes, the rest of the configuration file are relics from the past, and a bit confusing... We plan to remove most of them :) >> Any comments are wellcome! > >I hope this helps. > >You can go a long way with a mostly-empty nginx.conf. > >Note, though, that while I read this list, I'm not a nginx developer. Thanks a lot for your time and comments! From yatiohi at ideopolis.gr Wed Oct 8 08:20:27 2014 From: yatiohi at ideopolis.gr (Christos Trochalakis) Date: Wed, 8 Oct 2014 11:20:27 +0300 Subject: PHP fastcgi sninppet for Debian jessie In-Reply-To: <2223073.mHXbGrTN87@al> References: <20141007102256.GA12894@luke.ws.skroutz.gr> <2223073.mHXbGrTN87@al> Message-ID: <20141008082027.GB18868@luke.ws.skroutz.gr> Hello Peter, On Tue, Oct 07, 2014 at 01:33:26PM +0200, Peter Wu wrote: >On Tuesday 07 October 2014 13:22:56 Christos Trochalakis wrote: >> Hello all, >> >> We are thinking about shipping a php-fastcgi snippet with the upcoming jessie >> debian stable release. I wanted to bring that to your attention to avoid >> shipping a broken config file that will be difficult to revert. >> >> the snippet: >> http://anonscm.debian.org/cgit/collab-maint/nginx.git/tree/debian/conf/snippets/fastcgi-php.conf?h=php-fastcgi&id=87f23062 >> >> the default site config that references it: >> http://anonscm.debian.org/cgit/collab-maint/nginx.git/tree/debian/conf/sites-available/default?h=php-fastcgi&id=87f23062 >> >> Any comments are welcome! > >It looks functional (though the PATH_INFO config looks like a workaround for a >bug, couldn't that bug/feature be fixed instead?). > It's known and desired behaviour, as Maxim Dunin's says in http://trac.nginx.org/nginx/ticket/321. >Why do you have a separate fastcgi.conf while fastcgi_params already exists? >Actually, it seems that Igor Sysoev added this file in December 2009. Igor, why >is the file duplicated if the only difference is in SCRIPT_FILENAME? > This is a recent addition to the debian package, we wanted to be as close to upstream as possible. Less entropy is always good :) >Christos, the configuration example allows for execution of xyz.php files in the >document root. Another case is the use of frameworks which have a single >controller, such as FuelPHP and Laravel. In such cases, this nginx configuration >is sufficient (using the old config): > > location ~ ^/($|api/|user/|...) { > try_files $uri @php_router; > } > location @php_router { > include fastcgi_params; > fastcgi_param SCRIPT_FILENAME /path/to/php/controller.php; > fastcgi_pass unix:/var/run/php5-fpm.app.sock; > } > Yes, I wanted to catch the simplest setup of serving PHP files from your document root. Other, more complex, use cases will require a little bit of searching by the administrator. Thanks a lot for your comments :) From mdounin at mdounin.ru Wed Oct 8 12:58:48 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 8 Oct 2014 16:58:48 +0400 Subject: [Patch] SO_REUSEPORT support from master process In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AA37D0C4E@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AA37A52F5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AA37D0C4E@ORSMSX113.amr.corp.intel.com> Message-ID: <20141008125848.GA31276@mdounin.ru> Hello! On Tue, Oct 07, 2014 at 07:32:08PM +0000, Lu, Yingqi wrote: > Dear All, > > It has been quiet for a while on this patch. I am checking to > see if there is any questions/feedbacks/concerns we need to > address? > > Please let me know. Thanks very much for your help! Apart from style/coding issues, I disagree with the whole approach. As far as I understand the patch idea, it tries to introduce multiple listening sockets to avoid in-kernel lock contention. This is something that can be done completely in kernel though, and I see no reason to introduce any changes to nginx here. The approach previously discussed with Sepherosa Ziehau looks much more interesting. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Wed Oct 8 13:41:40 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 08 Oct 2014 13:41:40 +0000 Subject: [nginx] Upstream: fix $upstream_cache_last_modified variable. Message-ID: details: http://hg.nginx.org/nginx/rev/fd283aa92e04 branches: changeset: 5869:fd283aa92e04 user: Piotr Sikora date: Wed Oct 01 15:48:53 2014 -0700 description: Upstream: fix $upstream_cache_last_modified variable. Due to the u->headers_in.last_modified_time not being correctly initialized, this variable was evaluated to "Thu, 01 Jan 1970 00:00:00 GMT" for responses cached without the "Last-Modified" header which resulted in subsequent proxy requests being sent with "If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT" header. Signed-off-by: Piotr Sikora diffstat: src/http/ngx_http_upstream.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diffs (27 lines): diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -441,6 +441,7 @@ ngx_http_upstream_create(ngx_http_reques #endif u->headers_in.content_length_n = -1; + u->headers_in.last_modified_time = -1; return NGX_OK; } @@ -885,6 +886,7 @@ ngx_http_upstream_cache_send(ngx_http_re ngx_memzero(&u->headers_in, sizeof(ngx_http_upstream_headers_in_t)); u->headers_in.content_length_n = -1; + u->headers_in.last_modified_time = -1; if (ngx_list_init(&u->headers_in.headers, r->pool, 8, sizeof(ngx_table_elt_t)) @@ -1582,6 +1584,7 @@ ngx_http_upstream_reinit(ngx_http_reques ngx_memzero(&u->headers_in, sizeof(ngx_http_upstream_headers_in_t)); u->headers_in.content_length_n = -1; + u->headers_in.last_modified_time = -1; if (ngx_list_init(&u->headers_in.headers, r->pool, 8, sizeof(ngx_table_elt_t)) From mdounin at mdounin.ru Wed Oct 8 13:41:44 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 08 Oct 2014 13:41:44 +0000 Subject: [nginx] Core: fixed buffer overrun when hash max_size reached. Message-ID: details: http://hg.nginx.org/nginx/rev/5e72578e6503 branches: changeset: 5870:5e72578e6503 user: Yichun Zhang date: Thu Oct 02 12:00:17 2014 -0700 description: Core: fixed buffer overrun when hash max_size reached. diffstat: src/core/ngx_hash.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff --git a/src/core/ngx_hash.c b/src/core/ngx_hash.c --- a/src/core/ngx_hash.c +++ b/src/core/ngx_hash.c @@ -312,6 +312,8 @@ ngx_hash_init(ngx_hash_init_t *hinit, ng continue; } + size--; + ngx_log_error(NGX_LOG_WARN, hinit->pool->log, 0, "could not build optimal %s, you should increase " "either %s_max_size: %i or %s_bucket_size: %i; " From mdounin at mdounin.ru Wed Oct 8 13:41:46 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 08 Oct 2014 13:41:46 +0000 Subject: [nginx] Fixed possible buffer overrun in "too long header line" ... Message-ID: details: http://hg.nginx.org/nginx/rev/21043ce2a005 branches: changeset: 5871:21043ce2a005 user: Maxim Dounin date: Wed Oct 08 17:16:04 2014 +0400 description: Fixed possible buffer overrun in "too long header line" logging. Additionally, ellipsis now always added to make it clear that the header logged is incomplete. Reported by Daniil Bondarev. diffstat: src/http/ngx_http_request.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diffs (18 lines): diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c +++ b/src/http/ngx_http_request.c @@ -1227,12 +1227,11 @@ ngx_http_process_request_headers(ngx_eve if (len > NGX_MAX_ERROR_STR - 300) { len = NGX_MAX_ERROR_STR - 300; - p[len++] = '.'; p[len++] = '.'; p[len++] = '.'; } ngx_log_error(NGX_LOG_INFO, c->log, 0, - "client sent too long header line: \"%*s\"", - len, r->header_name_start); + "client sent too long header line: \"%*s...\"", + len, r->header_name_start); ngx_http_finalize_request(r, NGX_HTTP_REQUEST_HEADER_TOO_LARGE); From mdounin at mdounin.ru Wed Oct 8 13:41:53 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 8 Oct 2014 17:41:53 +0400 Subject: [PATCH] Upstream: fix $upstream_cache_last_modified variable In-Reply-To: References: Message-ID: <20141008134153.GB31276@mdounin.ru> Hello! On Wed, Oct 01, 2014 at 05:20:06PM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1412203733 25200 > # Wed Oct 01 15:48:53 2014 -0700 > # Node ID dbf6364ab44fcc6b5d1fbbe64c162135b18e9209 > # Parent a215d9021f137b9e2d4f69c37c7f992a2bef12c6 > Upstream: fix $upstream_cache_last_modified variable. > > Due to the u->headers_in.last_modified_time not being correctly initialized, > this variable was evaluated to "Thu, 01 Jan 1970 00:00:00 GMT" for responses > cached without the "Last-Modified" header which resulted in subsequent proxy > requests being sent with "If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT" > header. Committed, thanks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Wed Oct 8 13:42:39 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 8 Oct 2014 17:42:39 +0400 Subject: [PATCH] Fix a buffer overflow issue in ngx_hash_t when exceeding limits In-Reply-To: References: Message-ID: <20141008134239.GC31276@mdounin.ru> Hello! On Thu, Oct 02, 2014 at 12:32:57PM -0700, Yichun Zhang (agentzh) wrote: > # HG changeset patch > # User Yichun Zhang > # Date 1412276417 25200 > # Thu Oct 02 12:00:17 2014 -0700 > # Node ID 4032b992f23b054c1a2cfb0be879330d2c6708e5 > # Parent 1ff0f68d9376e3d184d65814a6372856bf65cfcd > Hash: buffer overflow might happen when exceeding the pre-configured limits. Committed with the commit message modified to match style we use, thanks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Wed Oct 8 13:45:17 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 8 Oct 2014 17:45:17 +0400 Subject: Prevent buffer overrun on NGX_HTTP_REQUEST_HEADER_TOO_LARGE In-Reply-To: <665DDF9CF644F74F91FFF1C33140FB5424F32FB1@ex10-mbx-36009.ant.amazon.com> References: <665DDF9CF644F74F91FFF1C33140FB5424F32C6C@ex10-mbx-36009.ant.amazon.com> <20141007175446.GU69200@mdounin.ru> <665DDF9CF644F74F91FFF1C33140FB5424F32DAC@ex10-mbx-36009.ant.amazon.com> <20141007185744.GV69200@mdounin.ru> <665DDF9CF644F74F91FFF1C33140FB5424F32FB1@ex10-mbx-36009.ant.amazon.com> Message-ID: <20141008134517.GD31276@mdounin.ru> Hello! On Tue, Oct 07, 2014 at 07:11:24PM +0000, Bondarev, Daniil wrote: > Hey Maxim, > > > On the other hand, looking into this more closely, I tend to think > > that ellipsis should be always added to make it clear that the > > header logged is incomplete. > > Agree, good point! > > Patch looks good to me, only note - you can reduce amount of lines by ngx_min, > if you wish: I think the code is more readable as is, without ngx_min(). The patch comitted, thanks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Wed Oct 8 14:24:41 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 8 Oct 2014 18:24:41 +0400 Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141004.195156.801414609668336877.kou@cozmixng.org> References: <20141004.195156.801414609668336877.kou@cozmixng.org> Message-ID: <20141008142441.GE31276@mdounin.ru> Hello! On Sat, Oct 04, 2014 at 07:51:56PM +0900, Kouhei Sutou wrote: > Hi, > > I'm trying cross compiling nginx for Windows on Debian > GNU/Linux. The current nginx doesn't support it. > I created a patch to do it. With the attached patch, I can > cross compile nginx for Windows on Debian GNU/Linux. > Please review the attached patch. In short: changes to ngx_wsasend_chain.c and ngx_iocp_module.c look fine, changes to ngx_win32_config.h look wrong. See below for more comments. [...] > diff --git a/src/event/modules/ngx_iocp_module.c b/src/event/modules/ngx_iocp_module.c > --- a/src/event/modules/ngx_iocp_module.c > +++ b/src/event/modules/ngx_iocp_module.c > @@ -242,17 +242,17 @@ ngx_int_t ngx_iocp_process_events(ngx_cy > ngx_event_ovlp_t *ovlp; > > if (timer == NGX_TIMER_INFINITE) { > timer = INFINITE; > } > > ngx_log_debug1(NGX_LOG_DEBUG_EVENT, cycle->log, 0, "iocp timer: %M", timer); > > - rc = GetQueuedCompletionStatus(iocp, &bytes, (LPDWORD) &key, > + rc = GetQueuedCompletionStatus(iocp, &bytes, (PULONG_PTR) &key, > (LPOVERLAPPED *) &ovlp, (u_long) timer); > > if (rc == 0) { > err = ngx_errno; > } else { > err = 0; > } > Looks fine, though I think it should be a separate patch. Please see http://nginx.org/en/docs/contributing_changes.html for hints how to make sure you'll name will be preserved in commit history. > diff --git a/src/os/win32/ngx_win32_config.h b/src/os/win32/ngx_win32_config.h > --- a/src/os/win32/ngx_win32_config.h > +++ b/src/os/win32/ngx_win32_config.h > @@ -33,18 +33,17 @@ > > #include > #include /* ipv6 */ > #include > #include > #include /* offsetof() */ > > #ifdef __GNUC__ > -/* GCC MinGW's stdio.h includes sys/types.h */ > -#define _OFF_T_ > +#define _FILE_OFFSET_BITS 64 > #endif > > #include > #include > #include > #include > #include > > @@ -131,25 +130,31 @@ typedef int int32_t; > typedef unsigned short int uint16_t; > #define ngx_libc_cdecl > > #endif > > typedef __int64 int64_t; > typedef unsigned __int64 uint64_t; > > -#ifndef __WATCOMC__ > +#ifdef __WATCOMC__ > +/* do nothing */ > +#elif defined(__GNUC__) > +#include > +#else > typedef int intptr_t; > typedef u_int uintptr_t; > #endif I think that ngx_win32_config.h part needs a bit more work. In particular, "#include " should be placed with other includes if it's needed. > +#ifndef __GNUC__ > /* Windows defines off_t as long, which is 32-bit */ > -typedef __int64 off_t; > -#define _OFF_T_DEFINED > +typedef __int64 off64_t; > +#define _OFF64_T_DEFINED > +#endif Defining off64_t looks very wrong for me. There is nothing in nginx which use it, and it shouldn't be defined. Additionally, this breaks at least compilation with MSVC. [...] > --- a/src/os/win32/ngx_wsasend_chain.c > +++ b/src/os/win32/ngx_wsasend_chain.c > @@ -29,17 +29,17 @@ ngx_wsasend_chain(ngx_connection_t *c, n > wev = c->write; > > if (!wev->ready) { > return in; > } > > /* the maximum limit size is the maximum u_long value - the page size */ > > - if (limit == 0 || limit > NGX_MAX_UINT32_VALUE - ngx_pagesize) { > + if (limit == 0 || limit > (off_t) (NGX_MAX_UINT32_VALUE - ngx_pagesize)) { > limit = NGX_MAX_UINT32_VALUE - ngx_pagesize; > } > > send = 0; > > /* > * WSABUFs must be 4-byte aligned otherwise > * WSASend() will return undocumented WSAEINVAL error. > @@ -151,17 +151,17 @@ ngx_overlapped_wsasend_chain(ngx_connect > "wev->complete: %d", wev->complete); > > if (!wev->complete) { > > /* post the overlapped WSASend() */ > > /* the maximum limit size is the maximum u_long value - the page size */ > > - if (limit == 0 || limit > NGX_MAX_UINT32_VALUE - ngx_pagesize) { > + if (limit == 0 || limit > (off_t) (NGX_MAX_UINT32_VALUE - ngx_pagesize)) { > limit = NGX_MAX_UINT32_VALUE - ngx_pagesize; > } > > /* > * WSABUFs must be 4-byte aligned otherwise > * WSASend() will return undocumented WSAEINVAL error. > */ > Looks fine, though should be a separate patch. See above. -- Maxim Dounin http://nginx.org/ From yingqi.lu at intel.com Wed Oct 8 17:05:19 2014 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Wed, 8 Oct 2014 17:05:19 +0000 Subject: [Patch] SO_REUSEPORT support from master process In-Reply-To: <20141008125848.GA31276@mdounin.ru> References: <9ACD5B67AAC5594CB6268234CF29CF9AA37A52F5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AA37D0C4E@ORSMSX113.amr.corp.intel.com> <20141008125848.GA31276@mdounin.ru> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA37D1997@ORSMSX113.amr.corp.intel.com> Hi Maxim, Thanks for letting us know. Our updated patch is located at http://forum.nginx.org/read.php?29,253446,253446#msg-253446 It supposes to address all the style issues and fixes the restart and binary upgrade issues. This is just a FYI in case you are not aware of. Thanks, Yingqi -----Original Message----- From: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Maxim Dounin Sent: Wednesday, October 08, 2014 5:59 AM To: nginx-devel at nginx.org Subject: Re: [Patch] SO_REUSEPORT support from master process Hello! On Tue, Oct 07, 2014 at 07:32:08PM +0000, Lu, Yingqi wrote: > Dear All, > > It has been quiet for a while on this patch. I am checking to see if > there is any questions/feedbacks/concerns we need to address? > > Please let me know. Thanks very much for your help! Apart from style/coding issues, I disagree with the whole approach. As far as I understand the patch idea, it tries to introduce multiple listening sockets to avoid in-kernel lock contention. This is something that can be done completely in kernel though, and I see no reason to introduce any changes to nginx here. The approach previously discussed with Sepherosa Ziehau looks much more interesting. -- Maxim Dounin http://nginx.org/ _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel From kpariani at zimbra.com Wed Oct 8 17:54:35 2014 From: kpariani at zimbra.com (Kunal Pariani) Date: Wed, 8 Oct 2014 12:54:35 -0500 (CDT) Subject: [PATCH] Update mail parsing to be per protocol (imap/pop/smtp) In-Reply-To: <1819431299.101798.1412790873785.JavaMail.zimbra@zimbra.com> References: <1273114156.2387485.1410986989995.JavaMail.zimbra@zimbra.com> <20140918110032.GO91749@mdounin.ru> <1644005643.2543169.1411061568638.JavaMail.zimbra@zimbra.com> <1349223799.3017631.1411600472282.JavaMail.zimbra@zimbra.com> Message-ID: <166026479.101799.1412790875955.JavaMail.zimbra@zimbra.com> Any responses on this please ? Thanks -Kunal ----- Original Message ----- From: "Kunal Pariani" To: "nginx-devel" Sent: Wednesday, September 24, 2014 4:14:32 PM Subject: Re: [PATCH] Update mail parsing to be per protocol (imap/pop/smtp) Some comments received on the patch from Filipe I can add that I think your patch is valuable. But half of the changes are not related to the main patch goals: - expected args count check - additional error messages. Just a quick review of your patch: - it is too long to be easily reviewed. - please keep the blank lines from original code. ( lines with just a - remove them ). - moving and grouping of enum has no added value from my point of view. Because of the ngx_mail_set__parse_state_start() and ngx_mail_set_imap_parse_state_argument() methods, you need to push the enums declarations out of parse() methods. => use a macro to replace the ngx_mail_set__parse_state_start method. => not sure that a macro will be compliant with nginx standard. - ngx_mail_set_imap_parse_state_start() and co. has no value added, unless you want to put a breakpoint to catch these step. - same about ngx_mail_set_imap_parse_state_argument() and co. Can i know the possibility of this patch being considered if i make the above suggested changes or any others if there ? Thanks -Kunal ----- Original Message ----- From: "Kunal Pariani" To: "nginx-devel" Sent: Thursday, September 18, 2014 10:32:48 AM Subject: Re: [PATCH] Update mail parsing to be per protocol (imap/pop/smtp) The patch certainly adds more intelligence in the mail parser per protocol (adding more specific states & error codes) and seems valid even with the latest nginx mainline. Thanks -Kunal ----- Original Message ----- From: "Maxim Dounin" To: "nginx-devel" Sent: Thursday, September 18, 2014 4:00:32 AM Subject: Re: [PATCH] Update mail parsing to be per protocol (imap/pop/smtp) Hello! On Wed, Sep 17, 2014 at 03:49:50PM -0500, Kunal Pariani wrote: > Hello, > This patch adds separate mail parsing states per protocol > (imap/pop/smtp), specific error codes for parsing/auth/login > failures. Also includes some minor bug fixes in the mail parsing > code. Requesting to include this patch as Zimbra has been using > this since nginx 0.5.37, so it's quite heavily tested. Mail parsing code a bit changed since 0.5.37, so this is more like an argument against. In any case, it's not clear what the patch tries to achieve. It's highly unlikely that it will be considered, see http://nginx.org/en/docs/contributing_changes.html. -- Maxim Dounin http://nginx.org/ _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel From yingqi.lu at intel.com Wed Oct 8 18:24:28 2014 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Wed, 8 Oct 2014 18:24:28 +0000 Subject: [Patch] SO_REUSEPORT support from master process In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AA37D1997@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AA37A52F5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AA37D0C4E@ORSMSX113.amr.corp.intel.com> <20141008125848.GA31276@mdounin.ru> <9ACD5B67AAC5594CB6268234CF29CF9AA37D1997@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA37D1B44@ORSMSX113.amr.corp.intel.com> One more comment from me: duplicate listen sockets in kernel is not a trivia thing to do and it may take long time before people can see it. Addressing it Nginx may not be as ideal as in kernel, but at least user can see the performance improvement sooner. In fact, we see up to 48% performance improvement on modern Intel system. Just my two cents. Again, thanks very much for everyone for helping us review this. Thanks, Yingqi -----Original Message----- From: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Lu, Yingqi Sent: Wednesday, October 08, 2014 10:05 AM To: nginx-devel at nginx.org Subject: RE: [Patch] SO_REUSEPORT support from master process Hi Maxim, Thanks for letting us know. Our updated patch is located at http://forum.nginx.org/read.php?29,253446,253446#msg-253446 It supposes to address all the style issues and fixes the restart and binary upgrade issues. This is just a FYI in case you are not aware of. Thanks, Yingqi -----Original Message----- From: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Maxim Dounin Sent: Wednesday, October 08, 2014 5:59 AM To: nginx-devel at nginx.org Subject: Re: [Patch] SO_REUSEPORT support from master process Hello! On Tue, Oct 07, 2014 at 07:32:08PM +0000, Lu, Yingqi wrote: > Dear All, > > It has been quiet for a while on this patch. I am checking to see if > there is any questions/feedbacks/concerns we need to address? > > Please let me know. Thanks very much for your help! Apart from style/coding issues, I disagree with the whole approach. As far as I understand the patch idea, it tries to introduce multiple listening sockets to avoid in-kernel lock contention. This is something that can be done completely in kernel though, and I see no reason to introduce any changes to nginx here. The approach previously discussed with Sepherosa Ziehau looks much more interesting. -- Maxim Dounin http://nginx.org/ _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Wed Oct 8 18:34:56 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 8 Oct 2014 22:34:56 +0400 Subject: [PATCH] Update mail parsing to be per protocol (imap/pop/smtp) In-Reply-To: <166026479.101799.1412790875955.JavaMail.zimbra@zimbra.com> References: <1273114156.2387485.1410986989995.JavaMail.zimbra@zimbra.com> <20140918110032.GO91749@mdounin.ru> <1644005643.2543169.1411061568638.JavaMail.zimbra@zimbra.com> <1349223799.3017631.1411600472282.JavaMail.zimbra@zimbra.com> <166026479.101799.1412790875955.JavaMail.zimbra@zimbra.com> Message-ID: <20141008183456.GJ31276@mdounin.ru> Hello! On Wed, Oct 08, 2014 at 12:54:35PM -0500, Kunal Pariani wrote: > Any responses on this please ? The original problem still applies: it's not clear what the patch tries to achieve. Quoting http://nginx.org/en/docs/contributing_changes.html: : Try to make it clear why the suggested change is needed, and : provide a use case, if possible. -- Maxim Dounin http://nginx.org/ From kpariani at zimbra.com Wed Oct 8 19:21:05 2014 From: kpariani at zimbra.com (Kunal Pariani) Date: Wed, 8 Oct 2014 14:21:05 -0500 (CDT) Subject: [PATCH] Update mail parsing to be per protocol (imap/pop/smtp) In-Reply-To: <330110650.105518.1412796031520.JavaMail.zimbra@zimbra.com> References: <1273114156.2387485.1410986989995.JavaMail.zimbra@zimbra.com> <20140918110032.GO91749@mdounin.ru> <1644005643.2543169.1411061568638.JavaMail.zimbra@zimbra.com> <1349223799.3017631.1411600472282.JavaMail.zimbra@zimbra.com> <166026479.101799.1412790875955.JavaMail.zimbra@zimbra.com> <20141008183456.GJ31276@mdounin.ru> Message-ID: <1708574986.105528.1412796065990.JavaMail.zimbra@zimbra.com> Here the main goals of the patch 1. Adds separate mail parsing states per protocol (imap/pop/smtp) 2. Expected args count check 3. Additional error messages (specific error codes for parsing/auth/login failures & the handling in case of these failures) Thanks -Kunal ----- Original Message ----- From: "Maxim Dounin" To: "nginx-devel" Sent: Wednesday, October 8, 2014 11:34:56 AM Subject: Re: [PATCH] Update mail parsing to be per protocol (imap/pop/smtp) Hello! On Wed, Oct 08, 2014 at 12:54:35PM -0500, Kunal Pariani wrote: > Any responses on this please ? The original problem still applies: it's not clear what the patch tries to achieve. Quoting http://nginx.org/en/docs/contributing_changes.html: : Try to make it clear why the suggested change is needed, and : provide a use case, if possible. -- Maxim Dounin http://nginx.org/ _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel From kpariani at zimbra.com Wed Oct 8 21:31:59 2014 From: kpariani at zimbra.com (Kunal Pariani) Date: Wed, 8 Oct 2014 16:31:59 -0500 (CDT) Subject: Support for ssl_certificate_dir ? Message-ID: <437522766.115663.1412803919848.JavaMail.zimbra@zimbra.com> Hello, Currently nginx doesn't have the capability to define the directory containing all the ssl certificates. This requires all the chained certificates need to be munched together in the right order in a single file and specified using the ssl_certificate directive (http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate). Any plans of adding the option for of having a 'ssl_certicate_dir' maybe which would have the path where all the cert files reside ? Also the ordering of the certs will be done by the proxy itself instead of being done manually ? Thanks -Kunal -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard at fussenegger.info Thu Oct 9 08:36:10 2014 From: richard at fussenegger.info (Richard Fussenegger, BSc) Date: Thu, 09 Oct 2014 10:36:10 +0200 Subject: Session Ticket Rotation In-Reply-To: <20140922123809.GE80160@mdounin.ru> References: <54200A7F.3030408@fussenegger.info> <20140922123809.GE80160@mdounin.ru> Message-ID: <543648FA.7070002@fussenegger.info> Hello Maxim! On 9/22/2014 2:38 PM, Maxim Dounin wrote: > Hello! > > On Mon, Sep 22, 2014 at 01:39:43PM +0200, Richard Fussenegger, BSc wrote: > > The main problem here is how to share keys between worker > processes, to ensure different workers will be able to decrypt > tickets. So automatic rotation of ticket keys will likely require > shared SSL session cache to be configured as well, and using a SSL > session cache to store ticket keys. Does this mean that a ticket key isn't shared among workers if one is using a single nginx instance with e.g. four workers? Or is the sharing of that ticket key handled by a single SSL_CTX in OpenSSL? Richard -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4237 bytes Desc: S/MIME Cryptographic Signature URL: From mdounin at mdounin.ru Thu Oct 9 11:46:57 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 9 Oct 2014 15:46:57 +0400 Subject: [PATCH] Update mail parsing to be per protocol (imap/pop/smtp) In-Reply-To: <1708574986.105528.1412796065990.JavaMail.zimbra@zimbra.com> References: <1273114156.2387485.1410986989995.JavaMail.zimbra@zimbra.com> <20140918110032.GO91749@mdounin.ru> <1644005643.2543169.1411061568638.JavaMail.zimbra@zimbra.com> <1349223799.3017631.1411600472282.JavaMail.zimbra@zimbra.com> <166026479.101799.1412790875955.JavaMail.zimbra@zimbra.com> <20141008183456.GJ31276@mdounin.ru> <1708574986.105528.1412796065990.JavaMail.zimbra@zimbra.com> Message-ID: <20141009114656.GL31276@mdounin.ru> Hello! On Wed, Oct 08, 2014 at 02:21:05PM -0500, Kunal Pariani wrote: > Here the main goals of the patch > 1. Adds separate mail parsing states per protocol (imap/pop/smtp) This doesn't looks like a goal. > 2. Expected args count check What's the difference? Why do you think it's needed? > 3. Additional error messages (specific error codes for parsing/auth/login failures & the handling in case of these failures) Which error messages? Why do you think it's needed? -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Thu Oct 9 11:53:58 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 9 Oct 2014 15:53:58 +0400 Subject: Session Ticket Rotation In-Reply-To: <543648FA.7070002@fussenegger.info> References: <54200A7F.3030408@fussenegger.info> <20140922123809.GE80160@mdounin.ru> <543648FA.7070002@fussenegger.info> Message-ID: <20141009115358.GM31276@mdounin.ru> Hello! On Thu, Oct 09, 2014 at 10:36:10AM +0200, Richard Fussenegger, BSc wrote: > Hello Maxim! > > On 9/22/2014 2:38 PM, Maxim Dounin wrote: > >Hello! > > > >On Mon, Sep 22, 2014 at 01:39:43PM +0200, Richard Fussenegger, BSc wrote: > > > >The main problem here is how to share keys between worker > >processes, to ensure different workers will be able to decrypt > >tickets. So automatic rotation of ticket keys will likely require > >shared SSL session cache to be configured as well, and using a SSL > >session cache to store ticket keys. > > Does this mean that a ticket key isn't shared among workers if one is using > a single nginx instance with e.g. four workers? Or is the sharing of that > ticket key handled by a single SSL_CTX in OpenSSL? As of now, ticket keys are created (or read from files specified) during configuration parsing, when SSL_CTX is created. All workers inherit the same configuration from master during fork(), and hence will have identical ticket keys. -- Maxim Dounin http://nginx.org/ From peter at lekensteyn.nl Thu Oct 9 16:02:16 2014 From: peter at lekensteyn.nl (Peter Wu) Date: Thu, 09 Oct 2014 18:02:16 +0200 Subject: About the UNIX socket path length check Message-ID: <1425960.2AC8navbJM@al> Hi, In src/core/ngx_inet.c we have this code: if (len > sizeof(saun->sun_path)) { u->err = "too long path in the unix domain socket"; return NGX_ERROR; } But a NUL-terminated string requires one byte more. Is it allowed to omit the NUL byte when the string would not fit otherwise? If not, this should be changed to: if (len >= sizeof(saun->sun_path)) { u->err = "too long path in the unix domain socket"; return NGX_ERROR; } -- Kind regards, Peter https://lekensteyn.nl From ru at nginx.com Thu Oct 9 16:41:37 2014 From: ru at nginx.com (Ruslan Ermilov) Date: Thu, 9 Oct 2014 20:41:37 +0400 Subject: About the UNIX socket path length check In-Reply-To: <1425960.2AC8navbJM@al> References: <1425960.2AC8navbJM@al> Message-ID: <20141009164137.GT61641@lo0.su> On Thu, Oct 09, 2014 at 06:02:16PM +0200, Peter Wu wrote: > Hi, > > In src/core/ngx_inet.c we have this code: > > if (len > sizeof(saun->sun_path)) { > u->err = "too long path in the unix domain socket"; > return NGX_ERROR; > } > > But a NUL-terminated string requires one byte more. Is it allowed to > omit the NUL byte when the string would not fit otherwise? If not, > this should be changed to: > > if (len >= sizeof(saun->sun_path)) { > u->err = "too long path in the unix domain socket"; > return NGX_ERROR; > } The "len" you quote includes the slot for NUL due to u->host.len = len++; And ngx_strncpy() always NULL-terminates what it writes. From mdounin at mdounin.ru Thu Oct 9 17:13:22 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 9 Oct 2014 21:13:22 +0400 Subject: About the UNIX socket path length check In-Reply-To: <1425960.2AC8navbJM@al> References: <1425960.2AC8navbJM@al> Message-ID: <20141009171322.GO31276@mdounin.ru> Hello! On Thu, Oct 09, 2014 at 06:02:16PM +0200, Peter Wu wrote: > Hi, > > In src/core/ngx_inet.c we have this code: > > if (len > sizeof(saun->sun_path)) { > u->err = "too long path in the unix domain socket"; > return NGX_ERROR; > } > > But a NUL-terminated string requires one byte more. Is it allowed to > omit the NUL byte when the string would not fit otherwise? If not, > this should be changed to: > > if (len >= sizeof(saun->sun_path)) { > u->err = "too long path in the unix domain socket"; > return NGX_ERROR; > } You've missed the "len++" above. -- Maxim Dounin http://nginx.org/ From nzt4567 at gmx.com Fri Oct 10 14:08:46 2014 From: nzt4567 at gmx.com (NZT) Date: Fri, 10 Oct 2014 16:08:46 +0200 Subject: close client connection Message-ID: <1412950126.9783.3.camel@gmx.com> Greetings to all of you, I've been struggling with this for a while and I don't know where else to ask - if this has already been answered somewhere please forgive me for asking again - I couldn't find the answer. The question is quick and simple - how to properly close a connection with a client when I have an ongoing subrequest which is talking to the upstream? The client does not need the data from the subrequest, I just want to cache them and nginx won't send FIN,ACK to the client even after closing the connection with the upstream. Regards, Tomas Kvasnicka From agentzh at gmail.com Fri Oct 10 20:57:43 2014 From: agentzh at gmail.com (Yichun Zhang (agentzh)) Date: Fri, 10 Oct 2014 13:57:43 -0700 Subject: Session Ticket Rotation In-Reply-To: <54200A7F.3030408@fussenegger.info> References: <54200A7F.3030408@fussenegger.info> Message-ID: Hello! On Mon, Sep 22, 2014 at 4:39 AM, Richard Fussenegger, BSc wrote: > I'd like to implement built-in session ticket rotation. I know that it this > was discussed before but it was never implemented. Right now a custom > external ticket key system is supported. Admins with single installations > and not enough knowledge about the topic are left with keys that are valid > for the complete lifetime nginx is running. > Fortunately this does not have to be in the nginx core :) We're using the ngx_lua module [1] to periodically update the session ticket keys from external shared data services (like memcached). To be more specific, we're using ngx_lua's init_worker_by_lua [2] to create a re-occurring timer (via ngx.timer.at [3]) and fetch a new ticket key from external data sources via the nonblocking lua-resty-memcached library [4] and add that into the existing queue used by OpenSSL via LuaJIT FFI [5]. Also, we use the lua_shared_dict [6] to reduce traffic to the external data source online. No patches are needed for the nginx core :) In this "add-on" implementation, the ticket keys are also shared across all our machines. Best regards, -agentzh [1] https://github.com/openresty/lua-nginx-module [2] https://github.com/openresty/lua-nginx-module#init_worker_by_lua [3] https://github.com/openresty/lua-nginx-module#ngxtimerat [4] https://github.com/openresty/lua-resty-memcached [5] http://luajit.org/ext_ffi.html [6] https://github.com/openresty/lua-nginx-module#lua_shared_dict From richard at fussenegger.info Fri Oct 10 21:09:05 2014 From: richard at fussenegger.info (Richard Fussenegger, BSc) Date: Fri, 10 Oct 2014 23:09:05 +0200 Subject: Session Ticket Rotation In-Reply-To: References: <54200A7F.3030408@fussenegger.info> Message-ID: <54384AF1.20604@fussenegger.info> Hello and many thanks for your reply. On 10/10/2014 10:57 PM, Yichun Zhang (agentzh) wrote: > Hello! > > Fortunately this does not have to be in the nginx core :) I came to the conclusion that this shouldn't even be in nginx core. OpenSSL should be updated. Of course a few nginx changes would come with that (exposing some configuration settings). There's more that's problematic with session tickets in OpenSSL: only AES128-CBC-SHA256 while ignoring chosen cipher. Let's hope that some OpenSSL / C crack will update that at some point in the future. Luckily we have now a few forks and ReSSL might be just the project that brings us a real interface to work against. > We're using the ngx_lua module [1] to periodically update the session > ticket keys from external shared data services (like memcached). > > To be more specific, we're using ngx_lua's init_worker_by_lua [2] to > create a re-occurring timer (via ngx.timer.at [3]) and fetch a new > ticket key from external data sources via the nonblocking > lua-resty-memcached library [4] and add that into the existing queue > used by OpenSSL via LuaJIT FFI [5]. > > Also, we use the lua_shared_dict [6] to reduce traffic to the external > data source online. > > No patches are needed for the nginx core :) > > In this "add-on" implementation, the ticket keys are also shared > across all our machines. > > Best regards, > -agentzh > > [1] https://github.com/openresty/lua-nginx-module > [2] https://github.com/openresty/lua-nginx-module#init_worker_by_lua > [3] https://github.com/openresty/lua-nginx-module#ngxtimerat > [4] https://github.com/openresty/lua-resty-memcached > [5] http://luajit.org/ext_ffi.html > [6] https://github.com/openresty/lua-nginx-module#lua_shared_dict I'm currently working on a solution that only relies on a POSIX compatible shell interpreter without any additions to nginx. The only requirement is version 1.5.7 (which you have as well). https://github.com/Fleshgrinder/nginx-session-ticket-key-rotation It's work in progress right now and will only be Debian / Ubuntu compatible when it's finished. I'll also integrate syncing in clusters and compatibility with other operating systems if I find the time. @anyone If you have the time to review my work and approach, please do so. I'd love feedback! Regards Richard PS: You should open source your solution. :) -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4237 bytes Desc: S/MIME Cryptographic Signature URL: From kou at cozmixng.org Sat Oct 11 01:41:23 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Sat, 11 Oct 2014 10:41:23 +0900 (JST) Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141004.195156.801414609668336877.kou@cozmixng.org> References: <20141004.195156.801414609668336877.kou@cozmixng.org> Message-ID: <20141011.104123.843568327634645522.kou@cozmixng.org> Hi, Could anyone review the patch? In <20141004.195156.801414609668336877.kou at cozmixng.org> "[PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Sat, 04 Oct 2014 19:51:56 +0900 (JST), Kouhei Sutou wrote: > Hi, > > I'm trying cross compiling nginx for Windows on Debian > GNU/Linux. The current nginx doesn't support it. > I created a patch to do it. With the attached patch, I can > cross compile nginx for Windows on Debian GNU/Linux. > Please review the attached patch. > > The current status (without patch) > ---------------------------------- > > Compiler: > > % x86_64-w64-mingw32-gcc --version > x86_64-w64-mingw32-gcc (GCC) 4.9.1 > Copyright (C) 2014 Free Software Foundation, Inc. > This is free software; see the source for copying conditions. There is NO > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > > Configure command line: > > % auto/configure \ > --prefix=/tmp/local \ > --crossbuild=win32 \ > --with-cc=x86_64-w64-mingw32-gcc \ > --with-cpp=x86_64-w64-mingw32-c++ \ > --without-http_rewrite_module \ > --without-http_gzip_module > > Build result: > > % make > make -f objs/Makefile > make[1]: Entering directory '/home/kou/work/c/nginx.win32' > x86_64-w64-mingw32-gcc -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I src/core -I src/event -I src/event/modules -I src/os/win32 -I objs \ > -o objs/src/core/nginx.o \ > src/core/nginx.c > In file included from /usr/share/mingw-w64/include/stdio.h:80:0, > from src/os/win32/ngx_win32_config.h:45, > from src/core/ngx_config.h:38, > from src/core/nginx.c:8: > /usr/share/mingw-w64/include/_mingw_off_t.h:26:9: error: unknown type name ?off32_t? > typedef off32_t off_t; > ^ > In file included from src/os/win32/ngx_win32_config.h:45:0, > from src/core/ngx_config.h:38, > from src/core/nginx.c:8: > /usr/share/mingw-w64/include/stdio.h:463:28: error: unknown type name ?_off_t? > int fseeko(FILE* stream, _off_t offset, int whence); > ^ > /usr/share/mingw-w64/include/stdio.h:474:3: error: unknown type name ?_off_t? > _off_t ftello(FILE * stream); > ^ > In file included from src/core/ngx_config.h:38:0, > from src/core/nginx.c:8: > src/os/win32/ngx_win32_config.h:140:29: error: conflicting types for ?intptr_t? > typedef int intptr_t; > ^ > In file included from /usr/share/mingw-w64/include/windows.h:9:0, > from src/os/win32/ngx_win32_config.h:28, > from src/core/ngx_config.h:38, > from src/core/nginx.c:8: > /usr/share/mingw-w64/include/_mingw.h:399:35: note: previous declaration of ?intptr_t? was here > __MINGW_EXTENSION typedef __int64 intptr_t; > ^ > In file included from src/core/ngx_config.h:38:0, > from src/core/nginx.c:8: > src/os/win32/ngx_win32_config.h:141:29: error: conflicting types for ?uintptr_t? > typedef u_int uintptr_t; > ^ > In file included from /usr/share/mingw-w64/include/windows.h:9:0, > from src/os/win32/ngx_win32_config.h:28, > from src/core/ngx_config.h:38, > from src/core/nginx.c:8: > /usr/share/mingw-w64/include/_mingw.h:412:44: note: previous declaration of ?uintptr_t? was here > __MINGW_EXTENSION typedef unsigned __int64 uintptr_t; > ^ > In file included from src/core/ngx_config.h:38:0, > from src/core/nginx.c:8: > src/os/win32/ngx_win32_config.h:146:29: error: conflicting types for ?off_t? > typedef __int64 off_t; > ^ > In file included from /usr/share/mingw-w64/include/stdio.h:80:0, > from src/os/win32/ngx_win32_config.h:45, > from src/core/ngx_config.h:38, > from src/core/nginx.c:8: > /usr/share/mingw-w64/include/_mingw_off_t.h:26:17: note: previous declaration of ?off_t? was here > typedef off32_t off_t; > ^ > In file included from src/core/ngx_config.h:38:0, > from src/core/nginx.c:8: > src/os/win32/ngx_win32_config.h:167:29: error: conflicting types for ?ssize_t? > typedef int ssize_t; > ^ > In file included from /usr/share/mingw-w64/include/windows.h:9:0, > from src/os/win32/ngx_win32_config.h:28, > from src/core/ngx_config.h:38, > from src/core/nginx.c:8: > /usr/share/mingw-w64/include/_mingw.h:387:35: note: previous declaration of ?ssize_t? was here > __MINGW_EXTENSION typedef __int64 ssize_t; > ^ > objs/Makefile:323: recipe for target 'objs/src/core/nginx.o' failed > make[1]: *** [objs/src/core/nginx.o] Error 1 > make[1]: Leaving directory '/home/kou/work/c/nginx.win32' > Makefile:8: recipe for target 'build' failed > make: *** [build] Error 2 > > > How to solve > ------------ > > I use the system type definitions instead of custom type > definitions for off_t, intptr_t and ssize_t to solve the > errors. > > "#define _FILE_OFFSET_BITS 64" is for defining off_t as a > 64bit type. (nginx requires off_t as a 64bit type.) > > See the attached patch for details. > > Note > ---- > > The main change is the change for > src/os/win32/ngx_win32_config.h. > > The changes of the followings are just trivial build errors: > > * src/event/modules/ngx_iocp_module.c > * src/os/win32/ngx_wsasend_chain.c > > The change of src/event/modules/ngx_iocp_module.c is for > fixing the following warning: > > x86_64-w64-mingw32-gcc -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I src/core -I src/event -I src/event/modules -I src/os/win32 -I objs \ > -o objs/src/event/modules/ngx_iocp_module.o \ > src/event/modules/ngx_iocp_module.c > src/event/modules/ngx_iocp_module.c: In function 'ngx_iocp_process_events': > src/event/modules/ngx_iocp_module.c:250:10: error: passing argument 3 of 'GetQueuedCompletionStatus' from incompatible pointer type [-Werror] > rc = GetQueuedCompletionStatus(iocp, &bytes, (LPDWORD) &key, > ^ > In file included from /usr/share/mingw-w64/include/winbase.h:21:0, > from /usr/share/mingw-w64/include/windows.h:70, > from src/os/win32/ngx_win32_config.h:28, > from src/core/ngx_config.h:38, > from src/event/modules/ngx_iocp_module.c:8: > /usr/share/mingw-w64/include/ioapiset.h:20:29: note: expected 'PULONG_PTR' but argument is of type 'DWORD *' > WINBASEAPI WINBOOL WINAPI GetQueuedCompletionStatus (HANDLE CompletionPort, LPDWORD lpNumberOfBytesTransferred, PULONG_PTR lpCompletionKey, LPOVERLAPPED *lpOverlapped, DWORD dwMilliseconds); > ^ > cc1: all warnings being treated as errors > > GetQueuedCompletionStatus() document on MSDN says the > following signature: > > http://msdn.microsoft.com/en-us/library/windows/desktop/aa364986.aspx > > BOOL WINAPI GetQueuedCompletionStatus( > _In_ HANDLE CompletionPort, > _Out_ LPDWORD lpNumberOfBytes, > _Out_ PULONG_PTR lpCompletionKey, > _Out_ LPOVERLAPPED *lpOverlapped, > _In_ DWORD dwMilliseconds > ); > > In the latest specification, the type of the third argument > (lpCompletionKey) is PULONG_PTR not LPDWORD. So the attached > patch casts to PULONG_PTR instead of LPDWORD. > > > The change of src/os/win32/ngx_wsasend_chain.c is for > fixing the following warnings: > > x86_64-w64-mingw32-gcc -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I src/core -I src/event -I src/event/modules -I src/os/win32 -I objs \ > -o objs/src/os/win32/ngx_wsasend_chain.o \ > src/os/win32/ngx_wsasend_chain.c > src/os/win32/ngx_wsasend_chain.c: In function 'ngx_wsasend_chain': > src/os/win32/ngx_wsasend_chain.c:37:29: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare] > if (limit == 0 || limit > NGX_MAX_UINT32_VALUE - ngx_pagesize) { > ^ > src/os/win32/ngx_wsasend_chain.c: In function 'ngx_overlapped_wsasend_chain': > src/os/win32/ngx_wsasend_chain.c:159:33: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare] > if (limit == 0 || limit > NGX_MAX_UINT32_VALUE - ngx_pagesize) { > ^ > cc1: all warnings being treated as errors > objs/Makefile:673: recipe for target 'objs/src/os/win32/ngx_wsasend_chain.o' failed > > > > I don't test the patch with Visual Studio. Sorry. > I hope that someone try the patch with Visual Studio. > > > Thanks, > -- > kou From vbart at nginx.com Sat Oct 11 09:17:36 2014 From: vbart at nginx.com (Valentin V. Bartenev) Date: Sat, 11 Oct 2014 13:17:36 +0400 Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141011.104123.843568327634645522.kou@cozmixng.org> References: <20141004.195156.801414609668336877.kou@cozmixng.org> <20141011.104123.843568327634645522.kou@cozmixng.org> Message-ID: <9179053.BLszIlEqpy@vbart-laptop> On Saturday 11 October 2014 10:41:23 Kouhei Sutou wrote: > Hi, > > Could anyone review the patch? > > In <20141004.195156.801414609668336877.kou at cozmixng.org> > "[PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Sat, 04 Oct 2014 19:51:56 +0900 (JST), > Kouhei Sutou wrote: > [..] http://mailman.nginx.org/pipermail/nginx-devel/2014-October/006054.html wbr, Valentin V. Bartenev From kou at cozmixng.org Sat Oct 11 12:02:28 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Sat, 11 Oct 2014 21:02:28 +0900 (JST) Subject: [PATCH] Fix wrong type cast in ngx_iocp_module.c Message-ID: <20141011.210228.1918325823879194614.kou@cozmixng.org> Hi, The attached patch fix the following warning: x86_64-w64-mingw32-gcc -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I src/core -I src/event -I src/event/modules -I src/os/win32 -I objs \ -o objs/src/event/modules/ngx_iocp_module.o \ src/event/modules/ngx_iocp_module.c src/event/modules/ngx_iocp_module.c: In function 'ngx_iocp_process_events': src/event/modules/ngx_iocp_module.c:250:10: error: passing argument 3 of 'GetQueuedCompletionStatus' from incompatible pointer type [-Werror] rc = GetQueuedCompletionStatus(iocp, &bytes, (LPDWORD) &key, ^ In file included from /usr/share/mingw-w64/include/winbase.h:21:0, from /usr/share/mingw-w64/include/windows.h:70, from src/os/win32/ngx_win32_config.h:28, from src/core/ngx_config.h:38, from src/event/modules/ngx_iocp_module.c:8: /usr/share/mingw-w64/include/ioapiset.h:20:29: note: expected 'PULONG_PTR' but argument is of type 'DWORD *' WINBASEAPI WINBOOL WINAPI GetQueuedCompletionStatus (HANDLE CompletionPort, LPDWORD lpNumberOfBytesTransferred, PULONG_PTR lpCompletionKey, LPOVERLAPPED *lpOverlapped, DWORD dwMilliseconds); ^ cc1: all warnings being treated as errors This patch is separated from http://mailman.nginx.org/pipermail/nginx-devel/2014-October/006028.html It is suggested at http://mailman.nginx.org/pipermail/nginx-devel/2014-October/006054.html Thanks, -- kou -------------- next part -------------- A non-text attachment was scrubbed... Name: ngx-iocp-module-fix-wrong-type-cast.patch Type: text/x-patch Size: 1473 bytes Desc: not available URL: From kou at cozmixng.org Sat Oct 11 12:11:30 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Sat, 11 Oct 2014 21:11:30 +0900 (JST) Subject: [PATCH] Win32: suppressed sign-compare warnings Message-ID: <20141011.211130.518298213992700583.kou@cozmixng.org> Hi, The attached patch fixes the following warnings: x86_64-w64-mingw32-gcc -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I src/core -I src/event -I src/event/modules -I src/os/win32 -I objs \ -o objs/src/os/win32/ngx_wsasend_chain.o \ src/os/win32/ngx_wsasend_chain.c src/os/win32/ngx_wsasend_chain.c: In function 'ngx_wsasend_chain': src/os/win32/ngx_wsasend_chain.c:37:29: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare] if (limit == 0 || limit > NGX_MAX_UINT32_VALUE - ngx_pagesize) { ^ src/os/win32/ngx_wsasend_chain.c: In function 'ngx_overlapped_wsasend_chain': src/os/win32/ngx_wsasend_chain.c:159:33: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare] if (limit == 0 || limit > NGX_MAX_UINT32_VALUE - ngx_pagesize) { ^ cc1: all warnings being treated as errors objs/Makefile:673: recipe for target 'objs/src/os/win32/ngx_wsasend_chain.o' failed This patch is separated from http://mailman.nginx.org/pipermail/nginx-devel/2014-October/006028.html It was suggested at http://mailman.nginx.org/pipermail/nginx-devel/2014-October/006054.html Thanks, -- kou -------------- next part -------------- A non-text attachment was scrubbed... Name: win32-suppressed-sign-compare-warnings.patch Type: text/x-patch Size: 2757 bytes Desc: not available URL: From kou at cozmixng.org Sat Oct 11 12:29:02 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Sat, 11 Oct 2014 21:29:02 +0900 (JST) Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141008142441.GE31276@mdounin.ru> References: <20141004.195156.801414609668336877.kou@cozmixng.org> <20141008142441.GE31276@mdounin.ru> Message-ID: <20141011.212902.1900691365943898058.kou@cozmixng.org> Hi, Sorry... I missed your reply... I attach a new patch that is applied your comments. In <20141008142441.GE31276 at mdounin.ru> "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Wed, 8 Oct 2014 18:24:41 +0400, Maxim Dounin wrote: >> diff --git a/src/event/modules/ngx_iocp_module.c b/src/event/modules/ngx_iocp_module.c >> --- a/src/event/modules/ngx_iocp_module.c >> +++ b/src/event/modules/ngx_iocp_module.c ... > > Looks fine, though I think it should be a separate patch. I see! I did: http://mailman.nginx.org/pipermail/nginx-devel/2014-October/006072.html >> diff --git a/src/os/win32/ngx_win32_config.h b/src/os/win32/ngx_win32_config.h >> --- a/src/os/win32/ngx_win32_config.h >> +++ b/src/os/win32/ngx_win32_config.h ... >> @@ -131,25 +130,31 @@ typedef int int32_t; >> typedef unsigned short int uint16_t; >> #define ngx_libc_cdecl >> >> #endif >> >> typedef __int64 int64_t; >> typedef unsigned __int64 uint64_t; >> >> -#ifndef __WATCOMC__ >> +#ifdef __WATCOMC__ >> +/* do nothing */ >> +#elif defined(__GNUC__) >> +#include >> +#else >> typedef int intptr_t; >> typedef u_int uintptr_t; >> #endif > > I think that ngx_win32_config.h part needs a bit more work. In > particular, "#include " should be placed with other > includes if it's needed. I see. The attached patch puts "#include " after "#include ". >> +#ifndef __GNUC__ >> /* Windows defines off_t as long, which is 32-bit */ >> -typedef __int64 off_t; >> -#define _OFF_T_DEFINED >> +typedef __int64 off64_t; >> +#define _OFF64_T_DEFINED >> +#endif > > Defining off64_t looks very wrong for me. There is nothing in > nginx which use it, and it shouldn't be defined. > > Additionally, this breaks at least compilation with MSVC. You're absolutely right. This type name change is needless. (off64_t was introduced in my first try but it was removed in my second try. So I should remove the change before sending my patch.) The attached patch just adds "#ifdef __GNUC__" and "#endif" >> --- a/src/os/win32/ngx_wsasend_chain.c >> +++ b/src/os/win32/ngx_wsasend_chain.c ... > > Looks fine, though should be a separate patch. See above. I did: http://mailman.nginx.org/pipermail/nginx-devel/2014-October/006073.html Thanks, -- kou -------------- next part -------------- A non-text attachment was scrubbed... Name: win32-made-build-able-with-mingw-w64.patch Type: text/x-patch Size: 7140 bytes Desc: not available URL: From kou at cozmixng.org Sat Oct 11 12:29:45 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Sat, 11 Oct 2014 21:29:45 +0900 (JST) Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <9179053.BLszIlEqpy@vbart-laptop> References: <20141004.195156.801414609668336877.kou@cozmixng.org> <20141011.104123.843568327634645522.kou@cozmixng.org> <9179053.BLszIlEqpy@vbart-laptop> Message-ID: <20141011.212945.209793435386040021.kou@cozmixng.org> Hi, In <9179053.BLszIlEqpy at vbart-laptop> "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Sat, 11 Oct 2014 13:17:36 +0400, "Valentin V. Bartenev" wrote: > On Saturday 11 October 2014 10:41:23 Kouhei Sutou wrote: >> Hi, >> >> Could anyone review the patch? >> >> In <20141004.195156.801414609668336877.kou at cozmixng.org> >> "[PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Sat, 04 Oct 2014 19:51:56 +0900 (JST), >> Kouhei Sutou wrote: >> > [..] > > http://mailman.nginx.org/pipermail/nginx-devel/2014-October/006054.html Oh... I missed it... Thanks for notifying it! -- kou From kunalvjti at gmail.com Mon Oct 13 19:41:20 2014 From: kunalvjti at gmail.com (Kunal Pariani) Date: Mon, 13 Oct 2014 12:41:20 -0700 Subject: Support for ssl_certificate_dir ? In-Reply-To: <437522766.115663.1412803919848.JavaMail.zimbra@zimbra.com> References: <437522766.115663.1412803919848.JavaMail.zimbra@zimbra.com> Message-ID: 444 on this one too ? Thanks -Kunal On Wed, Oct 8, 2014 at 2:31 PM, Kunal Pariani wrote: > Hello, > Currently nginx doesn't have the capability to define the directory > containing all the ssl certificates. This requires all the chained > certificates need to be munched together in the right order in a single > file and specified using the ssl_certificate directive ( > http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate). > Any plans of adding the option for of having a 'ssl_certicate_dir' maybe > which would have the path where all the cert files reside ? Also the > ordering of the certs will be done by the proxy itself instead of being > done manually ? > > Thanks > -Kunal > > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon Oct 13 20:17:26 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 14 Oct 2014 00:17:26 +0400 Subject: Support for ssl_certificate_dir ? In-Reply-To: References: <437522766.115663.1412803919848.JavaMail.zimbra@zimbra.com> Message-ID: <20141013201726.GG31276@mdounin.ru> Hello! On Mon, Oct 13, 2014 at 12:41:20PM -0700, Kunal Pariani wrote: > 444 on this one too ? > > Thanks > -Kunal > > On Wed, Oct 8, 2014 at 2:31 PM, Kunal Pariani wrote: > > > Hello, > > Currently nginx doesn't have the capability to define the directory > > containing all the ssl certificates. This requires all the chained > > certificates need to be munched together in the right order in a single > > file and specified using the ssl_certificate directive ( > > http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate). > > Any plans of adding the option for of having a 'ssl_certicate_dir' maybe > > which would have the path where all the cert files reside ? Also the > > ordering of the certs will be done by the proxy itself instead of being > > done manually ? Short answer: no plans. Long answer: The ssl_trusted_certificate directive can be used to specify a PEM file with certs to be loaded into a certificate store. These certs will be used by OpenSSL to complete the certificate chain automatically. This OpenSSL behaviour is believed to cause more harm than good though, because a) it can add unneed certs to the chain in many cases, and b) the chain is built on runtime, and hence consumes CPU. And there are no plans to add support of directories to ssl_trusted_certificate, as well to other similar directives. -- Maxim Dounin http://nginx.org/ From kpariani at zimbra.com Mon Oct 13 21:17:04 2014 From: kpariani at zimbra.com (Kunal Pariani) Date: Mon, 13 Oct 2014 16:17:04 -0500 (CDT) Subject: Support for ssl_certificate_dir ? In-Reply-To: <20141013201726.GG31276@mdounin.ru> References: <437522766.115663.1412803919848.JavaMail.zimbra@zimbra.com> <20141013201726.GG31276@mdounin.ru> Message-ID: <264337458.537750.1413235024404.JavaMail.zimbra@zimbra.com> Thank you. -Kunal ----- Original Message ----- From: "Maxim Dounin" To: "nginx-devel" Sent: Monday, October 13, 2014 1:17:26 PM Subject: Re: Support for ssl_certificate_dir ? Hello! On Mon, Oct 13, 2014 at 12:41:20PM -0700, Kunal Pariani wrote: > 444 on this one too ? > > Thanks > -Kunal > > On Wed, Oct 8, 2014 at 2:31 PM, Kunal Pariani wrote: > > > Hello, > > Currently nginx doesn't have the capability to define the directory > > containing all the ssl certificates. This requires all the chained > > certificates need to be munched together in the right order in a single > > file and specified using the ssl_certificate directive ( > > http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_certificate). > > Any plans of adding the option for of having a 'ssl_certicate_dir' maybe > > which would have the path where all the cert files reside ? Also the > > ordering of the certs will be done by the proxy itself instead of being > > done manually ? Short answer: no plans. Long answer: The ssl_trusted_certificate directive can be used to specify a PEM file with certs to be loaded into a certificate store. These certs will be used by OpenSSL to complete the certificate chain automatically. This OpenSSL behaviour is believed to cause more harm than good though, because a) it can add unneed certs to the chain in many cases, and b) the chain is built on runtime, and hence consumes CPU. And there are no plans to add support of directories to ssl_trusted_certificate, as well to other similar directives. -- Maxim Dounin http://nginx.org/ _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Tue Oct 14 00:19:44 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 14 Oct 2014 00:19:44 +0000 Subject: [nginx] Win32: fixed wrong type cast. Message-ID: details: http://hg.nginx.org/nginx/rev/de7ec5eef6d2 branches: changeset: 5872:de7ec5eef6d2 user: Kouhei Sutou date: Sat Oct 11 20:56:35 2014 +0900 description: Win32: fixed wrong type cast. GetQueuedCompletionStatus() document on MSDN says the following signature: http://msdn.microsoft.com/en-us/library/windows/desktop/aa364986.aspx BOOL WINAPI GetQueuedCompletionStatus( _In_ HANDLE CompletionPort, _Out_ LPDWORD lpNumberOfBytes, _Out_ PULONG_PTR lpCompletionKey, _Out_ LPOVERLAPPED *lpOverlapped, _In_ DWORD dwMilliseconds ); In the latest specification, the type of the third argument (lpCompletionKey) is PULONG_PTR not LPDWORD. diffstat: src/event/modules/ngx_iocp_module.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/src/event/modules/ngx_iocp_module.c b/src/event/modules/ngx_iocp_module.c --- a/src/event/modules/ngx_iocp_module.c +++ b/src/event/modules/ngx_iocp_module.c @@ -247,7 +247,7 @@ ngx_int_t ngx_iocp_process_events(ngx_cy ngx_log_debug1(NGX_LOG_DEBUG_EVENT, cycle->log, 0, "iocp timer: %M", timer); - rc = GetQueuedCompletionStatus(iocp, &bytes, (LPDWORD) &key, + rc = GetQueuedCompletionStatus(iocp, &bytes, (PULONG_PTR) &key, (LPOVERLAPPED *) &ovlp, (u_long) timer); if (rc == 0) { From mdounin at mdounin.ru Tue Oct 14 00:19:47 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 14 Oct 2014 00:19:47 +0000 Subject: [nginx] Win32: suppressed warnings by "-Werror=sign-compare". Message-ID: details: http://hg.nginx.org/nginx/rev/35b8e5e98508 branches: changeset: 5873:35b8e5e98508 user: Kouhei Sutou date: Sat Oct 11 21:09:29 2014 +0900 description: Win32: suppressed warnings by "-Werror=sign-compare". diffstat: src/os/win32/ngx_wsasend_chain.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (21 lines): diff --git a/src/os/win32/ngx_wsasend_chain.c b/src/os/win32/ngx_wsasend_chain.c --- a/src/os/win32/ngx_wsasend_chain.c +++ b/src/os/win32/ngx_wsasend_chain.c @@ -34,7 +34,7 @@ ngx_wsasend_chain(ngx_connection_t *c, n /* the maximum limit size is the maximum u_long value - the page size */ - if (limit == 0 || limit > NGX_MAX_UINT32_VALUE - ngx_pagesize) { + if (limit == 0 || limit > (off_t) (NGX_MAX_UINT32_VALUE - ngx_pagesize)) { limit = NGX_MAX_UINT32_VALUE - ngx_pagesize; } @@ -156,7 +156,7 @@ ngx_overlapped_wsasend_chain(ngx_connect /* the maximum limit size is the maximum u_long value - the page size */ - if (limit == 0 || limit > NGX_MAX_UINT32_VALUE - ngx_pagesize) { + if (limit == 0 || limit > (off_t) (NGX_MAX_UINT32_VALUE - ngx_pagesize)) { limit = NGX_MAX_UINT32_VALUE - ngx_pagesize; } From mdounin at mdounin.ru Tue Oct 14 00:22:18 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 14 Oct 2014 04:22:18 +0400 Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141011.212902.1900691365943898058.kou@cozmixng.org> References: <20141004.195156.801414609668336877.kou@cozmixng.org> <20141008142441.GE31276@mdounin.ru> <20141011.212902.1900691365943898058.kou@cozmixng.org> Message-ID: <20141014002218.GH31276@mdounin.ru> Hello! On Sat, Oct 11, 2014 at 09:29:02PM +0900, Kouhei Sutou wrote: > Hi, > > Sorry... I missed your reply... > > I attach a new patch that is applied your comments. > > In <20141008142441.GE31276 at mdounin.ru> > "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Wed, 8 Oct 2014 18:24:41 +0400, > Maxim Dounin wrote: > > >> diff --git a/src/event/modules/ngx_iocp_module.c b/src/event/modules/ngx_iocp_module.c > >> --- a/src/event/modules/ngx_iocp_module.c > >> +++ b/src/event/modules/ngx_iocp_module.c > ... > > > > Looks fine, though I think it should be a separate patch. > > I see! > I did: http://mailman.nginx.org/pipermail/nginx-devel/2014-October/006072.html This and ngx_wsasend_chain.c patches committed (with minor changes to commit logs), thanks. [...] > # HG changeset patch > # User Kouhei Sutou > # Date 1413030061 -32400 > # Sat Oct 11 21:21:01 2014 +0900 > # Node ID 68c7ba284f40a5d97f774380cf4cf80259b15585 > # Parent 21043ce2a0058154687a779b4c9b0f241fa7e88a > Win32: made build-able with MinGW-w64 > > Compiler: > > % x86_64-w64-mingw32-gcc --version > x86_64-w64-mingw32-gcc (GCC) 4.9.1 While this patch fixes compilation with MinGW-w64 gcc, it breaks compilation with MinGW gcc, which isn't good. [...] > #include > #include > #include > +#ifdef __GNUC__ > +#include /* intptr_t and uintptr_t */ > +#endif > #include > #include Minor nit: the comment here looks unneeded. [...] > typedef __int64 int64_t; > typedef unsigned __int64 uint64_t; > > -#ifndef __WATCOMC__ > +#if defined(__WATCOMC__) || defined(__GNUC__) > +/* do nothing */ > +#else > typedef int intptr_t; > typedef u_int uintptr_t; > #endif The construction looks awfull, it should be #if !defined(...) && !defined(...) ... #endif instead. [...] > #define NGX_PTR_SIZE 4 > #define NGX_SIZE_T_LEN (sizeof("-2147483648") - 1) > #define NGX_MAX_SIZE_T_VALUE 2147483647 Note well: the ngx_win32_config.h defines various constants, including ones quoted, assuming 32-bit build. 64-bit build will be flawed unless these constands are changed appropriately. -- Maxim Dounin http://nginx.org/ From donatas.abraitis at gmail.com Tue Oct 14 13:28:17 2014 From: donatas.abraitis at gmail.com (Donatas Abraitis) Date: Tue, 14 Oct 2014 16:28:17 +0300 Subject: listen with ssl Message-ID: Hello, is it possible to make nginx listen like this: 0.0.0.0:443 127.0.0.1:443 Because using such confuration I'm getting error bind() .. Address already in use. -- Donatas -------------- next part -------------- An HTML attachment was scrubbed... URL: From vbart at nginx.com Tue Oct 14 13:38:24 2014 From: vbart at nginx.com (Valentin V. Bartenev) Date: Tue, 14 Oct 2014 17:38:24 +0400 Subject: listen with ssl In-Reply-To: References: Message-ID: <1693520.Xe4Tsy0c1Y@vbart-workstation> On Tuesday 14 October 2014 16:28:17 Donatas Abraitis wrote: > Hello, > > is it possible to make nginx listen like this: > > 0.0.0.0:443 > 127.0.0.1:443 > > Because using such confuration I'm getting error bind() .. Address already > in use. > events {} http { server { listen 0.0.0.0:443 ssl; } server { listen 127.0.0.1:443 ssl; } } It works for me perfectly. You should consult with the documentation: http://nginx.org/r/listen And if it will not help, then please specify more details. wbr, Valentin V. Bartenev From kou at cozmixng.org Tue Oct 14 13:42:33 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Tue, 14 Oct 2014 22:42:33 +0900 (JST) Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141014002218.GH31276@mdounin.ru> References: <20141008142441.GE31276@mdounin.ru> <20141011.212902.1900691365943898058.kou@cozmixng.org> <20141014002218.GH31276@mdounin.ru> Message-ID: <20141014.224233.819691702702722336.kou@cozmixng.org> Hi, In <20141014002218.GH31276 at mdounin.ru> "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Tue, 14 Oct 2014 04:22:18 +0400, Maxim Dounin wrote: >> >> diff --git a/src/event/modules/ngx_iocp_module.c b/src/event/modules/ngx_iocp_module.c >> >> --- a/src/event/modules/ngx_iocp_module.c >> >> +++ b/src/event/modules/ngx_iocp_module.c >> ... >> > >> > Looks fine, though I think it should be a separate patch. >> >> I see! >> I did: http://mailman.nginx.org/pipermail/nginx-devel/2014-October/006072.html > > This and ngx_wsasend_chain.c patches committed (with minor changes > to commit logs), thanks. Thanks! >> Compiler: >> >> % x86_64-w64-mingw32-gcc --version >> x86_64-w64-mingw32-gcc (GCC) 4.9.1 > > While this patch fixes compilation with MinGW-w64 gcc, it breaks > compilation with MinGW gcc, which isn't good. Oh... Could you show me the build errors? I don't have MinGW on my machine. >> #include >> #include >> #include >> +#ifdef __GNUC__ >> +#include /* intptr_t and uintptr_t */ >> +#endif >> #include >> #include > > Minor nit: the comment here looks unneeded. OK. I removed it in the attached patch. >> typedef __int64 int64_t; >> typedef unsigned __int64 uint64_t; >> >> -#ifndef __WATCOMC__ >> +#if defined(__WATCOMC__) || defined(__GNUC__) >> +/* do nothing */ >> +#else >> typedef int intptr_t; >> typedef u_int uintptr_t; >> #endif > > The construction looks awfull, it should be > > #if !defined(...) && !defined(...) > ... > #endif > > instead. I used the style in the attached patch. > [...] > >> #define NGX_PTR_SIZE 4 >> #define NGX_SIZE_T_LEN (sizeof("-2147483648") - 1) >> #define NGX_MAX_SIZE_T_VALUE 2147483647 > > Note well: the ngx_win32_config.h defines various constants, > including ones quoted, assuming 32-bit build. 64-bit build will > be flawed unless these constands are changed appropriately. I confirmed size of them with the following change: diff -r 9d6d7b34f93b src/core/nginx.c --- a/src/core/nginx.c Tue Oct 14 22:36:23 2014 +0900 +++ b/src/core/nginx.c Tue Oct 14 22:36:23 2014 +0900 @@ -201,16 +201,21 @@ static char **ngx_os_environ; int ngx_cdecl main(int argc, char *const *argv) { ngx_int_t i; ngx_log_t *log; ngx_cycle_t *cycle, init_cycle; ngx_core_conf_t *ccf; + printf("%Iu:%Iu:%Iu\n", + sizeof(size_t), + sizeof(void *), + sizeof(time_t)); + ngx_debug_init(); if (ngx_strerror_init() != NGX_OK) { return 1; } if (ngx_get_options(argc, argv) != NGX_OK) { return 1; I run objs/nginx on Wine and got the following output: 8:8:8 So I added the following change to the attached patch: +#if defined(__GNUC__) && defined(_WIN64) +#define NGX_PTR_SIZE 8 +#define NGX_SIZE_T_LEN (sizeof("-9223372036854775808") - 1) +#define NGX_MAX_SIZE_T_VALUE 9223372036854775807 +#define NGX_TIME_T_LEN (sizeof("-9223372036854775808") - 1) +#define NGX_TIME_T_SIZE 8 +#else Could you review my new patch? Thanks, -- kou -------------- next part -------------- A non-text attachment was scrubbed... Name: win32-made-build-able-with-mingw-w64-v2.patch Type: text/x-patch Size: 7549 bytes Desc: not available URL: From mdounin at mdounin.ru Tue Oct 14 14:10:22 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 14 Oct 2014 18:10:22 +0400 Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141014.224233.819691702702722336.kou@cozmixng.org> References: <20141008142441.GE31276@mdounin.ru> <20141011.212902.1900691365943898058.kou@cozmixng.org> <20141014002218.GH31276@mdounin.ru> <20141014.224233.819691702702722336.kou@cozmixng.org> Message-ID: <20141014141022.GL31276@mdounin.ru> Hello! On Tue, Oct 14, 2014 at 10:42:33PM +0900, Kouhei Sutou wrote: > >> Compiler: > >> > >> % x86_64-w64-mingw32-gcc --version > >> x86_64-w64-mingw32-gcc (GCC) 4.9.1 > > > > While this patch fixes compilation with MinGW-w64 gcc, it breaks > > compilation with MinGW gcc, which isn't good. > > Oh... > Could you show me the build errors? I don't have MinGW on my > machine. It ends up using 32-bit off_t as it doesn't understand _FILE_OFFSET_BITS, and then build fails due to this. No exact error message as I've already reverted VM where MinGW gcc was installed to a previous state, sorry. [...] > # HG changeset patch > # User Kouhei Sutou > # Date 1413293783 -32400 > # Tue Oct 14 22:36:23 2014 +0900 > # Node ID 9d6d7b34f93bfc5363e16f0340a136a27755354b > # Parent 35b8e5e985083976ba62914fc5953dbd2a3b954b > Win32: made build-able with MinGW-w64 > > Compiler: > > % x86_64-w64-mingw32-gcc --version > x86_64-w64-mingw32-gcc (GCC) 4.9.1 > Copyright (C) 2014 Free Software Foundation, Inc. > This is free software; see the source for copying conditions. There is NO > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. [...] It should be fine to add a dot to the first line (and probably "gcc"), and drop everything else. I.e., Win32: made build-able with MinGW-w64 gcc. [...] > +#if defined(__GNUC__) && defined(_WIN64) Shouldn't it be for all compilers, not just gcc? > +#define NGX_PTR_SIZE 8 > +#define NGX_SIZE_T_LEN (sizeof("-9223372036854775808") - 1) > +#define NGX_MAX_SIZE_T_VALUE 9223372036854775807 > +#define NGX_TIME_T_LEN (sizeof("-9223372036854775808") - 1) > +#define NGX_TIME_T_SIZE 8 > +#else > #define NGX_PTR_SIZE 4 > #define NGX_SIZE_T_LEN (sizeof("-2147483648") - 1) > #define NGX_MAX_SIZE_T_VALUE 2147483647 > #define NGX_TIME_T_LEN (sizeof("-2147483648") - 1) > #define NGX_TIME_T_SIZE 4 > +#endif > #define NGX_OFF_T_LEN (sizeof("-9223372036854775807") - 1) > #define NGX_MAX_OFF_T_VALUE 9223372036854775807 > #define NGX_SIG_ATOMIC_T_SIZE 4 Style nit: this needs more empty lines before/after preprocessor directives. -- Maxim Dounin http://nginx.org/ From arut at nginx.com Tue Oct 14 14:39:32 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 14 Oct 2014 14:39:32 +0000 Subject: [nginx] Upstream: proxy_force_ranges and friends. Message-ID: details: http://hg.nginx.org/nginx/rev/2aff40b3dea2 branches: changeset: 5874:2aff40b3dea2 user: Roman Arutyunyan date: Tue Oct 14 18:32:01 2014 +0400 description: Upstream: proxy_force_ranges and friends. The directives enable byte ranges for both cached and uncached responses regardless of backend headers. diffstat: src/http/modules/ngx_http_fastcgi_module.c | 11 +++++++++++ src/http/modules/ngx_http_proxy_module.c | 11 +++++++++++ src/http/modules/ngx_http_scgi_module.c | 11 +++++++++++ src/http/modules/ngx_http_uwsgi_module.c | 11 +++++++++++ src/http/ngx_http_upstream.c | 15 +++++++++++++++ src/http/ngx_http_upstream.h | 1 + 6 files changed, 60 insertions(+), 0 deletions(-) diffs (183 lines): diff -r 35b8e5e98508 -r 2aff40b3dea2 src/http/modules/ngx_http_fastcgi_module.c --- a/src/http/modules/ngx_http_fastcgi_module.c Sat Oct 11 21:09:29 2014 +0900 +++ b/src/http/modules/ngx_http_fastcgi_module.c Tue Oct 14 18:32:01 2014 +0400 @@ -326,6 +326,13 @@ static ngx_command_t ngx_http_fastcgi_c offsetof(ngx_http_fastcgi_loc_conf_t, upstream.busy_buffers_size_conf), NULL }, + { ngx_string("fastcgi_force_ranges"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_fastcgi_loc_conf_t, upstream.force_ranges), + NULL }, + #if (NGX_HTTP_CACHE) { ngx_string("fastcgi_cache"), @@ -2332,6 +2339,7 @@ ngx_http_fastcgi_create_loc_conf(ngx_con conf->upstream.next_upstream_tries = NGX_CONF_UNSET_UINT; conf->upstream.buffering = NGX_CONF_UNSET; conf->upstream.ignore_client_abort = NGX_CONF_UNSET; + conf->upstream.force_ranges = NGX_CONF_UNSET; conf->upstream.local = NGX_CONF_UNSET_PTR; @@ -2413,6 +2421,9 @@ ngx_http_fastcgi_merge_loc_conf(ngx_conf ngx_conf_merge_value(conf->upstream.ignore_client_abort, prev->upstream.ignore_client_abort, 0); + ngx_conf_merge_value(conf->upstream.force_ranges, + prev->upstream.force_ranges, 0); + ngx_conf_merge_ptr_value(conf->upstream.local, prev->upstream.local, NULL); diff -r 35b8e5e98508 -r 2aff40b3dea2 src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c Sat Oct 11 21:09:29 2014 +0900 +++ b/src/http/modules/ngx_http_proxy_module.c Tue Oct 14 18:32:01 2014 +0400 @@ -389,6 +389,13 @@ static ngx_command_t ngx_http_proxy_com offsetof(ngx_http_proxy_loc_conf_t, upstream.busy_buffers_size_conf), NULL }, + { ngx_string("proxy_force_ranges"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_proxy_loc_conf_t, upstream.force_ranges), + NULL }, + #if (NGX_HTTP_CACHE) { ngx_string("proxy_cache"), @@ -2472,6 +2479,7 @@ ngx_http_proxy_create_loc_conf(ngx_conf_ conf->upstream.next_upstream_tries = NGX_CONF_UNSET_UINT; conf->upstream.buffering = NGX_CONF_UNSET; conf->upstream.ignore_client_abort = NGX_CONF_UNSET; + conf->upstream.force_ranges = NGX_CONF_UNSET; conf->upstream.local = NGX_CONF_UNSET_PTR; @@ -2568,6 +2576,9 @@ ngx_http_proxy_merge_loc_conf(ngx_conf_t ngx_conf_merge_value(conf->upstream.ignore_client_abort, prev->upstream.ignore_client_abort, 0); + ngx_conf_merge_value(conf->upstream.force_ranges, + prev->upstream.force_ranges, 0); + ngx_conf_merge_ptr_value(conf->upstream.local, prev->upstream.local, NULL); diff -r 35b8e5e98508 -r 2aff40b3dea2 src/http/modules/ngx_http_scgi_module.c --- a/src/http/modules/ngx_http_scgi_module.c Sat Oct 11 21:09:29 2014 +0900 +++ b/src/http/modules/ngx_http_scgi_module.c Tue Oct 14 18:32:01 2014 +0400 @@ -183,6 +183,13 @@ static ngx_command_t ngx_http_scgi_comma offsetof(ngx_http_scgi_loc_conf_t, upstream.busy_buffers_size_conf), NULL }, + { ngx_string("scgi_force_ranges"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_scgi_loc_conf_t, upstream.force_ranges), + NULL }, + #if (NGX_HTTP_CACHE) { ngx_string("scgi_cache"), @@ -1091,6 +1098,7 @@ ngx_http_scgi_create_loc_conf(ngx_conf_t conf->upstream.next_upstream_tries = NGX_CONF_UNSET_UINT; conf->upstream.buffering = NGX_CONF_UNSET; conf->upstream.ignore_client_abort = NGX_CONF_UNSET; + conf->upstream.force_ranges = NGX_CONF_UNSET; conf->upstream.local = NGX_CONF_UNSET_PTR; @@ -1167,6 +1175,9 @@ ngx_http_scgi_merge_loc_conf(ngx_conf_t ngx_conf_merge_value(conf->upstream.ignore_client_abort, prev->upstream.ignore_client_abort, 0); + ngx_conf_merge_value(conf->upstream.force_ranges, + prev->upstream.force_ranges, 0); + ngx_conf_merge_ptr_value(conf->upstream.local, prev->upstream.local, NULL); diff -r 35b8e5e98508 -r 2aff40b3dea2 src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c Sat Oct 11 21:09:29 2014 +0900 +++ b/src/http/modules/ngx_http_uwsgi_module.c Tue Oct 14 18:32:01 2014 +0400 @@ -238,6 +238,13 @@ static ngx_command_t ngx_http_uwsgi_comm offsetof(ngx_http_uwsgi_loc_conf_t, upstream.busy_buffers_size_conf), NULL }, + { ngx_string("uwsgi_force_ranges"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_uwsgi_loc_conf_t, upstream.force_ranges), + NULL }, + #if (NGX_HTTP_CACHE) { ngx_string("uwsgi_cache"), @@ -1271,6 +1278,7 @@ ngx_http_uwsgi_create_loc_conf(ngx_conf_ conf->upstream.next_upstream_tries = NGX_CONF_UNSET_UINT; conf->upstream.buffering = NGX_CONF_UNSET; conf->upstream.ignore_client_abort = NGX_CONF_UNSET; + conf->upstream.force_ranges = NGX_CONF_UNSET; conf->upstream.local = NGX_CONF_UNSET_PTR; @@ -1354,6 +1362,9 @@ ngx_http_uwsgi_merge_loc_conf(ngx_conf_t ngx_conf_merge_value(conf->upstream.ignore_client_abort, prev->upstream.ignore_client_abort, 0); + ngx_conf_merge_value(conf->upstream.force_ranges, + prev->upstream.force_ranges, 0); + ngx_conf_merge_ptr_value(conf->upstream.local, prev->upstream.local, NULL); diff -r 35b8e5e98508 -r 2aff40b3dea2 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Sat Oct 11 21:09:29 2014 +0900 +++ b/src/http/ngx_http_upstream.c Tue Oct 14 18:32:01 2014 +0400 @@ -2276,6 +2276,17 @@ ngx_http_upstream_process_headers(ngx_ht r->disable_not_modified = !u->cacheable; + if (u->conf->force_ranges) { + r->allow_ranges = 1; + r->single_range = 1; + +#if (NGX_HTTP_CACHE) + if (r->cached) { + r->single_range = 0; + } +#endif + } + u->length = -1; return NGX_OK; @@ -4393,6 +4404,10 @@ ngx_http_upstream_copy_allow_ranges(ngx_ { ngx_table_elt_t *ho; + if (r->upstream->conf->force_ranges) { + return NGX_OK; + } + #if (NGX_HTTP_CACHE) if (r->cached) { diff -r 35b8e5e98508 -r 2aff40b3dea2 src/http/ngx_http_upstream.h --- a/src/http/ngx_http_upstream.h Sat Oct 11 21:09:29 2014 +0900 +++ b/src/http/ngx_http_upstream.h Tue Oct 14 18:32:01 2014 +0400 @@ -162,6 +162,7 @@ typedef struct { ngx_flag_t ignore_client_abort; ngx_flag_t intercept_errors; ngx_flag_t cyclic_temp_file; + ngx_flag_t force_ranges; ngx_path_t *temp_path; From kou at cozmixng.org Tue Oct 14 15:23:33 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Wed, 15 Oct 2014 00:23:33 +0900 (JST) Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141014141022.GL31276@mdounin.ru> References: <20141014002218.GH31276@mdounin.ru> <20141014.224233.819691702702722336.kou@cozmixng.org> <20141014141022.GL31276@mdounin.ru> Message-ID: <20141015.002333.334908315950383141.kou@cozmixng.org> Hi, Thanks for reviewing. I've updated my patch. In <20141014141022.GL31276 at mdounin.ru> "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Tue, 14 Oct 2014 18:10:22 +0400, Maxim Dounin wrote: >> > While this patch fixes compilation with MinGW-w64 gcc, it breaks >> > compilation with MinGW gcc, which isn't good. >> >> Oh... >> Could you show me the build errors? I don't have MinGW on my >> machine. > > It ends up using 32-bit off_t as it doesn't understand > _FILE_OFFSET_BITS, and then build fails due to this. > > No exact error message as I've already reverted VM where MinGW gcc > was installed to a previous state, sorry. Thanks for the information. I changed my patch to apply for MinGW-w64 gcc. But there is no macro to distinguish MinGW-w64 gcc and MinGW gcc... So I added MGX_MINGW_W64 and MGX_MINGW macros by auto/define and use ngx_auto_config.h in ngx_win32_config.h. How do you think about this approach? Other ngx_*_config.h include ngx_auto_config.h after some "#include"s but now ngx_win32_config.h includes ngx_auto_config.h *before* some "#include"s. Because I want to use NGX_MINGW_W64 macro to change "#include". >> Win32: made build-able with MinGW-w64 > > It should be fine to add a dot to the first line (and probably > "gcc"), and drop everything else. I.e., > > Win32: made build-able with MinGW-w64 gcc. OK! I've changed it. >> +#if defined(__GNUC__) && defined(_WIN64) > > Shouldn't it be for all compilers, not just gcc? I don't know about it because I didn't check it on Visual C++... But I removed "defined(__GNUC__)" check. >> +#define NGX_PTR_SIZE 8 >> +#define NGX_SIZE_T_LEN (sizeof("-9223372036854775808") - 1) >> +#define NGX_MAX_SIZE_T_VALUE 9223372036854775807 >> +#define NGX_TIME_T_LEN (sizeof("-9223372036854775808") - 1) >> +#define NGX_TIME_T_SIZE 8 >> +#else >> #define NGX_PTR_SIZE 4 >> #define NGX_SIZE_T_LEN (sizeof("-2147483648") - 1) >> #define NGX_MAX_SIZE_T_VALUE 2147483647 >> #define NGX_TIME_T_LEN (sizeof("-2147483648") - 1) >> #define NGX_TIME_T_SIZE 4 >> +#endif >> #define NGX_OFF_T_LEN (sizeof("-9223372036854775807") - 1) >> #define NGX_MAX_OFF_T_VALUE 9223372036854775807 >> #define NGX_SIG_ATOMIC_T_SIZE 4 > > Style nit: this needs more empty lines before/after preprocessor > directives. It means the following? #ifdef XXX #define A #define B #else #define C #define D #endif Or the following? #ifdef XXX #define A #define B #else #define C #define D #endif Other codes use the latter in the file, so I used the latter style. Thanks, -- kou -------------- next part -------------- A non-text attachment was scrubbed... Name: win32-made-build-able-with-mingw-w64-v3.patch Type: text/x-patch Size: 9090 bytes Desc: not available URL: From mdounin at mdounin.ru Tue Oct 14 16:11:36 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 14 Oct 2014 20:11:36 +0400 Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141015.002333.334908315950383141.kou@cozmixng.org> References: <20141014002218.GH31276@mdounin.ru> <20141014.224233.819691702702722336.kou@cozmixng.org> <20141014141022.GL31276@mdounin.ru> <20141015.002333.334908315950383141.kou@cozmixng.org> Message-ID: <20141014161135.GM31276@mdounin.ru> Hello! On Wed, Oct 15, 2014 at 12:23:33AM +0900, Kouhei Sutou wrote: > Hi, > > Thanks for reviewing. I've updated my patch. > > In <20141014141022.GL31276 at mdounin.ru> > "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Tue, 14 Oct 2014 18:10:22 +0400, > Maxim Dounin wrote: > > >> > While this patch fixes compilation with MinGW-w64 gcc, it breaks > >> > compilation with MinGW gcc, which isn't good. > >> > >> Oh... > >> Could you show me the build errors? I don't have MinGW on my > >> machine. > > > > It ends up using 32-bit off_t as it doesn't understand > > _FILE_OFFSET_BITS, and then build fails due to this. > > > > No exact error message as I've already reverted VM where MinGW gcc > > was installed to a previous state, sorry. > > Thanks for the information. > I changed my patch to apply for MinGW-w64 gcc. > > But there is no macro to distinguish MinGW-w64 gcc and MinGW > gcc... So I added MGX_MINGW_W64 and MGX_MINGW macros by > auto/define and use ngx_auto_config.h in ngx_win32_config.h. > > How do you think about this approach? It doesn't look like a good approach for me. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Tue Oct 14 16:20:15 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 14 Oct 2014 16:20:15 +0000 Subject: [nginx] Style. Message-ID: details: http://hg.nginx.org/nginx/rev/d91b98232e4c branches: changeset: 5875:d91b98232e4c user: Maxim Dounin date: Tue Oct 14 20:19:36 2014 +0400 description: Style. diffstat: src/os/win32/ngx_wsasend_chain.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff --git a/src/os/win32/ngx_wsasend_chain.c b/src/os/win32/ngx_wsasend_chain.c --- a/src/os/win32/ngx_wsasend_chain.c +++ b/src/os/win32/ngx_wsasend_chain.c @@ -156,7 +156,8 @@ ngx_overlapped_wsasend_chain(ngx_connect /* the maximum limit size is the maximum u_long value - the page size */ - if (limit == 0 || limit > (off_t) (NGX_MAX_UINT32_VALUE - ngx_pagesize)) { + if (limit == 0 || limit > (off_t) (NGX_MAX_UINT32_VALUE - ngx_pagesize)) + { limit = NGX_MAX_UINT32_VALUE - ngx_pagesize; } From gunnlaugur at gmail.com Tue Oct 14 16:58:20 2014 From: gunnlaugur at gmail.com (Gunnlaugur Thor Briem) Date: Tue, 14 Oct 2014 16:58:20 +0000 Subject: [PATCH] Clarify meaning of limit_conn in SPDY connections Message-ID: <3674e10a9e22a622998b.1413305900@toodee.local> # HG changeset patch # User Gunnlaugur Thor Briem # Date 1413305660 0 # Tue Oct 14 16:54:20 2014 +0000 # Node ID 3674e10a9e22a622998b65badfe01da34579bb65 # Parent 2096ecf6de02bc9e8ae920c45c59bf6a4e2e38fb Clarify meaning of limit_conn in SPDY connections Be clear about limit_conn applying to concurrent requests, not connections. The distinction matters for SPDY connections. diff -r 2096ecf6de02 -r 3674e10a9e22 xml/en/docs/http/ngx_http_limit_conn_module.xml --- a/xml/en/docs/http/ngx_http_limit_conn_module.xml Mon Oct 13 17:32:03 2014 +0400 +++ b/xml/en/docs/http/ngx_http_limit_conn_module.xml Tue Oct 14 16:54:20 2014 +0000 @@ -62,7 +62,7 @@ Sets the shared memory zone -and the maximum allowed number of connections for a given key value. +and the maximum allowed number of concurrent requests for a given key value. When this limit is exceeded, the server will return the error in reply to a request. @@ -75,15 +75,22 @@ limit_conn addr 1; } -allow only one connection per an IP address at a time. +allow only one request per an IP address at a time. + +In SPDY connections, multiple requests may occur in parallel on the same +connection; in that case, limit_conn applies to the number +of parallel requests, not the number of connections (despite the name). In +non-SPDY HTTP/HTTPS connections, there is only one request at a time per +connection, so this distinction is not necessary. + When several limit_conn directives are specified, any configured limit will apply. For example, the following configuration will limit the number -of connections to the server per a client IP and, at the same time, -the total number of connections to the virtual host: +of concurrent requests to the server per a client IP and, at the same time, +the total number of concurrent requests to the virtual host: limit_conn_zone $binary_remote_addr zone=perip:10m; limit_conn_zone $server_name zone=perserver:10m; From kou at cozmixng.org Wed Oct 15 12:21:09 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Wed, 15 Oct 2014 21:21:09 +0900 (JST) Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141014161135.GM31276@mdounin.ru> References: <20141014141022.GL31276@mdounin.ru> <20141015.002333.334908315950383141.kou@cozmixng.org> <20141014161135.GM31276@mdounin.ru> Message-ID: <20141015.212109.839031045647992737.kou@cozmixng.org> Hi, In <20141014161135.GM31276 at mdounin.ru> "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Tue, 14 Oct 2014 20:11:36 +0400, Maxim Dounin wrote: >> But there is no macro to distinguish MinGW-w64 gcc and MinGW >> gcc... So I added MGX_MINGW_W64 and MGX_MINGW macros by >> auto/define and use ngx_auto_config.h in ngx_win32_config.h. >> >> How do you think about this approach? > > It doesn't look like a good approach for me. OK. I use another approach. I add only 64bit version MinGW-w64 GCC support. We can distinguish 64bit version MinGW-w64 GCC by the following expression: defined(__GNUC__) && defined(_WIN64) Here is a summary of GCC support for Windows build: The current code base (without patch): * MinGW GCC: Supported * MinGW-w64 GCC (32bit): Not supported * MinGW-w64 GCC (64bit): Not supported With the previous patch (rejected approach): * MinGW GCC: Supported * MinGW-w64 GCC (32bit): Supported * MinGW-w64 GCC (64bit): Supported With the attached patch: * MinGW GCC: Supported * MinGW-w64 GCC (32bit): Not supported * MinGW-w64 GCC (64bit): Supported Could you review it? Thanks, -- kou -------------- next part -------------- A non-text attachment was scrubbed... Name: win32-made-build-able-with-64bit-version-mingw-w64-v4.patch Type: text/x-patch Size: 7962 bytes Desc: not available URL: From mdounin at mdounin.ru Wed Oct 15 18:58:12 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 15 Oct 2014 18:58:12 +0000 Subject: [nginx] Gzip, gunzip: flush busy buffers if any. Message-ID: details: http://hg.nginx.org/nginx/rev/973fded4f461 branches: changeset: 5876:973fded4f461 user: Maxim Dounin date: Wed Oct 15 22:57:23 2014 +0400 description: Gzip, gunzip: flush busy buffers if any. Previous code resulted in transfer stalls when client happened to read all the data in buffers at once, while all gzip buffers were exhausted (but ctx->nomem wasn't set). Make sure to call next body filter at least once per call if there are busy buffers. Additionally, handling of calls with NULL chain was changed to follow the same logic, i.e., next body filter is only called with NULL chain if there are busy buffers. This is expected to fix "output chain is empty" alerts as reported by some users after c52a761a2029 (1.5.7). diffstat: src/http/modules/ngx_http_gunzip_filter_module.c | 10 ++++++++-- src/http/modules/ngx_http_gzip_filter_module.c | 10 ++++++++-- 2 files changed, 16 insertions(+), 4 deletions(-) diffs (96 lines): diff --git a/src/http/modules/ngx_http_gunzip_filter_module.c b/src/http/modules/ngx_http_gunzip_filter_module.c --- a/src/http/modules/ngx_http_gunzip_filter_module.c +++ b/src/http/modules/ngx_http_gunzip_filter_module.c @@ -175,6 +175,7 @@ static ngx_int_t ngx_http_gunzip_body_filter(ngx_http_request_t *r, ngx_chain_t *in) { int rc; + ngx_uint_t flush; ngx_chain_t *cl; ngx_http_gunzip_ctx_t *ctx; @@ -199,7 +200,7 @@ ngx_http_gunzip_body_filter(ngx_http_req } } - if (ctx->nomem || in == NULL) { + if (ctx->nomem) { /* flush busy buffers */ @@ -212,6 +213,10 @@ ngx_http_gunzip_body_filter(ngx_http_req ngx_chain_update_chains(r->pool, &ctx->free, &ctx->busy, &cl, (ngx_buf_tag_t) &ngx_http_gunzip_filter_module); ctx->nomem = 0; + flush = 0; + + } else { + flush = ctx->busy ? 1 : 0; } for ( ;; ) { @@ -258,7 +263,7 @@ ngx_http_gunzip_body_filter(ngx_http_req /* rc == NGX_AGAIN */ } - if (ctx->out == NULL) { + if (ctx->out == NULL && !flush) { return ctx->busy ? NGX_AGAIN : NGX_OK; } @@ -276,6 +281,7 @@ ngx_http_gunzip_body_filter(ngx_http_req "gunzip out: %p", ctx->out); ctx->nomem = 0; + flush = 0; if (ctx->done) { return rc; diff --git a/src/http/modules/ngx_http_gzip_filter_module.c b/src/http/modules/ngx_http_gzip_filter_module.c --- a/src/http/modules/ngx_http_gzip_filter_module.c +++ b/src/http/modules/ngx_http_gzip_filter_module.c @@ -316,6 +316,7 @@ static ngx_int_t ngx_http_gzip_body_filter(ngx_http_request_t *r, ngx_chain_t *in) { int rc; + ngx_uint_t flush; ngx_chain_t *cl; ngx_http_gzip_ctx_t *ctx; @@ -372,7 +373,7 @@ ngx_http_gzip_body_filter(ngx_http_reque r->connection->buffered |= NGX_HTTP_GZIP_BUFFERED; } - if (ctx->nomem || in == NULL) { + if (ctx->nomem) { /* flush busy buffers */ @@ -385,6 +386,10 @@ ngx_http_gzip_body_filter(ngx_http_reque ngx_chain_update_chains(r->pool, &ctx->free, &ctx->busy, &cl, (ngx_buf_tag_t) &ngx_http_gzip_filter_module); ctx->nomem = 0; + flush = 0; + + } else { + flush = ctx->busy ? 1 : 0; } for ( ;; ) { @@ -432,7 +437,7 @@ ngx_http_gzip_body_filter(ngx_http_reque /* rc == NGX_AGAIN */ } - if (ctx->out == NULL) { + if (ctx->out == NULL && !flush) { ngx_http_gzip_filter_free_copy_buf(r, ctx); return ctx->busy ? NGX_AGAIN : NGX_OK; @@ -457,6 +462,7 @@ ngx_http_gzip_body_filter(ngx_http_reque ctx->last_out = &ctx->out; ctx->nomem = 0; + flush = 0; if (ctx->done) { return rc; From mdounin at mdounin.ru Fri Oct 17 14:45:12 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 17 Oct 2014 18:45:12 +0400 Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141015.212109.839031045647992737.kou@cozmixng.org> References: <20141014141022.GL31276@mdounin.ru> <20141015.002333.334908315950383141.kou@cozmixng.org> <20141014161135.GM31276@mdounin.ru> <20141015.212109.839031045647992737.kou@cozmixng.org> Message-ID: <20141017144512.GF35211@mdounin.ru> Hello! On Wed, Oct 15, 2014 at 09:21:09PM +0900, Kouhei Sutou wrote: > Hi, > > In <20141014161135.GM31276 at mdounin.ru> > "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Tue, 14 Oct 2014 20:11:36 +0400, > Maxim Dounin wrote: > > >> But there is no macro to distinguish MinGW-w64 gcc and MinGW > >> gcc... So I added MGX_MINGW_W64 and MGX_MINGW macros by > >> auto/define and use ngx_auto_config.h in ngx_win32_config.h. > >> > >> How do you think about this approach? > > > > It doesn't look like a good approach for me. > > OK. I use another approach. > > I add only 64bit version MinGW-w64 GCC support. We can > distinguish 64bit version MinGW-w64 GCC by the following > expression: > > defined(__GNUC__) && defined(_WIN64) The MinGW-w64 wiki suggests the __MINGW64_VERSION_MAJOR macro can be used to distinguish MinGW-w64 and MinGW: http://sourceforge.net/p/mingw-w64/wiki2/Answer%20Check%20For%20Mingw-w64/ -- Maxim Dounin http://nginx.org/ From kou at cozmixng.org Fri Oct 17 15:42:50 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Sat, 18 Oct 2014 00:42:50 +0900 (JST) Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141017144512.GF35211@mdounin.ru> References: <20141014161135.GM31276@mdounin.ru> <20141015.212109.839031045647992737.kou@cozmixng.org> <20141017144512.GF35211@mdounin.ru> Message-ID: <20141018.004250.1469014097857676892.kou@cozmixng.org> Hi, In <20141017144512.GF35211 at mdounin.ru> "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Fri, 17 Oct 2014 18:45:12 +0400, Maxim Dounin wrote: >> I add only 64bit version MinGW-w64 GCC support. We can >> distinguish 64bit version MinGW-w64 GCC by the following >> expression: >> >> defined(__GNUC__) && defined(_WIN64) > > The MinGW-w64 wiki suggests the __MINGW64_VERSION_MAJOR macro can be > used to distinguish MinGW-w64 and MinGW: > > http://sourceforge.net/p/mingw-w64/wiki2/Answer%20Check%20For%20Mingw-w64/ Thanks for the information. I don't know about it. (I just checked only pre-defined macros by "x86_64-w64-mingw32-gcc -dM -xc -E /dev/null"...) I confirmed the the approach can work with both 32bit version and 64bit version MinGW-w64 GCCs. Could you merge the attached patch? Thanks, -- kou -------------- next part -------------- A non-text attachment was scrubbed... Name: win32-made-build-able-with-mingw-w64-v5.patch Type: text/x-patch Size: 8089 bytes Desc: not available URL: From peter at lekensteyn.nl Sun Oct 19 13:15:07 2014 From: peter at lekensteyn.nl (Peter Wu) Date: Sun, 19 Oct 2014 15:15:07 +0200 Subject: About the UNIX socket path length check In-Reply-To: <20141009164137.GT61641@lo0.su> References: <1425960.2AC8navbJM@al> <20141009164137.GT61641@lo0.su> Message-ID: <4490478.6EJpHRWj5Q@al> On Thursday 09 October 2014 20:41:37 Ruslan Ermilov wrote: > On Thu, Oct 09, 2014 at 06:02:16PM +0200, Peter Wu wrote: > > Hi, > > > > In src/core/ngx_inet.c we have this code: > > > > if (len > sizeof(saun->sun_path)) { > > u->err = "too long path in the unix domain socket"; > > return NGX_ERROR; > > } > > > > But a NUL-terminated string requires one byte more. Is it allowed to > > omit the NUL byte when the string would not fit otherwise? If not, > > this should be changed to: > > > > if (len >= sizeof(saun->sun_path)) { > > u->err = "too long path in the unix domain socket"; > > return NGX_ERROR; > > } > > The "len" you quote includes the slot for NUL due to > > u->host.len = len++; > > And ngx_strncpy() always NULL-terminates what it writes. Thank you and Maxim for the clarification, I missed the len++. Peter From peter at lekensteyn.nl Sun Oct 19 14:21:23 2014 From: peter at lekensteyn.nl (Peter Wu) Date: Sun, 19 Oct 2014 16:21:23 +0200 Subject: [PATCH] Contrib: add more directives to vim syntax Message-ID: <2127392.887hlZJyHY@al> Hi, While editing a config file I noticed that ssl_stapling was not highlighted. Please find the attached patch to update the directives in the vim syntax highlighting file. The script to extract all directives is also attached. I also compared the output against the list on http://nginx.org/en/docs/dirindex.html and noticed that some directives are missing on that page, and that some typos are present (for example, fascgi_force_ranges instead of "fastcgi_force_ranges"). Kind regards, Peter Wu https://lekensteyn.nl/ -------------- next part -------------- A non-text attachment was scrubbed... Name: Contrib__add_more_directives_to_vim_syntax.patch Type: text/x-patch Size: 24881 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: find-directives.sh Type: application/x-shellscript Size: 1381 bytes Desc: not available URL: From piotr at cloudflare.com Mon Oct 20 09:56:29 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Mon, 20 Oct 2014 02:56:29 -0700 Subject: [PATCH] Upstream: add $upstream_cache_age variable In-Reply-To: <68F0B9EB-6B1E-4EFC-878B-75B59CB07872@nginx.com> References: <8CF786D5-0834-4D2A-B7CB-BBA96A16293A@nginx.com> <68F0B9EB-6B1E-4EFC-878B-75B59CB07872@nginx.com> Message-ID: Hey Roman, sorry for late reply. > Another issue i?ve found reading the RFC. > > If the reason for $upstream_cache_age is adding HTTP Age response header then it makes > sense to improve age calculation algorithm. It is and it isn't ;) There are two use cases for this variable: 1. Exposing it as "Age" or "X-Cache-Age" HTTP response header. The RFC describes "Age" as the time that passed since the response was generated at the origin server, taking into account the information from the intermediary proxies (in practice - just the previous one). As you've already noticed, this variable shows time that passed since the response was received (or revalidated) and ignores information from the intermediary proxies, so using it for as the "Age" header isn't strictly RFC-conformant, but it's "good enough" and it's actually what most people we've been dealing with expect to see. 2. Logging of the cached file's age to the access logs. The more information you have in the logs when you need them, the better... That should go without saying ;) Best regards, Piotr Sikora From flevionnois at gmail.com Tue Oct 21 07:59:04 2014 From: flevionnois at gmail.com (Franck Levionnois) Date: Tue, 21 Oct 2014 09:59:04 +0200 Subject: [PATCH] SSL support for the mail proxy module In-Reply-To: <1497581210.2187358.1410913427096.JavaMail.zimbra@zimbra.com> References: <5c2524403ab7c870b1fa.1410650071@zdev-vm048.eng.zimbra.com> <8B3FA009-99B4-43AD-A207-5BE41FA58ECD@gmail.com> <607846621.1950104.1410852617470.JavaMail.zimbra@zimbra.com> <20140916120306.GF59236@mdounin.ru> <897609716.2187306.1410913349045.JavaMail.zimbra@zimbra.com> <1497581210.2187358.1410913427096.JavaMail.zimbra@zimbra.com> Message-ID: Hello, The patch below has been submited some months ago. It do about the same, and it support to return a name by the auth script. Kind regards Franck Levionnois. ---------- Forwarded message ---------- From: Date: 2014-01-24 21:40 GMT+01:00 Subject: [PATCH 1 of 1] Mail: added support for SSL client certificate To: nginx-devel at nginx.org # HG changeset patch # User Franck Levionnois # Date 1390577176 -3600 # Fri Jan 24 16:26:16 2014 +0100 # Node ID d7b8381c200e300c2b6729574f4c2a b537804f56 # Parent a387ce36744aa36b50e8171dbf01ef716748327e Mail: added support for SSL client certificate Add support for SSL module like HTTP. Added mail configuration directives (like http): ssl_verify_client, ssl_verify_depth, ssl_client_certificate, ssl_trusted_certificate, ssl_crl Added headers: Auth-Certificate, Auth-Certificate-Verify, Auth-Issuer-DN, Auth-Subject-DN, Auth-Subject-Serial diff -r a387ce36744a -r d7b8381c200e src/mail/ngx_mail_auth_http_module.c --- a/src/mail/ngx_mail_auth_http_module.c Thu Jan 23 22:09:59 2014 +0900 +++ b/src/mail/ngx_mail_auth_http_module.c Fri Jan 24 16:26:16 2014 +0100 @@ -1135,6 +1135,32 @@ "mail auth http dummy handler"); } +#if (NGX_MAIL_SSL) +static ngx_int_t +ngx_ssl_get_certificate_oneline(ngx_connection_t *c, ngx_pool_t *pool, + ngx_str_t *b64_cert) +{ + ngx_str_t pemCert; + if (ngx_ssl_get_raw_certificate(c, pool, &pemCert) != NGX_OK) { + return NGX_ERROR; + } + + if (pemCert.len == 0) { + b64_cert->len = 0; + return NGX_OK; + } + + b64_cert->len = ngx_base64_encoded_length(pemCert.len); + b64_cert->data = ngx_palloc( pool, b64_cert->len); + if (b64_cert->data == NULL) { + b64_cert->len = 0; + return NGX_ERROR; + } + ngx_encode_base64(b64_cert, &pemCert); + + return NGX_OK; +} +#endif static ngx_buf_t * ngx_mail_auth_http_create_request(ngx_mail_session_t *s, ngx_pool_t *pool, @@ -1142,7 +1168,9 @@ { size_t len; ngx_buf_t *b; - ngx_str_t login, passwd; + ngx_str_t login, passwd, client_cert, client_verify, + client_subject, client_issuer, + client_serial; ngx_mail_core_srv_conf_t *cscf; if (ngx_mail_auth_http_escape(pool, &s->login, &login) != NGX_OK) { @@ -1155,6 +1183,42 @@ cscf = ngx_mail_get_module_srv_conf(s, ngx_mail_core_module); +#if (NGX_MAIL_SSL) + if (s->connection->ssl) { + if (ngx_ssl_get_client_verify(s->connection, pool, + &client_verify) != NGX_OK) { + return NULL; + } + + if (ngx_ssl_get_subject_dn(s->connection, pool, + &client_subject) != NGX_OK) { + return NULL; + } + + if (ngx_ssl_get_issuer_dn(s->connection, pool, + &client_issuer) != NGX_OK) { + return NULL; + } + + if (ngx_ssl_get_serial_number(s->connection, pool, + &client_serial) != NGX_OK) { + return NULL; + } + + if (ngx_ssl_get_certificate_oneline(s->connection, pool, + &client_cert) != NGX_OK) { + return NULL; + } + } else { + client_verify.len = 0; + client_issuer.len = 0; + client_subject.len = 0; + client_serial.len = 0; + client_cert.len = 0; + } + +#endif + len = sizeof("GET ") - 1 + ahcf->uri.len + sizeof(" HTTP/1.0" CRLF) - 1 + sizeof("Host: ") - 1 + ahcf->host_header.len + sizeof(CRLF) - 1 + sizeof("Auth-Method: ") - 1 @@ -1163,6 +1227,18 @@ + sizeof("Auth-User: ") - 1 + login.len + sizeof(CRLF) - 1 + sizeof("Auth-Pass: ") - 1 + passwd.len + sizeof(CRLF) - 1 + sizeof("Auth-Salt: ") - 1 + s->salt.len +#if (NGX_MAIL_SSL) + + sizeof("Auth-Certificate: ") - 1 + client_cert.len + + sizeof(CRLF) - 1 + + sizeof("Auth-Certificate-Verify: ") - 1 + client_verify.len + + sizeof(CRLF) - 1 + + sizeof("Auth-Issuer-DN: ") - 1 + client_issuer.len + + sizeof(CRLF) - 1 + + sizeof("Auth-Subject-DN: ") - 1 + client_subject.len + + sizeof(CRLF) - 1 + + sizeof("Auth-Subject-Serial: ") - 1 + client_serial.len + + sizeof(CRLF) - 1 +#endif + sizeof("Auth-Protocol: ") - 1 + cscf->protocol->name.len + sizeof(CRLF) - 1 + sizeof("Auth-Login-Attempt: ") - 1 + NGX_INT_T_LEN @@ -1212,7 +1288,43 @@ s->passwd.data = NULL; } - +#if (NGX_MAIL_SSL) + if ( client_cert.len ) + { + b->last = ngx_cpymem(b->last, "Auth-Certificate: ", + sizeof("Auth-Certificate: ") - 1); + b->last = ngx_copy(b->last, client_cert.data, client_cert.len); + *b->last++ = CR; *b->last++ = LF; + } + if ( client_verify.len ) + { + b->last = ngx_cpymem(b->last, "Auth-Certificate-Verify: ", + sizeof("Auth-Certificate-Verify: ") - 1); + b->last = ngx_copy(b->last, client_verify.data, client_verify.len); + *b->last++ = CR; *b->last++ = LF; + } + if ( client_issuer.len ) + { + b->last = ngx_cpymem(b->last, "Auth-Issuer-DN: ", + sizeof("Auth-Issuer-DN: ") - 1); + b->last = ngx_copy(b->last, client_issuer.data, client_issuer.len); + *b->last++ = CR; *b->last++ = LF; + } + if ( client_subject.len ) + { + b->last = ngx_cpymem(b->last, "Auth-Subject-DN: ", + sizeof("Auth-Subject-DN: ") - 1); + b->last = ngx_copy(b->last, client_subject.data, client_subject.len); + *b->last++ = CR; *b->last++ = LF; + } + if ( client_serial.len ) + { + b->last = ngx_cpymem(b->last, "Auth-Subject-Serial: ", + sizeof("Auth-Subject-Serial: ") - 1); + b->last = ngx_copy(b->last, client_serial.data, client_serial.len); + *b->last++ = CR; *b->last++ = LF; + } +#endif b->last = ngx_cpymem(b->last, "Auth-Protocol: ", sizeof("Auth-Protocol: ") - 1); b->last = ngx_cpymem(b->last, cscf->protocol->name.data, diff -r a387ce36744a -r d7b8381c200e src/mail/ngx_mail_handler.c --- a/src/mail/ngx_mail_handler.c Thu Jan 23 22:09:59 2014 +0900 +++ b/src/mail/ngx_mail_handler.c Fri Jan 24 16:26:16 2014 +0100 @@ -236,11 +236,59 @@ { ngx_mail_session_t *s; ngx_mail_core_srv_conf_t *cscf; +#if (NGX_MAIL_SSL) + ngx_mail_ssl_conf_t *sslcf; +#endif + + ngx_log_debug1(NGX_LOG_DEBUG_MAIL, c->log, 0, + "ngx_mail_ssl_handshake_handler handshaked: %d ", + c->ssl->handshaked ); if (c->ssl->handshaked) { s = c->data; +#if (NGX_MAIL_SSL) + sslcf = ngx_mail_get_module_srv_conf(s, ngx_mail_ssl_module); + if (sslcf->verify) { + long rc; + + rc = SSL_get_verify_result(c->ssl->connection); + + if (rc != X509_V_OK + && (sslcf->verify != 3 || !ngx_ssl_verify_error_optional(rc))) + { + ngx_log_error(NGX_LOG_INFO, c->log, 0, + "client SSL certificate verify error: (%l:%s)", + rc, X509_verify_cert_error_string(rc)); + + ngx_ssl_remove_cached_session(sslcf->ssl.ctx, + (SSL_get0_session(c->ssl->connection))); + + ngx_mail_close_connection(c); + return; + } + + if (sslcf->verify == 1) { + X509 *cert; + cert = SSL_get_peer_certificate(c->ssl->connection); + + if (cert == NULL) { + ngx_log_error(NGX_LOG_INFO, c->log, 0, + "client sent no required SSL certificate"); + + ngx_ssl_remove_cached_session(sslcf->ssl.ctx, + (SSL_get0_session(c->ssl->connection))); + + ngx_mail_close_connection(c); + return; + } + + X509_free(cert); + } + } +#endif + if (s->starttls) { cscf = ngx_mail_get_module_srv_conf(s, ngx_mail_core_module); @@ -276,6 +324,10 @@ s->protocol = cscf->protocol->type; + ngx_log_debug1(NGX_LOG_DEBUG_MAIL, c->log, 0, + "ngx_mail_init_session protocol: %d ", + cscf->protocol->type ); + s->ctx = ngx_pcalloc(c->pool, sizeof(void *) * ngx_mail_max_module); if (s->ctx == NULL) { ngx_mail_session_internal_server_error(s); diff -r a387ce36744a -r d7b8381c200e src/mail/ngx_mail_ssl_module.c --- a/src/mail/ngx_mail_ssl_module.c Thu Jan 23 22:09:59 2014 +0900 +++ b/src/mail/ngx_mail_ssl_module.c Fri Jan 24 16:26:16 2014 +0100 @@ -43,6 +43,13 @@ { ngx_null_string, 0 } }; +static ngx_conf_enum_t ngx_mail_ssl_verify[] = { + { ngx_string("off"), 0 }, + { ngx_string("on"), 1 }, + { ngx_string("optional"), 2 }, + { ngx_string("optional_no_ca"), 3 }, + { ngx_null_string, 0 } +}; static ngx_command_t ngx_mail_ssl_commands[] = { @@ -102,6 +109,34 @@ offsetof(ngx_mail_ssl_conf_t, ciphers), NULL }, + { ngx_string("ssl_verify_client"), + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_enum_slot, + NGX_MAIL_SRV_CONF_OFFSET, + offsetof(ngx_mail_ssl_conf_t, verify), + &ngx_mail_ssl_verify }, + + { ngx_string("ssl_verify_depth"), + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_1MORE, + ngx_conf_set_num_slot, + NGX_MAIL_SRV_CONF_OFFSET, + offsetof(ngx_mail_ssl_conf_t, verify_depth), + NULL }, + + { ngx_string("ssl_client_certificate"), + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_MAIL_SRV_CONF_OFFSET, + offsetof(ngx_mail_ssl_conf_t, client_certificate), + NULL }, + + { ngx_string("ssl_trusted_certificate"), + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_MAIL_SRV_CONF_OFFSET, + offsetof(ngx_mail_ssl_conf_t, trusted_certificate), + NULL }, + { ngx_string("ssl_prefer_server_ciphers"), NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_FLAG, ngx_conf_set_flag_slot, @@ -137,6 +172,13 @@ offsetof(ngx_mail_ssl_conf_t, session_timeout), NULL }, + { ngx_string("ssl_crl"), + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_MAIL_SRV_CONF_OFFSET, + offsetof(ngx_mail_ssl_conf_t, crl), + NULL }, + ngx_null_command }; @@ -196,6 +238,8 @@ scf->enable = NGX_CONF_UNSET; scf->starttls = NGX_CONF_UNSET_UINT; scf->prefer_server_ciphers = NGX_CONF_UNSET; + scf->verify = NGX_CONF_UNSET_UINT; + scf->verify_depth = NGX_CONF_UNSET_UINT; scf->builtin_session_cache = NGX_CONF_UNSET; scf->session_timeout = NGX_CONF_UNSET; scf->session_tickets = NGX_CONF_UNSET; @@ -228,11 +272,20 @@ (NGX_CONF_BITMASK_SET|NGX_SSL_SSLv3|NGX_SSL_TLSv1 |NGX_SSL_TLSv1_1|NGX_SSL_TLSv1_2)); + ngx_conf_merge_uint_value(conf->verify, prev->verify, 0); + ngx_conf_merge_uint_value(conf->verify_depth, prev->verify_depth, 1); + ngx_conf_merge_str_value(conf->certificate, prev->certificate, ""); ngx_conf_merge_str_value(conf->certificate_key, prev->certificate_key, ""); ngx_conf_merge_str_value(conf->dhparam, prev->dhparam, ""); + ngx_conf_merge_str_value(conf->client_certificate, prev->client_certificate, + ""); + ngx_conf_merge_str_value(conf->trusted_certificate, + prev->trusted_certificate, ""); + ngx_conf_merge_str_value(conf->crl, prev->crl, ""); + ngx_conf_merge_str_value(conf->ecdh_curve, prev->ecdh_curve, NGX_DEFAULT_ECDH_CURVE); @@ -318,6 +371,35 @@ return NGX_CONF_ERROR; } + if (conf->verify) { + + if (conf->client_certificate.len == 0 && conf->verify != 3) { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "no ssl_client_certificate for ssl_client_verify"); + return NGX_CONF_ERROR; + } + + if (ngx_ssl_client_certificate(cf, &conf->ssl, + &conf->client_certificate, + conf->verify_depth) + != NGX_OK) + { + return NGX_CONF_ERROR; + } + } + + if (ngx_ssl_trusted_certificate(cf, &conf->ssl, + &conf->trusted_certificate, + conf->verify_depth) + != NGX_OK) + { + return NGX_CONF_ERROR; + } + + if (ngx_ssl_crl(cf, &conf->ssl, &conf->crl) != NGX_OK) { + return NGX_CONF_ERROR; + } + if (conf->prefer_server_ciphers) { SSL_CTX_set_options(conf->ssl.ctx, SSL_OP_CIPHER_SERVER_PREFERENCE); } diff -r a387ce36744a -r d7b8381c200e src/mail/ngx_mail_ssl_module.h --- a/src/mail/ngx_mail_ssl_module.h Thu Jan 23 22:09:59 2014 +0900 +++ b/src/mail/ngx_mail_ssl_module.h Fri Jan 24 16:26:16 2014 +0100 @@ -28,6 +28,8 @@ ngx_uint_t starttls; ngx_uint_t protocols; + ngx_uint_t verify; + ngx_uint_t verify_depth; ssize_t builtin_session_cache; time_t session_timeout; @@ -36,6 +38,9 @@ ngx_str_t certificate_key; ngx_str_t dhparam; ngx_str_t ecdh_curve; + ngx_str_t client_certificate; + ngx_str_t trusted_certificate; + ngx_str_t crl; ngx_str_t ciphers; 2014-09-17 2:23 GMT+02:00 Kunal Pariani : > I guess these diffs can still be applied with limitation of having the > same ssl setting for all upstreams until the support to return a name by > the auth script is added later ? > > Thanks > -Kunal > > ----- Original Message ----- > From: "Maxim Dounin" > To: "nginx-devel" > Sent: Tuesday, September 16, 2014 5:03:06 AM > Subject: Re: [PATCH] SSL support for the mail proxy module > > Hello! > > On Tue, Sep 16, 2014 at 02:30:17AM -0500, Kunal Pariani wrote: > > > Updated the diffs after addressing the first 2 issues. > > Regarding the 3rd comment, you are correct. Only 1 set of ssl > > settings for all the mail backends with my patch. I guess this > > will be a limitation with the current mail proxy workflow ? Am > > not sure of the exact changes that will be required to address > > this issue completely. > > Probably, returning a name by the auth script is a way to go. > > -- > Maxim Dounin > http://nginx.org/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From piotr at cloudflare.com Tue Oct 21 11:16:01 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 21 Oct 2014 04:16:01 -0700 Subject: [PATCH] SPDY: stop emitting empty header values Message-ID: # HG changeset patch # User Piotr Sikora # Date 1413890111 25200 # Tue Oct 21 04:15:11 2014 -0700 # Node ID fbc2eb84bd266b64644b00a6454c4f79f241af5a # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 SPDY: stop emitting empty header values. Previously, nginx would emit empty values in a header with multiple, NULL-separated values. This is forbidden by the SPDY specification, which requires headers to have either a single (possibly empty) value or multiple, NULL-separated non-empty values. Signed-off-by: Piotr Sikora diff -r 973fded4f461 -r fbc2eb84bd26 src/http/ngx_http_spdy_filter_module.c --- a/src/http/ngx_http_spdy_filter_module.c Wed Oct 15 22:57:23 2014 +0400 +++ b/src/http/ngx_http_spdy_filter_module.c Tue Oct 21 04:15:11 2014 -0700 @@ -100,7 +100,7 @@ ngx_http_spdy_header_filter(ngx_http_req u_char *p, *buf, *last; ngx_buf_t *b; ngx_str_t host; - ngx_uint_t i, j, count, port; + ngx_uint_t i, j, count, port, sep; ngx_chain_t *cl; ngx_list_part_t *part, *pt; ngx_table_elt_t *header, *h; @@ -472,6 +472,7 @@ ngx_http_spdy_header_filter(ngx_http_req pt = part; h = header; + sep = header[i].value.len ? 1 : 0; for (j = i + 1; /* void */; j++) { @@ -493,9 +494,14 @@ ngx_http_spdy_header_filter(ngx_http_req continue; } - *last++ = '\0'; + if (h[j].value.len) { + if (sep) { + *last++ = '\0'; + } - last = ngx_cpymem(last, h[j].value.data, h[j].value.len); + last = ngx_cpymem(last, h[j].value.data, h[j].value.len); + sep = 1; + } h[j].hash = 2; } From mdounin at mdounin.ru Wed Oct 22 06:04:45 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 22 Oct 2014 10:04:45 +0400 Subject: [PATCH] Contrib: add more directives to vim syntax In-Reply-To: <2127392.887hlZJyHY@al> References: <2127392.887hlZJyHY@al> Message-ID: <20141022060445.GB44913@mdounin.ru> Hello! On Sun, Oct 19, 2014 at 04:21:23PM +0200, Peter Wu wrote: > Hi, > > While editing a config file I noticed that ssl_stapling was not highlighted. > Please find the attached patch to update the directives in the vim syntax > highlighting file. Thanks, it needs refreshing. > The script to extract all directives is also attached. I also compared the > output against the list on http://nginx.org/en/docs/dirindex.html and noticed > that some directives are missing on that page, Some directives are undocumented on purpose. E.g., acceptex_read directive you've added in your patch is from never worked unfinished acceptex event handling module. > and that some typos are present > (for example, fascgi_force_ranges instead of "fastcgi_force_ranges"). This means there is a typo in the directive name in docs. This on fixed, thanks: http://hg.nginx.org/nginx.org/rev/c63717ebc5bc (BTW, patches for docs are also welcome.) > # HG changeset patch > # User Peter Wu > # Date 1413727713 -7200 > # Sun Oct 19 16:08:33 2014 +0200 > # Node ID 052ec6744ffb2cce94c312a8a62d3ddc650dfe7d > # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 > Contrib: add more directives to vim syntax Please add a trailing dot here. > > uwsgi is not a third-party module anymore. 'split_clients' is a new > block directive type. The directives have automatically been extracted > from ngx_command_t arrays and are manually reviewed. > > Trailing spaces got removed at some places. > > Also mention an alternative path for the vim syntax files (which is in > use on Arch Linux). > > diff -r 973fded4f461 -r 052ec6744ffb contrib/README > --- a/contrib/README Wed Oct 15 22:57:23 2014 +0400 > +++ b/contrib/README Sun Oct 19 16:08:33 2014 +0200 > @@ -17,5 +17,5 @@ unicode2nginx by Maxim Dounin > vim by Evan Miller > > Syntax highlighting of nginx configuration for vim, to be > - placed into ~/.vim/. > + placed into ~/.vim/ or /usr/share/vim/vimfiles/. I would like to avoid platform-specific paths here. [...] > syn keyword ngxDirective epoll_events > syn keyword ngxDirective error_log > +syn keyword ngxDirective error_log We probably don't wont duplicate directives here and in other places. > +syn keyword ngxDirective etag > syn keyword ngxDirective eventport_events > syn keyword ngxDirective expires > syn keyword ngxDirective fastcgi_bind > +syn keyword ngxDirective fastcgi_buffering > +syn keyword ngxDirective fastcgi_buffers > syn keyword ngxDirective fastcgi_buffer_size > -syn keyword ngxDirective fastcgi_buffers The change in order looks strange, in lowercase fastcgi_buffer_size should be before fastcgi_buffers and fastcgi_buffering. [...] -- Maxim Dounin http://nginx.org/ From alexx.todorov at gmail.com Wed Oct 22 09:26:56 2014 From: alexx.todorov at gmail.com (Alexander Todorov) Date: Wed, 22 Oct 2014 12:26:56 +0300 Subject: What is the order of module execution and how can I change it? Message-ID: <54477860.5040600@gmail.com> Hello folks, I'm looking into nginx for developing custom modules and I'm missing some info related to the order of module execution. At http://www.evanmiller.org/nginx-modules-guide.html it says: " The order of their execution is determined at compile-time. Filters have the classic "CHAIN OF RESPONSIBILITY" design pattern: one filter is called, does its work, and then calls the next filter, until the final filter is called, and Nginx finishes up the response. " Not a mention about non-filter modules. Then when trying to build nginx from source I see a objs/ngx_modules.c file with: ngx_module_t *ngx_modules[] = { &ngx_core_module, &ngx_errlog_module, &ngx_conf_module, &ngx_events_module, &ngx_event_core_module, &ngx_epoll_module, &ngx_regex_module, &ngx_http_module, &ngx_http_core_module, &ngx_http_log_module, ... skip ... &ngx_http_headers_filter_module, &ngx_http_copy_filter_module, &ngx_http_range_body_filter_module, &ngx_http_not_modified_filter_module, NULL }; This looks like the order of execution of all modules, is this correct ? If so, where can I place my modules in this list ? Only at the end or at arbitrary positions? How do I do that (point me to docs is fine) ? For example I'd like to have a custom logging module which executes after ngx_http_log_module saving results to a DB for further analysis and BI purposes. Another example is if I'd want to minify HTML content (a filter module) how do I do that before gzip kicks in? Thanks for any comments or helpers. -- Alex From peter at lekensteyn.nl Wed Oct 22 13:39:31 2014 From: peter at lekensteyn.nl (Peter Wu) Date: Wed, 22 Oct 2014 15:39:31 +0200 Subject: [PATCH] Contrib: add more directives to vim syntax In-Reply-To: <20141022060445.GB44913@mdounin.ru> References: <2127392.887hlZJyHY@al> <20141022060445.GB44913@mdounin.ru> Message-ID: <3664659.SkH2gXEf4o@al> On Wednesday 22 October 2014 10:04:45 Maxim Dounin wrote: > > The script to extract all directives is also attached. I also compared the > > output against the list on http://nginx.org/en/docs/dirindex.html and noticed > > that some directives are missing on that page, > > Some directives are undocumented on purpose. E.g., acceptex_read > directive you've added in your patch is from never worked > unfinished acceptex event handling module. Ok, I'll keep it in the syntax file anyway, it is not documentation and does not hurt. Why not remove the file if it is not used? > > and that some typos are present > > (for example, fascgi_force_ranges instead of "fastcgi_force_ranges"). > > This means there is a typo in the directive name in docs. This on > fixed, thanks: > > http://hg.nginx.org/nginx.org/rev/c63717ebc5bc > > (BTW, patches for docs are also welcome.) Didn't know that the website was also in hg. Good to know in case I spot other errors. > > Contrib: add more directives to vim syntax > > Please add a trailing dot here. Done. > > --- a/contrib/README Wed Oct 15 22:57:23 2014 +0400 > > +++ b/contrib/README Sun Oct 19 16:08:33 2014 +0200 > > @@ -17,5 +17,5 @@ unicode2nginx by Maxim Dounin > > vim by Evan Miller > > > > Syntax highlighting of nginx configuration for vim, to be > > - placed into ~/.vim/. > > + placed into ~/.vim/ or /usr/share/vim/vimfiles/. > > I would like to avoid platform-specific paths here. Okay, I could not really find documentation on this so have just removed it. > > syn keyword ngxDirective epoll_events > > syn keyword ngxDirective error_log > > +syn keyword ngxDirective error_log > > We probably don't wont duplicate directives here and in other > places. Yeah, I originally sorted the list of directives without using the '--unique' option. This causes duplicate lines as error_log appears in the HTTP core and log modules. > > +syn keyword ngxDirective etag > > syn keyword ngxDirective eventport_events > > syn keyword ngxDirective expires > > syn keyword ngxDirective fastcgi_bind > > +syn keyword ngxDirective fastcgi_buffering > > +syn keyword ngxDirective fastcgi_buffers > > syn keyword ngxDirective fastcgi_buffer_size > > -syn keyword ngxDirective fastcgi_buffers > > The change in order looks strange, in lowercase > fastcgi_buffer_size should be before fastcgi_buffers and > fastcgi_buffering. Turns out to be a locale issue, forcing it with LC_ALL=C changed it to the previous ordering. Please find the updated patch attached. -- Kind regards, Peter https://lekensteyn.nl -------------- next part -------------- A non-text attachment was scrubbed... Name: Contrib__add_more_directives_to_vim_syntax_.patch Type: text/x-patch Size: 23180 bytes Desc: not available URL: From mdounin at mdounin.ru Wed Oct 22 16:38:40 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 22 Oct 2014 20:38:40 +0400 Subject: What is the order of module execution and how can I change it? In-Reply-To: <54477860.5040600@gmail.com> References: <54477860.5040600@gmail.com> Message-ID: <20141022163840.GD44913@mdounin.ru> Hello! On Wed, Oct 22, 2014 at 12:26:56PM +0300, Alexander Todorov wrote: > Hello folks, > I'm looking into nginx for developing custom modules and I'm missing some > info related to the order of module execution. > > At http://www.evanmiller.org/nginx-modules-guide.html it says: > " > The order of their execution is determined at compile-time. Filters have the > classic "CHAIN OF RESPONSIBILITY" design pattern: one filter is called, does > its work, and then calls the next filter, until the final filter is called, > and Nginx finishes up the response. > " > > Not a mention about non-filter modules. Then when trying to build nginx from > source I see a objs/ngx_modules.c file with: > > ngx_module_t *ngx_modules[] = { > &ngx_core_module, > &ngx_errlog_module, > &ngx_conf_module, > &ngx_events_module, > &ngx_event_core_module, > &ngx_epoll_module, > &ngx_regex_module, > &ngx_http_module, > &ngx_http_core_module, > &ngx_http_log_module, > > ... skip ... > > &ngx_http_headers_filter_module, > &ngx_http_copy_filter_module, > &ngx_http_range_body_filter_module, > &ngx_http_not_modified_filter_module, > NULL > }; > > > This looks like the order of execution of all modules, is this correct ? Not exactly. This is a list of all modules. It also defines module registration order (i.e., the order in which various init handlers are called). > If so, where can I place my modules in this list ? Only at the end or at > arbitrary positions? How do I do that (point me to docs is fine) ? The order is defined during ./configure. You may want to take a look at auto/modules if you want to understand it better. > For example I'd like to have a custom logging module which executes after > ngx_http_log_module saving results to a DB for further analysis and BI > purposes. Add your module to the HTTP_MODULES modules list, and then register to log phase in your module postconfiguration handler. > Another example is if I'd want to minify HTML content (a filter module) how > do I do that before gzip kicks in? Add your module to the HTTP_AUX_FILTER_MODULES list, and add a filter to header/body filter chains in your module postconfiguration handler. -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Fri Oct 24 11:29:27 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Fri, 24 Oct 2014 04:29:27 -0700 Subject: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases Message-ID: # HG changeset patch # User Piotr Sikora # Date 1414150080 25200 # Fri Oct 24 04:28:00 2014 -0700 # Node ID f71b843694fc2be7eabb9313aa82fb87e83210d6 # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 SSL: make ssl_password_file work with recent OpenSSL releases. Multiple passwords in a single ssl_password_file feature was broken after recent OpenSSL changes (commit 4aac102f75b517bdb56b1bcfd0a856052d559f6e). Affected OpenSSL releases: 0.9.8zc, 1.0.0o, 1.0.1j and 1.0.2-beta3. Signed-off-by: Piotr Sikora diff -r 973fded4f461 -r f71b843694fc src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Wed Oct 15 22:57:23 2014 +0400 +++ b/src/event/ngx_event_openssl.c Fri Oct 24 04:28:00 2014 -0700 @@ -410,8 +410,10 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ if (ERR_GET_LIB(n) == ERR_LIB_CIPHER && ERR_GET_REASON(n) == CIPHER_R_BAD_DECRYPT) #else - if (ERR_GET_LIB(n) == ERR_LIB_EVP - && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) + if ((ERR_GET_LIB(n) == ERR_LIB_PEM + && ERR_GET_REASON(n) == PEM_R_BAD_DECRYPT) + || (ERR_GET_LIB(n) == ERR_LIB_EVP + && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT)) #endif { ERR_clear_error(); From donatas.abraitis at gmail.com Fri Oct 24 19:02:08 2014 From: donatas.abraitis at gmail.com (Donatas Abraitis) Date: Fri, 24 Oct 2014 22:02:08 +0300 Subject: memory free on nginx reload Message-ID: Hello everyone, I'm trying to change behavior of one of the modules like ModSecurity. It doesn't free memory after reload, only after restart. static void ngx_http_modsecurity_terminate(ngx_cycle_t *cycle); ngx_http_modsecurity_terminate, /* exit process */ ngx_http_modsecurity_terminate, /* exit master */ static void ngx_http_modsecurity_terminate(ngx_cycle_t *cycle) { if (modsec_server) { modsecTerminate(); modsec_server = NULL; } } Backtrace is quite good, but I don't understand why it repeats two times (it's after nginx's reload): 0x506060 : ngx_http_modsecurity_terminate+0x0/0x30 [/usr/local/openresty/nginx/sbin/nginx] 0x4590be : ngx_worker_process_exit+0x2e/0x210 [/usr/local/openresty/nginx/sbin/nginx] 0x45989a : ngx_cache_manager_process_cycle+0x13a/0x180 [/usr/local/openresty/nginx/sbin/nginx] 0x457da4 : ngx_spawn_process+0x154/0x4d0 [/usr/local/openresty/nginx/sbin/nginx] 0x4596fa : ngx_start_cache_manager_processes+0x17a/0x1e0 [/usr/local/openresty/nginx/sbin/nginx] 0x45a00e : ngx_master_process_cycle+0x2ae/0x990 [/usr/local/openresty/nginx/sbin/nginx] 0x4388cf : main+0x77f/0xa10 [/usr/local/openresty/nginx/sbin/nginx] 0x7f30f2f69ec5 : __libc_start_main+0xf5/0x1c0 [/lib/x86_64-linux-gnu/ libc-2.19.so] 0x43b380 : _start+0x29/0x39 [/usr/local/openresty/nginx/sbin/nginx] 0x506060 : ngx_http_modsecurity_terminate+0x0/0x30 [/usr/local/openresty/nginx/sbin/nginx] 0x4590be : ngx_worker_process_exit+0x2e/0x210 [/usr/local/openresty/nginx/sbin/nginx] 0x4593cb : ngx_worker_process_cycle+0x12b/0x190 [/usr/local/openresty/nginx/sbin/nginx] 0x457da4 : ngx_spawn_process+0x154/0x4d0 [/usr/local/openresty/nginx/sbin/nginx] 0x459514 : ngx_start_worker_processes+0x74/0xe0 [/usr/local/openresty/nginx/sbin/nginx] 0x45a001 : ngx_master_process_cycle+0x2a1/0x990 [/usr/local/openresty/nginx/sbin/nginx] 0x4388cf : main+0x77f/0xa10 [/usr/local/openresty/nginx/sbin/nginx] 0x7f30f2f69ec5 : __libc_start_main+0xf5/0x1c0 [/lib/x86_64-linux-gnu/ libc-2.19.so] 0x43b380 : _start+0x29/0x39 [/usr/local/openresty/nginx/sbin/nginx] Any help would be appreciated! Or just any hints as well! -- Donatas -------------- next part -------------- An HTML attachment was scrubbed... URL: From wking at tremily.us Sat Oct 25 16:19:02 2014 From: wking at tremily.us (W. Trevor King) Date: Sat, 25 Oct 2014 09:19:02 -0700 Subject: [RFC] auth_request: Add auth_request_intercept_errors (on by default) Message-ID: # HG changeset patch # User W. Trevor King # Date 1414251802 25200 # Sat Oct 25 08:43:22 2014 -0700 # Node ID cffff97c4b7ce07175cf6ac1ac0c8b814d86d468 # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 auth_request: Add auth_request_intercept_errors (on by default). This allows you (with 'auth_request_intercept_errors off') to pass through errors >= 400 from the auth_request endpoint. You might want to do this if you want to return more explicit information about what went wrong (e.g., returning a 504 if the auth_request endpoint times out). diff -r 973fded4f461 -r cffff97c4b7c src/http/modules/ngx_http_auth_request_module.c --- a/src/http/modules/ngx_http_auth_request_module.c Wed Oct 15 22:57:23 2014 +0400 +++ b/src/http/modules/ngx_http_auth_request_module.c Sat Oct 25 08:43:22 2014 -0700 @@ -12,6 +12,7 @@ typedef struct { ngx_str_t uri; + ngx_flag_t intercept_errors; ngx_array_t *vars; } ngx_http_auth_request_conf_t; @@ -63,6 +64,13 @@ 0, NULL }, + { ngx_string("auth_request_intercept_errors"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_auth_request_conf_t, intercept_errors), + NULL }, + ngx_null_command }; @@ -167,6 +175,11 @@ return NGX_OK; } + if (ctx->status >= NGX_HTTP_BAD_REQUEST + && !arcf->intercept_errors) { + return ctx->status; + } + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "auth request unexpected status: %d", ctx->status); @@ -315,6 +328,7 @@ * conf->uri = { 0, NULL }; */ + conf->intercept_errors = NGX_CONF_UNSET; conf->vars = NGX_CONF_UNSET_PTR; return conf; @@ -328,6 +342,7 @@ ngx_http_auth_request_conf_t *conf = child; ngx_conf_merge_str_value(conf->uri, prev->uri, ""); + ngx_conf_merge_value(conf->intercept_errors, prev->intercept_errors, 1); ngx_conf_merge_ptr_value(conf->vars, prev->vars, NULL); return NGX_CONF_OK; From wking at tremily.us Sat Oct 25 16:27:39 2014 From: wking at tremily.us (W. Trevor King) Date: Sat, 25 Oct 2014 09:27:39 -0700 Subject: [RFC] auth_request: Add auth_request_intercept_errors (on by default) In-Reply-To: References: Message-ID: <20141025162739.GU15573@odin.tremily.us> The patch successfully passes through error codes, but I haven't been able to find a way to distinguish between auth-endpoint errors and errors from the authorized endpoint. For example, with: location = /auth { proxy_pass http://auth.example.com/; proxy_pass_request_body off; proxy_set_header Content-Length ""; proxy_set_header X-Original-URI $request_uri; proxy_read_timeout 5s; proxy_intercept_errors on; error_page 504 =504 @504-auth; } location / { auth_request /auth; auth_request_intercept_errors off; proxy_pass http://api.example.com/; } location @504-auth { return 504 "auth timeout"; } I get the generic 504 error when either auth.example.com or api.example.com times out. I expect this is due to the auth_request handler stripping the response body from the auth request, with a flow like: 1. Client requests /abc. 2. Nginx matches location /, and creates a sub-request for /auth. 3. Nginx matches location /auth, and sends a request to auth.example.com. 4. auth.example.com times out, Nginx creates a default 504 response. 5. The /auth error_page catches the 504 and adjusts the response body. 6. Nginx exits the /auth location, returning the adjusted 504 to the auth_request handler. 7. ngx_http_auth_request_handler strips the response body. 8. Nginx notices that the 504 doesn't have a body, and attaches the default body. 9. A 504 with the default body is returned to the client. I'm still trying to work out what's actually going on so I can respond with an auth-specific timeout message. However, I thought I'd send the current patch as is (since it works for folks who don't need an auth-specific messages and just want to pass through the error codes). Do folks besides me want this feature? Can anyone give me hints on auth-specific error messages? Thanks, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From flygoast at 126.com Mon Oct 27 07:27:03 2014 From: flygoast at 126.com (Gu Feng) Date: Mon, 27 Oct 2014 15:27:03 +0800 Subject: [PATCH] Upstream: limited next_upstream time and tries when resolving DNS Message-ID: # HG changeset patch # User Gu Feng # Date 1414394704 -28800 # Mon Oct 27 15:25:04 2014 +0800 # Node ID a7d62f0f3000d156477f556e5cb307f229fe6301 # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 Upstream: limited next_upstream time and tries when resolving DNS. When got multiple upstream IP addresses using DNS resolving, the number of upstreams tries and the maxinum time spent for these tries were not affected. This patch fixed it. diff -r 973fded4f461 -r a7d62f0f3000 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Wed Oct 15 22:57:23 2014 +0400 +++ b/src/http/ngx_http_upstream.c Mon Oct 27 15:25:04 2014 +0800 @@ -613,6 +613,14 @@ ngx_http_upstream_init_request(ngx_http_ return; } + u->peer.start_time = ngx_current_msec; + + if (u->conf->next_upstream_tries + && u->peer.tries > u->conf->next_upstream_tries) + { + u->peer.tries = u->conf->next_upstream_tries; + } + ngx_http_upstream_connect(r, u); return; @@ -974,6 +982,14 @@ ngx_http_upstream_resolve_handler(ngx_re ngx_resolve_name_done(ctx); ur->ctx = NULL; + u->peer.start_time = ngx_current_msec; + + if (u->conf->next_upstream_tries + && u->peer.tries > u->conf->next_upstream_tries) + { + u->peer.tries = u->conf->next_upstream_tries; + } + ngx_http_upstream_connect(r, u); failed: From ru at nginx.com Mon Oct 27 11:41:39 2014 From: ru at nginx.com (Ruslan Ermilov) Date: Mon, 27 Oct 2014 15:41:39 +0400 Subject: [PATCH] Upstream: limited next_upstream time and tries when resolving DNS In-Reply-To: References: Message-ID: <20141027114139.GA89713@lo0.su> On Mon, Oct 27, 2014 at 03:27:03PM +0800, Gu Feng wrote: > # HG changeset patch > # User Gu Feng > # Date 1414394704 -28800 > # Mon Oct 27 15:25:04 2014 +0800 > # Node ID a7d62f0f3000d156477f556e5cb307f229fe6301 > # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 > Upstream: limited next_upstream time and tries when resolving DNS. > > When got multiple upstream IP addresses using DNS resolving, the number of > upstreams tries and the maxinum time spent for these tries were not affected. > This patch fixed it. > > diff -r 973fded4f461 -r a7d62f0f3000 src/http/ngx_http_upstream.c > --- a/src/http/ngx_http_upstream.c Wed Oct 15 22:57:23 2014 +0400 > +++ b/src/http/ngx_http_upstream.c Mon Oct 27 15:25:04 2014 +0800 > @@ -613,6 +613,14 @@ ngx_http_upstream_init_request(ngx_http_ > return; > } > > + u->peer.start_time = ngx_current_msec; > + > + if (u->conf->next_upstream_tries > + && u->peer.tries > u->conf->next_upstream_tries) > + { > + u->peer.tries = u->conf->next_upstream_tries; > + } > + > ngx_http_upstream_connect(r, u); > > return; This part of the patch shouldn't be necessary - it's for the case when upstream is defined using variables but defines a single IP literal. > @@ -974,6 +982,14 @@ ngx_http_upstream_resolve_handler(ngx_re > ngx_resolve_name_done(ctx); > ur->ctx = NULL; > > + u->peer.start_time = ngx_current_msec; > + > + if (u->conf->next_upstream_tries > + && u->peer.tries > u->conf->next_upstream_tries) > + { > + u->peer.tries = u->conf->next_upstream_tries; > + } > + > ngx_http_upstream_connect(r, u); > > failed: From flygoast at 126.com Mon Oct 27 11:53:26 2014 From: flygoast at 126.com (Gu Feng) Date: Mon, 27 Oct 2014 19:53:26 +0800 Subject: [PATCH] Upstream: limited next_upstream time and tries when resolving DNS Message-ID: # HG changeset patch # User Gu Feng # Date 1414410723 -28800 # Mon Oct 27 19:52:03 2014 +0800 # Node ID ccc3b86835da54273c333e7d6a00f68e0b0e6c27 # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 Upstream: limited next_upstream time and tries when resolving DNS. When got multiple upstream IP addresses using DNS resolving, the number of upstreams tries and the maxinum time spent for these tries were not affected. This patch fixed it. diff -r 973fded4f461 -r ccc3b86835da src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Wed Oct 15 22:57:23 2014 +0400 +++ b/src/http/ngx_http_upstream.c Mon Oct 27 19:52:03 2014 +0800 @@ -974,6 +974,14 @@ ngx_http_upstream_resolve_handler(ngx_re ngx_resolve_name_done(ctx); ur->ctx = NULL; + u->peer.start_time = ngx_current_msec; + + if (u->conf->next_upstream_tries + && u->peer.tries > u->conf->next_upstream_tries) + { + u->peer.tries = u->conf->next_upstream_tries; + } + ngx_http_upstream_connect(r, u); failed: From pluknet at nginx.com Mon Oct 27 13:55:28 2014 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 27 Oct 2014 16:55:28 +0300 Subject: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases In-Reply-To: References: Message-ID: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> On Oct 24, 2014, at 3:29 PM, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1414150080 25200 > # Fri Oct 24 04:28:00 2014 -0700 > # Node ID f71b843694fc2be7eabb9313aa82fb87e83210d6 > # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 > SSL: make ssl_password_file work with recent OpenSSL releases. I updated the patch to cover the pem file in PKCS#8 format that is matched with PEM_STRING_PKCS8. diff -r 973fded4f461 -r 8c59ef63e7c0 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Wed Oct 15 22:57:23 2014 +0400 +++ b/src/event/ngx_event_openssl.c Mon Oct 27 13:19:01 2014 +0300 @@ -410,8 +410,12 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ if (ERR_GET_LIB(n) == ERR_LIB_CIPHER && ERR_GET_REASON(n) == CIPHER_R_BAD_DECRYPT) #else - if (ERR_GET_LIB(n) == ERR_LIB_EVP - && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) + if ((ERR_GET_LIB(n) == ERR_LIB_PEM + && ERR_GET_REASON(n) == PEM_R_BAD_DECRYPT) + || (ERR_GET_LIB(n) == ERR_LIB_EVP + && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) + || (ERR_GET_LIB(n) == ERR_LIB_PKCS12 + && ERR_GET_REASON(n) == PKCS12_R_PKCS12_CIPHERFINAL_ERROR)) #endif { ERR_clear_error(); diff -r 973fded4f461 -r 8c59ef63e7c0 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Wed Oct 15 22:57:23 2014 +0400 +++ b/src/event/ngx_event_openssl.h Mon Oct 27 13:19:01 2014 +0300 @@ -22,6 +22,7 @@ #include #endif #include +#include #ifndef OPENSSL_NO_OCSP #include #endif -- Sergey Kandaurov From vbart at nginx.com Mon Oct 27 14:01:58 2014 From: vbart at nginx.com (Valentin V. Bartenev) Date: Mon, 27 Oct 2014 17:01:58 +0300 Subject: [PATCH] SPDY: stop emitting empty header values In-Reply-To: References: Message-ID: <1424354.N8cOoFmhc0@vbart-workstation> On Tuesday 21 October 2014 04:16:01 Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1413890111 25200 > # Tue Oct 21 04:15:11 2014 -0700 > # Node ID fbc2eb84bd266b64644b00a6454c4f79f241af5a > # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 > SPDY: stop emitting empty header values. > > Previously, nginx would emit empty values in a header with multiple, > NULL-separated values. > > This is forbidden by the SPDY specification, which requires headers to > have either a single (possibly empty) value or multiple, NULL-separated > non-empty values. > > Signed-off-by: Piotr Sikora > > diff -r 973fded4f461 -r fbc2eb84bd26 src/http/ngx_http_spdy_filter_module.c > --- a/src/http/ngx_http_spdy_filter_module.c Wed Oct 15 22:57:23 2014 +0400 > +++ b/src/http/ngx_http_spdy_filter_module.c Tue Oct 21 04:15:11 2014 -0700 > @@ -100,7 +100,7 @@ ngx_http_spdy_header_filter(ngx_http_req > u_char *p, *buf, *last; > ngx_buf_t *b; > ngx_str_t host; > - ngx_uint_t i, j, count, port; > + ngx_uint_t i, j, count, port, sep; > ngx_chain_t *cl; > ngx_list_part_t *part, *pt; > ngx_table_elt_t *header, *h; > @@ -472,6 +472,7 @@ ngx_http_spdy_header_filter(ngx_http_req > > pt = part; > h = header; > + sep = header[i].value.len ? 1 : 0; > > for (j = i + 1; /* void */; j++) { > > @@ -493,9 +494,14 @@ ngx_http_spdy_header_filter(ngx_http_req > continue; > } > > - *last++ = '\0'; > + if (h[j].value.len) { > + if (sep) { > + *last++ = '\0'; > + } > > - last = ngx_cpymem(last, h[j].value.data, h[j].value.len); > + last = ngx_cpymem(last, h[j].value.data, h[j].value.len); > + sep = 1; > + } > > h[j].hash = 2; > } > IMHO, there is no need to introduce "sep" variable: diff -r a8d111bb6884 src/http/ngx_http_spdy_filter_module.c --- a/src/http/ngx_http_spdy_filter_module.c Tue Sep 30 17:20:32 2014 +0400 +++ b/src/http/ngx_http_spdy_filter_module.c Mon Oct 27 16:58:02 2014 +0300 @@ -493,9 +493,13 @@ ngx_http_spdy_header_filter(ngx_http_req continue; } - *last++ = '\0'; + if (h[j].value.len) { + if (last != p) { + *last++ = '\0'; + } - last = ngx_cpymem(last, h[j].value.data, h[j].value.len); + last = ngx_cpymem(last, h[j].value.data, h[j].value.len); + } h[j].hash = 2; } From mdounin at mdounin.ru Mon Oct 27 14:29:26 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 27 Oct 2014 18:29:26 +0400 Subject: [RFC] auth_request: Add auth_request_intercept_errors (on by default) In-Reply-To: <20141025162739.GU15573@odin.tremily.us> References: <20141025162739.GU15573@odin.tremily.us> Message-ID: <20141027142926.GG44913@mdounin.ru> Hello! On Sat, Oct 25, 2014 at 09:27:39AM -0700, W. Trevor King wrote: > The patch successfully passes through error codes, but I haven't been > able to find a way to distinguish between auth-endpoint errors and > errors from the authorized endpoint. For example, with: > > location = /auth { > proxy_pass http://auth.example.com/; > proxy_pass_request_body off; > proxy_set_header Content-Length ""; > proxy_set_header X-Original-URI $request_uri; > proxy_read_timeout 5s; > proxy_intercept_errors on; > error_page 504 =504 @504-auth; > } > > location / { > auth_request /auth; > auth_request_intercept_errors off; > proxy_pass http://api.example.com/; > } > > location @504-auth { > return 504 "auth timeout"; > } > > I get the generic 504 error when either auth.example.com or > api.example.com times out. I expect this is due to the auth_request > handler stripping the response body from the auth request, with a flow > like: [...] > Do folks besides me want this feature? Can anyone give me hints on > auth-specific error messages? Without any patches, you can do something like this (not tested though): location / { auth_request /auth; auth_request_set $auth_timeout $upstream_http_x_auth_timeout; error_page 403 = /auth_timeout; proxy_pass ... } location = /auth { error_page 504 = /auth_helper; proxy_pass http://auth.example.com; ... } location = /auth_helper { add_header X-Auth-Timeout 1 always; return 403; } location = /auth_timeout { if ($auth_timeout) { return 504; } return 403; } With this config, the 403 with a special header is used by auth endpoint to indicate timeouts, and a special error_page for 403 is used to distinguish between various reasons for 403. -- Maxim Dounin http://nginx.org/ From kou at cozmixng.org Mon Oct 27 15:01:26 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Tue, 28 Oct 2014 00:01:26 +0900 (JST) Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141018.004250.1469014097857676892.kou@cozmixng.org> References: <20141015.212109.839031045647992737.kou@cozmixng.org> <20141017144512.GF35211@mdounin.ru> <20141018.004250.1469014097857676892.kou@cozmixng.org> Message-ID: <20141028.000126.1606071279740532747.kou@cozmixng.org> Hi, Could you confirm the patch? http://mailman.nginx.org/pipermail/nginx-devel/2014-October/006094.html Thanks, -- kou In <20141018.004250.1469014097857676892.kou at cozmixng.org> "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Sat, 18 Oct 2014 00:42:50 +0900 (JST), Kouhei Sutou wrote: > Hi, > > In <20141017144512.GF35211 at mdounin.ru> > "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Fri, 17 Oct 2014 18:45:12 +0400, > Maxim Dounin wrote: > >>> I add only 64bit version MinGW-w64 GCC support. We can >>> distinguish 64bit version MinGW-w64 GCC by the following >>> expression: >>> >>> defined(__GNUC__) && defined(_WIN64) >> >> The MinGW-w64 wiki suggests the __MINGW64_VERSION_MAJOR macro can be >> used to distinguish MinGW-w64 and MinGW: >> >> http://sourceforge.net/p/mingw-w64/wiki2/Answer%20Check%20For%20Mingw-w64/ > > Thanks for the information. I don't know about it. > (I just checked only pre-defined macros by > "x86_64-w64-mingw32-gcc -dM -xc -E /dev/null"...) > > I confirmed the the approach can work with both 32bit > version and 64bit version MinGW-w64 GCCs. > > > Could you merge the attached patch? > > > Thanks, > -- > kou From mdounin at mdounin.ru Mon Oct 27 16:28:29 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 27 Oct 2014 19:28:29 +0300 Subject: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases In-Reply-To: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> References: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> Message-ID: <20141027162829.GC45418@mdounin.ru> Hello! On Mon, Oct 27, 2014 at 04:55:28PM +0300, Sergey Kandaurov wrote: > On Oct 24, 2014, at 3:29 PM, Piotr Sikora wrote: > > # HG changeset patch > > # User Piotr Sikora > > # Date 1414150080 25200 > > # Fri Oct 24 04:28:00 2014 -0700 > > # Node ID f71b843694fc2be7eabb9313aa82fb87e83210d6 > > # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 > > SSL: make ssl_password_file work with recent OpenSSL releases. > > I updated the patch to cover the pem file in PKCS#8 format > that is matched with PEM_STRING_PKCS8. > > diff -r 973fded4f461 -r 8c59ef63e7c0 src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c Wed Oct 15 22:57:23 2014 +0400 > +++ b/src/event/ngx_event_openssl.c Mon Oct 27 13:19:01 2014 +0300 > @@ -410,8 +410,12 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ > if (ERR_GET_LIB(n) == ERR_LIB_CIPHER > && ERR_GET_REASON(n) == CIPHER_R_BAD_DECRYPT) > #else > - if (ERR_GET_LIB(n) == ERR_LIB_EVP > - && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) > + if ((ERR_GET_LIB(n) == ERR_LIB_PEM > + && ERR_GET_REASON(n) == PEM_R_BAD_DECRYPT) > + || (ERR_GET_LIB(n) == ERR_LIB_EVP > + && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) > + || (ERR_GET_LIB(n) == ERR_LIB_PKCS12 > + && ERR_GET_REASON(n) == PKCS12_R_PKCS12_CIPHERFINAL_ERROR)) > #endif > { > ERR_clear_error(); Is OPENSSL_IS_BORINGSSL part also needs changes for PKCS12? Also, indentation looks wrong for me. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Mon Oct 27 17:47:11 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 27 Oct 2014 20:47:11 +0300 Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141028.000126.1606071279740532747.kou@cozmixng.org> References: <20141015.212109.839031045647992737.kou@cozmixng.org> <20141017144512.GF35211@mdounin.ru> <20141018.004250.1469014097857676892.kou@cozmixng.org> <20141028.000126.1606071279740532747.kou@cozmixng.org> Message-ID: <20141027174711.GD45418@mdounin.ru> Hello! On Tue, Oct 28, 2014 at 12:01:26AM +0900, Kouhei Sutou wrote: > Hi, > > Could you confirm the patch? > > http://mailman.nginx.org/pipermail/nginx-devel/2014-October/006094.html Sorry, I have been travelling to the nginx.conf conference and wasn't yet able to review the patch. Below are the patch after some polishing. Changes include: - the NGX_GNUC_64 macro was removed, __MINGW64_VERSION_MAJOR tested directly instead; - the #include <_mingw.h> removed as tests are after includes now, see [1]; - some style nits. Please test if it still works for you. [1] https://www.mail-archive.com/mingw-w64-public at lists.sourceforge.net/msg00638.html # HG changeset patch # User Kouhei Sutou # Date 1413375301 -32400 # Wed Oct 15 21:15:01 2014 +0900 # Node ID 839983fff39f58d151e582987f6d3bd9833efde6 # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 Win32: made build-able with MinGW-w64 gcc. diff -r 973fded4f461 -r 839983fff39f src/os/win32/ngx_win32_config.h --- a/src/os/win32/ngx_win32_config.h Wed Oct 15 22:57:23 2014 +0400 +++ b/src/os/win32/ngx_win32_config.h Wed Oct 15 21:15:01 2014 +0900 @@ -37,14 +37,24 @@ #include #include /* offsetof() */ -#ifdef __GNUC__ +#ifdef __MINGW64_VERSION_MAJOR + +/* GCC MinGW-w64 supports _FILE_OFFSET_BITS */ +#define _FILE_OFFSET_BITS 64 + +#elif defined __GNUC__ + /* GCC MinGW's stdio.h includes sys/types.h */ #define _OFF_T_ + #endif #include #include #include +#ifdef __MINGW64_VERSION_MAJOR +#include +#endif #include #include @@ -136,16 +146,21 @@ typedef unsigned short int uint16_t; typedef __int64 int64_t; typedef unsigned __int64 uint64_t; -#ifndef __WATCOMC__ +#if !defined(__WATCOMC__) && !defined(__MINGW64_VERSION_MAJOR) typedef int intptr_t; typedef u_int uintptr_t; #endif +#ifndef __MINGW64_VERSION_MAJOR + /* Windows defines off_t as long, which is 32-bit */ typedef __int64 off_t; #define _OFF_T_DEFINED +#endif + + #ifdef __WATCOMC__ /* off_t is redefined by sys/types.h used by zlib.h */ @@ -164,17 +179,35 @@ typedef unsigned int ino_t; #endif +#ifndef __MINGW64_VERSION_MAJOR typedef int ssize_t; +#endif + + typedef uint32_t in_addr_t; typedef u_short in_port_t; typedef int sig_atomic_t; +#ifdef _WIN64 + +#define NGX_PTR_SIZE 8 +#define NGX_SIZE_T_LEN (sizeof("-9223372036854775808") - 1) +#define NGX_MAX_SIZE_T_VALUE 9223372036854775807 +#define NGX_TIME_T_LEN (sizeof("-9223372036854775808") - 1) +#define NGX_TIME_T_SIZE 8 + +#else + #define NGX_PTR_SIZE 4 #define NGX_SIZE_T_LEN (sizeof("-2147483648") - 1) #define NGX_MAX_SIZE_T_VALUE 2147483647 #define NGX_TIME_T_LEN (sizeof("-2147483648") - 1) #define NGX_TIME_T_SIZE 4 + +#endif + + #define NGX_OFF_T_LEN (sizeof("-9223372036854775807") - 1) #define NGX_MAX_OFF_T_VALUE 9223372036854775807 #define NGX_SIG_ATOMIC_T_SIZE 4 -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Mon Oct 27 18:20:05 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 27 Oct 2014 18:20:05 +0000 Subject: [nginx] Cache: disable caching of responses with Vary (ticket #1... Message-ID: details: http://hg.nginx.org/nginx/rev/60fde1bc7236 branches: changeset: 5877:60fde1bc7236 user: Maxim Dounin date: Mon Oct 27 21:13:39 2014 +0300 description: Cache: disable caching of responses with Vary (ticket #118). The "proxy_ignore_header" directive now undersands the "Vary" parameter to ignore the header as needed. diffstat: src/http/ngx_http_upstream.c | 30 ++++++++++++++++++++++++++++++ src/http/ngx_http_upstream.h | 2 ++ 2 files changed, 32 insertions(+), 0 deletions(-) diffs (80 lines): diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -113,6 +113,8 @@ static ngx_int_t ngx_http_upstream_proce static ngx_int_t ngx_http_upstream_process_transfer_encoding(ngx_http_request_t *r, ngx_table_elt_t *h, ngx_uint_t offset); +static ngx_int_t ngx_http_upstream_process_vary(ngx_http_request_t *r, + ngx_table_elt_t *h, ngx_uint_t offset); static ngx_int_t ngx_http_upstream_copy_header_line(ngx_http_request_t *r, ngx_table_elt_t *h, ngx_uint_t offset); static ngx_int_t @@ -250,6 +252,10 @@ ngx_http_upstream_header_t ngx_http_ups ngx_http_upstream_ignore_header_line, 0, ngx_http_upstream_ignore_header_line, 0, 0 }, + { ngx_string("Vary"), + ngx_http_upstream_process_vary, 0, + ngx_http_upstream_copy_header_line, 0, 0 }, + { ngx_string("X-Powered-By"), ngx_http_upstream_ignore_header_line, 0, ngx_http_upstream_copy_header_line, 0, 0 }, @@ -407,6 +413,7 @@ ngx_conf_bitmask_t ngx_http_upstream_ig { ngx_string("Expires"), NGX_HTTP_UPSTREAM_IGN_EXPIRES }, { ngx_string("Cache-Control"), NGX_HTTP_UPSTREAM_IGN_CACHE_CONTROL }, { ngx_string("Set-Cookie"), NGX_HTTP_UPSTREAM_IGN_SET_COOKIE }, + { ngx_string("Vary"), NGX_HTTP_UPSTREAM_IGN_VARY }, { ngx_null_string, 0 } }; @@ -4139,6 +4146,29 @@ ngx_http_upstream_process_transfer_encod static ngx_int_t +ngx_http_upstream_process_vary(ngx_http_request_t *r, + ngx_table_elt_t *h, ngx_uint_t offset) +{ + ngx_http_upstream_t *u; + + u = r->upstream; + u->headers_in.vary = h; + +#if (NGX_HTTP_CACHE) + + if (u->conf->ignore_headers & NGX_HTTP_UPSTREAM_IGN_VARY) { + return NGX_OK; + } + + u->cacheable = 0; + +#endif + + return NGX_OK; +} + + +static ngx_int_t ngx_http_upstream_copy_header_line(ngx_http_request_t *r, ngx_table_elt_t *h, ngx_uint_t offset) { diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h --- a/src/http/ngx_http_upstream.h +++ b/src/http/ngx_http_upstream.h @@ -50,6 +50,7 @@ #define NGX_HTTP_UPSTREAM_IGN_XA_LIMIT_RATE 0x00000040 #define NGX_HTTP_UPSTREAM_IGN_XA_BUFFERING 0x00000080 #define NGX_HTTP_UPSTREAM_IGN_XA_CHARSET 0x00000100 +#define NGX_HTTP_UPSTREAM_IGN_VARY 0x00000200 typedef struct { @@ -244,6 +245,7 @@ typedef struct { ngx_table_elt_t *accept_ranges; ngx_table_elt_t *www_authenticate; ngx_table_elt_t *transfer_encoding; + ngx_table_elt_t *vary; #if (NGX_HTTP_GZIP) ngx_table_elt_t *content_encoding; From mdounin at mdounin.ru Mon Oct 27 18:20:08 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 27 Oct 2014 18:20:08 +0000 Subject: [nginx] Cache: hash of Vary headers now stored in cache. Message-ID: details: http://hg.nginx.org/nginx/rev/2c89956b6a76 branches: changeset: 5878:2c89956b6a76 user: Maxim Dounin date: Mon Oct 27 21:13:58 2014 +0300 description: Cache: hash of Vary headers now stored in cache. To cache responses with Vary, we now calculate hash of headers listed in Vary, and return the response from cache only if new request headers match. As of now, only one variant of the same resource can be stored in cache. diffstat: src/http/ngx_http_cache.h | 8 ++- src/http/ngx_http_file_cache.c | 135 +++++++++++++++++++++++++++++++++++++++++ src/http/ngx_http_upstream.c | 12 +++- 3 files changed, 153 insertions(+), 2 deletions(-) diffs (228 lines): diff --git a/src/http/ngx_http_cache.h b/src/http/ngx_http_cache.h --- a/src/http/ngx_http_cache.h +++ b/src/http/ngx_http_cache.h @@ -25,8 +25,9 @@ #define NGX_HTTP_CACHE_KEY_LEN 16 #define NGX_HTTP_CACHE_ETAG_LEN 42 +#define NGX_HTTP_CACHE_VARY_LEN 42 -#define NGX_HTTP_CACHE_VERSION 2 +#define NGX_HTTP_CACHE_VERSION 3 typedef struct { @@ -71,6 +72,8 @@ struct ngx_http_cache_s { time_t date; ngx_str_t etag; + ngx_str_t vary; + u_char variant[NGX_HTTP_CACHE_KEY_LEN]; size_t header_start; size_t body_start; @@ -112,6 +115,9 @@ typedef struct { u_short body_start; u_char etag_len; u_char etag[NGX_HTTP_CACHE_ETAG_LEN]; + u_char vary_len; + u_char vary[NGX_HTTP_CACHE_VARY_LEN]; + u_char variant[NGX_HTTP_CACHE_KEY_LEN]; } ngx_http_file_cache_header_t; diff --git a/src/http/ngx_http_file_cache.c b/src/http/ngx_http_file_cache.c --- a/src/http/ngx_http_file_cache.c +++ b/src/http/ngx_http_file_cache.c @@ -29,6 +29,10 @@ static ngx_http_file_cache_node_t * ngx_http_file_cache_lookup(ngx_http_file_cache_t *cache, u_char *key); static void ngx_http_file_cache_rbtree_insert_value(ngx_rbtree_node_t *temp, ngx_rbtree_node_t *node, ngx_rbtree_node_t *sentinel); +static void ngx_http_file_cache_vary(ngx_http_request_t *r, u_char *vary, + size_t len, u_char *hash); +static void ngx_http_file_cache_vary_header(ngx_http_request_t *r, + ngx_md5_t *md5, ngx_str_t *name); static void ngx_http_file_cache_cleanup(void *data); static time_t ngx_http_file_cache_forced_expire(ngx_http_file_cache_t *cache); static time_t ngx_http_file_cache_expire(ngx_http_file_cache_t *cache); @@ -519,6 +523,23 @@ ngx_http_file_cache_read(ngx_http_reques return NGX_DECLINED; } + if (h->vary_len > NGX_HTTP_CACHE_VARY_LEN) { + ngx_log_error(NGX_LOG_CRIT, r->connection->log, 0, + "cache file \"%s\" has incorrect vary length", + c->file.name.data); + return NGX_DECLINED; + } + + if (h->vary_len) { + ngx_http_file_cache_vary(r, h->vary, h->vary_len, c->variant); + + if (ngx_memcmp(c->variant, h->variant, NGX_HTTP_CACHE_KEY_LEN) != 0) { + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "http file cache vary mismatch"); + return NGX_DECLINED; + } + } + c->buf->last += n; c->valid_sec = h->valid_sec; @@ -870,6 +891,94 @@ ngx_http_file_cache_rbtree_insert_value( } +static void +ngx_http_file_cache_vary(ngx_http_request_t *r, u_char *vary, size_t len, + u_char *hash) +{ + u_char *p, *last; + ngx_str_t name; + ngx_md5_t md5; + u_char buf[NGX_HTTP_CACHE_VARY_LEN]; + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "http file cache vary: \"%*s\"", len, vary); + + ngx_md5_init(&md5); + + ngx_strlow(buf, vary, len); + + p = buf; + last = buf + len; + + while (p < last) { + + while (p < last && (*p == ' ' || *p == ',')) { p++; } + + name.data = p; + + while (p < last && *p != ',' && *p != ' ') { p++; } + + name.len = p - name.data; + + if (name.len == 0) { + break; + } + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "http file cache vary: %V", &name); + + ngx_md5_update(&md5, name.data, name.len); + ngx_md5_update(&md5, (u_char *) ":", sizeof(":") - 1); + + ngx_http_file_cache_vary_header(r, &md5, &name); + + ngx_md5_update(&md5, (u_char *) CRLF, sizeof(CRLF) - 1); + } + + ngx_md5_final(hash, &md5); +} + + +static void +ngx_http_file_cache_vary_header(ngx_http_request_t *r, ngx_md5_t *md5, + ngx_str_t *name) +{ + ngx_uint_t i; + ngx_list_part_t *part; + ngx_table_elt_t *header; + + part = &r->headers_in.headers.part; + header = part->elts; + + for (i = 0; /* void */ ; i++) { + + if (i >= part->nelts) { + if (part->next == NULL) { + break; + } + + part = part->next; + header = part->elts; + i = 0; + } + + if (header[i].hash == 0) { + continue; + } + + if (header[i].key.len != name->len) { + continue; + } + + if (ngx_strncasecmp(header[i].key.data, name->data, name->len) != 0) { + continue; + } + + ngx_md5_update(md5, header[i].value.data, header[i].value.len); + } +} + + void ngx_http_file_cache_set_header(ngx_http_request_t *r, u_char *buf) { @@ -901,6 +1010,19 @@ ngx_http_file_cache_set_header(ngx_http_ ngx_memcpy(h->etag, c->etag.data, c->etag.len); } + if (c->vary.len) { + if (c->vary.len > NGX_HTTP_CACHE_VARY_LEN) { + /* should not happen */ + c->vary.len = NGX_HTTP_CACHE_VARY_LEN; + } + + h->vary_len = (u_char) c->vary.len; + ngx_memcpy(h->vary, c->vary.data, c->vary.len); + + ngx_http_file_cache_vary(r, c->vary.data, c->vary.len, c->variant); + ngx_memcpy(h->variant, c->variant, NGX_HTTP_CACHE_KEY_LEN); + } + p = buf + sizeof(ngx_http_file_cache_header_t); p = ngx_cpymem(p, ngx_http_file_cache_key, sizeof(ngx_http_file_cache_key)); @@ -1093,6 +1215,19 @@ ngx_http_file_cache_update_header(ngx_ht ngx_memcpy(h.etag, c->etag.data, c->etag.len); } + if (c->vary.len) { + if (c->vary.len > NGX_HTTP_CACHE_VARY_LEN) { + /* should not happen */ + c->vary.len = NGX_HTTP_CACHE_VARY_LEN; + } + + h.vary_len = (u_char) c->vary.len; + ngx_memcpy(h.vary, c->vary.data, c->vary.len); + + ngx_http_file_cache_vary(r, c->vary.data, c->vary.len, c->variant); + ngx_memcpy(h.variant, c->variant, NGX_HTTP_CACHE_KEY_LEN); + } + (void) ngx_write_file(&file, (u_char *) &h, sizeof(ngx_http_file_cache_header_t), 0); diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -4160,7 +4160,17 @@ ngx_http_upstream_process_vary(ngx_http_ return NGX_OK; } - u->cacheable = 0; + if (r->cache == NULL) { + return NGX_OK; + } + + if (h->value.len > NGX_HTTP_CACHE_VARY_LEN + || (h->value.len == 1 && h->value.data[0] == '*')) + { + u->cacheable = 0; + } + + r->cache->vary = h->value; #endif From mdounin at mdounin.ru Mon Oct 27 18:20:10 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 27 Oct 2014 18:20:10 +0000 Subject: [nginx] Cache: c->reading flag introduced. Message-ID: details: http://hg.nginx.org/nginx/rev/c525c0454aa5 branches: changeset: 5879:c525c0454aa5 user: Maxim Dounin date: Mon Oct 27 21:14:07 2014 +0300 description: Cache: c->reading flag introduced. It replaces c->buf in checks in ngx_http_file_cache_open(), making it possible to reopen the file without clearing c->buf. No functional changes. diffstat: src/http/ngx_http_cache.h | 1 + src/http/ngx_http_file_cache.c | 5 ++++- 2 files changed, 5 insertions(+), 1 deletions(-) diffs (36 lines): diff --git a/src/http/ngx_http_cache.h b/src/http/ngx_http_cache.h --- a/src/http/ngx_http_cache.h +++ b/src/http/ngx_http_cache.h @@ -101,6 +101,7 @@ struct ngx_http_cache_s { unsigned updating:1; unsigned exists:1; unsigned temp_file:1; + unsigned reading:1; }; diff --git a/src/http/ngx_http_file_cache.c b/src/http/ngx_http_file_cache.c --- a/src/http/ngx_http_file_cache.c +++ b/src/http/ngx_http_file_cache.c @@ -259,7 +259,7 @@ ngx_http_file_cache_open(ngx_http_reques return NGX_AGAIN; } - if (c->buf) { + if (c->reading) { return ngx_http_file_cache_read(r, c); } @@ -620,9 +620,12 @@ ngx_http_file_cache_aio_read(ngx_http_re n = ngx_file_aio_read(&c->file, c->buf->pos, c->body_start, 0, r->pool); if (n != NGX_AGAIN) { + c->reading = 0; return n; } + c->reading = 1; + c->file.aio->data = r; c->file.aio->handler = ngx_http_cache_aio_event_handler; From mdounin at mdounin.ru Mon Oct 27 18:20:13 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 27 Oct 2014 18:20:13 +0000 Subject: [nginx] Cache: multiple variants of a resource now can be stored. Message-ID: details: http://hg.nginx.org/nginx/rev/78c49e243848 branches: changeset: 5880:78c49e243848 user: Maxim Dounin date: Mon Oct 27 21:14:10 2014 +0300 description: Cache: multiple variants of a resource now can be stored. If a variant stored can't be used to respond to a request, the variant hash is used as a secondary key. Additionally, if we previously switched to a secondary key, while storing a response to cache we check if the variant hash still apply. If not, we switch back to the original key, to handle cases when Vary changes. diffstat: src/http/ngx_http_cache.h | 2 + src/http/ngx_http_file_cache.c | 80 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 79 insertions(+), 3 deletions(-) diffs (154 lines): diff --git a/src/http/ngx_http_cache.h b/src/http/ngx_http_cache.h --- a/src/http/ngx_http_cache.h +++ b/src/http/ngx_http_cache.h @@ -65,6 +65,7 @@ struct ngx_http_cache_s { ngx_array_t keys; uint32_t crc32; u_char key[NGX_HTTP_CACHE_KEY_LEN]; + u_char main[NGX_HTTP_CACHE_KEY_LEN]; ngx_file_uniq_t uniq; time_t valid_sec; @@ -102,6 +103,7 @@ struct ngx_http_cache_s { unsigned exists:1; unsigned temp_file:1; unsigned reading:1; + unsigned secondary:1; }; diff --git a/src/http/ngx_http_file_cache.c b/src/http/ngx_http_file_cache.c --- a/src/http/ngx_http_file_cache.c +++ b/src/http/ngx_http_file_cache.c @@ -33,6 +33,8 @@ static void ngx_http_file_cache_vary(ngx size_t len, u_char *hash); static void ngx_http_file_cache_vary_header(ngx_http_request_t *r, ngx_md5_t *md5, ngx_str_t *name); +static ngx_int_t ngx_http_file_cache_reopen(ngx_http_request_t *r, + ngx_http_cache_t *c); static void ngx_http_file_cache_cleanup(void *data); static time_t ngx_http_file_cache_forced_expire(ngx_http_file_cache_t *cache); static time_t ngx_http_file_cache_expire(ngx_http_file_cache_t *cache); @@ -239,6 +241,8 @@ ngx_http_file_cache_create_key(ngx_http_ ngx_crc32_final(c->crc32); ngx_md5_final(c->key, &md5); + + ngx_memcpy(c->main, c->key, NGX_HTTP_CACHE_KEY_LEN); } @@ -536,7 +540,7 @@ ngx_http_file_cache_read(ngx_http_reques if (ngx_memcmp(c->variant, h->variant, NGX_HTTP_CACHE_KEY_LEN) != 0) { ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http file cache vary mismatch"); - return NGX_DECLINED; + return ngx_http_file_cache_reopen(r, c); } } @@ -907,6 +911,7 @@ ngx_http_file_cache_vary(ngx_http_reques "http file cache vary: \"%*s\"", len, vary); ngx_md5_init(&md5); + ngx_md5_update(&md5, r->cache->main, NGX_HTTP_CACHE_KEY_LEN); ngx_strlow(buf, vary, len); @@ -982,6 +987,40 @@ ngx_http_file_cache_vary_header(ngx_http } +static ngx_int_t +ngx_http_file_cache_reopen(ngx_http_request_t *r, ngx_http_cache_t *c) +{ + ngx_http_file_cache_t *cache; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->file.log, 0, + "http file cache reopen"); + + if (c->secondary) { + ngx_log_error(NGX_LOG_CRIT, r->connection->log, 0, + "cache file \"%s\" has incorrect vary hash", + c->file.name.data); + return NGX_DECLINED; + } + + cache = c->file_cache; + + ngx_shmtx_lock(&cache->shpool->mutex); + + c->node->count--; + c->node = NULL; + + ngx_shmtx_unlock(&cache->shpool->mutex); + + c->secondary = 1; + c->file.name.len = 0; + c->body_start = c->buf->end - c->buf->start; + + ngx_memcpy(c->key, c->variant, NGX_HTTP_CACHE_KEY_LEN); + + return ngx_http_file_cache_open(r); +} + + void ngx_http_file_cache_set_header(ngx_http_request_t *r, u_char *buf) { @@ -1024,6 +1063,9 @@ ngx_http_file_cache_set_header(ngx_http_ ngx_http_file_cache_vary(r, c->vary.data, c->vary.len, c->variant); ngx_memcpy(h->variant, c->variant, NGX_HTTP_CACHE_KEY_LEN); + + } else { + ngx_memzero(c->variant, NGX_HTTP_CACHE_KEY_LEN); } p = buf + sizeof(ngx_http_file_cache_header_t); @@ -1059,11 +1101,43 @@ ngx_http_file_cache_update(ngx_http_requ ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http file cache update"); + cache = c->file_cache; + + if (c->secondary + && ngx_memcmp(c->variant, c->key, NGX_HTTP_CACHE_KEY_LEN) != 0) + { + /* + * if the variant hash doesn't match one we used as a secondary + * cache key, switch back to the original key + */ + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "http file cache main key"); + + ngx_shmtx_lock(&cache->shpool->mutex); + + c->node->count--; + c->node->updating = 0; + c->node = NULL; + + ngx_shmtx_unlock(&cache->shpool->mutex); + + c->file.name.len = 0; + + ngx_memcpy(c->key, c->main, NGX_HTTP_CACHE_KEY_LEN); + + if (ngx_http_file_cache_exists(cache, c) == NGX_ERROR) { + return; + } + + if (ngx_http_file_cache_name(r, cache->path) != NGX_OK) { + return; + } + } + c->updated = 1; c->updating = 0; - cache = c->file_cache; - uniq = 0; fs_size = 0; From mdounin at mdounin.ru Mon Oct 27 18:20:16 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 27 Oct 2014 18:20:16 +0000 Subject: [nginx] Cache: normalization of some Vary headers. Message-ID: details: http://hg.nginx.org/nginx/rev/ee9230cd4bda branches: changeset: 5881:ee9230cd4bda user: Maxim Dounin date: Mon Oct 27 21:14:12 2014 +0300 description: Cache: normalization of some Vary headers. Spaces in Accept-Charset, Accept-Encoding, and Accept-Language headers are now ignored. As per syntax of these headers spaces can only appear in places where they are optional. diffstat: src/http/ngx_http_file_cache.c | 68 ++++++++++++++++++++++++++++++++++++++++- 1 files changed, 66 insertions(+), 2 deletions(-) diffs (88 lines): diff --git a/src/http/ngx_http_file_cache.c b/src/http/ngx_http_file_cache.c --- a/src/http/ngx_http_file_cache.c +++ b/src/http/ngx_http_file_cache.c @@ -951,10 +951,34 @@ static void ngx_http_file_cache_vary_header(ngx_http_request_t *r, ngx_md5_t *md5, ngx_str_t *name) { - ngx_uint_t i; + size_t len; + u_char *p, *start, *last; + ngx_uint_t i, multiple, normalize; ngx_list_part_t *part; ngx_table_elt_t *header; + multiple = 0; + normalize = 0; + + if (name->len == sizeof("Accept-Charset") - 1 + && ngx_strncasecmp(name->data, (u_char *) "Accept-Charset", + sizeof("Accept-Charset") - 1) == 0) + { + normalize = 1; + + } else if (name->len == sizeof("Accept-Encoding") - 1 + && ngx_strncasecmp(name->data, (u_char *) "Accept-Encoding", + sizeof("Accept-Encoding") - 1) == 0) + { + normalize = 1; + + } else if (name->len == sizeof("Accept-Language") - 1 + && ngx_strncasecmp(name->data, (u_char *) "Accept-Language", + sizeof("Accept-Language") - 1) == 0) + { + normalize = 1; + } + part = &r->headers_in.headers.part; header = part->elts; @@ -982,7 +1006,47 @@ ngx_http_file_cache_vary_header(ngx_http continue; } - ngx_md5_update(md5, header[i].value.data, header[i].value.len); + if (!normalize) { + + if (multiple) { + ngx_md5_update(md5, (u_char *) ",", sizeof(",") - 1); + } + + ngx_md5_update(md5, header[i].value.data, header[i].value.len); + + multiple = 1; + + continue; + } + + /* normalize spaces */ + + p = header[i].value.data; + start = p; + last = p + header[i].value.len; + + while (p < last) { + + while (p < last && (*p == ' ' || *p == ',')) { p++; } + + start = p; + + while (p < last && *p != ',' && *p != ' ') { p++; } + + len = p - start; + + if (len == 0) { + break; + } + + if (multiple) { + ngx_md5_update(md5, (u_char *) ",", sizeof(",") - 1); + } + + ngx_md5_update(md5, start, len); + + multiple = 1; + } } } From piotr at cloudflare.com Mon Oct 27 21:33:10 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Mon, 27 Oct 2014 14:33:10 -0700 Subject: [PATCH] SPDY: stop emitting empty header values In-Reply-To: <1424354.N8cOoFmhc0@vbart-workstation> References: <1424354.N8cOoFmhc0@vbart-workstation> Message-ID: Hey Valentin. > IMHO, there is no need to introduce "sep" variable: I think you're right, thanks! Best regards, Piotr Sikora From piotr at cloudflare.com Mon Oct 27 21:33:15 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Mon, 27 Oct 2014 14:33:15 -0700 Subject: [PATCH] SPDY: stop emitting empty header values In-Reply-To: <1424354.N8cOoFmhc0@vbart-workstation> References: <1424354.N8cOoFmhc0@vbart-workstation> Message-ID: # HG changeset patch # User Piotr Sikora # Date 1414445156 25200 # Mon Oct 27 14:25:56 2014 -0700 # Node ID e590d484a7f9d2750e2cd45ea16c601ae01cb607 # Parent ee9230cd4bda9f9e7e83f7baf37d66ad6bb9b0c1 SPDY: stop emitting empty header values. Previously, nginx would emit empty values in a header with multiple, NULL-separated values. This is forbidden by the SPDY specification, which requires headers to have either a single (possibly empty) value or multiple, NULL-separated non-empty values. Signed-off-by: Piotr Sikora diff -r ee9230cd4bda -r e590d484a7f9 src/http/ngx_http_spdy_filter_module.c --- a/src/http/ngx_http_spdy_filter_module.c Mon Oct 27 21:14:12 2014 +0300 +++ b/src/http/ngx_http_spdy_filter_module.c Mon Oct 27 14:25:56 2014 -0700 @@ -493,9 +493,13 @@ ngx_http_spdy_header_filter(ngx_http_req continue; } - *last++ = '\0'; + if (h[j].value.len) { + if (last != p) { + *last++ = '\0'; + } - last = ngx_cpymem(last, h[j].value.data, h[j].value.len); + last = ngx_cpymem(last, h[j].value.data, h[j].value.len); + } h[j].hash = 2; } From piotr at cloudflare.com Mon Oct 27 21:38:55 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Mon, 27 Oct 2014 14:38:55 -0700 Subject: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases In-Reply-To: <20141027162829.GC45418@mdounin.ru> References: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> <20141027162829.GC45418@mdounin.ru> Message-ID: Hey Maxim, > Is OPENSSL_IS_BORINGSSL part also needs changes for PKCS12? I'm pretty sure it does, I'll take care of it shortly. Also, I think that PKCS#12 support should be added in a separate commit, instead of mixing it with semi-related fix. Best regards, Piotr Sikora From piotr at cloudflare.com Mon Oct 27 23:51:37 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Mon, 27 Oct 2014 16:51:37 -0700 Subject: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases In-Reply-To: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> References: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> Message-ID: Hey Sergey, > diff -r 973fded4f461 -r 8c59ef63e7c0 src/event/ngx_event_openssl.h > --- a/src/event/ngx_event_openssl.h Wed Oct 15 22:57:23 2014 +0400 > +++ b/src/event/ngx_event_openssl.h Mon Oct 27 13:19:01 2014 +0300 > @@ -22,6 +22,7 @@ > #include > #endif > #include > +#include > #ifndef OPENSSL_NO_OCSP > #include > #endif This should be added after ocsp.h, ideally guarded for OpenSSL only, i.e.: +#ifndef OPENSSL_IS_BORINGSSL +#include +#endif I've looked into BoringSSL and it supports PKCS#8 private keys with encrypted with PKCS#12 compatible algorithms even now, without any changes in nginx, because it reports bad passwords using the same error code as for traditional private keys (i.e. CIPHER_R_BAD_DECRYPT). Best regards, Piotr Sikora From donatas.abraitis at gmail.com Tue Oct 28 06:30:15 2014 From: donatas.abraitis at gmail.com (Donatas Abraitis) Date: Tue, 28 Oct 2014 08:30:15 +0200 Subject: SIGHUP and ngx_module_t Message-ID: Hello, is it true, that SIGHUP is not handled by ngx_module_t /* exit process */ ? I mean if I have these defined: ngx_http_modsecurity_terminate, /* exit process */ ngx_http_modsecurity_terminate, /* exit master */ So, sending SIGHUP to the master it doesn't call the ngx_http_modsecurity_terminate(). If sending SIGTERM it does. How is it for real? If I send SIGHUP to the parent I get only: Program received signal SIGHUP, Hangup. If I send SIGTERM to the parent I get what I'm expected to get to SIGHUP as well: #0 modsecTerminate () at api.c:237 No locals. -- Donatas -------------- next part -------------- An HTML attachment was scrubbed... URL: From arut at nginx.com Tue Oct 28 09:35:44 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 28 Oct 2014 09:35:44 +0000 Subject: [nginx] Core: added limit to recv_chain(). Message-ID: details: http://hg.nginx.org/nginx/rev/ec81934727a1 branches: changeset: 5882:ec81934727a1 user: Roman Arutyunyan date: Tue Oct 28 12:29:58 2014 +0300 description: Core: added limit to recv_chain(). diffstat: src/event/ngx_event_openssl.c | 19 +++++++++++++++---- src/event/ngx_event_openssl.h | 2 +- src/event/ngx_event_pipe.c | 2 +- src/os/unix/ngx_aio_read_chain.c | 2 +- src/os/unix/ngx_os.h | 7 ++++--- src/os/unix/ngx_readv_chain.c | 20 ++++++++++++++++---- src/os/win32/ngx_os.h | 5 +++-- src/os/win32/ngx_wsarecv_chain.c | 22 +++++++++++++++++----- 8 files changed, 58 insertions(+), 21 deletions(-) diffs (227 lines): diff -r ee9230cd4bda -r ec81934727a1 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Mon Oct 27 21:14:12 2014 +0300 +++ b/src/event/ngx_event_openssl.c Tue Oct 28 12:29:58 2014 +0300 @@ -1185,10 +1185,10 @@ ngx_ssl_handshake_handler(ngx_event_t *e ssize_t -ngx_ssl_recv_chain(ngx_connection_t *c, ngx_chain_t *cl) +ngx_ssl_recv_chain(ngx_connection_t *c, ngx_chain_t *cl, off_t limit) { u_char *last; - ssize_t n, bytes; + ssize_t n, bytes, size; ngx_buf_t *b; bytes = 0; @@ -1197,8 +1197,19 @@ ngx_ssl_recv_chain(ngx_connection_t *c, last = b->last; for ( ;; ) { - - n = ngx_ssl_recv(c, last, b->end - last); + size = b->end - last; + + if (limit) { + if (bytes >= limit) { + return bytes; + } + + if (bytes + size > limit) { + size = (ssize_t) (limit - bytes); + } + } + + n = ngx_ssl_recv(c, last, size); if (n > 0) { last += n; diff -r ee9230cd4bda -r ec81934727a1 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Mon Oct 27 21:14:12 2014 +0300 +++ b/src/event/ngx_event_openssl.h Tue Oct 28 12:29:58 2014 +0300 @@ -194,7 +194,7 @@ ngx_int_t ngx_ssl_get_client_verify(ngx_ ngx_int_t ngx_ssl_handshake(ngx_connection_t *c); ssize_t ngx_ssl_recv(ngx_connection_t *c, u_char *buf, size_t size); ssize_t ngx_ssl_write(ngx_connection_t *c, u_char *data, size_t size); -ssize_t ngx_ssl_recv_chain(ngx_connection_t *c, ngx_chain_t *cl); +ssize_t ngx_ssl_recv_chain(ngx_connection_t *c, ngx_chain_t *cl, off_t limit); ngx_chain_t *ngx_ssl_send_chain(ngx_connection_t *c, ngx_chain_t *in, off_t limit); void ngx_ssl_free_buffer(ngx_connection_t *c); diff -r ee9230cd4bda -r ec81934727a1 src/event/ngx_event_pipe.c --- a/src/event/ngx_event_pipe.c Mon Oct 27 21:14:12 2014 +0300 +++ b/src/event/ngx_event_pipe.c Tue Oct 28 12:29:58 2014 +0300 @@ -270,7 +270,7 @@ ngx_event_pipe_read_upstream(ngx_event_p break; } - n = p->upstream->recv_chain(p->upstream, chain); + n = p->upstream->recv_chain(p->upstream, chain, 0); ngx_log_debug1(NGX_LOG_DEBUG_EVENT, p->log, 0, "pipe recv chain: %z", n); diff -r ee9230cd4bda -r ec81934727a1 src/os/unix/ngx_aio_read_chain.c --- a/src/os/unix/ngx_aio_read_chain.c Mon Oct 27 21:14:12 2014 +0300 +++ b/src/os/unix/ngx_aio_read_chain.c Tue Oct 28 12:29:58 2014 +0300 @@ -11,7 +11,7 @@ ssize_t -ngx_aio_read_chain(ngx_connection_t *c, ngx_chain_t *cl) +ngx_aio_read_chain(ngx_connection_t *c, ngx_chain_t *cl, off_t limit) { int n; u_char *buf, *prev; diff -r ee9230cd4bda -r ec81934727a1 src/os/unix/ngx_os.h --- a/src/os/unix/ngx_os.h Mon Oct 27 21:14:12 2014 +0300 +++ b/src/os/unix/ngx_os.h Tue Oct 28 12:29:58 2014 +0300 @@ -17,7 +17,8 @@ typedef ssize_t (*ngx_recv_pt)(ngx_connection_t *c, u_char *buf, size_t size); -typedef ssize_t (*ngx_recv_chain_pt)(ngx_connection_t *c, ngx_chain_t *in); +typedef ssize_t (*ngx_recv_chain_pt)(ngx_connection_t *c, ngx_chain_t *in, + off_t limit); typedef ssize_t (*ngx_send_pt)(ngx_connection_t *c, u_char *buf, size_t size); typedef ngx_chain_t *(*ngx_send_chain_pt)(ngx_connection_t *c, ngx_chain_t *in, off_t limit); @@ -41,7 +42,7 @@ ngx_int_t ngx_os_signal_process(ngx_cycl ssize_t ngx_unix_recv(ngx_connection_t *c, u_char *buf, size_t size); -ssize_t ngx_readv_chain(ngx_connection_t *c, ngx_chain_t *entry); +ssize_t ngx_readv_chain(ngx_connection_t *c, ngx_chain_t *entry, off_t limit); ssize_t ngx_udp_unix_recv(ngx_connection_t *c, u_char *buf, size_t size); ssize_t ngx_unix_send(ngx_connection_t *c, u_char *buf, size_t size); ngx_chain_t *ngx_writev_chain(ngx_connection_t *c, ngx_chain_t *in, @@ -49,7 +50,7 @@ ngx_chain_t *ngx_writev_chain(ngx_connec #if (NGX_HAVE_AIO) ssize_t ngx_aio_read(ngx_connection_t *c, u_char *buf, size_t size); -ssize_t ngx_aio_read_chain(ngx_connection_t *c, ngx_chain_t *cl); +ssize_t ngx_aio_read_chain(ngx_connection_t *c, ngx_chain_t *cl, off_t limit); ssize_t ngx_aio_write(ngx_connection_t *c, u_char *buf, size_t size); ngx_chain_t *ngx_aio_write_chain(ngx_connection_t *c, ngx_chain_t *in, off_t limit); diff -r ee9230cd4bda -r ec81934727a1 src/os/unix/ngx_readv_chain.c --- a/src/os/unix/ngx_readv_chain.c Mon Oct 27 21:14:12 2014 +0300 +++ b/src/os/unix/ngx_readv_chain.c Tue Oct 28 12:29:58 2014 +0300 @@ -11,7 +11,7 @@ ssize_t -ngx_readv_chain(ngx_connection_t *c, ngx_chain_t *chain) +ngx_readv_chain(ngx_connection_t *c, ngx_chain_t *chain, off_t limit) { u_char *prev; ssize_t n, size; @@ -66,8 +66,20 @@ ngx_readv_chain(ngx_connection_t *c, ngx /* coalesce the neighbouring bufs */ while (chain) { + n = chain->buf->end - chain->buf->last; + + if (limit) { + if (size >= limit) { + break; + } + + if (size + n > limit) { + n = (ssize_t) (limit - size); + } + } + if (prev == chain->buf->last) { - iov->iov_len += chain->buf->end - chain->buf->last; + iov->iov_len += n; } else { if (vec.nelts >= IOV_MAX) { @@ -80,10 +92,10 @@ ngx_readv_chain(ngx_connection_t *c, ngx } iov->iov_base = (void *) chain->buf->last; - iov->iov_len = chain->buf->end - chain->buf->last; + iov->iov_len = n; } - size += chain->buf->end - chain->buf->last; + size += n; prev = chain->buf->end; chain = chain->next; } diff -r ee9230cd4bda -r ec81934727a1 src/os/win32/ngx_os.h --- a/src/os/win32/ngx_os.h Mon Oct 27 21:14:12 2014 +0300 +++ b/src/os/win32/ngx_os.h Tue Oct 28 12:29:58 2014 +0300 @@ -17,7 +17,8 @@ typedef ssize_t (*ngx_recv_pt)(ngx_connection_t *c, u_char *buf, size_t size); -typedef ssize_t (*ngx_recv_chain_pt)(ngx_connection_t *c, ngx_chain_t *in); +typedef ssize_t (*ngx_recv_chain_pt)(ngx_connection_t *c, ngx_chain_t *in, + off_t limit); typedef ssize_t (*ngx_send_pt)(ngx_connection_t *c, u_char *buf, size_t size); typedef ngx_chain_t *(*ngx_send_chain_pt)(ngx_connection_t *c, ngx_chain_t *in, off_t limit); @@ -41,7 +42,7 @@ ssize_t ngx_overlapped_wsarecv(ngx_conne ssize_t ngx_udp_wsarecv(ngx_connection_t *c, u_char *buf, size_t size); ssize_t ngx_udp_overlapped_wsarecv(ngx_connection_t *c, u_char *buf, size_t size); -ssize_t ngx_wsarecv_chain(ngx_connection_t *c, ngx_chain_t *chain); +ssize_t ngx_wsarecv_chain(ngx_connection_t *c, ngx_chain_t *chain, off_t limit); ssize_t ngx_wsasend(ngx_connection_t *c, u_char *buf, size_t size); ssize_t ngx_overlapped_wsasend(ngx_connection_t *c, u_char *buf, size_t size); ngx_chain_t *ngx_wsasend_chain(ngx_connection_t *c, ngx_chain_t *in, diff -r ee9230cd4bda -r ec81934727a1 src/os/win32/ngx_wsarecv_chain.c --- a/src/os/win32/ngx_wsarecv_chain.c Mon Oct 27 21:14:12 2014 +0300 +++ b/src/os/win32/ngx_wsarecv_chain.c Tue Oct 28 12:29:58 2014 +0300 @@ -14,12 +14,12 @@ ssize_t -ngx_wsarecv_chain(ngx_connection_t *c, ngx_chain_t *chain) +ngx_wsarecv_chain(ngx_connection_t *c, ngx_chain_t *chain, off_t limit) { int rc; u_char *prev; u_long bytes, flags; - size_t size; + size_t n, size; ngx_err_t err; ngx_array_t vec; ngx_event_t *rev; @@ -41,8 +41,20 @@ ngx_wsarecv_chain(ngx_connection_t *c, n /* coalesce the neighbouring bufs */ while (chain) { + n = chain->buf->end - chain->buf->last; + + if (limit) { + if (size >= (size_t) limit) { + break; + } + + if (size + n > (size_t) limit) { + n = (size_t) limit - size; + } + } + if (prev == chain->buf->last) { - wsabuf->len += chain->buf->end - chain->buf->last; + wsabuf->len += n; } else { wsabuf = ngx_array_push(&vec); @@ -51,10 +63,10 @@ ngx_wsarecv_chain(ngx_connection_t *c, n } wsabuf->buf = (char *) chain->buf->last; - wsabuf->len = chain->buf->end - chain->buf->last; + wsabuf->len = n; } - size += chain->buf->end - chain->buf->last; + size += n; prev = chain->buf->end; chain = chain->next; } From arut at nginx.com Tue Oct 28 09:35:47 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 28 Oct 2014 09:35:47 +0000 Subject: [nginx] Upstream: proxy_limit_rate and friends. Message-ID: details: http://hg.nginx.org/nginx/rev/973ee2276300 branches: changeset: 5883:973ee2276300 user: Roman Arutyunyan date: Tue Oct 28 12:29:59 2014 +0300 description: Upstream: proxy_limit_rate and friends. The directives limit the upstream read rate. For example, "proxy_limit_rate 42" limits proxy upstream read rate to 42 bytes per second. diffstat: src/event/ngx_event_pipe.c | 41 +++++++++++++++++++++--- src/event/ngx_event_pipe.h | 3 + src/http/modules/ngx_http_fastcgi_module.c | 11 ++++++ src/http/modules/ngx_http_proxy_module.c | 11 ++++++ src/http/modules/ngx_http_scgi_module.c | 11 ++++++ src/http/modules/ngx_http_uwsgi_module.c | 11 ++++++ src/http/ngx_http_upstream.c | 50 +++++++++++++++++++++++++++-- src/http/ngx_http_upstream.h | 1 + 8 files changed, 130 insertions(+), 9 deletions(-) diffs (truncated from 332 to 300 lines): diff -r ec81934727a1 -r 973ee2276300 src/event/ngx_event_pipe.c --- a/src/event/ngx_event_pipe.c Tue Oct 28 12:29:58 2014 +0300 +++ b/src/event/ngx_event_pipe.c Tue Oct 28 12:29:59 2014 +0300 @@ -66,11 +66,13 @@ ngx_event_pipe(ngx_event_pipe_t *p, ngx_ return NGX_ABORT; } - if (rev->active && !rev->ready) { - ngx_add_timer(rev, p->read_timeout); + if (!rev->delayed) { + if (rev->active && !rev->ready) { + ngx_add_timer(rev, p->read_timeout); - } else if (rev->timer_set) { - ngx_del_timer(rev); + } else if (rev->timer_set) { + ngx_del_timer(rev); + } } } @@ -99,9 +101,11 @@ ngx_event_pipe(ngx_event_pipe_t *p, ngx_ static ngx_int_t ngx_event_pipe_read_upstream(ngx_event_pipe_t *p) { + off_t limit; ssize_t n, size; ngx_int_t rc; ngx_buf_t *b; + ngx_msec_t delay; ngx_chain_t *chain, *cl, *ln; if (p->upstream_eof || p->upstream_error || p->upstream_done) { @@ -169,6 +173,25 @@ ngx_event_pipe_read_upstream(ngx_event_p } #endif + if (p->limit_rate) { + if (p->upstream->read->delayed) { + break; + } + + limit = (off_t) p->limit_rate * (ngx_time() - p->start_sec + 1) + - p->read_length; + + if (limit <= 0) { + p->upstream->read->delayed = 1; + delay = (ngx_msec_t) (- limit * 1000 / p->limit_rate + 1); + ngx_add_timer(p->upstream->read, delay); + break; + } + + } else { + limit = 0; + } + if (p->free_raw_bufs) { /* use the free bufs if they exist */ @@ -270,7 +293,7 @@ ngx_event_pipe_read_upstream(ngx_event_p break; } - n = p->upstream->recv_chain(p->upstream, chain, 0); + n = p->upstream->recv_chain(p->upstream, chain, limit); ngx_log_debug1(NGX_LOG_DEBUG_EVENT, p->log, 0, "pipe recv chain: %z", n); @@ -301,6 +324,8 @@ ngx_event_pipe_read_upstream(ngx_event_p } } + delay = p->limit_rate ? (ngx_msec_t) n * 1000 / p->limit_rate : 0; + p->read_length += n; cl = chain; p->free_raw_bufs = NULL; @@ -337,6 +362,12 @@ ngx_event_pipe_read_upstream(ngx_event_p ln->next = p->free_raw_bufs; p->free_raw_bufs = cl; } + + if (delay > 0) { + p->upstream->read->delayed = 1; + ngx_add_timer(p->upstream->read, delay); + break; + } } #if (NGX_DEBUG) diff -r ec81934727a1 -r 973ee2276300 src/event/ngx_event_pipe.h --- a/src/event/ngx_event_pipe.h Tue Oct 28 12:29:58 2014 +0300 +++ b/src/event/ngx_event_pipe.h Tue Oct 28 12:29:59 2014 +0300 @@ -80,6 +80,9 @@ struct ngx_event_pipe_s { size_t preread_size; ngx_buf_t *buf_to_file; + size_t limit_rate; + time_t start_sec; + ngx_temp_file_t *temp_file; /* STUB */ int num; diff -r ec81934727a1 -r 973ee2276300 src/http/modules/ngx_http_fastcgi_module.c --- a/src/http/modules/ngx_http_fastcgi_module.c Tue Oct 28 12:29:58 2014 +0300 +++ b/src/http/modules/ngx_http_fastcgi_module.c Tue Oct 28 12:29:59 2014 +0300 @@ -333,6 +333,13 @@ static ngx_command_t ngx_http_fastcgi_c offsetof(ngx_http_fastcgi_loc_conf_t, upstream.force_ranges), NULL }, + { ngx_string("fastcgi_limit_rate"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_size_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_fastcgi_loc_conf_t, upstream.limit_rate), + NULL }, + #if (NGX_HTTP_CACHE) { ngx_string("fastcgi_cache"), @@ -2350,6 +2357,7 @@ ngx_http_fastcgi_create_loc_conf(ngx_con conf->upstream.send_lowat = NGX_CONF_UNSET_SIZE; conf->upstream.buffer_size = NGX_CONF_UNSET_SIZE; + conf->upstream.limit_rate = NGX_CONF_UNSET_SIZE; conf->upstream.busy_buffers_size_conf = NGX_CONF_UNSET_SIZE; conf->upstream.max_temp_file_size_conf = NGX_CONF_UNSET_SIZE; @@ -2446,6 +2454,9 @@ ngx_http_fastcgi_merge_loc_conf(ngx_conf prev->upstream.buffer_size, (size_t) ngx_pagesize); + ngx_conf_merge_size_value(conf->upstream.limit_rate, + prev->upstream.limit_rate, 0); + ngx_conf_merge_bufs_value(conf->upstream.bufs, prev->upstream.bufs, 8, ngx_pagesize); diff -r ec81934727a1 -r 973ee2276300 src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c Tue Oct 28 12:29:58 2014 +0300 +++ b/src/http/modules/ngx_http_proxy_module.c Tue Oct 28 12:29:59 2014 +0300 @@ -396,6 +396,13 @@ static ngx_command_t ngx_http_proxy_com offsetof(ngx_http_proxy_loc_conf_t, upstream.force_ranges), NULL }, + { ngx_string("proxy_limit_rate"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_size_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_proxy_loc_conf_t, upstream.limit_rate), + NULL }, + #if (NGX_HTTP_CACHE) { ngx_string("proxy_cache"), @@ -2490,6 +2497,7 @@ ngx_http_proxy_create_loc_conf(ngx_conf_ conf->upstream.send_lowat = NGX_CONF_UNSET_SIZE; conf->upstream.buffer_size = NGX_CONF_UNSET_SIZE; + conf->upstream.limit_rate = NGX_CONF_UNSET_SIZE; conf->upstream.busy_buffers_size_conf = NGX_CONF_UNSET_SIZE; conf->upstream.max_temp_file_size_conf = NGX_CONF_UNSET_SIZE; @@ -2601,6 +2609,9 @@ ngx_http_proxy_merge_loc_conf(ngx_conf_t prev->upstream.buffer_size, (size_t) ngx_pagesize); + ngx_conf_merge_size_value(conf->upstream.limit_rate, + prev->upstream.limit_rate, 0); + ngx_conf_merge_bufs_value(conf->upstream.bufs, prev->upstream.bufs, 8, ngx_pagesize); diff -r ec81934727a1 -r 973ee2276300 src/http/modules/ngx_http_scgi_module.c --- a/src/http/modules/ngx_http_scgi_module.c Tue Oct 28 12:29:58 2014 +0300 +++ b/src/http/modules/ngx_http_scgi_module.c Tue Oct 28 12:29:59 2014 +0300 @@ -190,6 +190,13 @@ static ngx_command_t ngx_http_scgi_comma offsetof(ngx_http_scgi_loc_conf_t, upstream.force_ranges), NULL }, + { ngx_string("scgi_limit_rate"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_size_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_scgi_loc_conf_t, upstream.limit_rate), + NULL }, + #if (NGX_HTTP_CACHE) { ngx_string("scgi_cache"), @@ -1109,6 +1116,7 @@ ngx_http_scgi_create_loc_conf(ngx_conf_t conf->upstream.send_lowat = NGX_CONF_UNSET_SIZE; conf->upstream.buffer_size = NGX_CONF_UNSET_SIZE; + conf->upstream.limit_rate = NGX_CONF_UNSET_SIZE; conf->upstream.busy_buffers_size_conf = NGX_CONF_UNSET_SIZE; conf->upstream.max_temp_file_size_conf = NGX_CONF_UNSET_SIZE; @@ -1200,6 +1208,9 @@ ngx_http_scgi_merge_loc_conf(ngx_conf_t prev->upstream.buffer_size, (size_t) ngx_pagesize); + ngx_conf_merge_size_value(conf->upstream.limit_rate, + prev->upstream.limit_rate, 0); + ngx_conf_merge_bufs_value(conf->upstream.bufs, prev->upstream.bufs, 8, ngx_pagesize); diff -r ec81934727a1 -r 973ee2276300 src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c Tue Oct 28 12:29:58 2014 +0300 +++ b/src/http/modules/ngx_http_uwsgi_module.c Tue Oct 28 12:29:59 2014 +0300 @@ -245,6 +245,13 @@ static ngx_command_t ngx_http_uwsgi_comm offsetof(ngx_http_uwsgi_loc_conf_t, upstream.force_ranges), NULL }, + { ngx_string("uwsgi_limit_rate"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_size_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_uwsgi_loc_conf_t, upstream.limit_rate), + NULL }, + #if (NGX_HTTP_CACHE) { ngx_string("uwsgi_cache"), @@ -1289,6 +1296,7 @@ ngx_http_uwsgi_create_loc_conf(ngx_conf_ conf->upstream.send_lowat = NGX_CONF_UNSET_SIZE; conf->upstream.buffer_size = NGX_CONF_UNSET_SIZE; + conf->upstream.limit_rate = NGX_CONF_UNSET_SIZE; conf->upstream.busy_buffers_size_conf = NGX_CONF_UNSET_SIZE; conf->upstream.max_temp_file_size_conf = NGX_CONF_UNSET_SIZE; @@ -1387,6 +1395,9 @@ ngx_http_uwsgi_merge_loc_conf(ngx_conf_t prev->upstream.buffer_size, (size_t) ngx_pagesize); + ngx_conf_merge_size_value(conf->upstream.limit_rate, + prev->upstream.limit_rate, 0); + ngx_conf_merge_bufs_value(conf->upstream.bufs, prev->upstream.bufs, 8, ngx_pagesize); diff -r ec81934727a1 -r 973ee2276300 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Tue Oct 28 12:29:58 2014 +0300 +++ b/src/http/ngx_http_upstream.c Tue Oct 28 12:29:59 2014 +0300 @@ -2579,6 +2579,8 @@ ngx_http_upstream_send_response(ngx_http p->downstream = c; p->pool = r->pool; p->log = c->log; + p->limit_rate = u->conf->limit_rate; + p->start_sec = ngx_time(); p->cacheable = u->cacheable || u->store; @@ -3253,21 +3255,61 @@ static void ngx_http_upstream_process_upstream(ngx_http_request_t *r, ngx_http_upstream_t *u) { + ngx_event_t *rev; + ngx_event_pipe_t *p; ngx_connection_t *c; c = u->peer.connection; + p = u->pipe; + rev = c->read; ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, "http upstream process upstream"); c->log->action = "reading upstream"; - if (c->read->timedout) { - u->pipe->upstream_error = 1; - ngx_connection_error(c, NGX_ETIMEDOUT, "upstream timed out"); + if (rev->timedout) { + + if (rev->delayed) { + + rev->timedout = 0; + rev->delayed = 0; + + if (!rev->ready) { + ngx_add_timer(rev, p->read_timeout); + + if (ngx_handle_read_event(rev, 0) != NGX_OK) { + ngx_http_upstream_finalize_request(r, u, NGX_ERROR); + } + + return; + } + + if (ngx_event_pipe(p, 0) == NGX_ABORT) { + ngx_http_upstream_finalize_request(r, u, NGX_ERROR); + return; + } + + } else { + p->upstream_error = 1; + ngx_connection_error(c, NGX_ETIMEDOUT, "upstream timed out"); From arut at nginx.com Tue Oct 28 09:35:50 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 28 Oct 2014 09:35:50 +0000 Subject: [nginx] Write filter: stored delay in a variable, no functional ... Message-ID: details: http://hg.nginx.org/nginx/rev/8486205d10db branches: changeset: 5884:8486205d10db user: Roman Arutyunyan date: Tue Oct 28 12:29:59 2014 +0300 description: Write filter: stored delay in a variable, no functional changes. The code is now similar to ngx_event_pipe_read_upstream(). diffstat: src/http/ngx_http_write_filter_module.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r 973ee2276300 -r 8486205d10db src/http/ngx_http_write_filter_module.c --- a/src/http/ngx_http_write_filter_module.c Tue Oct 28 12:29:59 2014 +0300 +++ b/src/http/ngx_http_write_filter_module.c Tue Oct 28 12:29:59 2014 +0300 @@ -219,8 +219,8 @@ ngx_http_write_filter(ngx_http_request_t if (limit <= 0) { c->write->delayed = 1; - ngx_add_timer(c->write, - (ngx_msec_t) (- limit * 1000 / r->limit_rate + 1)); + delay = (ngx_msec_t) (- limit * 1000 / r->limit_rate + 1); + ngx_add_timer(c->write, delay); c->buffered |= NGX_HTTP_WRITE_BUFFERED; From arut at nginx.com Tue Oct 28 12:48:14 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 28 Oct 2014 12:48:14 +0000 Subject: [nginx] Upstream: limited next_upstream time and tries when reso... Message-ID: details: http://hg.nginx.org/nginx/rev/5a042519bfe7 branches: changeset: 5885:5a042519bfe7 user: Gu Feng date: Mon Oct 27 19:52:03 2014 +0800 description: Upstream: limited next_upstream time and tries when resolving DNS. When got multiple upstream IP addresses using DNS resolving, the number of upstreams tries and the maxinum time spent for these tries were not affected. This patch fixed it. diffstat: src/http/ngx_http_upstream.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diffs (18 lines): diff -r 8486205d10db -r 5a042519bfe7 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Tue Oct 28 12:29:59 2014 +0300 +++ b/src/http/ngx_http_upstream.c Mon Oct 27 19:52:03 2014 +0800 @@ -981,6 +981,14 @@ ngx_http_upstream_resolve_handler(ngx_re ngx_resolve_name_done(ctx); ur->ctx = NULL; + u->peer.start_time = ngx_current_msec; + + if (u->conf->next_upstream_tries + && u->peer.tries > u->conf->next_upstream_tries) + { + u->peer.tries = u->conf->next_upstream_tries; + } + ngx_http_upstream_connect(r, u); failed: From mdounin at mdounin.ru Tue Oct 28 12:51:19 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 28 Oct 2014 15:51:19 +0300 Subject: SIGHUP and ngx_module_t In-Reply-To: References: Message-ID: <20141028125119.GK45418@mdounin.ru> Hello! On Tue, Oct 28, 2014 at 08:30:15AM +0200, Donatas Abraitis wrote: > Hello, > > is it true, that SIGHUP is not handled by ngx_module_t /* exit process */ ? > > I mean if I have these defined: > > ngx_http_modsecurity_terminate, /* exit process */ > ngx_http_modsecurity_terminate, /* exit master */ > > So, sending SIGHUP to the master it doesn't call the > ngx_http_modsecurity_terminate(). If sending SIGTERM it does. How is it for > real? The exit_process callback is called on process exit, not on SIGHUP. There is a difference, see here: http://nginx.org/en/docs/control.html -- Maxim Dounin http://nginx.org/ From arut at nginx.com Tue Oct 28 12:52:38 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 28 Oct 2014 15:52:38 +0300 Subject: [PATCH] Upstream: limited next_upstream time and tries when resolving DNS In-Reply-To: References: Message-ID: On 27 Oct 2014, at 14:53, Gu Feng wrote: > # HG changeset patch > # User Gu Feng > # Date 1414410723 -28800 > # Mon Oct 27 19:52:03 2014 +0800 > # Node ID ccc3b86835da54273c333e7d6a00f68e0b0e6c27 > # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 > Upstream: limited next_upstream time and tries when resolving DNS. > > When got multiple upstream IP addresses using DNS resolving, the number of > upstreams tries and the maxinum time spent for these tries were not affected. > This patch fixed it. > > diff -r 973fded4f461 -r ccc3b86835da src/http/ngx_http_upstream.c > --- a/src/http/ngx_http_upstream.c Wed Oct 15 22:57:23 2014 +0400 > +++ b/src/http/ngx_http_upstream.c Mon Oct 27 19:52:03 2014 +0800 > @@ -974,6 +974,14 @@ ngx_http_upstream_resolve_handler(ngx_re > ngx_resolve_name_done(ctx); > ur->ctx = NULL; > > + u->peer.start_time = ngx_current_msec; > + > + if (u->conf->next_upstream_tries > + && u->peer.tries > u->conf->next_upstream_tries) > + { > + u->peer.tries = u->conf->next_upstream_tries; > + } > + > ngx_http_upstream_connect(r, u); > > failed: > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > Thanks, committed. From kou at cozmixng.org Tue Oct 28 13:04:00 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Tue, 28 Oct 2014 22:04:00 +0900 (JST) Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141027174711.GD45418@mdounin.ru> References: <20141018.004250.1469014097857676892.kou@cozmixng.org> <20141028.000126.1606071279740532747.kou@cozmixng.org> <20141027174711.GD45418@mdounin.ru> Message-ID: <20141028.220400.1975564655674477555.kou@cozmixng.org> Hi, In <20141027174711.GD45418 at mdounin.ru> "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Mon, 27 Oct 2014 20:47:11 +0300, Maxim Dounin wrote: > - the NGX_GNUC_64 macro was removed, __MINGW64_VERSION_MAJOR > tested directly instead; > > - the #include <_mingw.h> removed as tests are after includes now, > see [1]; > > - some style nits. > > Please test if it still works for you. Thanks for providing your patch. I confirmed that it works well with both x86_64-w64-mingw32-gcc and i686-w64-mingw32-gcc! Please push the patch. Thanks, -- kou From mdounin at mdounin.ru Tue Oct 28 13:31:26 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 28 Oct 2014 13:31:26 +0000 Subject: [nginx] Win32: made build-able with MinGW-w64 gcc. Message-ID: details: http://hg.nginx.org/nginx/rev/5d4e4619982f branches: changeset: 5886:5d4e4619982f user: Kouhei Sutou date: Wed Oct 15 21:15:01 2014 +0900 description: Win32: made build-able with MinGW-w64 gcc. diffstat: src/os/win32/ngx_win32_config.h | 37 +++++++++++++++++++++++++++++++++++-- 1 files changed, 35 insertions(+), 2 deletions(-) diffs (88 lines): diff --git a/src/os/win32/ngx_win32_config.h b/src/os/win32/ngx_win32_config.h --- a/src/os/win32/ngx_win32_config.h +++ b/src/os/win32/ngx_win32_config.h @@ -37,14 +37,24 @@ #include #include /* offsetof() */ -#ifdef __GNUC__ +#ifdef __MINGW64_VERSION_MAJOR + +/* GCC MinGW-w64 supports _FILE_OFFSET_BITS */ +#define _FILE_OFFSET_BITS 64 + +#elif defined __GNUC__ + /* GCC MinGW's stdio.h includes sys/types.h */ #define _OFF_T_ + #endif #include #include #include +#ifdef __MINGW64_VERSION_MAJOR +#include +#endif #include #include @@ -136,16 +146,21 @@ typedef unsigned short int uint16_t; typedef __int64 int64_t; typedef unsigned __int64 uint64_t; -#ifndef __WATCOMC__ +#if !defined(__WATCOMC__) && !defined(__MINGW64_VERSION_MAJOR) typedef int intptr_t; typedef u_int uintptr_t; #endif +#ifndef __MINGW64_VERSION_MAJOR + /* Windows defines off_t as long, which is 32-bit */ typedef __int64 off_t; #define _OFF_T_DEFINED +#endif + + #ifdef __WATCOMC__ /* off_t is redefined by sys/types.h used by zlib.h */ @@ -164,17 +179,35 @@ typedef unsigned int ino_t; #endif +#ifndef __MINGW64_VERSION_MAJOR typedef int ssize_t; +#endif + + typedef uint32_t in_addr_t; typedef u_short in_port_t; typedef int sig_atomic_t; +#ifdef _WIN64 + +#define NGX_PTR_SIZE 8 +#define NGX_SIZE_T_LEN (sizeof("-9223372036854775808") - 1) +#define NGX_MAX_SIZE_T_VALUE 9223372036854775807 +#define NGX_TIME_T_LEN (sizeof("-9223372036854775808") - 1) +#define NGX_TIME_T_SIZE 8 + +#else + #define NGX_PTR_SIZE 4 #define NGX_SIZE_T_LEN (sizeof("-2147483648") - 1) #define NGX_MAX_SIZE_T_VALUE 2147483647 #define NGX_TIME_T_LEN (sizeof("-2147483648") - 1) #define NGX_TIME_T_SIZE 4 + +#endif + + #define NGX_OFF_T_LEN (sizeof("-9223372036854775807") - 1) #define NGX_MAX_OFF_T_VALUE 9223372036854775807 #define NGX_SIG_ATOMIC_T_SIZE 4 From mdounin at mdounin.ru Tue Oct 28 13:38:02 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 28 Oct 2014 16:38:02 +0300 Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141028.220400.1975564655674477555.kou@cozmixng.org> References: <20141018.004250.1469014097857676892.kou@cozmixng.org> <20141028.000126.1606071279740532747.kou@cozmixng.org> <20141027174711.GD45418@mdounin.ru> <20141028.220400.1975564655674477555.kou@cozmixng.org> Message-ID: <20141028133802.GL45418@mdounin.ru> Hello! On Tue, Oct 28, 2014 at 10:04:00PM +0900, Kouhei Sutou wrote: > Hi, > > In <20141027174711.GD45418 at mdounin.ru> > "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Mon, 27 Oct 2014 20:47:11 +0300, > Maxim Dounin wrote: > > > - the NGX_GNUC_64 macro was removed, __MINGW64_VERSION_MAJOR > > tested directly instead; > > > > - the #include <_mingw.h> removed as tests are after includes now, > > see [1]; > > > > - some style nits. > > > > Please test if it still works for you. > > Thanks for providing your patch. > > I confirmed that it works well with both > x86_64-w64-mingw32-gcc and i686-w64-mingw32-gcc! > > > Please push the patch. Committed, thanks. -- Maxim Dounin http://nginx.org/ From vbart at nginx.com Tue Oct 28 13:39:06 2014 From: vbart at nginx.com (Valentin V. Bartenev) Date: Tue, 28 Oct 2014 16:39:06 +0300 Subject: [PATCH] SPDY: stop emitting empty header values In-Reply-To: References: <1424354.N8cOoFmhc0@vbart-workstation> Message-ID: <6234114.T01q3fE4Uv@vbart-workstation> On Monday 27 October 2014 14:33:15 Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1414445156 25200 > # Mon Oct 27 14:25:56 2014 -0700 > # Node ID e590d484a7f9d2750e2cd45ea16c601ae01cb607 > # Parent ee9230cd4bda9f9e7e83f7baf37d66ad6bb9b0c1 > SPDY: stop emitting empty header values. > > Previously, nginx would emit empty values in a header with multiple, > NULL-separated values. > > This is forbidden by the SPDY specification, which requires headers to > have either a single (possibly empty) value or multiple, NULL-separated > non-empty values. > > Signed-off-by: Piotr Sikora > > diff -r ee9230cd4bda -r e590d484a7f9 src/http/ngx_http_spdy_filter_module.c > --- a/src/http/ngx_http_spdy_filter_module.c Mon Oct 27 21:14:12 2014 +0300 > +++ b/src/http/ngx_http_spdy_filter_module.c Mon Oct 27 14:25:56 2014 -0700 > @@ -493,9 +493,13 @@ ngx_http_spdy_header_filter(ngx_http_req > continue; > } > > - *last++ = '\0'; > + if (h[j].value.len) { > + if (last != p) { > + *last++ = '\0'; > + } > > - last = ngx_cpymem(last, h[j].value.data, h[j].value.len); > + last = ngx_cpymem(last, h[j].value.data, h[j].value.len); > + } > > h[j].hash = 2; > } > Ok. Ruslan? Maxim? Will you approve the patch for commit? wbr, Valentin V. Bartenev From kou at cozmixng.org Tue Oct 28 13:44:26 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Tue, 28 Oct 2014 22:44:26 +0900 (JST) Subject: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux In-Reply-To: <20141028133802.GL45418@mdounin.ru> References: <20141027174711.GD45418@mdounin.ru> <20141028.220400.1975564655674477555.kou@cozmixng.org> <20141028133802.GL45418@mdounin.ru> Message-ID: <20141028.224426.1385733153224259244.kou@cozmixng.org> Hi, In <20141028133802.GL45418 at mdounin.ru> "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" on Tue, 28 Oct 2014 16:38:02 +0300, Maxim Dounin wrote: >> > - the NGX_GNUC_64 macro was removed, __MINGW64_VERSION_MAJOR >> > tested directly instead; >> > >> > - the #include <_mingw.h> removed as tests are after includes now, >> > see [1]; >> > >> > - some style nits. >> > >> > Please test if it still works for you. >> >> Thanks for providing your patch. >> >> I confirmed that it works well with both >> x86_64-w64-mingw32-gcc and i686-w64-mingw32-gcc! >> >> >> Please push the patch. > > Committed, thanks. Thanks!!! -- kou From kou at cozmixng.org Tue Oct 28 14:19:17 2014 From: kou at cozmixng.org (Kouhei Sutou) Date: Tue, 28 Oct 2014 23:19:17 +0900 (JST) Subject: [PATCH] Add Win32 related files into tar.gz Message-ID: <20141028.231917.907259224060825208.kou@cozmixng.org> Hi, The current nginx can be built with MinGW-w64. Could you add Win32 related files into tar.gz? If tar.gz includes Win32 related file, I can build nginx for Windows from tar.gz. Thanks, -- kou -------------- next part -------------- A non-text attachment was scrubbed... Name: win32-add-win32-related-files-into-tar-gz.patch Type: text/x-patch Size: 986 bytes Desc: not available URL: From mdounin at mdounin.ru Tue Oct 28 14:52:33 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 28 Oct 2014 14:52:33 +0000 Subject: [nginx] Updated OpenSSL used for win32 builds. Message-ID: details: http://hg.nginx.org/nginx/rev/e40ce16b2c42 branches: changeset: 5887:e40ce16b2c42 user: Maxim Dounin date: Tue Oct 28 17:35:00 2014 +0300 description: Updated OpenSSL used for win32 builds. diffstat: misc/GNUmakefile | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/misc/GNUmakefile b/misc/GNUmakefile --- a/misc/GNUmakefile +++ b/misc/GNUmakefile @@ -5,7 +5,7 @@ NGINX = nginx-$(VER) TEMP = tmp OBJS = objs.msvc8 -OPENSSL = openssl-1.0.1i +OPENSSL = openssl-1.0.1j ZLIB = zlib-1.2.8 PCRE = pcre-8.35 From mdounin at mdounin.ru Tue Oct 28 15:17:19 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 28 Oct 2014 15:17:19 +0000 Subject: [nginx] nginx-1.7.7-RELEASE Message-ID: details: http://hg.nginx.org/nginx/rev/6d2fbc30f8a7 branches: changeset: 5888:6d2fbc30f8a7 user: Maxim Dounin date: Tue Oct 28 18:04:46 2014 +0300 description: nginx-1.7.7-RELEASE diffstat: docs/xml/nginx/changes.xml | 110 +++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 110 insertions(+), 0 deletions(-) diffs (120 lines): diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -5,6 +5,116 @@ + + + + +?????? nginx ????????? ??? ??????????? ?????? "Vary" +? ????????? ?????? ???????. + + +now nginx takes into account the "Vary" +header line in a backend response while caching. + + + + + +????????? proxy_force_ranges, fastcgi_force_ranges, +scgi_force_ranges ? uwsgi_force_ranges. + + +the "proxy_force_ranges", "fastcgi_force_ranges", +"scgi_force_ranges", and "uwsgi_force_ranges" directives. + + + + + +????????? proxy_limit_rate, fastcgi_limit_rate, +scgi_limit_rate ? uwsgi_limit_rate. + + +the "proxy_limit_rate", "fastcgi_limit_rate", +"scgi_limit_rate", and "uwsgi_limit_rate" directives. + + + + + +???????? Vary ???????? proxy_ignore_headers, fastcgi_ignore_headers, +scgi_ignore_headers ? uwsgi_ignore_headers. + + +the "Vary" parameter of the "proxy_ignore_headers", "fastcgi_ignore_headers", +"scgi_ignore_headers", and "uwsgi_ignore_headers" directives. + + + + + +????????? ????? ??????, ??????????? ?? ??????? +??? ?????????????????? ?????????????, +????? ?? ???????????? ???????, +???? ?????????????? ????????? gzip ??? gunzip. + + +the last part of a response received from a backend +with unbufferred proxy +might not be sent to a client +if "gzip" or "gunzip" directives were used. + + + + + +? ????????? proxy_cache_revalidate.
+??????? Piotr Sikora. + + +in the "proxy_cache_revalidate" directive.
+Thanks to Piotr Sikora. + + + + + +? ????????? ??????.
+??????? Yichun Zhang ? ??????? ?????????. + + +in error handling.
+Thanks to Yichun Zhang and Daniil Bondarev. + + + + + +? ?????????? +proxy_next_upstream_tries ? proxy_next_upstream_timeout.
+??????? Feng Gu. + + +in the "proxy_next_upstream_tries" and "proxy_next_upstream_timeout" +directives.
+Thanks to Feng Gu. + + + + + +nginx/Windows ?? ????????? ? MinGW-w64 gcc.
+??????? Kouhei Sutou. + + +nginx/Windows could not be built with MinGW-w64 gcc.
+Thanks to Kouhei Sutou. + + + + + + From mdounin at mdounin.ru Tue Oct 28 15:17:22 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 28 Oct 2014 15:17:22 +0000 Subject: [nginx] release-1.7.7 tag Message-ID: details: http://hg.nginx.org/nginx/rev/0103dc197a90 branches: changeset: 5889:0103dc197a90 user: Maxim Dounin date: Tue Oct 28 18:04:46 2014 +0300 description: release-1.7.7 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -376,3 +376,4 @@ fe7cd01828d5ca7491059f0690bb4453645eb28b cbb146b120296852e781079d5138b04495bab6df release-1.7.4 fe129aa02db9001d220f1db7c3c056f79482c111 release-1.7.5 a8d111bb68847f61d682a3c8792fecb2e52efa2c release-1.7.6 +6d2fbc30f8a7f70136cf08f32d5ff3179d524873 release-1.7.7 From donatas.abraitis at gmail.com Tue Oct 28 15:22:38 2014 From: donatas.abraitis at gmail.com (Donatas Abraitis) Date: Tue, 28 Oct 2014 17:22:38 +0200 Subject: nginx-devel Digest, Vol 60, Issue 34 In-Reply-To: References: Message-ID: Maxim, but is it possible somehow to implement to handle SIGHUP the same way as exit_process? On Tue, Oct 28, 2014 at 5:17 PM, wrote: > Send nginx-devel mailing list submissions to > nginx-devel at nginx.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.nginx.org/mailman/listinfo/nginx-devel > or, via email, send a message with subject or body 'help' to > nginx-devel-request at nginx.org > > You can reach the person managing the list at > nginx-devel-owner at nginx.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of nginx-devel digest..." > > > Today's Topics: > > 1. [nginx] Upstream: limited next_upstream time and tries when > reso... (Roman Arutyunyan) > 2. Re: SIGHUP and ngx_module_t (Maxim Dounin) > 3. Re: [PATCH] Upstream: limited next_upstream time and tries > when resolving DNS (Roman Arutyunyan) > 4. Re: [PATCH] Support cross compiling with MinGW-w64 on Debian > GNU/Linux (Kouhei Sutou) > 5. [nginx] Win32: made build-able with MinGW-w64 gcc. (Maxim Dounin) > 6. Re: [PATCH] Support cross compiling with MinGW-w64 on Debian > GNU/Linux (Maxim Dounin) > 7. Re: [PATCH] SPDY: stop emitting empty header values > (Valentin V. Bartenev) > 8. Re: [PATCH] Support cross compiling with MinGW-w64 on Debian > GNU/Linux (Kouhei Sutou) > 9. [PATCH] Add Win32 related files into tar.gz (Kouhei Sutou) > 10. [nginx] Updated OpenSSL used for win32 builds. (Maxim Dounin) > 11. [nginx] nginx-1.7.7-RELEASE (Maxim Dounin) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 28 Oct 2014 12:48:14 +0000 > From: Roman Arutyunyan > To: nginx-devel at nginx.org > Subject: [nginx] Upstream: limited next_upstream time and tries when > reso... > Message-ID: > > Content-Type: text/plain; charset="us-ascii" > > details: http://hg.nginx.org/nginx/rev/5a042519bfe7 > branches: > changeset: 5885:5a042519bfe7 > user: Gu Feng > date: Mon Oct 27 19:52:03 2014 +0800 > description: > Upstream: limited next_upstream time and tries when resolving DNS. > > When got multiple upstream IP addresses using DNS resolving, the number of > upstreams tries and the maxinum time spent for these tries were not > affected. > This patch fixed it. > > diffstat: > > src/http/ngx_http_upstream.c | 8 ++++++++ > 1 files changed, 8 insertions(+), 0 deletions(-) > > diffs (18 lines): > > diff -r 8486205d10db -r 5a042519bfe7 src/http/ngx_http_upstream.c > --- a/src/http/ngx_http_upstream.c Tue Oct 28 12:29:59 2014 +0300 > +++ b/src/http/ngx_http_upstream.c Mon Oct 27 19:52:03 2014 +0800 > @@ -981,6 +981,14 @@ ngx_http_upstream_resolve_handler(ngx_re > ngx_resolve_name_done(ctx); > ur->ctx = NULL; > > + u->peer.start_time = ngx_current_msec; > + > + if (u->conf->next_upstream_tries > + && u->peer.tries > u->conf->next_upstream_tries) > + { > + u->peer.tries = u->conf->next_upstream_tries; > + } > + > ngx_http_upstream_connect(r, u); > > failed: > > > > ------------------------------ > > Message: 2 > Date: Tue, 28 Oct 2014 15:51:19 +0300 > From: Maxim Dounin > To: nginx-devel at nginx.org > Subject: Re: SIGHUP and ngx_module_t > Message-ID: <20141028125119.GK45418 at mdounin.ru> > Content-Type: text/plain; charset=us-ascii > > Hello! > > On Tue, Oct 28, 2014 at 08:30:15AM +0200, Donatas Abraitis wrote: > > > Hello, > > > > is it true, that SIGHUP is not handled by ngx_module_t /* exit process > */ ? > > > > I mean if I have these defined: > > > > ngx_http_modsecurity_terminate, /* exit process */ > > ngx_http_modsecurity_terminate, /* exit master */ > > > > So, sending SIGHUP to the master it doesn't call the > > ngx_http_modsecurity_terminate(). If sending SIGTERM it does. How is it > for > > real? > > The exit_process callback is called on process exit, not on > SIGHUP. There is a difference, see here: > > http://nginx.org/en/docs/control.html > > -- > Maxim Dounin > http://nginx.org/ > > > > ------------------------------ > > Message: 3 > Date: Tue, 28 Oct 2014 15:52:38 +0300 > From: Roman Arutyunyan > To: nginx-devel at nginx.org > Subject: Re: [PATCH] Upstream: limited next_upstream time and tries > when resolving DNS > Message-ID: > Content-Type: text/plain; charset=us-ascii > > > On 27 Oct 2014, at 14:53, Gu Feng wrote: > > > # HG changeset patch > > # User Gu Feng > > # Date 1414410723 -28800 > > # Mon Oct 27 19:52:03 2014 +0800 > > # Node ID ccc3b86835da54273c333e7d6a00f68e0b0e6c27 > > # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 > > Upstream: limited next_upstream time and tries when resolving DNS. > > > > When got multiple upstream IP addresses using DNS resolving, the number > of > > upstreams tries and the maxinum time spent for these tries were not > affected. > > This patch fixed it. > > > > diff -r 973fded4f461 -r ccc3b86835da src/http/ngx_http_upstream.c > > --- a/src/http/ngx_http_upstream.c Wed Oct 15 22:57:23 2014 +0400 > > +++ b/src/http/ngx_http_upstream.c Mon Oct 27 19:52:03 2014 +0800 > > @@ -974,6 +974,14 @@ ngx_http_upstream_resolve_handler(ngx_re > > ngx_resolve_name_done(ctx); > > ur->ctx = NULL; > > > > + u->peer.start_time = ngx_current_msec; > > + > > + if (u->conf->next_upstream_tries > > + && u->peer.tries > u->conf->next_upstream_tries) > > + { > > + u->peer.tries = u->conf->next_upstream_tries; > > + } > > + > > ngx_http_upstream_connect(r, u); > > > > failed: > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > Thanks, committed. > > > ------------------------------ > > Message: 4 > Date: Tue, 28 Oct 2014 22:04:00 +0900 (JST) > From: Kouhei Sutou > To: nginx-devel at nginx.org > Subject: Re: [PATCH] Support cross compiling with MinGW-w64 on Debian > GNU/Linux > Message-ID: <20141028.220400.1975564655674477555.kou at cozmixng.org> > Content-Type: Text/Plain; charset=us-ascii > > Hi, > > In <20141027174711.GD45418 at mdounin.ru> > "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" > on Mon, 27 Oct 2014 20:47:11 +0300, > Maxim Dounin wrote: > > > - the NGX_GNUC_64 macro was removed, __MINGW64_VERSION_MAJOR > > tested directly instead; > > > > - the #include <_mingw.h> removed as tests are after includes now, > > see [1]; > > > > - some style nits. > > > > Please test if it still works for you. > > Thanks for providing your patch. > > I confirmed that it works well with both > x86_64-w64-mingw32-gcc and i686-w64-mingw32-gcc! > > > Please push the patch. > > > Thanks, > -- > kou > > > > ------------------------------ > > Message: 5 > Date: Tue, 28 Oct 2014 13:31:26 +0000 > From: Maxim Dounin > To: nginx-devel at nginx.org > Subject: [nginx] Win32: made build-able with MinGW-w64 gcc. > Message-ID: > > Content-Type: text/plain; charset="us-ascii" > > details: http://hg.nginx.org/nginx/rev/5d4e4619982f > branches: > changeset: 5886:5d4e4619982f > user: Kouhei Sutou > date: Wed Oct 15 21:15:01 2014 +0900 > description: > Win32: made build-able with MinGW-w64 gcc. > > diffstat: > > src/os/win32/ngx_win32_config.h | 37 > +++++++++++++++++++++++++++++++++++-- > 1 files changed, 35 insertions(+), 2 deletions(-) > > diffs (88 lines): > > diff --git a/src/os/win32/ngx_win32_config.h > b/src/os/win32/ngx_win32_config.h > --- a/src/os/win32/ngx_win32_config.h > +++ b/src/os/win32/ngx_win32_config.h > @@ -37,14 +37,24 @@ > #include > #include /* offsetof() */ > > -#ifdef __GNUC__ > +#ifdef __MINGW64_VERSION_MAJOR > + > +/* GCC MinGW-w64 supports _FILE_OFFSET_BITS */ > +#define _FILE_OFFSET_BITS 64 > + > +#elif defined __GNUC__ > + > /* GCC MinGW's stdio.h includes sys/types.h */ > #define _OFF_T_ > + > #endif > > #include > #include > #include > +#ifdef __MINGW64_VERSION_MAJOR > +#include > +#endif > #include > #include > > @@ -136,16 +146,21 @@ typedef unsigned short int uint16_t; > typedef __int64 int64_t; > typedef unsigned __int64 uint64_t; > > -#ifndef __WATCOMC__ > +#if !defined(__WATCOMC__) && !defined(__MINGW64_VERSION_MAJOR) > typedef int intptr_t; > typedef u_int uintptr_t; > #endif > > > +#ifndef __MINGW64_VERSION_MAJOR > + > /* Windows defines off_t as long, which is 32-bit */ > typedef __int64 off_t; > #define _OFF_T_DEFINED > > +#endif > + > + > #ifdef __WATCOMC__ > > /* off_t is redefined by sys/types.h used by zlib.h */ > @@ -164,17 +179,35 @@ typedef unsigned int ino_t; > #endif > > > +#ifndef __MINGW64_VERSION_MAJOR > typedef int ssize_t; > +#endif > + > + > typedef uint32_t in_addr_t; > typedef u_short in_port_t; > typedef int sig_atomic_t; > > > +#ifdef _WIN64 > + > +#define NGX_PTR_SIZE 8 > +#define NGX_SIZE_T_LEN (sizeof("-9223372036854775808") - 1) > +#define NGX_MAX_SIZE_T_VALUE 9223372036854775807 > +#define NGX_TIME_T_LEN (sizeof("-9223372036854775808") - 1) > +#define NGX_TIME_T_SIZE 8 > + > +#else > + > #define NGX_PTR_SIZE 4 > #define NGX_SIZE_T_LEN (sizeof("-2147483648") - 1) > #define NGX_MAX_SIZE_T_VALUE 2147483647 > #define NGX_TIME_T_LEN (sizeof("-2147483648") - 1) > #define NGX_TIME_T_SIZE 4 > + > +#endif > + > + > #define NGX_OFF_T_LEN (sizeof("-9223372036854775807") - 1) > #define NGX_MAX_OFF_T_VALUE 9223372036854775807 > #define NGX_SIG_ATOMIC_T_SIZE 4 > > > > ------------------------------ > > Message: 6 > Date: Tue, 28 Oct 2014 16:38:02 +0300 > From: Maxim Dounin > To: nginx-devel at nginx.org > Subject: Re: [PATCH] Support cross compiling with MinGW-w64 on Debian > GNU/Linux > Message-ID: <20141028133802.GL45418 at mdounin.ru> > Content-Type: text/plain; charset=us-ascii > > Hello! > > On Tue, Oct 28, 2014 at 10:04:00PM +0900, Kouhei Sutou wrote: > > > Hi, > > > > In <20141027174711.GD45418 at mdounin.ru> > > "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian > GNU/Linux" on Mon, 27 Oct 2014 20:47:11 +0300, > > Maxim Dounin wrote: > > > > > - the NGX_GNUC_64 macro was removed, __MINGW64_VERSION_MAJOR > > > tested directly instead; > > > > > > - the #include <_mingw.h> removed as tests are after includes now, > > > see [1]; > > > > > > - some style nits. > > > > > > Please test if it still works for you. > > > > Thanks for providing your patch. > > > > I confirmed that it works well with both > > x86_64-w64-mingw32-gcc and i686-w64-mingw32-gcc! > > > > > > Please push the patch. > > Committed, thanks. > > -- > Maxim Dounin > http://nginx.org/ > > > > ------------------------------ > > Message: 7 > Date: Tue, 28 Oct 2014 16:39:06 +0300 > From: "Valentin V. Bartenev" > To: nginx-devel at nginx.org > Cc: Maxim Dounin > Subject: Re: [PATCH] SPDY: stop emitting empty header values > Message-ID: <6234114.T01q3fE4Uv at vbart-workstation> > Content-Type: text/plain; charset="us-ascii" > > On Monday 27 October 2014 14:33:15 Piotr Sikora wrote: > > # HG changeset patch > > # User Piotr Sikora > > # Date 1414445156 25200 > > # Mon Oct 27 14:25:56 2014 -0700 > > # Node ID e590d484a7f9d2750e2cd45ea16c601ae01cb607 > > # Parent ee9230cd4bda9f9e7e83f7baf37d66ad6bb9b0c1 > > SPDY: stop emitting empty header values. > > > > Previously, nginx would emit empty values in a header with multiple, > > NULL-separated values. > > > > This is forbidden by the SPDY specification, which requires headers to > > have either a single (possibly empty) value or multiple, NULL-separated > > non-empty values. > > > > Signed-off-by: Piotr Sikora > > > > diff -r ee9230cd4bda -r e590d484a7f9 > src/http/ngx_http_spdy_filter_module.c > > --- a/src/http/ngx_http_spdy_filter_module.c Mon Oct 27 21:14:12 2014 > +0300 > > +++ b/src/http/ngx_http_spdy_filter_module.c Mon Oct 27 14:25:56 2014 > -0700 > > @@ -493,9 +493,13 @@ ngx_http_spdy_header_filter(ngx_http_req > > continue; > > } > > > > - *last++ = '\0'; > > + if (h[j].value.len) { > > + if (last != p) { > > + *last++ = '\0'; > > + } > > > > - last = ngx_cpymem(last, h[j].value.data, h[j].value.len); > > + last = ngx_cpymem(last, h[j].value.data, > h[j].value.len); > > + } > > > > h[j].hash = 2; > > } > > > > > Ok. Ruslan? Maxim? Will you approve the patch for commit? > > wbr, Valentin V. Bartenev > > > > ------------------------------ > > Message: 8 > Date: Tue, 28 Oct 2014 22:44:26 +0900 (JST) > From: Kouhei Sutou > To: nginx-devel at nginx.org > Subject: Re: [PATCH] Support cross compiling with MinGW-w64 on Debian > GNU/Linux > Message-ID: <20141028.224426.1385733153224259244.kou at cozmixng.org> > Content-Type: Text/Plain; charset=us-ascii > > Hi, > > In <20141028133802.GL45418 at mdounin.ru> > "Re: [PATCH] Support cross compiling with MinGW-w64 on Debian GNU/Linux" > on Tue, 28 Oct 2014 16:38:02 +0300, > Maxim Dounin wrote: > > >> > - the NGX_GNUC_64 macro was removed, __MINGW64_VERSION_MAJOR > >> > tested directly instead; > >> > > >> > - the #include <_mingw.h> removed as tests are after includes now, > >> > see [1]; > >> > > >> > - some style nits. > >> > > >> > Please test if it still works for you. > >> > >> Thanks for providing your patch. > >> > >> I confirmed that it works well with both > >> x86_64-w64-mingw32-gcc and i686-w64-mingw32-gcc! > >> > >> > >> Please push the patch. > > > > Committed, thanks. > > Thanks!!! > > -- > kou > > > > ------------------------------ > > Message: 9 > Date: Tue, 28 Oct 2014 23:19:17 +0900 (JST) > From: Kouhei Sutou > To: nginx-devel at nginx.org > Subject: [PATCH] Add Win32 related files into tar.gz > Message-ID: <20141028.231917.907259224060825208.kou at cozmixng.org> > Content-Type: text/plain; charset="us-ascii" > > Hi, > > The current nginx can be built with MinGW-w64. Could you add > Win32 related files into tar.gz? If tar.gz includes Win32 > related file, I can build nginx for Windows from tar.gz. > > > Thanks, > -- > kou > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: win32-add-win32-related-files-into-tar-gz.patch > Type: text/x-patch > Size: 986 bytes > Desc: not available > URL: < > http://mailman.nginx.org/pipermail/nginx-devel/attachments/20141028/e15b926d/attachment-0001.bin > > > > ------------------------------ > > Message: 10 > Date: Tue, 28 Oct 2014 14:52:33 +0000 > From: Maxim Dounin > To: nginx-devel at nginx.org > Subject: [nginx] Updated OpenSSL used for win32 builds. > Message-ID: > > Content-Type: text/plain; charset="us-ascii" > > details: http://hg.nginx.org/nginx/rev/e40ce16b2c42 > branches: > changeset: 5887:e40ce16b2c42 > user: Maxim Dounin > date: Tue Oct 28 17:35:00 2014 +0300 > description: > Updated OpenSSL used for win32 builds. > > diffstat: > > misc/GNUmakefile | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diffs (12 lines): > > diff --git a/misc/GNUmakefile b/misc/GNUmakefile > --- a/misc/GNUmakefile > +++ b/misc/GNUmakefile > @@ -5,7 +5,7 @@ NGINX = nginx-$(VER) > TEMP = tmp > > OBJS = objs.msvc8 > -OPENSSL = openssl-1.0.1i > +OPENSSL = openssl-1.0.1j > ZLIB = zlib-1.2.8 > PCRE = pcre-8.35 > > > > > ------------------------------ > > Message: 11 > Date: Tue, 28 Oct 2014 15:17:19 +0000 > From: Maxim Dounin > To: nginx-devel at nginx.org > Subject: [nginx] nginx-1.7.7-RELEASE > Message-ID: > > Content-Type: text/plain; charset="utf-8" > > details: http://hg.nginx.org/nginx/rev/6d2fbc30f8a7 > branches: > changeset: 5888:6d2fbc30f8a7 > user: Maxim Dounin > date: Tue Oct 28 18:04:46 2014 +0300 > description: > nginx-1.7.7-RELEASE > > diffstat: > > docs/xml/nginx/changes.xml | 110 > +++++++++++++++++++++++++++++++++++++++++++++ > 1 files changed, 110 insertions(+), 0 deletions(-) > > diffs (120 lines): > > diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml > --- a/docs/xml/nginx/changes.xml > +++ b/docs/xml/nginx/changes.xml > @@ -5,6 +5,116 @@ > > > > + > + > + > + > +?????? nginx ????????? ??? ??????????? ?????? "Vary" > +? ????????? ?????? ???????. > + > + > +now nginx takes into account the "Vary" > +header line in a backend response while caching. > + > + > + > + > + > +????????? proxy_force_ranges, fastcgi_force_ranges, > +scgi_force_ranges ? uwsgi_force_ranges. > + > + > +the "proxy_force_ranges", "fastcgi_force_ranges", > +"scgi_force_ranges", and "uwsgi_force_ranges" directives. > + > + > + > + > + > +????????? proxy_limit_rate, fastcgi_limit_rate, > +scgi_limit_rate ? uwsgi_limit_rate. > + > + > +the "proxy_limit_rate", "fastcgi_limit_rate", > +"scgi_limit_rate", and "uwsgi_limit_rate" directives. > + > + > + > + > + > +???????? Vary ???????? proxy_ignore_headers, fastcgi_ignore_headers, > +scgi_ignore_headers ? uwsgi_ignore_headers. > + > + > +the "Vary" parameter of the "proxy_ignore_headers", > "fastcgi_ignore_headers", > +"scgi_ignore_headers", and "uwsgi_ignore_headers" directives. > + > + > + > + > + > +????????? ????? ??????, ??????????? ?? ??????? > +??? ?????????????????? ?????????????, > +????? ?? ???????????? ???????, > +???? ?????????????? ????????? gzip ??? gunzip. > + > + > +the last part of a response received from a backend > +with unbufferred proxy > +might not be sent to a client > +if "gzip" or "gunzip" directives were used. > + > + > + > + > + > +? ????????? proxy_cache_revalidate.
> +??????? Piotr Sikora. > + > + > +in the "proxy_cache_revalidate" directive.
> +Thanks to Piotr Sikora. > + > + > + > + > + > +? ????????? ??????.
> +??????? Yichun Zhang ? ??????? ?????????. > + > + > +in error handling.
> +Thanks to Yichun Zhang and Daniil Bondarev. > + > + > + > + > + > +? ?????????? > +proxy_next_upstream_tries ? proxy_next_upstream_timeout.
> +??????? Feng Gu. > + > + > +in the "proxy_next_upstream_tries" and "proxy_next_upstream_timeout" > +directives.
> +Thanks to Feng Gu. > + > + > + > + > + > +nginx/Windows ?? ????????? ? MinGW-w64 gcc.
> +??????? Kouhei Sutou. > + > + > +nginx/Windows could not be built with MinGW-w64 gcc.
> +Thanks to Kouhei Sutou. > + > + > + > + > + > + > > > > > ------------------------------ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > End of nginx-devel Digest, Vol 60, Issue 34 > ******************************************* > -- Donatas -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Tue Oct 28 15:25:41 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 28 Oct 2014 18:25:41 +0300 Subject: nginx-devel Digest, Vol 60, Issue 34 In-Reply-To: References: Message-ID: <20141028152541.GQ45418@mdounin.ru> Hello! On Tue, Oct 28, 2014 at 05:22:38PM +0200, Donatas Abraitis wrote: By replying to digests, you ask everyone to not answer. Please don't do this. [...] -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Tue Oct 28 15:41:30 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 28 Oct 2014 18:41:30 +0300 Subject: [PATCH] Add Win32 related files into tar.gz In-Reply-To: <20141028.231917.907259224060825208.kou@cozmixng.org> References: <20141028.231917.907259224060825208.kou@cozmixng.org> Message-ID: <20141028154130.GW45418@mdounin.ru> Hello! On Tue, Oct 28, 2014 at 11:19:17PM +0900, Kouhei Sutou wrote: > Hi, > > The current nginx can be built with MinGW-w64. Could you add > Win32 related files into tar.gz? If tar.gz includes Win32 > related file, I can build nginx for Windows from tar.gz. I would rather not. They are omitted intentionally, and support for one more compiler in addition to 4 previously supported doesn't change things much. -- Maxim Dounin http://nginx.org/ From ru at nginx.com Tue Oct 28 18:01:19 2014 From: ru at nginx.com (Ruslan Ermilov) Date: Tue, 28 Oct 2014 22:01:19 +0400 Subject: [nginx] Cache: multiple variants of a resource now can be stored. In-Reply-To: References: Message-ID: <20141028180119.GC56368@lo0.su> On Mon, Oct 27, 2014 at 06:20:13PM +0000, Maxim Dounin wrote: > details: http://hg.nginx.org/nginx/rev/78c49e243848 > branches: > changeset: 5880:78c49e243848 > user: Maxim Dounin > date: Mon Oct 27 21:14:10 2014 +0300 > description: > Cache: multiple variants of a resource now can be stored. ???????? ? ????? ???????????? ?? ??? ????, ????????? ??????????? ? ????? ?? ????? ? ???, ? ?????? ?? ?????? ???. ???? ??? ???????, ?????? ????? ?????????? ????. ??????? ??? ???? ??????, ????? ??? ??? ????-?? ??????????. > If a variant stored can't be used to respond to a request, the variant > hash is used as a secondary key. ??? ??? ????????????? ?????? ???????? ????????. :) "a variant stored" - ??? ???????? ??????? ? ????, ? ???????? ???? ??????????? ?? ???????? ???? Vary-???. "can't be used" - ??? ?????? ??? Vary-??? ??????? ?? ?????? ? ???, ??? ???????? ? ???? ???????? ????????. ???? ??? ???, ?? ???????????? ????? ???? ? ?????? ??? Vary-??? ? ?????? ????? ??????? ? ???? ? ???? ??????. > Additionally, if we previously switched to a secondary key, while storing > a response to cache we check if the variant hash still apply. If not, we > switch back to the original key, to handle cases when Vary changes. ????? ? ?? ????? ??? "if the variant hash still apply". ??? ??? ??????, ????? ? ???? ??? ???? ???????? ???????, ??? ?? ?? ????????, ?????? ?????????? ????????? ? ???? ???? ???, ?????????? ??????, ?????? ??? ????????????, ? ?????????? ??? ? ???? ?????? ???????? Vary. ? ???? ?????? ???????? ??????? ????????????????. From vbart at nginx.com Tue Oct 28 19:03:14 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Tue, 28 Oct 2014 19:03:14 +0000 Subject: [nginx] Version bump. Message-ID: details: http://hg.nginx.org/nginx/rev/049c3ea24480 branches: changeset: 5890:049c3ea24480 user: Valentin Bartenev date: Tue Oct 28 21:58:49 2014 +0300 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r 0103dc197a90 -r 049c3ea24480 src/core/nginx.h --- a/src/core/nginx.h Tue Oct 28 18:04:46 2014 +0300 +++ b/src/core/nginx.h Tue Oct 28 21:58:49 2014 +0300 @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1007007 -#define NGINX_VERSION "1.7.7" +#define nginx_version 1007008 +#define NGINX_VERSION "1.7.8" #define NGINX_VER "nginx/" NGINX_VERSION #ifdef NGX_BUILD From vbart at nginx.com Tue Oct 28 19:03:17 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Tue, 28 Oct 2014 19:03:17 +0000 Subject: [nginx] SPDY: stop emitting multiple empty header values. Message-ID: details: http://hg.nginx.org/nginx/rev/87ada3ba1392 branches: changeset: 5891:87ada3ba1392 user: Piotr Sikora date: Mon Oct 27 14:25:56 2014 -0700 description: SPDY: stop emitting multiple empty header values. Previously, nginx would emit empty values in a header with multiple, NULL-separated values. This is forbidden by the SPDY specification, which requires headers to have either a single (possibly empty) value or multiple, NULL-separated non-empty values. Signed-off-by: Piotr Sikora diffstat: src/http/ngx_http_spdy_filter_module.c | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-) diffs (19 lines): diff -r 049c3ea24480 -r 87ada3ba1392 src/http/ngx_http_spdy_filter_module.c --- a/src/http/ngx_http_spdy_filter_module.c Tue Oct 28 21:58:49 2014 +0300 +++ b/src/http/ngx_http_spdy_filter_module.c Mon Oct 27 14:25:56 2014 -0700 @@ -493,9 +493,13 @@ ngx_http_spdy_header_filter(ngx_http_req continue; } - *last++ = '\0'; + if (h[j].value.len) { + if (last != p) { + *last++ = '\0'; + } - last = ngx_cpymem(last, h[j].value.data, h[j].value.len); + last = ngx_cpymem(last, h[j].value.data, h[j].value.len); + } h[j].hash = 2; } From vbart at nginx.com Tue Oct 28 19:05:33 2014 From: vbart at nginx.com (Valentin V. Bartenev) Date: Tue, 28 Oct 2014 22:05:33 +0300 Subject: [PATCH] SPDY: stop emitting empty header values In-Reply-To: References: <1424354.N8cOoFmhc0@vbart-workstation> Message-ID: <5196756.cLkcAQNVuk@vbart-laptop> On Monday 27 October 2014 14:33:15 Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1414445156 25200 > # Mon Oct 27 14:25:56 2014 -0700 > # Node ID e590d484a7f9d2750e2cd45ea16c601ae01cb607 > # Parent ee9230cd4bda9f9e7e83f7baf37d66ad6bb9b0c1 > SPDY: stop emitting empty header values. > > Previously, nginx would emit empty values in a header with multiple, > NULL-separated values. > > This is forbidden by the SPDY specification, which requires headers to > have either a single (possibly empty) value or multiple, NULL-separated > non-empty values. > > Signed-off-by: Piotr Sikora [..] Thanks. Committed: http://hg.nginx.org/nginx/rev/87ada3ba1392 wbr, Valentin V. Bartenev From ru at nginx.com Tue Oct 28 19:15:51 2014 From: ru at nginx.com (Ruslan Ermilov) Date: Tue, 28 Oct 2014 23:15:51 +0400 Subject: [nginx] Cache: multiple variants of a resource now can be stored. In-Reply-To: <20141028180119.GC56368@lo0.su> References: <20141028180119.GC56368@lo0.su> Message-ID: <20141028191551.GF56368@lo0.su> On Tue, Oct 28, 2014 at 10:01:19PM +0400, Ruslan Ermilov wrote: > On Mon, Oct 27, 2014 at 06:20:13PM +0000, Maxim Dounin wrote: > > details: http://hg.nginx.org/nginx/rev/78c49e243848 > > branches: > > changeset: 5880:78c49e243848 > > user: Maxim Dounin > > date: Mon Oct 27 21:14:10 2014 +0300 > > description: > > Cache: multiple variants of a resource now can be stored. > > ???????? ? ????? ???????????? ?? ??? ????, ????????? > ??????????? ? ????? ?? ????? ? ???, ? ?????? ?? ?????? > ???. ???? ??? ???????, ?????? ????? ?????????? ????. > ??????? ??? ???? ??????, ????? ??? ??? ????-?? ??????????. [...] That wasn't meant to be sent here, sorry about that. From piotr at cloudflare.com Tue Oct 28 22:03:21 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 28 Oct 2014 15:03:21 -0700 Subject: [nginx] Updated OpenSSL used for win32 builds. In-Reply-To: References: Message-ID: Hey Maxim, > Updated OpenSSL used for win32 builds. Uhm, committing this before merging my patch [1] broke password-protected private keys (using multiple passwords in the same ssl_password_file feature) on win32 for the 1.7.7 release. [1] http://mailman.nginx.org/pipermail/nginx-devel/2014-October/006104.html Best regards, Piotr Sikora From pdn at cryptopro.ru Wed Oct 29 14:48:07 2014 From: pdn at cryptopro.ru (Dmitrii Pichulin) Date: Wed, 29 Oct 2014 17:48:07 +0300 Subject: [PATCH] allow to use engine keyform for server private key In-Reply-To: <20140812004334.GN1849@mdounin.ru> References: <20140801165839.GT1849@mdounin.ru> <53E8483C.8020507@cryptopro.ru> <20140812004334.GN1849@mdounin.ru> Message-ID: <5450FE27.2070702@cryptopro.ru> For example, the testing steps for ubuntu user "nx" with nginx > openssl > engine_pkcs11 > softhsm: 1) -install softhsm (apt-get install softhsm); -edit config (/etc/softhsm/softhsm.conf with: "0:/home/nx/slot0.db"); -init token (softhsm --init-token --slot 0 --label "NginxZero"); 2) -install opensc (apt-get install opensc); -generate key in token from step 1) ( pkcs11-tool --module=/usr/lib/softhsm/libsofthsm.so \ -l -k -d 0 -a nx_key_0 --key-type rsa:2048 ) 3) -install engine_pkcs11 (apt-get install libengine-pkcs11-openssl) 4) -install openssl (apt-get install openssl libssl-dev) -edit config ( /etc/ssl/openssl.cnf insert after "oid_section = new_oids": openssl_conf = openssl_def [openssl_def] engines = engine_section [engine_section] pkcs11 = pkcs11_section [pkcs11_section] engine_id = pkcs11 dynamic_path = /usr/lib/engines/engine_pkcs11.so MODULE_PATH = /usr/lib/softhsm/libsofthsm.so init = 0 ) -make self-signed certificate ( openssl req -engine pkcs11 -new -key id_00 -keyform engine \ -out req.pem -text -x509 -subj "/CN=NginxZero" openssl x509 -engine pkcs11 -signkey slot_0-id_00 \ -keyform engine -in req.pem -out cert.pem ) 5) -build nginx with this patch and with-http_ssl_module -edit config ( nginx.conf section main: ssl_engine pkcs11; nginx.conf section http: server { listen *:443; server_name localhost; ssl on; ssl_certificate /home/nx/cert.pem; ssl_certificate_key engine:pkcs11:slot_0-id_00; ssl_protocols SSLv2 SSLv3 TLSv1; ssl_ciphers AES128-SHA:AES256-SHA:RC4-SHA:DES-CBC3-SHA:RC4-MD5; ssl_prefer_server_ciphers on; charset UTF-8; location / { root /home/nx/www; index index.html index.htm; } } ) 6) run/test nginx On 12.08.2014 4:43, Maxim Dounin wrote: > Hello! > > On Mon, Aug 11, 2014 at 08:36:12AM +0400, Dmitrii Pichulin wrote: > >> What is the current statusof this patch? >> >> Let us know if it has any issues. > > As far as I see, there are no serious problems left. There are > various minor style issues though (like incorrect patch subject, > use of "char" instead of "u_char", and so on). I'm going to > cleanup these as time permits. Please ping again if it won't > happen in a week or two. > > (Meanwhile, you may try to clean things yourself and post an > updated patch.) > > BTW, if you have a good example of an engine to test with, this > may be also helpful. > From mdounin at mdounin.ru Wed Oct 29 16:46:48 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 29 Oct 2014 19:46:48 +0300 Subject: [nginx] Updated OpenSSL used for win32 builds. In-Reply-To: References: Message-ID: <20141029164648.GJ45418@mdounin.ru> Hello! On Tue, Oct 28, 2014 at 03:03:21PM -0700, Piotr Sikora wrote: > Hey Maxim, > > > Updated OpenSSL used for win32 builds. > > Uhm, committing this before merging my patch [1] broke > password-protected private keys (using multiple passwords in the same > ssl_password_file feature) on win32 for the 1.7.7 release. Yes, though I suppose that number of affected users is close to zero. And, after all, win32 version is considered to be beta[2]. On the other hand, not updating OpenSSL will mean that all win32 users will not get various OpenSSL security fixes, which is believed to be worse. Sergey is working on ssl_password_file patches, they are likely to be merged shortly. [2] http://nginx.org/en/docs/windows.html -- Maxim Dounin http://nginx.org/ From pluknet at nginx.com Wed Oct 29 18:09:57 2014 From: pluknet at nginx.com (Sergey Kandaurov) Date: Wed, 29 Oct 2014 21:09:57 +0300 Subject: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases In-Reply-To: References: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> <20141027162829.GC45418@mdounin.ru> Message-ID: On Oct 28, 2014, at 12:38 AM, Piotr Sikora wrote: > Also, I think that PKCS#12 support should be added in a separate > commit, instead of mixing it with semi-related fix. How so? It is not about adding error handling support for yet another format, because it was already there but was broken for the same reason after 1.0.1j. Below is an updated patch with fixed header inclusion I?d like to commit. # HG changeset patch # User Piotr Sikora # Date 1414150080 25200 # Fri Oct 24 04:28:00 2014 -0700 # Node ID 8316cb9139f5f5b0fb39969006d68708f22f312d # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 SSL: make ssl_password_file work with recent OpenSSL releases. Multiple passwords in a single ssl_password_file feature was broken after recent OpenSSL changes (commit 4aac102f75b517bdb56b1bcfd0a856052d559f6e). Affected OpenSSL releases: 0.9.8zc, 1.0.0o, 1.0.1j and 1.0.2-beta3. Signed-off-by: Piotr Sikora diff -r 973fded4f461 -r 8316cb9139f5 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Wed Oct 15 22:57:23 2014 +0400 +++ b/src/event/ngx_event_openssl.c Fri Oct 24 04:28:00 2014 -0700 @@ -410,8 +410,12 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ if (ERR_GET_LIB(n) == ERR_LIB_CIPHER && ERR_GET_REASON(n) == CIPHER_R_BAD_DECRYPT) #else - if (ERR_GET_LIB(n) == ERR_LIB_EVP - && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) + if ((ERR_GET_LIB(n) == ERR_LIB_PEM + && ERR_GET_REASON(n) == PEM_R_BAD_DECRYPT) + || (ERR_GET_LIB(n) == ERR_LIB_EVP + && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) + || (ERR_GET_LIB(n) == ERR_LIB_PKCS12 + && ERR_GET_REASON(n) == PKCS12_R_PKCS12_CIPHERFINAL_ERROR)) #endif { ERR_clear_error(); diff -r 973fded4f461 -r 8316cb9139f5 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Wed Oct 15 22:57:23 2014 +0400 +++ b/src/event/ngx_event_openssl.h Fri Oct 24 04:28:00 2014 -0700 @@ -25,6 +25,9 @@ #ifndef OPENSSL_NO_OCSP #include #endif +#ifndef OPENSSL_IS_BORINGSSL +#include +#endif #include #include #include -- Sergey Kandaurov From jadas at akamai.com Wed Oct 29 18:15:11 2014 From: jadas at akamai.com (Das, Jagannath) Date: Wed, 29 Oct 2014 23:45:11 +0530 Subject: NGINX Persistent Connection Limit Value Message-ID: Hi Folks, Provided the scalable architecture we have today, Is it possible that we may hit the connection limit issue using the persistent connection flags like the keepalive_timeout/keepalive_requests due to too many open connections? Thanks, Jagannath -------------- next part -------------- An HTML attachment was scrubbed... URL: From pluknet at nginx.com Wed Oct 29 19:11:27 2014 From: pluknet at nginx.com (Sergey Kandaurov) Date: Wed, 29 Oct 2014 22:11:27 +0300 Subject: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases In-Reply-To: References: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> <20141027162829.GC45418@mdounin.ru> Message-ID: <02A7426A-6236-4791-90B0-272AC50CFE91@nginx.com> On Oct 29, 2014, at 9:09 PM, Sergey Kandaurov wrote: > On Oct 28, 2014, at 12:38 AM, Piotr Sikora wrote: >> Also, I think that PKCS#12 support should be added in a separate >> commit, instead of mixing it with semi-related fix. > > How so? It is not about adding error handling support for yet another format, > because it was already there but was broken for the same reason after 1.0.1j. > > Below is an updated patch with fixed header inclusion I?d like to commit. Alternatively, remove error collection that doesn?t really make much sense. It?s an open set and we cannot predict for sure all of them enumerated. # HG changeset patch # User Sergey Kandaurov # Date 1414150080 25200 # Fri Oct 24 04:28:00 2014 -0700 # Node ID 47728601e542cab9406ea323576f6df5adfff193 # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 SSL: simplified ssl_password_file error handling. Instead of collecting a number of the possible SSL_CTX_use_PrivateKey_file() error codes that becomes more and more difficult with the rising variety of OpenSSL versions and its derivatives, just continue with the next passphrase. This fixes multiple passwords in a single ssl_password_file that was broken after recent OpenSSL changes (commit 4aac102f75b517bdb56b1bcfd0a856052d559f6e). diff -r 973fded4f461 -r 47728601e542 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Wed Oct 15 22:57:23 2014 +0400 +++ b/src/event/ngx_event_openssl.c Fri Oct 24 04:28:00 2014 -0700 @@ -404,20 +404,9 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ } if (--tries) { - n = ERR_peek_error(); - -#ifdef OPENSSL_IS_BORINGSSL - if (ERR_GET_LIB(n) == ERR_LIB_CIPHER - && ERR_GET_REASON(n) == CIPHER_R_BAD_DECRYPT) -#else - if (ERR_GET_LIB(n) == ERR_LIB_EVP - && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) -#endif - { - ERR_clear_error(); - SSL_CTX_set_default_passwd_cb_userdata(ssl->ctx, ++pwd); - continue; - } + ERR_clear_error(); + SSL_CTX_set_default_passwd_cb_userdata(ssl->ctx, ++pwd); + continue; } ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, -- Sergey Kandaurov From mdounin at mdounin.ru Wed Oct 29 19:11:59 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 29 Oct 2014 22:11:59 +0300 Subject: NGINX Persistent Connection Limit Value In-Reply-To: References: Message-ID: <20141029191159.GR45418@mdounin.ru> Hello! On Wed, Oct 29, 2014 at 11:45:11PM +0530, Das, Jagannath wrote: > Hi Folks, > Provided the scalable architecture we have today, Is it > possible that we may hit the connection limit issue using > the persistent connection flags like the > keepalive_timeout/keepalive_requests due to too many open > connections? I wonder how it's related to the nginx-devel@ mailing list. You may want to ask in the nginx@ list instead. -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Wed Oct 29 19:18:35 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 29 Oct 2014 12:18:35 -0700 Subject: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases In-Reply-To: References: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> <20141027162829.GC45418@mdounin.ru> Message-ID: Hey Sergey, > How so? It is not about adding error handling support for yet another format, > because it was already there but was broken for the same reason after 1.0.1j. Oh, I see. I misread your original comment and I thought you were adding support for it. > Below is an updated patch with fixed header inclusion I?d like to commit. > > # HG changeset patch > # User Piotr Sikora > # Date 1414150080 25200 > # Fri Oct 24 04:28:00 2014 -0700 > # Node ID 8316cb9139f5f5b0fb39969006d68708f22f312d > # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 > SSL: make ssl_password_file work with recent OpenSSL releases. > > Multiple passwords in a single ssl_password_file feature was broken after > recent OpenSSL changes (commit 4aac102f75b517bdb56b1bcfd0a856052d559f6e). > > Affected OpenSSL releases: 0.9.8zc, 1.0.0o, 1.0.1j and 1.0.2-beta3. > > Signed-off-by: Piotr Sikora > > diff -r 973fded4f461 -r 8316cb9139f5 src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c Wed Oct 15 22:57:23 2014 +0400 > +++ b/src/event/ngx_event_openssl.c Fri Oct 24 04:28:00 2014 -0700 > @@ -410,8 +410,12 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ > if (ERR_GET_LIB(n) == ERR_LIB_CIPHER > && ERR_GET_REASON(n) == CIPHER_R_BAD_DECRYPT) > #else > - if (ERR_GET_LIB(n) == ERR_LIB_EVP > - && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) > + if ((ERR_GET_LIB(n) == ERR_LIB_PEM > + && ERR_GET_REASON(n) == PEM_R_BAD_DECRYPT) > + || (ERR_GET_LIB(n) == ERR_LIB_EVP > + && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) > + || (ERR_GET_LIB(n) == ERR_LIB_PKCS12 > + && ERR_GET_REASON(n) == PKCS12_R_PKCS12_CIPHERFINAL_ERROR)) > #endif > { > ERR_clear_error(); > diff -r 973fded4f461 -r 8316cb9139f5 src/event/ngx_event_openssl.h > --- a/src/event/ngx_event_openssl.h Wed Oct 15 22:57:23 2014 +0400 > +++ b/src/event/ngx_event_openssl.h Fri Oct 24 04:28:00 2014 -0700 > @@ -25,6 +25,9 @@ > #ifndef OPENSSL_NO_OCSP > #include > #endif > +#ifndef OPENSSL_IS_BORINGSSL > +#include > +#endif > #include > #include > #include Looks good, thanks. Best regards, Piotr Sikora From piotr at cloudflare.com Wed Oct 29 19:27:40 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 29 Oct 2014 12:27:40 -0700 Subject: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases In-Reply-To: <02A7426A-6236-4791-90B0-272AC50CFE91@nginx.com> References: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> <20141027162829.GC45418@mdounin.ru> <02A7426A-6236-4791-90B0-272AC50CFE91@nginx.com> Message-ID: Hey Sergey, > Alternatively, remove error collection that doesn?t really make much sense. > It?s an open set and we cannot predict for sure all of them enumerated. That's also a valid approach, but the commit is wrong. > # HG changeset patch > # User Sergey Kandaurov > # Date 1414150080 25200 > # Fri Oct 24 04:28:00 2014 -0700 > # Node ID 47728601e542cab9406ea323576f6df5adfff193 > # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 > SSL: simplified ssl_password_file error handling. > > Instead of collecting a number of the possible SSL_CTX_use_PrivateKey_file() > error codes that becomes more and more difficult with the rising variety of > OpenSSL versions and its derivatives, just continue with the next passphrase. > > This fixes multiple passwords in a single ssl_password_file that was broken > after recent OpenSSL changes (commit 4aac102f75b517bdb56b1bcfd0a856052d559f6e). Not even "Reported by"? :P > diff -r 973fded4f461 -r 47728601e542 src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c Wed Oct 15 22:57:23 2014 +0400 > +++ b/src/event/ngx_event_openssl.c Fri Oct 24 04:28:00 2014 -0700 > @@ -404,20 +404,9 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ > } > > if (--tries) { > - n = ERR_peek_error(); n is being used to print the final error, after the use of the last password fails, so you cannot completely remove it. You should probably just move it after the if loop. Best regards, Piotr Sikora From jadas at akamai.com Thu Oct 30 03:23:48 2014 From: jadas at akamai.com (Das, Jagannath) Date: Thu, 30 Oct 2014 08:53:48 +0530 Subject: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases In-Reply-To: References: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> <20141027162829.GC45418@mdounin.ru> Message-ID: Hi Sergey, Are you also looking to have a fix for the POODLE vulnerability? Thanks, Jagannath From: Sergey Kandaurov > Reply-To: "nginx-devel at nginx.org" > Date: Wednesday, October 29, 2014 at 11:39 PM To: "nginx-devel at nginx.org" > Subject: Re: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases On Oct 28, 2014, at 12:38 AM, Piotr Sikora > wrote: Also, I think that PKCS#12 support should be added in a separate commit, instead of mixing it with semi-related fix. How so? It is not about adding error handling support for yet another format, because it was already there but was broken for the same reason after 1.0.1j. Below is an updated patch with fixed header inclusion I?d like to commit. # HG changeset patch # User Piotr Sikora > # Date 1414150080 25200 # Fri Oct 24 04:28:00 2014 -0700 # Node ID 8316cb9139f5f5b0fb39969006d68708f22f312d # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 SSL: make ssl_password_file work with recent OpenSSL releases. Multiple passwords in a single ssl_password_file feature was broken after recent OpenSSL changes (commit 4aac102f75b517bdb56b1bcfd0a856052d559f6e). Affected OpenSSL releases: 0.9.8zc, 1.0.0o, 1.0.1j and 1.0.2-beta3. Signed-off-by: Piotr Sikora > diff -r 973fded4f461 -r 8316cb9139f5 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Wed Oct 15 22:57:23 2014 +0400 +++ b/src/event/ngx_event_openssl.c Fri Oct 24 04:28:00 2014 -0700 @@ -410,8 +410,12 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ if (ERR_GET_LIB(n) == ERR_LIB_CIPHER && ERR_GET_REASON(n) == CIPHER_R_BAD_DECRYPT) #else - if (ERR_GET_LIB(n) == ERR_LIB_EVP - && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) + if ((ERR_GET_LIB(n) == ERR_LIB_PEM + && ERR_GET_REASON(n) == PEM_R_BAD_DECRYPT) + || (ERR_GET_LIB(n) == ERR_LIB_EVP + && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) + || (ERR_GET_LIB(n) == ERR_LIB_PKCS12 + && ERR_GET_REASON(n) == PKCS12_R_PKCS12_CIPHERFINAL_ERROR)) #endif { ERR_clear_error(); diff -r 973fded4f461 -r 8316cb9139f5 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Wed Oct 15 22:57:23 2014 +0400 +++ b/src/event/ngx_event_openssl.h Fri Oct 24 04:28:00 2014 -0700 @@ -25,6 +25,9 @@ #ifndef OPENSSL_NO_OCSP #include #endif +#ifndef OPENSSL_IS_BORINGSSL +#include +#endif #include #include #include -- Sergey Kandaurov _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From piotr at cloudflare.com Thu Oct 30 04:14:01 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 29 Oct 2014 21:14:01 -0700 Subject: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases In-Reply-To: References: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> <20141027162829.GC45418@mdounin.ru> Message-ID: Hi, > Are you also looking to have a fix for the POODLE vulnerability? The only fix for the POODLE vulnerability is to disable SSLv3 (I'm going to send a patch for that in a moment). However, if you meant TLS_FALLBACK_SCSV, then it's not needed, because nginx doesn't do "downgrade dance". Best regards, Piotr Sikora From piotr at cloudflare.com Thu Oct 30 04:17:04 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 29 Oct 2014 21:17:04 -0700 Subject: [PATCH] SSL: don't enable SSLv3 by default Message-ID: # HG changeset patch # User Piotr Sikora # Date 1414642398 25200 # Wed Oct 29 21:13:18 2014 -0700 # Node ID bf17486e5d30574b870926b76c1d6f421e4def75 # Parent 87ada3ba1392fadaf4d9193b5d345c248be32f77 SSL: don't enable SSLv3 by default. Prodded by Jagannath Das. Signed-off-by: Piotr Sikora diff -r 87ada3ba1392 -r bf17486e5d30 src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c Mon Oct 27 14:25:56 2014 -0700 +++ b/src/http/modules/ngx_http_proxy_module.c Wed Oct 29 21:13:18 2014 -0700 @@ -2815,9 +2815,8 @@ ngx_http_proxy_merge_loc_conf(ngx_conf_t prev->upstream.ssl_session_reuse, 1); ngx_conf_merge_bitmask_value(conf->ssl_protocols, prev->ssl_protocols, - (NGX_CONF_BITMASK_SET|NGX_SSL_SSLv3 - |NGX_SSL_TLSv1|NGX_SSL_TLSv1_1 - |NGX_SSL_TLSv1_2)); + (NGX_CONF_BITMASK_SET|NGX_SSL_TLSv1 + |NGX_SSL_TLSv1_1|NGX_SSL_TLSv1_2)); ngx_conf_merge_str_value(conf->ssl_ciphers, prev->ssl_ciphers, "DEFAULT"); diff -r 87ada3ba1392 -r bf17486e5d30 src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Mon Oct 27 14:25:56 2014 -0700 +++ b/src/http/modules/ngx_http_ssl_module.c Wed Oct 29 21:13:18 2014 -0700 @@ -561,7 +561,7 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * prev->prefer_server_ciphers, 0); ngx_conf_merge_bitmask_value(conf->protocols, prev->protocols, - (NGX_CONF_BITMASK_SET|NGX_SSL_SSLv3|NGX_SSL_TLSv1 + (NGX_CONF_BITMASK_SET|NGX_SSL_TLSv1 |NGX_SSL_TLSv1_1|NGX_SSL_TLSv1_2)); ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size, diff -r 87ada3ba1392 -r bf17486e5d30 src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c Mon Oct 27 14:25:56 2014 -0700 +++ b/src/http/modules/ngx_http_uwsgi_module.c Wed Oct 29 21:13:18 2014 -0700 @@ -1598,9 +1598,8 @@ ngx_http_uwsgi_merge_loc_conf(ngx_conf_t prev->upstream.ssl_session_reuse, 1); ngx_conf_merge_bitmask_value(conf->ssl_protocols, prev->ssl_protocols, - (NGX_CONF_BITMASK_SET|NGX_SSL_SSLv3 - |NGX_SSL_TLSv1|NGX_SSL_TLSv1_1 - |NGX_SSL_TLSv1_2)); + (NGX_CONF_BITMASK_SET|NGX_SSL_TLSv1 + |NGX_SSL_TLSv1_1|NGX_SSL_TLSv1_2)); ngx_conf_merge_str_value(conf->ssl_ciphers, prev->ssl_ciphers, "DEFAULT"); diff -r 87ada3ba1392 -r bf17486e5d30 src/mail/ngx_mail_ssl_module.c --- a/src/mail/ngx_mail_ssl_module.c Mon Oct 27 14:25:56 2014 -0700 +++ b/src/mail/ngx_mail_ssl_module.c Wed Oct 29 21:13:18 2014 -0700 @@ -235,7 +235,7 @@ ngx_mail_ssl_merge_conf(ngx_conf_t *cf, prev->prefer_server_ciphers, 0); ngx_conf_merge_bitmask_value(conf->protocols, prev->protocols, - (NGX_CONF_BITMASK_SET|NGX_SSL_SSLv3|NGX_SSL_TLSv1 + (NGX_CONF_BITMASK_SET|NGX_SSL_TLSv1 |NGX_SSL_TLSv1_1|NGX_SSL_TLSv1_2)); ngx_conf_merge_str_value(conf->certificate, prev->certificate, ""); From piotr at cloudflare.com Thu Oct 30 09:41:47 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Thu, 30 Oct 2014 02:41:47 -0700 Subject: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases In-Reply-To: References: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> <20141027162829.GC45418@mdounin.ru> <02A7426A-6236-4791-90B0-272AC50CFE91@nginx.com> Message-ID: Hey Sergey, > n is being used to print the final error, after the use of the last > password fails, so you cannot completely remove it. You should > probably just move it after the if loop. Nevermind, that n is being used only in the code I used when debugging this issue. Sorry for the noise. Best regards, Piotr Sikora From piotr at cloudflare.com Thu Oct 30 11:31:37 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Thu, 30 Oct 2014 04:31:37 -0700 Subject: [PATCH] Upstream: add "proxy_ssl_certificate" and friends Message-ID: # HG changeset patch # User Piotr Sikora # Date 1414668641 25200 # Thu Oct 30 04:30:41 2014 -0700 # Node ID bb14c7659efb32d1d1f651bdf54a8c8157ef67f9 # Parent 87ada3ba1392fadaf4d9193b5d345c248be32f77 Upstream: add "proxy_ssl_certificate" and friends. Signed-off-by: Piotr Sikora diff -r 87ada3ba1392 -r bb14c7659efb src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c Mon Oct 27 14:25:56 2014 -0700 +++ b/src/http/modules/ngx_http_proxy_module.c Thu Oct 30 04:30:41 2014 -0700 @@ -84,6 +84,9 @@ typedef struct { ngx_uint_t ssl_verify_depth; ngx_str_t ssl_trusted_certificate; ngx_str_t ssl_crl; + ngx_str_t ssl_certificate; + ngx_str_t ssl_certificate_key; + ngx_array_t *ssl_passwords; #endif } ngx_http_proxy_loc_conf_t; @@ -169,6 +172,8 @@ static ngx_int_t ngx_http_proxy_rewrite_ ngx_http_proxy_rewrite_t *pr, ngx_str_t *regex, ngx_uint_t caseless); #if (NGX_HTTP_SSL) +static char *ngx_http_proxy_ssl_password_file(ngx_conf_t *cf, + ngx_command_t *cmd, void *conf); static ngx_int_t ngx_http_proxy_set_ssl(ngx_conf_t *cf, ngx_http_proxy_loc_conf_t *plcf); #endif @@ -626,6 +631,27 @@ static ngx_command_t ngx_http_proxy_com offsetof(ngx_http_proxy_loc_conf_t, ssl_crl), NULL }, + { ngx_string("proxy_ssl_certificate"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_proxy_loc_conf_t, ssl_certificate), + NULL }, + + { ngx_string("proxy_ssl_certificate_key"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_proxy_loc_conf_t, ssl_certificate_key), + NULL }, + + { ngx_string("proxy_ssl_password_file"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_http_proxy_ssl_password_file, + NGX_HTTP_LOC_CONF_OFFSET, + 0, + NULL }, + #endif ngx_null_command @@ -2479,6 +2505,8 @@ ngx_http_proxy_create_loc_conf(ngx_conf_ * conf->ssl_ciphers = { 0, NULL }; * conf->ssl_trusted_certificate = { 0, NULL }; * conf->ssl_crl = { 0, NULL }; + * conf->ssl_certificate = { 0, NULL }; + * conf->ssl_certificate_key = { 0, NULL }; */ conf->upstream.store = NGX_CONF_UNSET; @@ -2527,6 +2555,7 @@ ngx_http_proxy_create_loc_conf(ngx_conf_ conf->upstream.ssl_server_name = NGX_CONF_UNSET; conf->upstream.ssl_verify = NGX_CONF_UNSET; conf->ssl_verify_depth = NGX_CONF_UNSET_UINT; + conf->ssl_passwords = NGX_CONF_UNSET_PTR; #endif /* "proxy_cyclic_temp_file" is disabled */ @@ -2836,6 +2865,12 @@ ngx_http_proxy_merge_loc_conf(ngx_conf_t prev->ssl_trusted_certificate, ""); ngx_conf_merge_str_value(conf->ssl_crl, prev->ssl_crl, ""); + ngx_conf_merge_str_value(conf->ssl_certificate, + prev->ssl_certificate, ""); + ngx_conf_merge_str_value(conf->ssl_certificate_key, + prev->ssl_certificate_key, ""); + ngx_conf_merge_ptr_value(conf->ssl_passwords, prev->ssl_passwords, NULL); + if (conf->ssl && ngx_http_proxy_set_ssl(cf, conf) != NGX_OK) { return NGX_CONF_ERROR; } @@ -3868,6 +3903,29 @@ ngx_http_proxy_lowat_check(ngx_conf_t *c #if (NGX_HTTP_SSL) +static char * +ngx_http_proxy_ssl_password_file(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +{ + ngx_http_proxy_loc_conf_t *plcf = conf; + + ngx_str_t *value; + + if (plcf->ssl_passwords != NGX_CONF_UNSET_PTR) { + return "is duplicate"; + } + + value = cf->args->elts; + + plcf->ssl_passwords = ngx_ssl_read_password_file(cf, &value[1]); + + if (plcf->ssl_passwords == NULL) { + return NGX_CONF_ERROR; + } + + return NGX_CONF_OK; +} + + static ngx_int_t ngx_http_proxy_set_ssl(ngx_conf_t *cf, ngx_http_proxy_loc_conf_t *plcf) { @@ -3894,6 +3952,23 @@ ngx_http_proxy_set_ssl(ngx_conf_t *cf, n cln->handler = ngx_ssl_cleanup_ctx; cln->data = plcf->upstream.ssl; + if (plcf->ssl_certificate.len) { + + if (plcf->ssl_certificate_key.len == 0) { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "no \"proxy_ssl_certificate_key\" is defined " + "for certificate \"%V\"", &plcf->ssl_certificate); + return NGX_ERROR; + } + + if (ngx_ssl_certificate(cf, plcf->upstream.ssl, &plcf->ssl_certificate, + &plcf->ssl_certificate_key, plcf->ssl_passwords) + != NGX_OK) + { + return NGX_ERROR; + } + } + if (SSL_CTX_set_cipher_list(plcf->upstream.ssl->ctx, (const char *) plcf->ssl_ciphers.data) == 0) diff -r 87ada3ba1392 -r bb14c7659efb src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c Mon Oct 27 14:25:56 2014 -0700 +++ b/src/http/modules/ngx_http_uwsgi_module.c Thu Oct 30 04:30:41 2014 -0700 @@ -42,6 +42,9 @@ typedef struct { ngx_uint_t ssl_verify_depth; ngx_str_t ssl_trusted_certificate; ngx_str_t ssl_crl; + ngx_str_t ssl_certificate; + ngx_str_t ssl_certificate_key; + ngx_array_t *ssl_passwords; #endif } ngx_http_uwsgi_loc_conf_t; @@ -76,6 +79,8 @@ static char *ngx_http_uwsgi_cache_key(ng #endif #if (NGX_HTTP_SSL) +static char *ngx_http_uwsgi_ssl_password_file(ngx_conf_t *cf, + ngx_command_t *cmd, void *conf); static ngx_int_t ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, ngx_http_uwsgi_loc_conf_t *uwcf); #endif @@ -482,6 +487,27 @@ static ngx_command_t ngx_http_uwsgi_comm offsetof(ngx_http_uwsgi_loc_conf_t, ssl_crl), NULL }, + { ngx_string("uwsgi_ssl_certificate"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_uwsgi_loc_conf_t, ssl_certificate), + NULL }, + + { ngx_string("uwsgi_ssl_certificate_key"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_uwsgi_loc_conf_t, ssl_certificate_key), + NULL }, + + { ngx_string("uwsgi_ssl_password_file"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_http_uwsgi_ssl_password_file, + NGX_HTTP_LOC_CONF_OFFSET, + 0, + NULL }, + #endif ngx_null_command @@ -1326,6 +1352,7 @@ ngx_http_uwsgi_create_loc_conf(ngx_conf_ conf->upstream.ssl_server_name = NGX_CONF_UNSET; conf->upstream.ssl_verify = NGX_CONF_UNSET; conf->ssl_verify_depth = NGX_CONF_UNSET_UINT; + conf->ssl_passwords = NGX_CONF_UNSET_PTR; #endif /* "uwsgi_cyclic_temp_file" is disabled */ @@ -1619,6 +1646,12 @@ ngx_http_uwsgi_merge_loc_conf(ngx_conf_t prev->ssl_trusted_certificate, ""); ngx_conf_merge_str_value(conf->ssl_crl, prev->ssl_crl, ""); + ngx_conf_merge_str_value(conf->ssl_certificate, + prev->ssl_certificate, ""); + ngx_conf_merge_str_value(conf->ssl_certificate_key, + prev->ssl_certificate_key, ""); + ngx_conf_merge_ptr_value(conf->ssl_passwords, prev->ssl_passwords, NULL); + if (conf->ssl && ngx_http_uwsgi_set_ssl(cf, conf) != NGX_OK) { return NGX_CONF_ERROR; } @@ -2109,6 +2142,29 @@ ngx_http_uwsgi_cache_key(ngx_conf_t *cf, #if (NGX_HTTP_SSL) +static char * +ngx_http_uwsgi_ssl_password_file(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +{ + ngx_http_uwsgi_loc_conf_t *uwcf = conf; + + ngx_str_t *value; + + if (uwcf->ssl_passwords != NGX_CONF_UNSET_PTR) { + return "is duplicate"; + } + + value = cf->args->elts; + + uwcf->ssl_passwords = ngx_ssl_read_password_file(cf, &value[1]); + + if (uwcf->ssl_passwords == NULL) { + return NGX_CONF_ERROR; + } + + return NGX_CONF_OK; +} + + static ngx_int_t ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, ngx_http_uwsgi_loc_conf_t *uwcf) { @@ -2135,6 +2191,23 @@ ngx_http_uwsgi_set_ssl(ngx_conf_t *cf, n cln->handler = ngx_ssl_cleanup_ctx; cln->data = uwcf->upstream.ssl; + if (uwcf->ssl_certificate.len) { + + if (uwcf->ssl_certificate_key.len == 0) { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "no \"uwsgi_ssl_certificate_key\" is defined " + "for certificate \"%V\"", &uwcf->ssl_certificate); + return NGX_ERROR; + } + + if (ngx_ssl_certificate(cf, uwcf->upstream.ssl, &uwcf->ssl_certificate, + &uwcf->ssl_certificate_key, uwcf->ssl_passwords) + != NGX_OK) + { + return NGX_ERROR; + } + } + if (SSL_CTX_set_cipher_list(uwcf->upstream.ssl->ctx, (const char *) uwcf->ssl_ciphers.data) == 0) From mdounin at mdounin.ru Thu Oct 30 13:47:35 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 30 Oct 2014 16:47:35 +0300 Subject: [PATCH] SSL: don't enable SSLv3 by default In-Reply-To: References: Message-ID: <20141030134735.GY45418@mdounin.ru> Hello! On Wed, Oct 29, 2014 at 09:17:04PM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1414642398 25200 > # Wed Oct 29 21:13:18 2014 -0700 > # Node ID bf17486e5d30574b870926b76c1d6f421e4def75 > # Parent 87ada3ba1392fadaf4d9193b5d345c248be32f77 > SSL: don't enable SSLv3 by default. This was discussed excessively both in the office here and in Russian mailing list a while ago, and consensus is that we are not changing the default for now. Rationale is as follows: - SSLv3 is still important from compatibility point of view, there are various clients which doesn't support (or enable by default) anything better; - Mitigation for POODLE is already good and improving, including fallback protection via TLS_FALLBACK_SCSV and anti-POODLE record splitting; so, basically, modern browsers are not affected. -- Maxim Dounin http://nginx.org/ From richard at fussenegger.info Thu Oct 30 14:05:18 2014 From: richard at fussenegger.info (Richard Fussenegger) Date: Thu, 30 Oct 2014 15:05:18 +0100 Subject: [PATCH] SSL: don't enable SSLv3 by default In-Reply-To: <20141030134735.GY45418@mdounin.ru> References: <20141030134735.GY45418@mdounin.ru> Message-ID: <5452459E.5090502@fussenegger.info> The rationale may make sense depending on the priorities, but shouldn't the default configuration target generic applications? Generic applications don't need compatibility with ancient software (only IE6 on XP actually /needs/ SSLv3, don't know about libraries though). Administrators who need the support can still enable it and make use of SCSV. And don't forget that 'modern browser' applies to IE up to 11, FF up to 34, Chrome up to ? (couldn't find the exact version) of which actually not a single one has SCSV support and they won't get it! Providing compatibility for ancient out-of-life software and supporting a serious vulnerability for widely used (some ESR) software seems a bit dangerous to me. The default configuration should protect the /wanna-be/ administrators. All others will most likely tune their config no matter what is supplied. Best, Richard On 10/30/2014 2:47 PM, Maxim Dounin wrote: > Hello! > > On Wed, Oct 29, 2014 at 09:17:04PM -0700, Piotr Sikora wrote: > >> # HG changeset patch >> # User Piotr Sikora >> # Date 1414642398 25200 >> # Wed Oct 29 21:13:18 2014 -0700 >> # Node ID bf17486e5d30574b870926b76c1d6f421e4def75 >> # Parent 87ada3ba1392fadaf4d9193b5d345c248be32f77 >> SSL: don't enable SSLv3 by default. > This was discussed excessively both in the office here and > in Russian mailing list a while ago, and consensus is that we are > not changing the default for now. > > Rationale is as follows: > > - SSLv3 is still important from compatibility point of view, there > are various clients which doesn't support (or enable by default) > anything better; > > - Mitigation for POODLE is already good and improving, including > fallback protection via TLS_FALLBACK_SCSV and anti-POODLE record > splitting; so, basically, modern browsers are not affected. > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4237 bytes Desc: S/MIME Cryptographic Signature URL: From pluknet at nginx.com Thu Oct 30 14:39:28 2014 From: pluknet at nginx.com (Sergey Kandaurov) Date: Thu, 30 Oct 2014 14:39:28 +0000 Subject: [nginx] SSL: simplified ssl_password_file error handling. Message-ID: details: http://hg.nginx.org/nginx/rev/42520df85ebb branches: changeset: 5892:42520df85ebb user: Sergey Kandaurov date: Fri Oct 24 04:28:00 2014 -0700 description: SSL: simplified ssl_password_file error handling. Instead of collecting a number of the possible SSL_CTX_use_PrivateKey_file() error codes that becomes more and more difficult with the rising variety of OpenSSL versions and its derivatives, just continue with the next password. Multiple passwords in a single ssl_password_file feature was broken after recent OpenSSL changes (commit 4aac102f75b517bdb56b1bcfd0a856052d559f6e). Affected OpenSSL releases: 0.9.8zc, 1.0.0o, 1.0.1j and 1.0.2-beta3. Reported by Piotr Sikora. diffstat: src/event/ngx_event_openssl.c | 17 +++-------------- 1 files changed, 3 insertions(+), 14 deletions(-) diffs (27 lines): diff -r 87ada3ba1392 -r 42520df85ebb src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Mon Oct 27 14:25:56 2014 -0700 +++ b/src/event/ngx_event_openssl.c Fri Oct 24 04:28:00 2014 -0700 @@ -404,20 +404,9 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ } if (--tries) { - n = ERR_peek_error(); - -#ifdef OPENSSL_IS_BORINGSSL - if (ERR_GET_LIB(n) == ERR_LIB_CIPHER - && ERR_GET_REASON(n) == CIPHER_R_BAD_DECRYPT) -#else - if (ERR_GET_LIB(n) == ERR_LIB_EVP - && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) -#endif - { - ERR_clear_error(); - SSL_CTX_set_default_passwd_cb_userdata(ssl->ctx, ++pwd); - continue; - } + ERR_clear_error(); + SSL_CTX_set_default_passwd_cb_userdata(ssl->ctx, ++pwd); + continue; } ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, From pluknet at nginx.com Thu Oct 30 14:40:10 2014 From: pluknet at nginx.com (Sergey Kandaurov) Date: Thu, 30 Oct 2014 17:40:10 +0300 Subject: [PATCH] SSL: make ssl_password_file work with recent OpenSSL releases In-Reply-To: References: <2C5B0D5C-6A77-46BA-98C9-7F472572270D@nginx.com> <20141027162829.GC45418@mdounin.ru> <02A7426A-6236-4791-90B0-272AC50CFE91@nginx.com> Message-ID: <98E908A9-19D4-4670-82E5-F906926D6C25@nginx.com> On Oct 29, 2014, at 10:27 PM, Piotr Sikora wrote: > Hey Sergey, > >> Alternatively, remove error collection that doesn?t really make much sense. >> It?s an open set and we cannot predict for sure all of them enumerated. > > That's also a valid approach, but the commit is wrong. > So that the issue is resolved now. >> # HG changeset patch >> # User Sergey Kandaurov >> # Date 1414150080 25200 >> # Fri Oct 24 04:28:00 2014 -0700 >> # Node ID 47728601e542cab9406ea323576f6df5adfff193 >> # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 >> SSL: simplified ssl_password_file error handling. >> >> Instead of collecting a number of the possible SSL_CTX_use_PrivateKey_file() >> error codes that becomes more and more difficult with the rising variety of >> OpenSSL versions and its derivatives, just continue with the next passphrase. >> >> This fixes multiple passwords in a single ssl_password_file that was broken >> after recent OpenSSL changes (commit 4aac102f75b517bdb56b1bcfd0a856052d559f6e). > > Not even "Reported by"? :P > Sure. Committed, thank you. -- Sergey Kandaurov From mdounin at mdounin.ru Thu Oct 30 15:26:11 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 30 Oct 2014 18:26:11 +0300 Subject: [PATCH] SSL: don't enable SSLv3 by default In-Reply-To: <5452459E.5090502@fussenegger.info> References: <20141030134735.GY45418@mdounin.ru> <5452459E.5090502@fussenegger.info> Message-ID: <20141030152611.GZ45418@mdounin.ru> Hello! On Thu, Oct 30, 2014 at 03:05:18PM +0100, Richard Fussenegger wrote: > The rationale may make sense depending on the priorities, but shouldn't the > default configuration target generic applications? Generic applications > don't need compatibility with ancient software (only IE6 on XP actually > /needs/ SSLv3, don't know about libraries though). That's excatly the point: the default is for generic case, and in general there is nothing wrong with supporting SSLv3 as long as nothing better is available. And there are various clients which don't support anything better, including IE6 on XP. The bad thing with POODLE is actually that due to fallback code in browsers it used to affect modern browsers. This problem goes away gradually. > Administrators who need the support can still enable it and make use of > SCSV. And don't forget that 'modern browser' applies to IE up to 11, FF up > to 34, Chrome up to ? (couldn't find the exact version) of which actually > not a single one has SCSV support and they won't get it! Providing As of now, the problem doesn't affect at least: - latest versions of Chrome (TLS_FALLBACK_SCSV); - latest versions of Opera (TLS_FALLBACK_SCSV, anti-POODLE record splitting); - latest versions of Safari (no block ciphers over SSLv3); - latest (upcoming?) versions of Firefox (disabled fallback to SSLv3); - upcoming versions of IE (announced plans to disable fallback to SSLv3). This basically covers all modern browsers (or at least almost all). Talking about not updated versions from security point of view is mostly pointless, as there are multiple security problems fixed on a regular basis, and not updated means not secure. -- Maxim Dounin http://nginx.org/ From richard at fussenegger.info Thu Oct 30 15:30:46 2014 From: richard at fussenegger.info (Richard Fussenegger) Date: Thu, 30 Oct 2014 16:30:46 +0100 Subject: [PATCH] SSL: don't enable SSLv3 by default In-Reply-To: <20141030152611.GZ45418@mdounin.ru> References: <20141030134735.GY45418@mdounin.ru> <5452459E.5090502@fussenegger.info> <20141030152611.GZ45418@mdounin.ru> Message-ID: <545259A6.10504@fussenegger.info> On 10/30/2014 4:26 PM, Maxim Dounin wrote: > And there are various clients which > don't support anything better, including IE6 on XP. > [...] > Talking about not updated versions from security point of > view is mostly pointless, as there are multiple security problems > fixed on a regular basis, and not updated means not secure. Well, that's actually my point. Those old libraries and clients shouldn't be supported since they are, well, old. Like the old versions of the others. Also note that SSLv3's RFC has status HISTORIC. The guys over at the IETF TLS list are talking about deprecating it, but some parties argue that the HISTORIC status is equivalent to deprecation. Richard -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4237 bytes Desc: S/MIME Cryptographic Signature URL: From mdounin at mdounin.ru Thu Oct 30 15:47:05 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 30 Oct 2014 18:47:05 +0300 Subject: [PATCH] SSL: don't enable SSLv3 by default In-Reply-To: <545259A6.10504@fussenegger.info> References: <20141030134735.GY45418@mdounin.ru> <5452459E.5090502@fussenegger.info> <20141030152611.GZ45418@mdounin.ru> <545259A6.10504@fussenegger.info> Message-ID: <20141030154705.GB45418@mdounin.ru> Hello! On Thu, Oct 30, 2014 at 04:30:46PM +0100, Richard Fussenegger wrote: > On 10/30/2014 4:26 PM, Maxim Dounin wrote: > >And there are various clients which > >don't support anything better, including IE6 on XP. > >[...] > >Talking about not updated versions from security point of > >view is mostly pointless, as there are multiple security problems > >fixed on a regular basis, and not updated means not secure. > > Well, that's actually my point. Those old libraries and clients shouldn't be > supported since they are, well, old. Like the old versions of the others. There is still compatibility point of view, and from this point of view it's important to be able to talk to old versions of browsers. To be able to show a message like "update your browser, it's too old", to deliver updates to them, or whatever. -- Maxim Dounin http://nginx.org/ From richard at fussenegger.info Thu Oct 30 15:54:47 2014 From: richard at fussenegger.info (Richard Fussenegger) Date: Thu, 30 Oct 2014 16:54:47 +0100 Subject: [PATCH] SSL: don't enable SSLv3 by default In-Reply-To: <20141030154705.GB45418@mdounin.ru> References: <20141030134735.GY45418@mdounin.ru> <5452459E.5090502@fussenegger.info> <20141030152611.GZ45418@mdounin.ru> <545259A6.10504@fussenegger.info> <20141030154705.GB45418@mdounin.ru> Message-ID: <54525F47.80308@fussenegger.info> On 10/30/2014 4:47 PM, Maxim Dounin wrote: > Hello! > > There is still compatibility point of view, and from this point of > view it's important to be able to talk to old versions of > browsers. To be able to show a message like "update your browser, > it's too old", to deliver updates to them, or whatever. I'd love to live in such a world, instead huge companies like Microsoft are not capable of deploying valid certificates on their most frequented websites. Seems like they've fixed it now, well you could always try to download something from Oracle's download.oracle.com domain and *bam* enjoy your certificate warning. Assuming you actually receive it, since almost all browsers decide to simply ignore it because they don't want to break the Internet. I'm not the one to decide, but I still think that a major software like nginx should stand out by proper reactions to security threads and RFC statuses. However, I hope you react at least after the deprecation RFC is out. Richard -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4237 bytes Desc: S/MIME Cryptographic Signature URL: From mdounin at mdounin.ru Thu Oct 30 16:59:26 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 30 Oct 2014 16:59:26 +0000 Subject: [nginx] Contrib: add more directives to vim syntax. Message-ID: details: http://hg.nginx.org/nginx/rev/fa4161fe8254 branches: changeset: 5893:fa4161fe8254 user: Peter Wu date: Wed Oct 22 15:31:19 2014 +0200 description: Contrib: add more directives to vim syntax. uwsgi is not a third-party module anymore. 'split_clients' is a new block directive type. The directives were automatically extracted from the ngx_command_t arrays, sorted in the C locale and inserted. Trailing spaces got removed at some places. diffstat: contrib/vim/syntax/nginx.vim | 233 +++++++++++++++++++++++++++++++++--------- 1 files changed, 183 insertions(+), 50 deletions(-) diffs (truncated from 499 to 300 lines): diff --git a/contrib/vim/syntax/nginx.vim b/contrib/vim/syntax/nginx.vim --- a/contrib/vim/syntax/nginx.vim +++ b/contrib/vim/syntax/nginx.vim @@ -31,6 +31,7 @@ syn keyword ngxDirectiveBlock limit_exce syn keyword ngxDirectiveBlock if contained syn keyword ngxDirectiveBlock geo contained syn keyword ngxDirectiveBlock map contained +syn keyword ngxDirectiveBlock split_clients contained syn keyword ngxDirectiveImportant include syn keyword ngxDirectiveImportant root @@ -41,6 +42,8 @@ syn keyword ngxDirectiveImportant intern syn keyword ngxDirectiveImportant proxy_pass syn keyword ngxDirectiveImportant memcached_pass syn keyword ngxDirectiveImportant fastcgi_pass +syn keyword ngxDirectiveImportant scgi_pass +syn keyword ngxDirectiveImportant uwsgi_pass syn keyword ngxDirectiveImportant try_files syn keyword ngxDirectiveControl break @@ -53,12 +56,15 @@ syn keyword ngxDirectiveError post_actio syn keyword ngxDirectiveDeprecated connections syn keyword ngxDirectiveDeprecated imap +syn keyword ngxDirectiveDeprecated limit_zone syn keyword ngxDirectiveDeprecated open_file_cache_retest syn keyword ngxDirectiveDeprecated optimize_server_names syn keyword ngxDirectiveDeprecated satisfy_any +syn keyword ngxDirectiveDeprecated so_keepalive syn keyword ngxDirective accept_mutex syn keyword ngxDirective accept_mutex_delay +syn keyword ngxDirective acceptex_read syn keyword ngxDirective access_log syn keyword ngxDirective add_after_body syn keyword ngxDirective add_before_body @@ -74,11 +80,14 @@ syn keyword ngxDirective auth_basic_user syn keyword ngxDirective auth_http syn keyword ngxDirective auth_http_header syn keyword ngxDirective auth_http_timeout +syn keyword ngxDirective auth_request +syn keyword ngxDirective auth_request_set syn keyword ngxDirective autoindex syn keyword ngxDirective autoindex_exact_size syn keyword ngxDirective autoindex_localtime syn keyword ngxDirective charset syn keyword ngxDirective charset_types +syn keyword ngxDirective chunked_transfer_encoding syn keyword ngxDirective client_body_buffer_size syn keyword ngxDirective client_body_in_file_only syn keyword ngxDirective client_body_in_single_buffer @@ -102,32 +111,44 @@ syn keyword ngxDirective devpoll_changes syn keyword ngxDirective devpoll_events syn keyword ngxDirective directio syn keyword ngxDirective directio_alignment +syn keyword ngxDirective disable_symlinks syn keyword ngxDirective empty_gif syn keyword ngxDirective env syn keyword ngxDirective epoll_events syn keyword ngxDirective error_log +syn keyword ngxDirective etag syn keyword ngxDirective eventport_events syn keyword ngxDirective expires syn keyword ngxDirective fastcgi_bind syn keyword ngxDirective fastcgi_buffer_size +syn keyword ngxDirective fastcgi_buffering syn keyword ngxDirective fastcgi_buffers syn keyword ngxDirective fastcgi_busy_buffers_size syn keyword ngxDirective fastcgi_cache +syn keyword ngxDirective fastcgi_cache_bypass syn keyword ngxDirective fastcgi_cache_key +syn keyword ngxDirective fastcgi_cache_lock +syn keyword ngxDirective fastcgi_cache_lock_timeout syn keyword ngxDirective fastcgi_cache_methods syn keyword ngxDirective fastcgi_cache_min_uses syn keyword ngxDirective fastcgi_cache_path +syn keyword ngxDirective fastcgi_cache_revalidate syn keyword ngxDirective fastcgi_cache_use_stale syn keyword ngxDirective fastcgi_cache_valid syn keyword ngxDirective fastcgi_catch_stderr syn keyword ngxDirective fastcgi_connect_timeout +syn keyword ngxDirective fastcgi_force_ranges syn keyword ngxDirective fastcgi_hide_header syn keyword ngxDirective fastcgi_ignore_client_abort syn keyword ngxDirective fastcgi_ignore_headers syn keyword ngxDirective fastcgi_index syn keyword ngxDirective fastcgi_intercept_errors +syn keyword ngxDirective fastcgi_keep_conn syn keyword ngxDirective fastcgi_max_temp_file_size syn keyword ngxDirective fastcgi_next_upstream +syn keyword ngxDirective fastcgi_next_upstream_timeout +syn keyword ngxDirective fastcgi_next_upstream_tries +syn keyword ngxDirective fastcgi_no_cache syn keyword ngxDirective fastcgi_param syn keyword ngxDirective fastcgi_pass_header syn keyword ngxDirective fastcgi_pass_request_body @@ -140,12 +161,15 @@ syn keyword ngxDirective fastcgi_store syn keyword ngxDirective fastcgi_store_access syn keyword ngxDirective fastcgi_temp_file_write_size syn keyword ngxDirective fastcgi_temp_path -syn keyword ngxDirective fastcgi_upstream_fail_timeout -syn keyword ngxDirective fastcgi_upstream_max_fails syn keyword ngxDirective flv syn keyword ngxDirective geoip_city syn keyword ngxDirective geoip_country +syn keyword ngxDirective geoip_org +syn keyword ngxDirective geoip_proxy +syn keyword ngxDirective geoip_proxy_recursive syn keyword ngxDirective google_perftools_profiles +syn keyword ngxDirective gunzip +syn keyword ngxDirective gunzip_buffers syn keyword ngxDirective gzip syn keyword ngxDirective gzip_buffers syn keyword ngxDirective gzip_comp_level @@ -159,30 +183,40 @@ syn keyword ngxDirective gzip_static syn keyword ngxDirective gzip_types syn keyword ngxDirective gzip_vary syn keyword ngxDirective gzip_window +syn keyword ngxDirective hash syn keyword ngxDirective if_modified_since syn keyword ngxDirective ignore_invalid_headers syn keyword ngxDirective image_filter syn keyword ngxDirective image_filter_buffer +syn keyword ngxDirective image_filter_interlace syn keyword ngxDirective image_filter_jpeg_quality +syn keyword ngxDirective image_filter_sharpen syn keyword ngxDirective image_filter_transparency syn keyword ngxDirective imap_auth syn keyword ngxDirective imap_capabilities syn keyword ngxDirective imap_client_buffer syn keyword ngxDirective index +syn keyword ngxDirective iocp_threads syn keyword ngxDirective ip_hash +syn keyword ngxDirective keepalive +syn keyword ngxDirective keepalive_disable syn keyword ngxDirective keepalive_requests syn keyword ngxDirective keepalive_timeout syn keyword ngxDirective kqueue_changes syn keyword ngxDirective kqueue_events syn keyword ngxDirective large_client_header_buffers +syn keyword ngxDirective least_conn syn keyword ngxDirective limit_conn syn keyword ngxDirective limit_conn_log_level +syn keyword ngxDirective limit_conn_status +syn keyword ngxDirective limit_conn_zone syn keyword ngxDirective limit_rate syn keyword ngxDirective limit_rate_after syn keyword ngxDirective limit_req syn keyword ngxDirective limit_req_log_level +syn keyword ngxDirective limit_req_status syn keyword ngxDirective limit_req_zone -syn keyword ngxDirective limit_zone +syn keyword ngxDirective lingering_close syn keyword ngxDirective lingering_time syn keyword ngxDirective lingering_timeout syn keyword ngxDirective lock_file @@ -192,21 +226,27 @@ syn keyword ngxDirective log_subrequest syn keyword ngxDirective map_hash_bucket_size syn keyword ngxDirective map_hash_max_size syn keyword ngxDirective master_process +syn keyword ngxDirective max_ranges syn keyword ngxDirective memcached_bind syn keyword ngxDirective memcached_buffer_size syn keyword ngxDirective memcached_connect_timeout +syn keyword ngxDirective memcached_gzip_flag syn keyword ngxDirective memcached_next_upstream +syn keyword ngxDirective memcached_next_upstream_timeout +syn keyword ngxDirective memcached_next_upstream_tries syn keyword ngxDirective memcached_read_timeout syn keyword ngxDirective memcached_send_timeout -syn keyword ngxDirective memcached_upstream_fail_timeout -syn keyword ngxDirective memcached_upstream_max_fails syn keyword ngxDirective merge_slashes syn keyword ngxDirective min_delete_depth syn keyword ngxDirective modern_browser syn keyword ngxDirective modern_browser_value +syn keyword ngxDirective mp4 +syn keyword ngxDirective mp4_buffer_size +syn keyword ngxDirective mp4_max_buffer_size syn keyword ngxDirective msie_padding syn keyword ngxDirective msie_refresh syn keyword ngxDirective multi_accept +syn keyword ngxDirective mysql_test syn keyword ngxDirective open_file_cache syn keyword ngxDirective open_file_cache_errors syn keyword ngxDirective open_file_cache_events @@ -215,6 +255,7 @@ syn keyword ngxDirective open_file_cache syn keyword ngxDirective open_log_file_cache syn keyword ngxDirective output_buffers syn keyword ngxDirective override_charset +syn keyword ngxDirective pcre_jit syn keyword ngxDirective perl syn keyword ngxDirective perl_modules syn keyword ngxDirective perl_require @@ -223,6 +264,7 @@ syn keyword ngxDirective pid syn keyword ngxDirective pop3_auth syn keyword ngxDirective pop3_capabilities syn keyword ngxDirective port_in_redirect +syn keyword ngxDirective post_acceptex syn keyword ngxDirective postpone_gzipping syn keyword ngxDirective postpone_output syn keyword ngxDirective protocol @@ -234,22 +276,33 @@ syn keyword ngxDirective proxy_buffering syn keyword ngxDirective proxy_buffers syn keyword ngxDirective proxy_busy_buffers_size syn keyword ngxDirective proxy_cache +syn keyword ngxDirective proxy_cache_bypass syn keyword ngxDirective proxy_cache_key +syn keyword ngxDirective proxy_cache_lock +syn keyword ngxDirective proxy_cache_lock_timeout syn keyword ngxDirective proxy_cache_methods syn keyword ngxDirective proxy_cache_min_uses syn keyword ngxDirective proxy_cache_path +syn keyword ngxDirective proxy_cache_revalidate syn keyword ngxDirective proxy_cache_use_stale syn keyword ngxDirective proxy_cache_valid syn keyword ngxDirective proxy_connect_timeout +syn keyword ngxDirective proxy_cookie_domain +syn keyword ngxDirective proxy_cookie_path +syn keyword ngxDirective proxy_force_ranges syn keyword ngxDirective proxy_headers_hash_bucket_size syn keyword ngxDirective proxy_headers_hash_max_size syn keyword ngxDirective proxy_hide_header +syn keyword ngxDirective proxy_http_version syn keyword ngxDirective proxy_ignore_client_abort syn keyword ngxDirective proxy_ignore_headers syn keyword ngxDirective proxy_intercept_errors syn keyword ngxDirective proxy_max_temp_file_size syn keyword ngxDirective proxy_method syn keyword ngxDirective proxy_next_upstream +syn keyword ngxDirective proxy_next_upstream_timeout +syn keyword ngxDirective proxy_next_upstream_tries +syn keyword ngxDirective proxy_no_cache syn keyword ngxDirective proxy_pass_error_message syn keyword ngxDirective proxy_pass_header syn keyword ngxDirective proxy_pass_request_body @@ -260,18 +313,27 @@ syn keyword ngxDirective proxy_send_lowa syn keyword ngxDirective proxy_send_timeout syn keyword ngxDirective proxy_set_body syn keyword ngxDirective proxy_set_header +syn keyword ngxDirective proxy_ssl_ciphers +syn keyword ngxDirective proxy_ssl_crl +syn keyword ngxDirective proxy_ssl_name +syn keyword ngxDirective proxy_ssl_protocols +syn keyword ngxDirective proxy_ssl_server_name syn keyword ngxDirective proxy_ssl_session_reuse +syn keyword ngxDirective proxy_ssl_trusted_certificate +syn keyword ngxDirective proxy_ssl_verify +syn keyword ngxDirective proxy_ssl_verify_depth syn keyword ngxDirective proxy_store syn keyword ngxDirective proxy_store_access syn keyword ngxDirective proxy_temp_file_write_size syn keyword ngxDirective proxy_temp_path syn keyword ngxDirective proxy_timeout -syn keyword ngxDirective proxy_upstream_fail_timeout -syn keyword ngxDirective proxy_upstream_max_fails syn keyword ngxDirective random_index syn keyword ngxDirective read_ahead syn keyword ngxDirective real_ip_header +syn keyword ngxDirective real_ip_recursive syn keyword ngxDirective recursive_error_pages +syn keyword ngxDirective referer_hash_bucket_size +syn keyword ngxDirective referer_hash_max_size syn keyword ngxDirective request_pool_size syn keyword ngxDirective reset_timedout_connection syn keyword ngxDirective resolver @@ -282,6 +344,45 @@ syn keyword ngxDirective rtsig_overflow_ syn keyword ngxDirective rtsig_overflow_threshold syn keyword ngxDirective rtsig_signo syn keyword ngxDirective satisfy +syn keyword ngxDirective scgi_bind +syn keyword ngxDirective scgi_buffer_size +syn keyword ngxDirective scgi_buffering +syn keyword ngxDirective scgi_buffers +syn keyword ngxDirective scgi_busy_buffers_size +syn keyword ngxDirective scgi_cache +syn keyword ngxDirective scgi_cache_bypass +syn keyword ngxDirective scgi_cache_key +syn keyword ngxDirective scgi_cache_lock +syn keyword ngxDirective scgi_cache_lock_timeout +syn keyword ngxDirective scgi_cache_methods +syn keyword ngxDirective scgi_cache_min_uses +syn keyword ngxDirective scgi_cache_path +syn keyword ngxDirective scgi_cache_revalidate +syn keyword ngxDirective scgi_cache_use_stale +syn keyword ngxDirective scgi_cache_valid +syn keyword ngxDirective scgi_connect_timeout +syn keyword ngxDirective scgi_force_ranges +syn keyword ngxDirective scgi_hide_header +syn keyword ngxDirective scgi_ignore_client_abort +syn keyword ngxDirective scgi_ignore_headers +syn keyword ngxDirective scgi_intercept_errors +syn keyword ngxDirective scgi_max_temp_file_size +syn keyword ngxDirective scgi_next_upstream +syn keyword ngxDirective scgi_next_upstream_timeout +syn keyword ngxDirective scgi_next_upstream_tries +syn keyword ngxDirective scgi_no_cache +syn keyword ngxDirective scgi_param +syn keyword ngxDirective scgi_pass_header +syn keyword ngxDirective scgi_pass_request_body From mdounin at mdounin.ru Thu Oct 30 17:00:31 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 30 Oct 2014 20:00:31 +0300 Subject: [PATCH] Contrib: add more directives to vim syntax In-Reply-To: <3664659.SkH2gXEf4o@al> References: <2127392.887hlZJyHY@al> <20141022060445.GB44913@mdounin.ru> <3664659.SkH2gXEf4o@al> Message-ID: <20141030170030.GE45418@mdounin.ru> Hello! On Wed, Oct 22, 2014 at 03:39:31PM +0200, Peter Wu wrote: [...] > # HG changeset patch > # User Peter Wu > # Date 1413984679 -7200 > # Wed Oct 22 15:31:19 2014 +0200 > # Node ID 9c2253baa56b8f9a22f24b8650ca0861d99f24fa > # Parent 973fded4f461f3a397779b3a1dc80881b1b34974 > Contrib: add more directives to vim syntax. > > uwsgi is not a third-party module anymore. 'split_clients' is a new > block directive type. The directives were automatically extracted from > the ngx_command_t arrays, sorted in the C locale and inserted. > > Trailing spaces got removed at some places. Committed, thanks. -- Maxim Dounin http://nginx.org/ From kpariani at zimbra.com Thu Oct 30 18:25:04 2014 From: kpariani at zimbra.com (Kunal Pariani) Date: Thu, 30 Oct 2014 13:25:04 -0500 (CDT) Subject: [PATCH] SSL support for the mail proxy module In-Reply-To: <655993562.4651608.1414693499438.JavaMail.zimbra@zimbra.com> References: <5c2524403ab7c870b1fa.1410650071@zdev-vm048.eng.zimbra.com> <8B3FA009-99B4-43AD-A207-5BE41FA58ECD@gmail.com> <607846621.1950104.1410852617470.JavaMail.zimbra@zimbra.com> <20140916120306.GF59236@mdounin.ru> <897609716.2187306.1410913349045.JavaMail.zimbra@zimbra.com> <1497581210.2187358.1410913427096.JavaMail.zimbra@zimbra.com> Message-ID: <1672749644.4651636.1414693504176.JavaMail.zimbra@zimbra.com> Hello, Any reason for this patch not being committed upstream yet ? Thanks -Kunal From: "Franck Levionnois" To: "nginx-devel" , "Kunal Pariani" Sent: Tuesday, October 21, 2014 12:59:04 AM Subject: Re: [PATCH] SSL support for the mail proxy module Hello, The patch below has been submited some months ago. It do about the same, and it support to return a name by the auth script. Kind regards Franck Levionnois. ---------- Forwarded message ---------- From: < flevionnois at gmail.com > Date: 2014-01-24 21:40 GMT+01:00 Subject: [PATCH 1 of 1] Mail: added support for SSL client certificate To: nginx-devel at nginx.org # HG changeset patch # User Franck Levionnois < flevionnois at gmail.com > # Date 1390577176 -3600 # Fri Jan 24 16:26:16 2014 +0100 # Node ID d7b8381c200e300c2b6729574f4c2a b537804f56 # Parent a387ce36744aa36b50e8171dbf01ef716748327e Mail: added support for SSL client certificate Add support for SSL module like HTTP. Added mail configuration directives (like http): ssl_verify_client, ssl_verify_depth, ssl_client_certificate, ssl_trusted_certificate, ssl_crl Added headers: Auth-Certificate, Auth-Certificate-Verify, Auth-Issuer-DN, Auth-Subject-DN, Auth-Subject-Serial diff -r a387ce36744a -r d7b8381c200e src/mail/ngx_mail_auth_http_module.c --- a/src/mail/ngx_mail_auth_http_module.c Thu Jan 23 22:09:59 2014 +0900 +++ b/src/mail/ngx_mail_auth_http_module.c Fri Jan 24 16:26:16 2014 +0100 @@ -1135,6 +1135,32 @@ "mail auth http dummy handler"); } +#if (NGX_MAIL_SSL) +static ngx_int_t +ngx_ssl_get_certificate_oneline(ngx_connection_t *c, ngx_pool_t *pool, + ngx_str_t *b64_cert) +{ + ngx_str_t pemCert; + if (ngx_ssl_get_raw_certificate(c, pool, &pemCert) != NGX_OK) { + return NGX_ERROR; + } + + if (pemCert.len == 0) { + b64_cert->len = 0; + return NGX_OK; + } + + b64_cert->len = ngx_base64_encoded_length(pemCert.len); + b64_cert->data = ngx_palloc( pool, b64_cert->len); + if (b64_cert->data == NULL) { + b64_cert->len = 0; + return NGX_ERROR; + } + ngx_encode_base64(b64_cert, &pemCert); + + return NGX_OK; +} +#endif static ngx_buf_t * ngx_mail_auth_http_create_request(ngx_mail_session_t *s, ngx_pool_t *pool, @@ -1142,7 +1168,9 @@ { size_t len; ngx_buf_t *b; - ngx_str_t login, passwd; + ngx_str_t login, passwd, client_cert, client_verify, + client_subject, client_issuer, + client_serial; ngx_mail_core_srv_conf_t *cscf; if (ngx_mail_auth_http_escape(pool, &s->login, &login) != NGX_OK) { @@ -1155,6 +1183,42 @@ cscf = ngx_mail_get_module_srv_conf(s, ngx_mail_core_module); +#if (NGX_MAIL_SSL) + if (s->connection->ssl) { + if (ngx_ssl_get_client_verify(s->connection, pool, + &client_verify) != NGX_OK) { + return NULL; + } + + if (ngx_ssl_get_subject_dn(s->connection, pool, + &client_subject) != NGX_OK) { + return NULL; + } + + if (ngx_ssl_get_issuer_dn(s->connection, pool, + &client_issuer) != NGX_OK) { + return NULL; + } + + if (ngx_ssl_get_serial_number(s->connection, pool, + &client_serial) != NGX_OK) { + return NULL; + } + + if (ngx_ssl_get_certificate_oneline(s->connection, pool, + &client_cert) != NGX_OK) { + return NULL; + } + } else { + client_verify.len = 0; + client_issuer.len = 0; + client_subject.len = 0; + client_serial.len = 0; + client_cert.len = 0; + } + +#endif + len = sizeof("GET ") - 1 + ahcf->uri.len + sizeof(" HTTP/1.0" CRLF) - 1 + sizeof("Host: ") - 1 + ahcf->host_header.len + sizeof(CRLF) - 1 + sizeof("Auth-Method: ") - 1 @@ -1163,6 +1227,18 @@ + sizeof("Auth-User: ") - 1 + login.len + sizeof(CRLF) - 1 + sizeof("Auth-Pass: ") - 1 + passwd.len + sizeof(CRLF) - 1 + sizeof("Auth-Salt: ") - 1 + s->salt.len +#if (NGX_MAIL_SSL) + + sizeof("Auth-Certificate: ") - 1 + client_cert.len + + sizeof(CRLF) - 1 + + sizeof("Auth-Certificate-Verify: ") - 1 + client_verify.len + + sizeof(CRLF) - 1 + + sizeof("Auth-Issuer-DN: ") - 1 + client_issuer.len + + sizeof(CRLF) - 1 + + sizeof("Auth-Subject-DN: ") - 1 + client_subject.len + + sizeof(CRLF) - 1 + + sizeof("Auth-Subject-Serial: ") - 1 + client_serial.len + + sizeof(CRLF) - 1 +#endif + sizeof("Auth-Protocol: ") - 1 + cscf->protocol->name.len + sizeof(CRLF) - 1 + sizeof("Auth-Login-Attempt: ") - 1 + NGX_INT_T_LEN @@ -1212,7 +1288,43 @@ s->passwd.data = NULL; } - +#if (NGX_MAIL_SSL) + if ( client_cert.len ) + { + b->last = ngx_cpymem(b->last, "Auth-Certificate: ", + sizeof("Auth-Certificate: ") - 1); + b->last = ngx_copy(b->last, client_cert.data, client_cert.len); + *b->last++ = CR; *b->last++ = LF; + } + if ( client_verify.len ) + { + b->last = ngx_cpymem(b->last, "Auth-Certificate-Verify: ", + sizeof("Auth-Certificate-Verify: ") - 1); + b->last = ngx_copy(b->last, client_verify.data, client_verify.len); + *b->last++ = CR; *b->last++ = LF; + } + if ( client_issuer.len ) + { + b->last = ngx_cpymem(b->last, "Auth-Issuer-DN: ", + sizeof("Auth-Issuer-DN: ") - 1); + b->last = ngx_copy(b->last, client_issuer.data, client_issuer.len); + *b->last++ = CR; *b->last++ = LF; + } + if ( client_subject.len ) + { + b->last = ngx_cpymem(b->last, "Auth-Subject-DN: ", + sizeof("Auth-Subject-DN: ") - 1); + b->last = ngx_copy(b->last, client_subject.data, client_subject.len); + *b->last++ = CR; *b->last++ = LF; + } + if ( client_serial.len ) + { + b->last = ngx_cpymem(b->last, "Auth-Subject-Serial: ", + sizeof("Auth-Subject-Serial: ") - 1); + b->last = ngx_copy(b->last, client_serial.data, client_serial.len); + *b->last++ = CR; *b->last++ = LF; + } +#endif b->last = ngx_cpymem(b->last, "Auth-Protocol: ", sizeof("Auth-Protocol: ") - 1); b->last = ngx_cpymem(b->last, cscf->protocol->name.data, diff -r a387ce36744a -r d7b8381c200e src/mail/ngx_mail_handler.c --- a/src/mail/ngx_mail_handler.c Thu Jan 23 22:09:59 2014 +0900 +++ b/src/mail/ngx_mail_handler.c Fri Jan 24 16:26:16 2014 +0100 @@ -236,11 +236,59 @@ { ngx_mail_session_t *s; ngx_mail_core_srv_conf_t *cscf; +#if (NGX_MAIL_SSL) + ngx_mail_ssl_conf_t *sslcf; +#endif + + ngx_log_debug1(NGX_LOG_DEBUG_MAIL, c->log, 0, + "ngx_mail_ssl_handshake_handler handshaked: %d ", + c->ssl->handshaked ); if (c->ssl->handshaked) { s = c->data; +#if (NGX_MAIL_SSL) + sslcf = ngx_mail_get_module_srv_conf(s, ngx_mail_ssl_module); + if (sslcf->verify) { + long rc; + + rc = SSL_get_verify_result(c->ssl->connection); + + if (rc != X509_V_OK + && (sslcf->verify != 3 || !ngx_ssl_verify_error_optional(rc))) + { + ngx_log_error(NGX_LOG_INFO, c->log, 0, + "client SSL certificate verify error: (%l:%s)", + rc, X509_verify_cert_error_string(rc)); + + ngx_ssl_remove_cached_session(sslcf->ssl.ctx, + (SSL_get0_session(c->ssl->connection))); + + ngx_mail_close_connection(c); + return; + } + + if (sslcf->verify == 1) { + X509 *cert; + cert = SSL_get_peer_certificate(c->ssl->connection); + + if (cert == NULL) { + ngx_log_error(NGX_LOG_INFO, c->log, 0, + "client sent no required SSL certificate"); + + ngx_ssl_remove_cached_session(sslcf->ssl.ctx, + (SSL_get0_session(c->ssl->connection))); + + ngx_mail_close_connection(c); + return; + } + + X509_free(cert); + } + } +#endif + if (s->starttls) { cscf = ngx_mail_get_module_srv_conf(s, ngx_mail_core_module); @@ -276,6 +324,10 @@ s->protocol = cscf->protocol->type; + ngx_log_debug1(NGX_LOG_DEBUG_MAIL, c->log, 0, + "ngx_mail_init_session protocol: %d ", + cscf->protocol->type ); + s->ctx = ngx_pcalloc(c->pool, sizeof(void *) * ngx_mail_max_module); if (s->ctx == NULL) { ngx_mail_session_internal_server_error(s); diff -r a387ce36744a -r d7b8381c200e src/mail/ngx_mail_ssl_module.c --- a/src/mail/ngx_mail_ssl_module.c Thu Jan 23 22:09:59 2014 +0900 +++ b/src/mail/ngx_mail_ssl_module.c Fri Jan 24 16:26:16 2014 +0100 @@ -43,6 +43,13 @@ { ngx_null_string, 0 } }; +static ngx_conf_enum_t ngx_mail_ssl_verify[] = { + { ngx_string("off"), 0 }, + { ngx_string("on"), 1 }, + { ngx_string("optional"), 2 }, + { ngx_string("optional_no_ca"), 3 }, + { ngx_null_string, 0 } +}; static ngx_command_t ngx_mail_ssl_commands[] = { @@ -102,6 +109,34 @@ offsetof(ngx_mail_ssl_conf_t, ciphers), NULL }, + { ngx_string("ssl_verify_client"), + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_enum_slot, + NGX_MAIL_SRV_CONF_OFFSET, + offsetof(ngx_mail_ssl_conf_t, verify), + &ngx_mail_ssl_verify }, + + { ngx_string("ssl_verify_depth"), + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_1MORE, + ngx_conf_set_num_slot, + NGX_MAIL_SRV_CONF_OFFSET, + offsetof(ngx_mail_ssl_conf_t, verify_depth), + NULL }, + + { ngx_string("ssl_client_certificate"), + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_MAIL_SRV_CONF_OFFSET, + offsetof(ngx_mail_ssl_conf_t, client_certificate), + NULL }, + + { ngx_string("ssl_trusted_certificate"), + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_MAIL_SRV_CONF_OFFSET, + offsetof(ngx_mail_ssl_conf_t, trusted_certificate), + NULL }, + { ngx_string("ssl_prefer_server_ciphers"), NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_FLAG, ngx_conf_set_flag_slot, @@ -137,6 +172,13 @@ offsetof(ngx_mail_ssl_conf_t, session_timeout), NULL }, + { ngx_string("ssl_crl"), + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_MAIL_SRV_CONF_OFFSET, + offsetof(ngx_mail_ssl_conf_t, crl), + NULL }, + ngx_null_command }; @@ -196,6 +238,8 @@ scf->enable = NGX_CONF_UNSET; scf->starttls = NGX_CONF_UNSET_UINT; scf->prefer_server_ciphers = NGX_CONF_UNSET; + scf->verify = NGX_CONF_UNSET_UINT; + scf->verify_depth = NGX_CONF_UNSET_UINT; scf->builtin_session_cache = NGX_CONF_UNSET; scf->session_timeout = NGX_CONF_UNSET; scf->session_tickets = NGX_CONF_UNSET; @@ -228,11 +272,20 @@ (NGX_CONF_BITMASK_SET|NGX_SSL_SSLv3|NGX_SSL_TLSv1 |NGX_SSL_TLSv1_1|NGX_SSL_TLSv1_2)); + ngx_conf_merge_uint_value(conf->verify, prev->verify, 0); + ngx_conf_merge_uint_value(conf->verify_depth, prev->verify_depth, 1); + ngx_conf_merge_str_value(conf->certificate, prev->certificate, ""); ngx_conf_merge_str_value(conf->certificate_key, prev->certificate_key, ""); ngx_conf_merge_str_value(conf->dhparam, prev->dhparam, ""); + ngx_conf_merge_str_value(conf->client_certificate, prev->client_certificate, + ""); + ngx_conf_merge_str_value(conf->trusted_certificate, + prev->trusted_certificate, ""); + ngx_conf_merge_str_value(conf->crl, prev->crl, ""); + ngx_conf_merge_str_value(conf->ecdh_curve, prev->ecdh_curve, NGX_DEFAULT_ECDH_CURVE); @@ -318,6 +371,35 @@ return NGX_CONF_ERROR; } + if (conf->verify) { + + if (conf->client_certificate.len == 0 && conf->verify != 3) { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "no ssl_client_certificate for ssl_client_verify"); + return NGX_CONF_ERROR; + } + + if (ngx_ssl_client_certificate(cf, &conf->ssl, + &conf->client_certificate, + conf->verify_depth) + != NGX_OK) + { + return NGX_CONF_ERROR; + } + } + + if (ngx_ssl_trusted_certificate(cf, &conf->ssl, + &conf->trusted_certificate, + conf->verify_depth) + != NGX_OK) + { + return NGX_CONF_ERROR; + } + + if (ngx_ssl_crl(cf, &conf->ssl, &conf->crl) != NGX_OK) { + return NGX_CONF_ERROR; + } + if (conf->prefer_server_ciphers) { SSL_CTX_set_options(conf->ssl.ctx, SSL_OP_CIPHER_SERVER_PREFERENCE); } diff -r a387ce36744a -r d7b8381c200e src/mail/ngx_mail_ssl_module.h --- a/src/mail/ngx_mail_ssl_module.h Thu Jan 23 22:09:59 2014 +0900 +++ b/src/mail/ngx_mail_ssl_module.h Fri Jan 24 16:26:16 2014 +0100 @@ -28,6 +28,8 @@ ngx_uint_t starttls; ngx_uint_t protocols; + ngx_uint_t verify; + ngx_uint_t verify_depth; ssize_t builtin_session_cache; time_t session_timeout; @@ -36,6 +38,9 @@ ngx_str_t certificate_key; ngx_str_t dhparam; ngx_str_t ecdh_curve; + ngx_str_t client_certificate; + ngx_str_t trusted_certificate; + ngx_str_t crl; ngx_str_t ciphers; 2014-09-17 2:23 GMT+02:00 Kunal Pariani < kpariani at zimbra.com > : I guess these diffs can still be applied with limitation of having the same ssl setting for all upstreams until the support to return a name by the auth script is added later ? Thanks -Kunal ----- Original Message ----- From: "Maxim Dounin" < mdounin at mdounin.ru > To: "nginx-devel" < nginx-devel at nginx.org > Sent: Tuesday, September 16, 2014 5:03:06 AM Subject: Re: [PATCH] SSL support for the mail proxy module Hello! On Tue, Sep 16, 2014 at 02:30:17AM -0500, Kunal Pariani wrote: > Updated the diffs after addressing the first 2 issues. > Regarding the 3rd comment, you are correct. Only 1 set of ssl > settings for all the mail backends with my patch. I guess this > will be a limitation with the current mail proxy workflow ? Am > not sure of the exact changes that will be required to address > this issue completely. Probably, returning a name by the auth script is a way to go. -- Maxim Dounin http://nginx.org/ _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From yar at nginx.com Thu Oct 30 19:03:09 2014 From: yar at nginx.com (Yaroslav Zhuravlev) Date: Thu, 30 Oct 2014 23:03:09 +0400 Subject: [PATCH] Clarify meaning of limit_conn in SPDY connections In-Reply-To: <3674e10a9e22a622998b.1413305900@toodee.local> References: <3674e10a9e22a622998b.1413305900@toodee.local> Message-ID: On 14 Oct 2014, at 20:58, Gunnlaugur Thor Briem wrote: > # HG changeset patch > # User Gunnlaugur Thor Briem > # Date 1413305660 0 > # Tue Oct 14 16:54:20 2014 +0000 > # Node ID 3674e10a9e22a622998b65badfe01da34579bb65 > # Parent 2096ecf6de02bc9e8ae920c45c59bf6a4e2e38fb > Clarify meaning of limit_conn in SPDY connections > > Be clear about limit_conn applying to concurrent requests, not connections. > The distinction matters for SPDY connections. > [?] Hi Gunnlaugur! Thank you for your patch. I wouldn?t change ?connections? to "requests? because it is already reflected in the module description: "Not all connections are counted. A connection is counted only if it has a request processed by the server and the whole request header has already been read.? However, The part about SPDY was committed with some changes: http://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn Best regards, Yaroslav Zhuravlev From yingqi.lu at intel.com Thu Oct 30 22:24:35 2014 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Thu, 30 Oct 2014 22:24:35 +0000 Subject: [Patch] SO_REUSEPORT support from master process In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AA37D1B44@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AA37A52F5@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AA37D0C4E@ORSMSX113.amr.corp.intel.com> <20141008125848.GA31276@mdounin.ru> <9ACD5B67AAC5594CB6268234CF29CF9AA37D1997@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AA37D1B44@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA37EA433@ORSMSX113.amr.corp.intel.com> Hi All, We tested the dragonfly approach on Linux (RHEL 6.5 with kernel 3.13.9). We used the same testing environment for both our patch and the dragonfly patch. Here is what we found: 1. Our patch has 36% better performance (operations/sec) comparing to dragonfly patch. 2. Our patch has 53% lower response time comparing to dragonfly approach even at 36% higher throughput level. 3. Our patch can scale the CPU utilization and frequency to the max capacity while dragonfly patch cannot. 4. Our patch does not have any issues with "upgrade binary on the fly". However, dragonfly patch creates a spike in the response time during the upgrade. It also has lots of connection timeouts/losses with high load. Above findings are based on Linux OS. Thanks, Yingqi -----Original Message----- From: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Lu, Yingqi Sent: Wednesday, October 08, 2014 11:24 AM To: nginx-devel at nginx.org Subject: RE: [Patch] SO_REUSEPORT support from master process One more comment from me: duplicate listen sockets in kernel is not a trivia thing to do and it may take long time before people can see it. Addressing it Nginx may not be as ideal as in kernel, but at least user can see the performance improvement sooner. In fact, we see up to 48% performance improvement on modern Intel system. Just my two cents. Again, thanks very much for everyone for helping us review this. Thanks, Yingqi -----Original Message----- From: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Lu, Yingqi Sent: Wednesday, October 08, 2014 10:05 AM To: nginx-devel at nginx.org Subject: RE: [Patch] SO_REUSEPORT support from master process Hi Maxim, Thanks for letting us know. Our updated patch is located at http://forum.nginx.org/read.php?29,253446,253446#msg-253446 It supposes to address all the style issues and fixes the restart and binary upgrade issues. This is just a FYI in case you are not aware of. Thanks, Yingqi -----Original Message----- From: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Maxim Dounin Sent: Wednesday, October 08, 2014 5:59 AM To: nginx-devel at nginx.org Subject: Re: [Patch] SO_REUSEPORT support from master process Hello! On Tue, Oct 07, 2014 at 07:32:08PM +0000, Lu, Yingqi wrote: > Dear All, > > It has been quiet for a while on this patch. I am checking to see if > there is any questions/feedbacks/concerns we need to address? > > Please let me know. Thanks very much for your help! Apart from style/coding issues, I disagree with the whole approach. As far as I understand the patch idea, it tries to introduce multiple listening sockets to avoid in-kernel lock contention. This is something that can be done completely in kernel though, and I see no reason to introduce any changes to nginx here. The approach previously discussed with Sepherosa Ziehau looks much more interesting. -- Maxim Dounin http://nginx.org/ _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel From vbart at nginx.com Thu Oct 30 22:24:39 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Thu, 30 Oct 2014 22:24:39 +0000 Subject: [nginx] Events: removed broken thread support from event timers. Message-ID: details: http://hg.nginx.org/nginx/rev/1f513d7f1b45 branches: changeset: 5894:1f513d7f1b45 user: Valentin Bartenev date: Mon Aug 25 13:34:39 2014 +0400 description: Events: removed broken thread support from event timers. It's mostly dead code. And the idea of thread support for this task has been deprecated. diffstat: src/core/ngx_rbtree.c | 6 ++---- src/core/ngx_rbtree.h | 6 ++---- src/event/ngx_event_timer.c | 34 ++-------------------------------- src/event/ngx_event_timer.h | 15 +-------------- src/os/unix/ngx_process_cycle.c | 1 - 5 files changed, 7 insertions(+), 55 deletions(-) diffs (175 lines): diff -r fa4161fe8254 -r 1f513d7f1b45 src/core/ngx_rbtree.c --- a/src/core/ngx_rbtree.c Wed Oct 22 15:31:19 2014 +0200 +++ b/src/core/ngx_rbtree.c Mon Aug 25 13:34:39 2014 +0400 @@ -22,8 +22,7 @@ static ngx_inline void ngx_rbtree_right_ void -ngx_rbtree_insert(ngx_thread_volatile ngx_rbtree_t *tree, - ngx_rbtree_node_t *node) +ngx_rbtree_insert(ngx_rbtree_t *tree, ngx_rbtree_node_t *node) { ngx_rbtree_node_t **root, *temp, *sentinel; @@ -155,8 +154,7 @@ ngx_rbtree_insert_timer_value(ngx_rbtree void -ngx_rbtree_delete(ngx_thread_volatile ngx_rbtree_t *tree, - ngx_rbtree_node_t *node) +ngx_rbtree_delete(ngx_rbtree_t *tree, ngx_rbtree_node_t *node) { ngx_uint_t red; ngx_rbtree_node_t **root, *sentinel, *subst, *temp, *w; diff -r fa4161fe8254 -r 1f513d7f1b45 src/core/ngx_rbtree.h --- a/src/core/ngx_rbtree.h Wed Oct 22 15:31:19 2014 +0200 +++ b/src/core/ngx_rbtree.h Mon Aug 25 13:34:39 2014 +0400 @@ -48,10 +48,8 @@ struct ngx_rbtree_s { (tree)->insert = i -void ngx_rbtree_insert(ngx_thread_volatile ngx_rbtree_t *tree, - ngx_rbtree_node_t *node); -void ngx_rbtree_delete(ngx_thread_volatile ngx_rbtree_t *tree, - ngx_rbtree_node_t *node); +void ngx_rbtree_insert(ngx_rbtree_t *tree, ngx_rbtree_node_t *node); +void ngx_rbtree_delete(ngx_rbtree_t *tree, ngx_rbtree_node_t *node); void ngx_rbtree_insert_value(ngx_rbtree_node_t *root, ngx_rbtree_node_t *node, ngx_rbtree_node_t *sentinel); void ngx_rbtree_insert_timer_value(ngx_rbtree_node_t *root, diff -r fa4161fe8254 -r 1f513d7f1b45 src/event/ngx_event_timer.c --- a/src/event/ngx_event_timer.c Wed Oct 22 15:31:19 2014 +0200 +++ b/src/event/ngx_event_timer.c Mon Aug 25 13:34:39 2014 +0400 @@ -10,13 +10,8 @@ #include -#if (NGX_THREADS) -ngx_mutex_t *ngx_event_timer_mutex; -#endif - - -ngx_thread_volatile ngx_rbtree_t ngx_event_timer_rbtree; -static ngx_rbtree_node_t ngx_event_timer_sentinel; +ngx_rbtree_t ngx_event_timer_rbtree; +static ngx_rbtree_node_t ngx_event_timer_sentinel; /* * the event timer rbtree may contain the duplicate keys, however, @@ -30,20 +25,6 @@ ngx_event_timer_init(ngx_log_t *log) ngx_rbtree_init(&ngx_event_timer_rbtree, &ngx_event_timer_sentinel, ngx_rbtree_insert_timer_value); -#if (NGX_THREADS) - - if (ngx_event_timer_mutex) { - ngx_event_timer_mutex->log = log; - return NGX_OK; - } - - ngx_event_timer_mutex = ngx_mutex_init(log, 0); - if (ngx_event_timer_mutex == NULL) { - return NGX_ERROR; - } - -#endif - return NGX_OK; } @@ -58,15 +39,11 @@ ngx_event_find_timer(void) return NGX_TIMER_INFINITE; } - ngx_mutex_lock(ngx_event_timer_mutex); - root = ngx_event_timer_rbtree.root; sentinel = ngx_event_timer_rbtree.sentinel; node = ngx_rbtree_min(root, sentinel); - ngx_mutex_unlock(ngx_event_timer_mutex); - timer = (ngx_msec_int_t) (node->key - ngx_current_msec); return (ngx_msec_t) (timer > 0 ? timer : 0); @@ -82,9 +59,6 @@ ngx_event_expire_timers(void) sentinel = ngx_event_timer_rbtree.sentinel; for ( ;; ) { - - ngx_mutex_lock(ngx_event_timer_mutex); - root = ngx_event_timer_rbtree.root; if (root == sentinel) { @@ -104,8 +78,6 @@ ngx_event_expire_timers(void) ngx_rbtree_delete(&ngx_event_timer_rbtree, &ev->timer); - ngx_mutex_unlock(ngx_event_timer_mutex); - #if (NGX_DEBUG) ev->timer.left = NULL; ev->timer.right = NULL; @@ -123,6 +95,4 @@ ngx_event_expire_timers(void) break; } - - ngx_mutex_unlock(ngx_event_timer_mutex); } diff -r fa4161fe8254 -r 1f513d7f1b45 src/event/ngx_event_timer.h --- a/src/event/ngx_event_timer.h Wed Oct 22 15:31:19 2014 +0200 +++ b/src/event/ngx_event_timer.h Mon Aug 25 13:34:39 2014 +0400 @@ -24,12 +24,7 @@ ngx_msec_t ngx_event_find_timer(void); void ngx_event_expire_timers(void); -#if (NGX_THREADS) -extern ngx_mutex_t *ngx_event_timer_mutex; -#endif - - -extern ngx_thread_volatile ngx_rbtree_t ngx_event_timer_rbtree; +extern ngx_rbtree_t ngx_event_timer_rbtree; static ngx_inline void @@ -39,12 +34,8 @@ ngx_event_del_timer(ngx_event_t *ev) "event timer del: %d: %M", ngx_event_ident(ev->data), ev->timer.key); - ngx_mutex_lock(ngx_event_timer_mutex); - ngx_rbtree_delete(&ngx_event_timer_rbtree, &ev->timer); - ngx_mutex_unlock(ngx_event_timer_mutex); - #if (NGX_DEBUG) ev->timer.left = NULL; ev->timer.right = NULL; @@ -89,12 +80,8 @@ ngx_event_add_timer(ngx_event_t *ev, ngx "event timer add: %d: %M:%M", ngx_event_ident(ev->data), timer, ev->timer.key); - ngx_mutex_lock(ngx_event_timer_mutex); - ngx_rbtree_insert(&ngx_event_timer_rbtree, &ev->timer); - ngx_mutex_unlock(ngx_event_timer_mutex); - ev->timer_set = 1; } diff -r fa4161fe8254 -r 1f513d7f1b45 src/os/unix/ngx_process_cycle.c --- a/src/os/unix/ngx_process_cycle.c Wed Oct 22 15:31:19 2014 +0200 +++ b/src/os/unix/ngx_process_cycle.c Mon Aug 25 13:34:39 2014 +0400 @@ -1213,7 +1213,6 @@ ngx_wakeup_worker_threads(ngx_cycle_t *c /* STUB */ ngx_done_events(cycle); - ngx_mutex_destroy(ngx_event_timer_mutex); return; } From vbart at nginx.com Thu Oct 30 22:24:42 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Thu, 30 Oct 2014 22:24:42 +0000 Subject: [nginx] Events: simplified cycle in ngx_event_expire_timers(). Message-ID: details: http://hg.nginx.org/nginx/rev/0f53e5fb7205 branches: changeset: 5895:0f53e5fb7205 user: Valentin Bartenev date: Mon Aug 25 13:37:06 2014 +0400 description: Events: simplified cycle in ngx_event_expire_timers(). diffstat: src/event/ngx_event_timer.c | 34 ++++++++++++++++------------------ 1 files changed, 16 insertions(+), 18 deletions(-) diffs (52 lines): diff -r 1f513d7f1b45 -r 0f53e5fb7205 src/event/ngx_event_timer.c --- a/src/event/ngx_event_timer.c Mon Aug 25 13:34:39 2014 +0400 +++ b/src/event/ngx_event_timer.c Mon Aug 25 13:37:06 2014 +0400 @@ -67,32 +67,30 @@ ngx_event_expire_timers(void) node = ngx_rbtree_min(root, sentinel); - /* node->key <= ngx_current_time */ + /* node->key > ngx_current_time */ - if ((ngx_msec_int_t) (node->key - ngx_current_msec) <= 0) { - ev = (ngx_event_t *) ((char *) node - offsetof(ngx_event_t, timer)); + if ((ngx_msec_int_t) (node->key - ngx_current_msec) > 0) { + return; + } - ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0, - "event timer del: %d: %M", - ngx_event_ident(ev->data), ev->timer.key); + ev = (ngx_event_t *) ((char *) node - offsetof(ngx_event_t, timer)); - ngx_rbtree_delete(&ngx_event_timer_rbtree, &ev->timer); + ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0, + "event timer del: %d: %M", + ngx_event_ident(ev->data), ev->timer.key); + + ngx_rbtree_delete(&ngx_event_timer_rbtree, &ev->timer); #if (NGX_DEBUG) - ev->timer.left = NULL; - ev->timer.right = NULL; - ev->timer.parent = NULL; + ev->timer.left = NULL; + ev->timer.right = NULL; + ev->timer.parent = NULL; #endif - ev->timer_set = 0; + ev->timer_set = 0; - ev->timedout = 1; + ev->timedout = 1; - ev->handler(ev); - - continue; - } - - break; + ev->handler(ev); } } From vbart at nginx.com Thu Oct 30 22:24:45 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Thu, 30 Oct 2014 22:24:45 +0000 Subject: [nginx] Events: introduced cancelable timers. Message-ID: details: http://hg.nginx.org/nginx/rev/3efdd7788bb0 branches: changeset: 5896:3efdd7788bb0 user: Valentin Bartenev date: Wed Aug 13 22:45:04 2014 +0400 description: Events: introduced cancelable timers. diffstat: src/event/ngx_event.h | 2 + src/event/ngx_event_timer.c | 42 ++++++++++++++++++++++++++++++++++++++++ src/event/ngx_event_timer.h | 1 + src/os/unix/ngx_process_cycle.c | 2 + src/os/win32/ngx_process_cycle.c | 2 + 5 files changed, 49 insertions(+), 0 deletions(-) diffs (96 lines): diff -r 0f53e5fb7205 -r 3efdd7788bb0 src/event/ngx_event.h --- a/src/event/ngx_event.h Mon Aug 25 13:37:06 2014 +0400 +++ b/src/event/ngx_event.h Wed Aug 13 22:45:04 2014 +0400 @@ -136,6 +136,8 @@ struct ngx_event_s { unsigned channel:1; unsigned resolver:1; + unsigned cancelable:1; + #if 0 diff -r 0f53e5fb7205 -r 3efdd7788bb0 src/event/ngx_event_timer.c --- a/src/event/ngx_event_timer.c Mon Aug 25 13:37:06 2014 +0400 +++ b/src/event/ngx_event_timer.c Wed Aug 13 22:45:04 2014 +0400 @@ -94,3 +94,45 @@ ngx_event_expire_timers(void) ev->handler(ev); } } + + +void +ngx_event_cancel_timers(void) +{ + ngx_event_t *ev; + ngx_rbtree_node_t *node, *root, *sentinel; + + sentinel = ngx_event_timer_rbtree.sentinel; + + for ( ;; ) { + root = ngx_event_timer_rbtree.root; + + if (root == sentinel) { + return; + } + + node = ngx_rbtree_min(root, sentinel); + + ev = (ngx_event_t *) ((char *) node - offsetof(ngx_event_t, timer)); + + if (!ev->cancelable) { + return; + } + + ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0, + "event timer cancel: %d: %M", + ngx_event_ident(ev->data), ev->timer.key); + + ngx_rbtree_delete(&ngx_event_timer_rbtree, &ev->timer); + +#if (NGX_DEBUG) + ev->timer.left = NULL; + ev->timer.right = NULL; + ev->timer.parent = NULL; +#endif + + ev->timer_set = 0; + + ev->handler(ev); + } +} diff -r 0f53e5fb7205 -r 3efdd7788bb0 src/event/ngx_event_timer.h --- a/src/event/ngx_event_timer.h Mon Aug 25 13:37:06 2014 +0400 +++ b/src/event/ngx_event_timer.h Wed Aug 13 22:45:04 2014 +0400 @@ -22,6 +22,7 @@ ngx_int_t ngx_event_timer_init(ngx_log_t *log); ngx_msec_t ngx_event_find_timer(void); void ngx_event_expire_timers(void); +void ngx_event_cancel_timers(void); extern ngx_rbtree_t ngx_event_timer_rbtree; diff -r 0f53e5fb7205 -r 3efdd7788bb0 src/os/unix/ngx_process_cycle.c --- a/src/os/unix/ngx_process_cycle.c Mon Aug 25 13:37:06 2014 +0400 +++ b/src/os/unix/ngx_process_cycle.c Wed Aug 13 22:45:04 2014 +0400 @@ -809,6 +809,8 @@ ngx_worker_process_cycle(ngx_cycle_t *cy } } + ngx_event_cancel_timers(); + if (ngx_event_timer_rbtree.root == ngx_event_timer_rbtree.sentinel) { ngx_log_error(NGX_LOG_NOTICE, cycle->log, 0, "exiting"); diff -r 0f53e5fb7205 -r 3efdd7788bb0 src/os/win32/ngx_process_cycle.c --- a/src/os/win32/ngx_process_cycle.c Mon Aug 25 13:37:06 2014 +0400 +++ b/src/os/win32/ngx_process_cycle.c Wed Aug 13 22:45:04 2014 +0400 @@ -821,6 +821,8 @@ ngx_worker_thread(void *data) } } + ngx_event_cancel_timers(); + if (ngx_event_timer_rbtree.root == ngx_event_timer_rbtree.sentinel) { From vbart at nginx.com Thu Oct 30 22:24:47 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Thu, 30 Oct 2014 22:24:47 +0000 Subject: [nginx] Access log: cancel the flush timer on graceful shutdown. Message-ID: details: http://hg.nginx.org/nginx/rev/dff86e2246a5 branches: changeset: 5897:dff86e2246a5 user: Valentin Bartenev date: Mon Aug 25 13:41:31 2014 +0400 description: Access log: cancel the flush timer on graceful shutdown. Previously, it could prevent a worker process from exiting for up to the configured flush timeout. diffstat: src/http/modules/ngx_http_log_module.c | 16 +++++++++++++++- 1 files changed, 15 insertions(+), 1 deletions(-) diffs (36 lines): diff -r 3efdd7788bb0 -r dff86e2246a5 src/http/modules/ngx_http_log_module.c --- a/src/http/modules/ngx_http_log_module.c Wed Aug 13 22:45:04 2014 +0400 +++ b/src/http/modules/ngx_http_log_module.c Mon Aug 25 13:41:31 2014 +0400 @@ -744,10 +744,23 @@ ngx_http_log_flush(ngx_open_file_t *file static void ngx_http_log_flush_handler(ngx_event_t *ev) { + ngx_open_file_t *file; + ngx_http_log_buf_t *buffer; + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ev->log, 0, "http log buffer flush handler"); - ngx_http_log_flush(ev->data, ev->log); + if (ev->timedout) { + ngx_http_log_flush(ev->data, ev->log); + return; + } + + /* cancel the flush timer for graceful shutdown */ + + file = ev->data; + buffer = file->data; + + buffer->event = NULL; } @@ -1411,6 +1424,7 @@ process_formats: buffer->event->data = log->file; buffer->event->handler = ngx_http_log_flush_handler; buffer->event->log = &cf->cycle->new_log; + buffer->event->cancelable = 1; buffer->flush = flush; } From piotr at cloudflare.com Thu Oct 30 23:33:09 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Thu, 30 Oct 2014 16:33:09 -0700 Subject: [PATCH] SSL: don't enable SSLv3 by default In-Reply-To: <20141030134735.GY45418@mdounin.ru> References: <20141030134735.GY45418@mdounin.ru> Message-ID: Hey Maxim, > - SSLv3 is still important from compatibility point of view, there > are various clients which doesn't support (or enable by default) > anything better; But is it, really? All major browsers (Chrome [1], Firefox [2], IE [3], Opera [4]) either already disabled SSLv3 or are about to do it. Huge chunk of websites (>42% of Alexa's top 10.000 [5]) requires at least TLSv1.0, including major properties like Facebook, Twitter [6], Wikipedia [7] and websites that are using one of the popular CDNs (CloudFlare [8], Akamai [9], MaxCDN [10], Fastly [11]). OpenBSD and LibreSSL disabled SSLv3 by default [12]. Furthermore, when we disabled SSLv3 across our network [8] and gave website owners the ability to opt-in back to it... less than 0.001% did re-enable it. Hopefully that list is long enough to convince you that SSLv3 is not really important... Definitely not important enough to be enabled by default, because that's what the commit changes, people can still enable SSLv3 in the conf if they really need to. [1] https://groups.google.com/a/chromium.org/forum/#!topic/security-dev/Vnhy9aKM_l4 [2] https://blog.mozilla.org/security/2014/10/14/the-poodle-attack-and-the-end-of-ssl-3-0/ [3] http://azure.microsoft.com/blog/2014/10/29/protecting-against-the-ssl-3-0-vulnerability/ [4] http://blogs.opera.com/security/2014/10/security-changes-opera-25-poodle-attacks/ [5] https://8ack.de/ssl/ [6] https://twitter.com/twittersecurity/status/522190947782643712 [7] https://blog.wikimedia.org/2014/10/17/protecting-users-against-poodle-by-removing-ssl-3-0-support/ [8] https://blog.cloudflare.com/sslv3-support-disabled-by-default-due-to-vulnerability/ [9] https://blogs.akamai.com/2014/10/poodle-faq-what-akamai-customers-need-to-know.html [10] https://www.maxcdn.com/blog/delivery-sslv3-disabled/ [11] http://www.fastly.com/blog/fastly-update-POODLE/ [12] http://marc.info/?l=openbsd-cvs&m=141339479327258&w=2 Best regards, Piotr Sikora From mdounin at mdounin.ru Fri Oct 31 04:24:26 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 31 Oct 2014 07:24:26 +0300 Subject: [PATCH] SSL: don't enable SSLv3 by default In-Reply-To: References: <20141030134735.GY45418@mdounin.ru> Message-ID: <20141031042426.GA17248@mdounin.ru> Hello! On Thu, Oct 30, 2014 at 04:33:09PM -0700, Piotr Sikora wrote: > Hey Maxim, > > > - SSLv3 is still important from compatibility point of view, there > > are various clients which doesn't support (or enable by default) > > anything better; > > But is it, really? > > All major browsers (Chrome [1], Firefox [2], IE [3], Opera [4]) either > already disabled SSLv3 or are about to do it. AFAIK, the only browser already disabled SSLv3 for now is Opera 12, an obsolete Presto-based branch. The links provided suggests the same. (This is mostly unrelated though, as from nginx point of view it's the number of clients without anything better than SSLv3 is important.) > Huge chunk of websites (>42% of Alexa's top 10.000 [5]) requires at > least TLSv1.0, including major properties like Facebook, Twitter [6], > Wikipedia [7] and websites that are using one of the popular CDNs > (CloudFlare [8], Akamai [9], MaxCDN [10], Fastly [11]). The 42% here means, on the other hand, that 58% are still available via SSLv3, including Google, Youtube, Amazon, Microsoft and so on. While 42% is a good number, I'm pretty sure the question is different. > OpenBSD and LibreSSL disabled SSLv3 by default [12]. > > Furthermore, when we disabled SSLv3 across our network [8] and gave > website owners the ability to opt-in back to it... less than 0.001% > did re-enable it. And the comments there suggests people have problems with at least libcurl. On the other hand, I'm pretty sure that php scripts using libcurl with SSLv3 aren't vulnerable to POODLE. > Hopefully that list is long enough to convince you that SSLv3 is not > really important... Definitely not important enough to be enabled by > default, because that's what the commit changes, people can still > enable SSLv3 in the conf if they really need to. As previously said, this was alrady discussed excessively and we decided to preserve the default for now. We'll likely reconsider the change later. -- Maxim Dounin http://nginx.org/ From andrew at nginx.com Fri Oct 31 13:34:06 2014 From: andrew at nginx.com (Andrew Alexeev) Date: Fri, 31 Oct 2014 16:34:06 +0300 Subject: [PATCH] SSL: don't enable SSLv3 by default In-Reply-To: <20141031042426.GA17248@mdounin.ru> References: <20141030134735.GY45418@mdounin.ru> <20141031042426.GA17248@mdounin.ru> Message-ID: <4D40287A-4022-46C9-8D9B-D4B196DC5963@nginx.com> On Oct 31, 2014, at 7:24 AM, Maxim Dounin wrote: > Hello! > > On Thu, Oct 30, 2014 at 04:33:09PM -0700, Piotr Sikora wrote: > >> Hey Maxim, >> >>> - SSLv3 is still important from compatibility point of view, there >>> are various clients which doesn't support (or enable by default) >>> anything better; >> >> But is it, really? >> >> All major browsers (Chrome [1], Firefox [2], IE [3], Opera [4]) either >> already disabled SSLv3 or are about to do it. > > AFAIK, the only browser already disabled SSLv3 for now is Opera > 12, an obsolete Presto-based branch. The links provided suggests > the same. > > (This is mostly unrelated though, as from nginx point of view it's > the number of clients without anything better than SSLv3 is > important.) > >> Huge chunk of websites (>42% of Alexa's top 10.000 [5]) requires at >> least TLSv1.0, including major properties like Facebook, Twitter [6], >> Wikipedia [7] and websites that are using one of the popular CDNs >> (CloudFlare [8], Akamai [9], MaxCDN [10], Fastly [11]). > > The 42% here means, on the other hand, that 58% are still > available via SSLv3, including Google, Youtube, Amazon, Microsoft > and so on. While 42% is a good number, I'm pretty sure the > question is different. As a minor comment, some interesting stats here http://news.netcraft.com/archives/2014/10/15/googles-poodle-affects-oodles.html >> OpenBSD and LibreSSL disabled SSLv3 by default [12]. >> >> Furthermore, when we disabled SSLv3 across our network [8] and gave >> website owners the ability to opt-in back to it... less than 0.001% >> did re-enable it. > > And the comments there suggests people have problems with at least > libcurl. On the other hand, I'm pretty sure that php scripts > using libcurl with SSLv3 aren't vulnerable to POODLE. > >> Hopefully that list is long enough to convince you that SSLv3 is not >> really important... Definitely not important enough to be enabled by >> default, because that's what the commit changes, people can still >> enable SSLv3 in the conf if they really need to. > > As previously said, this was alrady discussed excessively and > we decided to preserve the default for now. We'll likely > reconsider the change later. > > -- > Maxim Dounin > http://nginx.org/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel >