From vbart at nginx.com Mon Mar 3 13:19:03 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Mon, 03 Mar 2014 13:19:03 +0000 Subject: [nginx] Gzip static: fixed NGX_CONF_FLAG misuse. Message-ID: details: http://hg.nginx.org/nginx/rev/ceecde39853f branches: changeset: 5587:ceecde39853f user: Valentin Bartenev date: Mon Mar 03 17:17:25 2014 +0400 description: Gzip static: fixed NGX_CONF_FLAG misuse. diffstat: src/http/modules/ngx_http_gzip_static_module.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r db6d07dbef3c -r ceecde39853f src/http/modules/ngx_http_gzip_static_module.c --- a/src/http/modules/ngx_http_gzip_static_module.c Fri Feb 28 20:17:01 2014 +0400 +++ b/src/http/modules/ngx_http_gzip_static_module.c Mon Mar 03 17:17:25 2014 +0400 @@ -38,7 +38,7 @@ static ngx_conf_enum_t ngx_http_gzip_st static ngx_command_t ngx_http_gzip_static_commands[] = { { ngx_string("gzip_static"), - NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, ngx_conf_set_enum_slot, NGX_HTTP_LOC_CONF_OFFSET, offsetof(ngx_http_gzip_static_conf_t, enable), From mbreid at thepei.com Mon Mar 3 17:11:22 2014 From: mbreid at thepei.com (Reid, Mike) Date: Mon, 3 Mar 2014 17:11:22 +0000 Subject: [alert] could not add new SSL session to the session cache while SSL handshaking Message-ID: I am experiencing the following in my error logs after a recent upgrade to NGiNX 1.5.10 (from 1.5.8), and also applying SSL / TLS updates as described on istlsfastyet.com [alert] 3319#0: *301399 could not add new SSL session to the session cache while SSL handshaking Any ideas on why these alerts would now be showing up? I am not sure how to address, or whether there should be cause for concern? NGiNX 1.5.10 w/ SPDY 3.1 # Previously 1.5.8, now including --with-http_spdy_module and using openssl-1.0.1f (previously openssl-1.0.1e without http_spdy_module) ssl_session_cache shared:SSL:10m; # No change ssl_buffer_size 1400; # New ssl_session_timeout 24h; # Previously 10m ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2; # No change If there's anything else I can provide to assist troubleshooting, please let me know. Thanks, - Mike -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon Mar 3 17:45:18 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 3 Mar 2014 21:45:18 +0400 Subject: [alert] could not add new SSL session to the session cache while SSL handshaking In-Reply-To: References: Message-ID: <20140303174518.GO34696@mdounin.ru> Hello! On Mon, Mar 03, 2014 at 05:11:22PM +0000, Reid, Mike wrote: > I am experiencing the following in my error logs after a recent > upgrade to NGiNX 1.5.10 (from 1.5.8), and also applying SSL / > TLS updates as described on istlsfastyet.com > > [alert] 3319#0: *301399 could not add new SSL session to the > session cache while SSL handshaking > > Any ideas on why these alerts would now be showing up? I am not > sure how to address, or whether there should be cause for > concern? > > NGiNX 1.5.10 w/ SPDY 3.1 # Previously 1.5.8, now including > --with-http_spdy_module and using openssl-1.0.1f (previously > openssl-1.0.1e without http_spdy_module) > ssl_session_cache shared:SSL:10m; # No change > ssl_buffer_size 1400; # New > ssl_session_timeout 24h; # Previously 10m > ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2; # No change You've changed SSL session timeout from 10 minutes to 24 hours, and this basically means that sessions will use 144 times more space in session cache. On the other hand, cache size wasn't changed - so you've run out of space in the cache configured. If there is no space in a cache nginx will try to drop one non-expired session from the cache, but it may not be enough to store a new session (as different sessions may occupy different space), resulting in alerts you've quoted. Note well that configuring ssl_buffer_size to 1400 isn't a good idea unless you are doing so for your own performance testing. See previous discussions for details. Overral, this doesn't looks relevant to nginx-devel at . Please use nginx@ for futher questions. -- Maxim Dounin http://nginx.org/ From mbreid at thepei.com Mon Mar 3 19:04:33 2014 From: mbreid at thepei.com (Reid, Mike) Date: Mon, 3 Mar 2014 19:04:33 +0000 Subject: [alert] could not add new SSL session to the session cache while SSL handshaking In-Reply-To: <20140303174518.GO34696@mdounin.ru> References: <20140303174518.GO34696@mdounin.ru> Message-ID: Thanks, Maxim. I appreciate it. Will do. On 3/3/14, 10:45 AM, "Maxim Dounin" wrote: >Hello! > >On Mon, Mar 03, 2014 at 05:11:22PM +0000, Reid, Mike wrote: > >> I am experiencing the following in my error logs after a recent >> upgrade to NGiNX 1.5.10 (from 1.5.8), and also applying SSL / >> TLS updates as described on istlsfastyet.com >> >> [alert] 3319#0: *301399 could not add new SSL session to the >> session cache while SSL handshaking >> >> Any ideas on why these alerts would now be showing up? I am not >> sure how to address, or whether there should be cause for >> concern? >> >> NGiNX 1.5.10 w/ SPDY 3.1 # Previously 1.5.8, now including >> --with-http_spdy_module and using openssl-1.0.1f (previously >> openssl-1.0.1e without http_spdy_module) >> ssl_session_cache shared:SSL:10m; # No change >> ssl_buffer_size 1400; # New >> ssl_session_timeout 24h; # Previously 10m >> ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2; # No change > >You've changed SSL session timeout from 10 minutes to 24 hours, >and this basically means that sessions will use 144 times more >space in session cache. On the other hand, cache size wasn't >changed - so you've run out of space in the cache configured. If >there is no space in a cache nginx will try to drop one >non-expired session from the cache, but it may not be enough to >store a new session (as different sessions may occupy different >space), resulting in alerts you've quoted. > >Note well that configuring ssl_buffer_size to 1400 isn't a good >idea unless you are doing so for your own performance testing. >See previous discussions for details. > >Overral, this doesn't looks relevant to nginx-devel at . Please use >nginx@ for futher questions. > >-- >Maxim Dounin >http://nginx.org/ > >_______________________________________________ >nginx-devel mailing list >nginx-devel at nginx.org >http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Tue Mar 4 15:18:25 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 04 Mar 2014 15:18:25 +0000 Subject: [nginx] Request body: avoid potential overflow. Message-ID: details: http://hg.nginx.org/nginx/rev/6fc3921025f0 branches: changeset: 5588:6fc3921025f0 user: Maxim Dounin date: Mon Mar 03 17:39:53 2014 +0400 description: Request body: avoid potential overflow. diffstat: src/http/ngx_http_request_body.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diffs (21 lines): diff --git a/src/http/ngx_http_request_body.c b/src/http/ngx_http_request_body.c --- a/src/http/ngx_http_request_body.c +++ b/src/http/ngx_http_request_body.c @@ -953,13 +953,13 @@ ngx_http_request_body_chunked_filter(ngx if (clcf->client_max_body_size && clcf->client_max_body_size - < r->headers_in.content_length_n + rb->chunked->size) + - r->headers_in.content_length_n < rb->chunked->size) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "client intended to send too large chunked " - "body: %O bytes", - r->headers_in.content_length_n - + rb->chunked->size); + "body: %O+%O bytes", + r->headers_in.content_length_n, + rb->chunked->size); r->lingering_close = 1; From mdounin at mdounin.ru Tue Mar 4 15:18:26 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 04 Mar 2014 15:18:26 +0000 Subject: [nginx] SPDY: constant number of preallocated structures for hea... Message-ID: details: http://hg.nginx.org/nginx/rev/a9116d9b63f9 branches: changeset: 5589:a9116d9b63f9 user: Valentin Bartenev date: Mon Mar 03 19:24:54 2014 +0400 description: SPDY: constant number of preallocated structures for headers. diffstat: src/http/ngx_http_spdy.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/src/http/ngx_http_spdy.c b/src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c +++ b/src/http/ngx_http_spdy.c @@ -1038,7 +1038,7 @@ ngx_http_spdy_state_headers(ngx_http_spd "spdy HEADERS block consists of %ui entries", sc->entries); - if (ngx_list_init(&r->headers_in.headers, r->pool, sc->entries + 3, + if (ngx_list_init(&r->headers_in.headers, r->pool, 20, sizeof(ngx_table_elt_t)) != NGX_OK) { From mdounin at mdounin.ru Tue Mar 4 15:18:28 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 04 Mar 2014 15:18:28 +0000 Subject: [nginx] SPDY: fixed potential integer overflow while parsing hea... Message-ID: details: http://hg.nginx.org/nginx/rev/6808ea2d69e4 branches: changeset: 5590:6808ea2d69e4 user: Valentin Bartenev date: Mon Mar 03 19:24:55 2014 +0400 description: SPDY: fixed potential integer overflow while parsing headers. Previously r->header_size was used to store length for a part of value that represents an individual already parsed HTTP header, while r->header_end pointed to the end of the whole value. Instead of storing length of a following name or value as pointer to a potential end address (r->header_name_end and r->header_end) that might be overflowed, now r->lowercase_index counter is used to store remaining length of a following unparsed field. It also fixes incorrect $body_bytes_sent value if a request is closed while parsing of the request header. Since r->header_size is intended for counting header size, thus abusing it for header parsing purpose was certainly a bad idea. diffstat: src/http/ngx_http_spdy.c | 62 ++++++++++++++++++++--------------------------- 1 files changed, 26 insertions(+), 36 deletions(-) diffs (196 lines): diff --git a/src/http/ngx_http_spdy.c b/src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c +++ b/src/http/ngx_http_spdy.c @@ -2325,7 +2325,7 @@ static ngx_int_t ngx_http_spdy_parse_header(ngx_http_request_t *r) { u_char *p, *end, ch; - ngx_uint_t len, hash; + ngx_uint_t hash; ngx_http_core_srv_conf_t *cscf; enum { @@ -2348,9 +2348,9 @@ ngx_http_spdy_parse_header(ngx_http_requ return NGX_AGAIN; } - len = ngx_spdy_frame_parse_uint32(p); - - if (!len) { + r->lowcase_index = ngx_spdy_frame_parse_uint32(p); + + if (r->lowcase_index == 0) { return NGX_HTTP_PARSE_INVALID_HEADER; } @@ -2359,8 +2359,6 @@ ngx_http_spdy_parse_header(ngx_http_requ p += NGX_SPDY_NV_NLEN_SIZE; - r->header_name_end = p + len; - r->lowcase_index = len; r->invalid_header = 0; state = sw_name; @@ -2369,16 +2367,16 @@ ngx_http_spdy_parse_header(ngx_http_requ case sw_name: - if (r->header_name_end > end) { + if ((ngx_uint_t) (end - p) < r->lowcase_index) { break; } cscf = ngx_http_get_module_srv_conf(r, ngx_http_core_module); r->header_name_start = p; + r->header_name_end = p + r->lowcase_index; if (p[0] == ':') { - r->lowcase_index--; p++; } @@ -2425,29 +2423,26 @@ ngx_http_spdy_parse_header(ngx_http_requ break; } - len = ngx_spdy_frame_parse_uint32(p); + r->lowcase_index = ngx_spdy_frame_parse_uint32(p); /* null-terminate header name */ *p = '\0'; p += NGX_SPDY_NV_VLEN_SIZE; - r->header_end = p + len; - state = sw_value; /* fall through */ case sw_value: - if (r->header_end > end) { + if ((ngx_uint_t) (end - p) < r->lowcase_index) { break; } r->header_start = p; - for ( /* void */ ; p != r->header_end; p++) { - + while (r->lowcase_index--) { ch = *p; if (ch == '\0') { @@ -2456,7 +2451,7 @@ ngx_http_spdy_parse_header(ngx_http_requ return NGX_ERROR; } - r->header_size = p - r->header_start; + r->header_end = p; r->header_in->pos = p + 1; return NGX_OK; @@ -2465,9 +2460,11 @@ ngx_http_spdy_parse_header(ngx_http_requ if (ch == CR || ch == LF) { return NGX_HTTP_PARSE_INVALID_HEADER; } + + p++; } - r->header_size = p - r->header_start; + r->header_end = p; r->header_in->pos = p; r->state = 0; @@ -2526,13 +2523,6 @@ ngx_http_spdy_alloc_large_header_buffer( buf->last = ngx_cpymem(new, old, rest); } - if (r->header_name_end > old) { - r->header_name_end = new + (r->header_name_end - old); - - } else if (r->header_end > old) { - r->header_end = new + (r->header_end - old); - } - r->header_in = buf; stream->header_buffers++; @@ -2563,14 +2553,14 @@ ngx_http_spdy_handle_request_header(ngx_ } if (r->header_name_start[0] == ':') { + r->header_name_start++; + for (i = 0; i < NGX_SPDY_REQUEST_HEADERS; i++) { sh = &ngx_http_spdy_request_headers[i]; if (sh->hash != r->header_hash - || sh->len != r->lowcase_index - || ngx_strncmp(sh->header, &r->header_name_start[1], - r->lowcase_index) - != 0) + || sh->len != r->header_name_end - r->header_name_start + || ngx_strncmp(sh->header, r->header_name_start, sh->len) != 0) { continue; } @@ -2590,10 +2580,10 @@ ngx_http_spdy_handle_request_header(ngx_ h->hash = r->header_hash; - h->key.len = r->lowcase_index; + h->key.len = r->header_name_end - r->header_name_start; h->key.data = r->header_name_start; - h->value.len = r->header_size; + h->value.len = r->header_end - r->header_start; h->value.data = r->header_start; h->lowcase_key = h->key.data; @@ -2653,7 +2643,7 @@ ngx_http_spdy_parse_method(ngx_http_requ return NGX_HTTP_PARSE_INVALID_HEADER; } - len = r->header_size; + len = r->header_end - r->header_start; r->method_name.len = len; r->method_name.data = r->header_start; @@ -2733,10 +2723,10 @@ ngx_http_spdy_parse_host(ngx_http_reques h->hash = r->header_hash; - h->key.len = r->lowcase_index; - h->key.data = &r->header_name_start[1]; - - h->value.len = r->header_size; + h->key.len = r->header_name_end - r->header_name_start; + h->key.data = r->header_name_start; + + h->value.len = r->header_end - r->header_start; h->value.data = r->header_start; h->lowcase_key = h->key.data; @@ -2778,7 +2768,7 @@ ngx_http_spdy_parse_version(ngx_http_req p = r->header_start; - if (r->header_size < 8 || !(ngx_str5cmp(p, 'H', 'T', 'T', 'P', '/'))) { + if (r->header_end - p < 8 || !(ngx_str5cmp(p, 'H', 'T', 'T', 'P', '/'))) { return NGX_HTTP_PARSE_INVALID_REQUEST; } @@ -2828,7 +2818,7 @@ ngx_http_spdy_parse_version(ngx_http_req r->http_minor = r->http_minor * 10 + ch - '0'; } - r->http_protocol.len = r->header_size; + r->http_protocol.len = r->header_end - r->header_start; r->http_protocol.data = r->header_start; r->http_version = r->http_major * 1000 + r->http_minor; From mdounin at mdounin.ru Tue Mar 4 15:18:29 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 04 Mar 2014 15:18:29 +0000 Subject: [nginx] nginx-1.5.11-RELEASE Message-ID: details: http://hg.nginx.org/nginx/rev/f995a10d4c7e branches: changeset: 5591:f995a10d4c7e user: Maxim Dounin date: Tue Mar 04 15:39:23 2014 +0400 description: nginx-1.5.11-RELEASE diffstat: docs/xml/nginx/changes.xml | 110 +++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 110 insertions(+), 0 deletions(-) diffs (120 lines): diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -5,6 +5,116 @@ + + + + +??? ????????? ?????????? ?????????? ??????? ??????? ngx_http_spdy_module +?? 32-?????? ?????????? ????? ???????????? ?????? ???????? ????????, +??? ???????????? ????? ????????? ? ?????????? ????????????? ???? +(CVE-2014-0088); +?????? ????????? ? 1.5.10.
+??????? Lucas Molas ?? Programa STIC, Fundaci?n Dr. Manuel +Sadosky, Buenos Aires, Argentina. +
+ +memory corruption might occur in a worker process on 32-bit platforms +while handling a specially crafted request by ngx_http_spdy_module, +potentially resulting in arbitrary code execution (CVE-2014-0088); +the bug had appeared in 1.5.10.
+Thanks to Lucas Molas, researcher at Programa STIC, Fundaci?n Dr. Manuel +Sadosky, Buenos Aires, Argentina. +
+
+ + + +?????????? $ssl_session_reused. + + +the $ssl_session_reused variable. + + + + + +????????? client_max_body_size ????? ?? ???????? +??? ?????? ???? ??????? ? ?????????????? chunked transfer encoding; +?????? ????????? ? 1.3.9.
+??????? Lucas Molas. +
+ +the "client_max_body_size" directive might not work +when reading a request body using chunked transfer encoding; +the bug had appeared in 1.3.9.
+Thanks to Lucas Molas. +
+
+ + + +??? ????????????? WebSocket-?????????? +? ??????? ???????? ??? ????????? segmentation fault. + + +a segmentation fault might occur in a worker process +when proxying WebSocket connections. + + + + + +? ??????? ???????? ??? ????????? segmentation fault, +???? ????????????? ?????? ngx_http_spdy_module ?? 32-?????? ??????????; +?????? ????????? ? 1.5.10. + + +a segmentation fault might occur in a worker process +if the ngx_http_spdy_module was used on 32-bit platforms; +the bug had appeared in 1.5.10. + + + + + +???????? ?????????? $upstream_status ????? ???? ????????, +???? ?????????????? ????????? proxy_cache_use_stale +??? proxy_cache_revalidate.
+??????? Piotr Sikora. +
+ +the $upstream_status variable might contain wrong data +if the "proxy_cache_use_stale" or "proxy_cache_revalidate" directives +were used.
+Thanks to Piotr Sikora. +
+
+ + + +? ??????? ???????? ??? ????????? segmentation fault, +???? ?????? ? ????? 400 ? ??????? ????????? error_page +???????????????? ? ??????????? location. + + +a segmentation fault might occur in a worker process +if errors with code 400 were redirected to a named location +using the "error_page" directive. + + + + + +nginx/Windows ?? ????????? ? Visual Studio 2013. + + +nginx/Windows could not be built with Visual Studio 2013. + + + +
+ + From mdounin at mdounin.ru Tue Mar 4 15:18:30 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 04 Mar 2014 15:18:30 +0000 Subject: [nginx] release-1.5.11 tag Message-ID: details: http://hg.nginx.org/nginx/rev/f6d457701d9e branches: changeset: 5592:f6d457701d9e user: Maxim Dounin date: Tue Mar 04 15:39:23 2014 +0400 description: release-1.5.11 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -366,3 +366,4 @@ 9ba2542d75bf62a3972278c63561fc2ef5ec573a eaa76f24975948b0ce8be01838d949122d44ed67 release-1.5.8 5a1759f33b7fa6270e1617c08d7e655b7b127f26 release-1.5.9 b798fc020e3a84ef68e6c9f47865a319c826d33c release-1.5.10 +f995a10d4c7e9a817157a6ce7b753297ad32897e release-1.5.11 From mdounin at mdounin.ru Tue Mar 4 15:18:32 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 04 Mar 2014 15:18:32 +0000 Subject: [nginx] Version bump. Message-ID: details: http://hg.nginx.org/nginx/rev/7a7c6104b474 branches: stable-1.4 changeset: 5593:7a7c6104b474 user: Maxim Dounin date: Mon Mar 03 17:22:28 2014 +0400 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff --git a/src/core/nginx.h b/src/core/nginx.h --- a/src/core/nginx.h +++ b/src/core/nginx.h @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1004005 -#define NGINX_VERSION "1.4.5" +#define nginx_version 1004006 +#define NGINX_VERSION "1.4.6" #define NGINX_VER "nginx/" NGINX_VERSION #define NGINX_VAR "NGINX" From mdounin at mdounin.ru Tue Mar 4 15:18:33 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 04 Mar 2014 15:18:33 +0000 Subject: [nginx] Upstream: ngx_post_event() instead of upgraded call (tic... Message-ID: details: http://hg.nginx.org/nginx/rev/c8a14fbd3ce9 branches: stable-1.4 changeset: 5594:c8a14fbd3ce9 user: Maxim Dounin date: Tue Feb 18 17:30:40 2014 +0400 description: Upstream: ngx_post_event() instead of upgraded call (ticket #503). If a request is finalized in the first call to the ngx_http_upstream_process_upgraded() function, e.g., because upstream server closed the connection for some reason, in the second call the u->peer.connection pointer will be null, resulting in segmentation fault. Fix is to avoid second direct call, and post event instead. This ensures that ngx_http_upstream_process_upgraded() won't be called again if a request is finalized. diffstat: src/http/ngx_http_upstream.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (13 lines): diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -2456,7 +2456,9 @@ ngx_http_upstream_upgrade(ngx_http_reque if (u->peer.connection->read->ready || u->buffer.pos != u->buffer.last) { + ngx_post_event(c->read, &ngx_posted_events); ngx_http_upstream_process_upgraded(r, 1, 1); + return; } ngx_http_upstream_process_upgraded(r, 0, 1); From mdounin at mdounin.ru Tue Mar 4 15:18:35 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 04 Mar 2014 15:18:35 +0000 Subject: [nginx] Request body: avoid potential overflow. Message-ID: details: http://hg.nginx.org/nginx/rev/8a2261c59f3b branches: stable-1.4 changeset: 5595:8a2261c59f3b user: Maxim Dounin date: Mon Mar 03 17:39:53 2014 +0400 description: Request body: avoid potential overflow. diffstat: src/http/ngx_http_request_body.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diffs (21 lines): diff --git a/src/http/ngx_http_request_body.c b/src/http/ngx_http_request_body.c --- a/src/http/ngx_http_request_body.c +++ b/src/http/ngx_http_request_body.c @@ -953,13 +953,13 @@ ngx_http_request_body_chunked_filter(ngx if (clcf->client_max_body_size && clcf->client_max_body_size - < r->headers_in.content_length_n + rb->chunked->size) + - r->headers_in.content_length_n < rb->chunked->size) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "client intended to send too large chunked " - "body: %O bytes", - r->headers_in.content_length_n - + rb->chunked->size); + "body: %O+%O bytes", + r->headers_in.content_length_n, + rb->chunked->size); r->lingering_close = 1; From mdounin at mdounin.ru Tue Mar 4 15:18:36 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 04 Mar 2014 15:18:36 +0000 Subject: [nginx] nginx-1.4.6-RELEASE Message-ID: details: http://hg.nginx.org/nginx/rev/e201159f5d4e branches: stable-1.4 changeset: 5596:e201159f5d4e user: Maxim Dounin date: Tue Mar 04 15:46:44 2014 +0400 description: nginx-1.4.6-RELEASE diffstat: docs/xml/nginx/changes.xml | 31 +++++++++++++++++++++++++++++++ 1 files changed, 31 insertions(+), 0 deletions(-) diffs (41 lines): diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -5,6 +5,37 @@ + + + + +????????? client_max_body_size ????? ?? ???????? +??? ?????? ???? ??????? ? ?????????????? chunked transfer encoding; +?????? ????????? ? 1.3.9.
+??????? Lucas Molas. +
+ +the "client_max_body_size" directive might not work +when reading a request body using chunked transfer encoding; +the bug had appeared in 1.3.9.
+Thanks to Lucas Molas. +
+
+ + + +??? ????????????? WebSocket-?????????? +? ??????? ???????? ??? ????????? segmentation fault. + + +a segmentation fault might occur in a worker process +when proxying WebSocket connections. + + + +
+ + From mdounin at mdounin.ru Tue Mar 4 15:18:38 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 04 Mar 2014 15:18:38 +0000 Subject: [nginx] release-1.4.6 tag Message-ID: details: http://hg.nginx.org/nginx/rev/4ded8129deec branches: stable-1.4 changeset: 5597:4ded8129deec user: Maxim Dounin date: Tue Mar 04 15:46:45 2014 +0400 description: release-1.4.6 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -360,3 +360,4 @@ 50f065641b4c52ced41fae1ce216c73aaf112306 69ffaca7795531e19f3827940cc28dca0b50d0b8 release-1.4.3 7e9543faf5f0a443ba605d9d483cf4721fae30a5 release-1.4.4 844b2af1d65cbb143e8ecaa1c3ad9968f60d1882 release-1.4.5 +e201159f5d4ebe7ddbaf1e9247c843022ec202f3 release-1.4.6 From arut at nginx.com Tue Mar 4 19:43:18 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 04 Mar 2014 19:43:18 +0000 Subject: [nginx] Version bump. Message-ID: details: http://hg.nginx.org/nginx/rev/375a861d946b branches: changeset: 5598:375a861d946b user: Roman Arutyunyan date: Tue Mar 04 23:38:31 2014 +0400 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r f6d457701d9e -r 375a861d946b src/core/nginx.h --- a/src/core/nginx.h Tue Mar 04 15:39:23 2014 +0400 +++ b/src/core/nginx.h Tue Mar 04 23:38:31 2014 +0400 @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1005011 -#define NGINX_VERSION "1.5.11" +#define nginx_version 1005012 +#define NGINX_VERSION "1.5.12" #define NGINX_VER "nginx/" NGINX_VERSION #define NGINX_VAR "NGINX" From arut at nginx.com Tue Mar 4 19:43:19 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 04 Mar 2014 19:43:19 +0000 Subject: [nginx] MIME: added video/mp2t and application/vnd.apple.mpegurl... Message-ID: details: http://hg.nginx.org/nginx/rev/68e41f0cb207 branches: changeset: 5599:68e41f0cb207 user: Roman Arutyunyan date: Tue Mar 04 23:40:35 2014 +0400 description: MIME: added video/mp2t and application/vnd.apple.mpegurl types. Both types are used by HLS. The values are taken from RFC 3555 and HLS specification respectively. diffstat: conf/mime.types | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (19 lines): diff -r 375a861d946b -r 68e41f0cb207 conf/mime.types --- a/conf/mime.types Tue Mar 04 23:38:31 2014 +0400 +++ b/conf/mime.types Tue Mar 04 23:40:35 2014 +0400 @@ -32,6 +32,7 @@ types { application/pdf pdf; application/postscript ps eps ai; application/rtf rtf; + application/vnd.apple.mpegurl m3u8; application/vnd.ms-excel xls; application/vnd.ms-fontobject eot; application/vnd.ms-powerpoint ppt; @@ -73,6 +74,7 @@ types { audio/x-realaudio ra; video/3gpp 3gpp 3gp; + video/mp2t ts; video/mp4 mp4; video/mpeg mpeg mpg; video/quicktime mov; From agentzh at gmail.com Thu Mar 6 00:55:59 2014 From: agentzh at gmail.com (Yichun Zhang (agentzh)) Date: Wed, 5 Mar 2014 16:55:59 -0800 Subject: Valgrind error in nginx 1.5.9 Message-ID: Hello! My EC2 test cluster caught a valgrind error in nginx 1.5.9 recently which I cannot easily reproduce by hand. Just FYI :) ==28619== Invalid read of size 1 ==28619== at 0x426B11: ngx_init_cycle (ngx_cycle.c:535) ==28619== by 0x437DA1: ngx_master_process_cycle (ngx_process_cycle.c:240) ==28619== by 0x419689: main (nginx.c:407) ==28619== Address 0x7f4bae9 is 23 bytes before a block of size 17 alloc'd ==28619== at 0x4C276FC: malloc (vg_replace_malloc.c:270) ==28619== by 0x433185: ngx_alloc (ngx_alloc.c:22) ==28619== by 0x41A980: ngx_malloc (ngx_palloc.c:124) ==28619== by 0x41A9D8: ngx_pnalloc (ngx_palloc.c:152) ==28619== by 0x426386: ngx_init_cycle (ngx_cycle.c:202) ==28619== by 0x437DA1: ngx_master_process_cycle (ngx_process_cycle.c:240) ==28619== by 0x419689: main (nginx.c:407) ==28619== Thanks! -agentzh From mdounin at mdounin.ru Thu Mar 6 03:37:26 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 6 Mar 2014 07:37:26 +0400 Subject: Valgrind error in nginx 1.5.9 In-Reply-To: References: Message-ID: <20140306033726.GX34696@mdounin.ru> Hello! On Wed, Mar 05, 2014 at 04:55:59PM -0800, Yichun Zhang (agentzh) wrote: > Hello! > > My EC2 test cluster caught a valgrind error in nginx 1.5.9 recently which > I cannot easily reproduce by hand. Just FYI :) > > ==28619== Invalid read of size 1 > ==28619== at 0x426B11: ngx_init_cycle (ngx_cycle.c:535) > ==28619== by 0x437DA1: ngx_master_process_cycle (ngx_process_cycle.c:240) > ==28619== by 0x419689: main (nginx.c:407) > ==28619== Address 0x7f4bae9 is 23 bytes before a block of size 17 alloc'd > ==28619== at 0x4C276FC: malloc (vg_replace_malloc.c:270) > ==28619== by 0x433185: ngx_alloc (ngx_alloc.c:22) > ==28619== by 0x41A980: ngx_malloc (ngx_palloc.c:124) > ==28619== by 0x41A9D8: ngx_pnalloc (ngx_palloc.c:152) > ==28619== by 0x426386: ngx_init_cycle (ngx_cycle.c:202) > ==28619== by 0x437DA1: ngx_master_process_cycle (ngx_process_cycle.c:240) > ==28619== by 0x419689: main (nginx.c:407) > ==28619== Thanks, looks like couple of typos in the code. diff --git a/src/core/ngx_cycle.c b/src/core/ngx_cycle.c --- a/src/core/ngx_cycle.c +++ b/src/core/ngx_cycle.c @@ -494,14 +494,14 @@ ngx_init_cycle(ngx_cycle_t *old_cycle) } if (ngx_cmp_sockaddr(nls[n].sockaddr, nls[n].socklen, - ls[i].sockaddr, ls[n].socklen, 1) + ls[i].sockaddr, ls[i].socklen, 1) == NGX_OK) { nls[n].fd = ls[i].fd; nls[n].previous = &ls[i]; ls[i].remain = 1; - if (ls[n].backlog != nls[i].backlog) { + if (ls[i].backlog != nls[n].backlog) { nls[n].listen = 1; } @@ -532,7 +532,7 @@ ngx_init_cycle(ngx_cycle_t *old_cycle) #if (NGX_HAVE_DEFERRED_ACCEPT && defined TCP_DEFER_ACCEPT) - if (ls[n].deferred_accept && !nls[n].deferred_accept) { + if (ls[i].deferred_accept && !nls[n].deferred_accept) { nls[n].delete_deferred = 1; } else if (ls[i].deferred_accept != nls[n].deferred_accept) -- Maxim Dounin http://nginx.org/ From fdasilvayy at gmail.com Thu Mar 6 09:58:06 2014 From: fdasilvayy at gmail.com (Filipe da Silva) Date: Thu, 06 Mar 2014 10:58:06 +0100 Subject: [PATCH] MIME: added application/xspf+xml MIME type (ticket #479) Message-ID: <24df9fa5868957c1fb9a.1394099886@HPC> # HG changeset patch # User Filipe da Silva # Date 1394099468 -3600 # Thu Mar 06 10:51:08 2014 +0100 # Node ID 24df9fa5868957c1fb9a2d1569271e0958327dad # Parent 68e41f0cb207212b3cff2023fa1723bb0930e2d3 MIME: added application/xspf+xml MIME type (ticket #479). diff -r 68e41f0cb207 -r 24df9fa58689 conf/mime.types --- a/conf/mime.types Tue Mar 04 23:40:35 2014 +0400 +++ b/conf/mime.types Thu Mar 06 10:51:08 2014 +0100 @@ -55,6 +55,7 @@ types { application/x-x509-ca-cert der pem crt; application/x-xpinstall xpi; application/xhtml+xml xhtml; + application/xspf+xml xspf; application/zip zip; application/octet-stream bin exe dll; From fdasilvayy at gmail.com Thu Mar 6 09:59:29 2014 From: fdasilvayy at gmail.com (Filipe da Silva) Date: Thu, 06 Mar 2014 10:59:29 +0100 Subject: [PATCH] Mail: send starttls flag value to auth script Message-ID: <51fd90f96449c23af007.1394099969@HPC> # HG changeset patch # User Filipe da Silva # Date 1394099468 -3600 # Thu Mar 06 10:51:08 2014 +0100 # Node ID 51fd90f96449c23af0076a19efbfdb1f88702125 # Parent 24df9fa5868957c1fb9a2d1569271e0958327dad Mail: send starttls flag value to auth script. Allow to do logging (if logging takes place in the auth script) and or force some users to use STARTTLS while others can use unencrypted connection. diff -r 24df9fa58689 -r 51fd90f96449 src/mail/ngx_mail_auth_http_module.c --- a/src/mail/ngx_mail_auth_http_module.c Thu Mar 06 10:51:08 2014 +0100 +++ b/src/mail/ngx_mail_auth_http_module.c Thu Mar 06 10:51:08 2014 +0100 @@ -1165,6 +1165,9 @@ ngx_mail_auth_http_create_request(ngx_ma + sizeof("Auth-Salt: ") - 1 + s->salt.len + sizeof("Auth-Protocol: ") - 1 + cscf->protocol->name.len + sizeof(CRLF) - 1 +#if (NGX_MAIL_SSL) + + sizeof("Auth-STARTTLS: ") - 1 + 1 + sizeof(CRLF) - 1 +#endif + sizeof("Auth-Login-Attempt: ") - 1 + NGX_INT_T_LEN + sizeof(CRLF) - 1 + sizeof("Client-IP: ") - 1 + s->connection->addr_text.len @@ -1219,6 +1222,13 @@ ngx_mail_auth_http_create_request(ngx_ma cscf->protocol->name.len); *b->last++ = CR; *b->last++ = LF; +#if (NGX_MAIL_SSL) + b->last = ngx_cpymem(b->last, "Auth-STARTTLS: ", + sizeof("Auth-STARTTLS: ") - 1); + *b->last++ = s->starttls ? '1' : '0' ; + *b->last++ = CR; *b->last++ = LF; +#endif + b->last = ngx_sprintf(b->last, "Auth-Login-Attempt: %ui" CRLF, s->login_attempt); -------------- next part -------------- A non-text attachment was scrubbed... Name: 2014-02-13_21-59-37.diff Type: text/x-patch Size: 1636 bytes Desc: not available URL: From pluknet at nginx.com Thu Mar 6 14:28:08 2014 From: pluknet at nginx.com (Sergey Kandaurov) Date: Thu, 06 Mar 2014 14:28:08 +0000 Subject: [nginx] Fixed format specifier in logging of "c->number". Message-ID: details: http://hg.nginx.org/nginx/rev/957d734362ed branches: changeset: 5600:957d734362ed user: Sergey Kandaurov date: Thu Mar 06 18:25:59 2014 +0400 description: Fixed format specifier in logging of "c->number". diffstat: src/core/ngx_resolver.c | 2 +- src/event/ngx_event_accept.c | 2 +- src/event/ngx_event_connect.c | 2 +- src/mail/ngx_mail_handler.c | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diffs (48 lines): diff -r 68e41f0cb207 -r 957d734362ed src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c Tue Mar 04 23:40:35 2014 +0400 +++ b/src/core/ngx_resolver.c Thu Mar 06 18:25:59 2014 +0400 @@ -3062,7 +3062,7 @@ ngx_udp_connect(ngx_udp_connection_t *uc #endif ngx_log_debug3(NGX_LOG_DEBUG_EVENT, &uc->log, 0, - "connect to %V, fd:%d #%d", &uc->server, s, c->number); + "connect to %V, fd:%d #%uA", &uc->server, s, c->number); rc = connect(s, uc->sockaddr, uc->socklen); diff -r 68e41f0cb207 -r 957d734362ed src/event/ngx_event_accept.c --- a/src/event/ngx_event_accept.c Tue Mar 04 23:40:35 2014 +0400 +++ b/src/event/ngx_event_accept.c Thu Mar 06 18:25:59 2014 +0400 @@ -344,7 +344,7 @@ ngx_event_accept(ngx_event_t *ev) #endif ngx_log_debug3(NGX_LOG_DEBUG_EVENT, log, 0, - "*%d accept: %V fd:%d", c->number, &c->addr_text, s); + "*%uA accept: %V fd:%d", c->number, &c->addr_text, s); if (ngx_add_conn && (ngx_event_flags & NGX_USE_EPOLL_EVENT) == 0) { if (ngx_add_conn(c) == NGX_ERROR) { diff -r 68e41f0cb207 -r 957d734362ed src/event/ngx_event_connect.c --- a/src/event/ngx_event_connect.c Tue Mar 04 23:40:35 2014 +0400 +++ b/src/event/ngx_event_connect.c Thu Mar 06 18:25:59 2014 +0400 @@ -122,7 +122,7 @@ ngx_event_connect_peer(ngx_peer_connecti } ngx_log_debug3(NGX_LOG_DEBUG_EVENT, pc->log, 0, - "connect to %V, fd:%d #%d", pc->name, s, c->number); + "connect to %V, fd:%d #%uA", pc->name, s, c->number); rc = connect(s, pc->sockaddr, pc->socklen); diff -r 68e41f0cb207 -r 957d734362ed src/mail/ngx_mail_handler.c --- a/src/mail/ngx_mail_handler.c Tue Mar 04 23:40:35 2014 +0400 +++ b/src/mail/ngx_mail_handler.c Thu Mar 06 18:25:59 2014 +0400 @@ -127,7 +127,7 @@ ngx_mail_init_connection(ngx_connection_ c->data = s; s->connection = c; - ngx_log_error(NGX_LOG_INFO, c->log, 0, "*%ui client %V connected to %V", + ngx_log_error(NGX_LOG_INFO, c->log, 0, "*%uA client %V connected to %V", c->number, &c->addr_text, s->addr_text); ctx = ngx_palloc(c->pool, sizeof(ngx_mail_log_ctx_t)); From mdounin at mdounin.ru Thu Mar 6 16:27:18 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 6 Mar 2014 20:27:18 +0400 Subject: [PATCH] Mail: send starttls flag value to auth script In-Reply-To: <51fd90f96449c23af007.1394099969@HPC> References: <51fd90f96449c23af007.1394099969@HPC> Message-ID: <20140306162718.GL34696@mdounin.ru> Hello! On Thu, Mar 06, 2014 at 10:59:29AM +0100, Filipe da Silva wrote: > # HG changeset patch > # User Filipe da Silva > # Date 1394099468 -3600 > # Thu Mar 06 10:51:08 2014 +0100 > # Node ID 51fd90f96449c23af0076a19efbfdb1f88702125 > # Parent 24df9fa5868957c1fb9a2d1569271e0958327dad > Mail: send starttls flag value to auth script. > > Allow to do logging (if logging takes place in the auth script) and or force > some users to use STARTTLS while others can use unencrypted connection. > > diff -r 24df9fa58689 -r 51fd90f96449 src/mail/ngx_mail_auth_http_module.c > --- a/src/mail/ngx_mail_auth_http_module.c Thu Mar 06 10:51:08 2014 +0100 > +++ b/src/mail/ngx_mail_auth_http_module.c Thu Mar 06 10:51:08 2014 +0100 > @@ -1165,6 +1165,9 @@ ngx_mail_auth_http_create_request(ngx_ma > + sizeof("Auth-Salt: ") - 1 + s->salt.len > + sizeof("Auth-Protocol: ") - 1 + cscf->protocol->name.len > + sizeof(CRLF) - 1 > +#if (NGX_MAIL_SSL) > + + sizeof("Auth-STARTTLS: ") - 1 + 1 + sizeof(CRLF) - 1 > +#endif > + sizeof("Auth-Login-Attempt: ") - 1 + NGX_INT_T_LEN > + sizeof(CRLF) - 1 > + sizeof("Client-IP: ") - 1 + s->connection->addr_text.len > @@ -1219,6 +1222,13 @@ ngx_mail_auth_http_create_request(ngx_ma > cscf->protocol->name.len); > *b->last++ = CR; *b->last++ = LF; > > +#if (NGX_MAIL_SSL) > + b->last = ngx_cpymem(b->last, "Auth-STARTTLS: ", > + sizeof("Auth-STARTTLS: ") - 1); > + *b->last++ = s->starttls ? '1' : '0' ; > + *b->last++ = CR; *b->last++ = LF; > +#endif > + > b->last = ngx_sprintf(b->last, "Auth-Login-Attempt: %ui" CRLF, > s->login_attempt); > > # HG changeset patch > # User Filipe da Silva > # Date 1394099468 -3600 > # Thu Mar 06 10:51:08 2014 +0100 > # Node ID 51fd90f96449c23af0076a19efbfdb1f88702125 > # Parent 24df9fa5868957c1fb9a2d1569271e0958327dad > Mail: send starttls flag value to auth script. > > Allow to do logging (if logging takes place in the auth script) and or force > some users to use STARTTLS while others can use unencrypted connection. > > diff -r 24df9fa58689 -r 51fd90f96449 src/mail/ngx_mail_auth_http_module.c > --- a/src/mail/ngx_mail_auth_http_module.c Thu Mar 06 10:51:08 2014 +0100 > +++ b/src/mail/ngx_mail_auth_http_module.c Thu Mar 06 10:51:08 2014 +0100 > @@ -1165,6 +1165,9 @@ ngx_mail_auth_http_create_request(ngx_ma > + sizeof("Auth-Salt: ") - 1 + s->salt.len > + sizeof("Auth-Protocol: ") - 1 + cscf->protocol->name.len > + sizeof(CRLF) - 1 > +#if (NGX_MAIL_SSL) > + + sizeof("Auth-STARTTLS: ") - 1 + 1 + sizeof(CRLF) - 1 > +#endif > + sizeof("Auth-Login-Attempt: ") - 1 + NGX_INT_T_LEN > + sizeof(CRLF) - 1 > + sizeof("Client-IP: ") - 1 + s->connection->addr_text.len > @@ -1219,6 +1222,13 @@ ngx_mail_auth_http_create_request(ngx_ma > cscf->protocol->name.len); > *b->last++ = CR; *b->last++ = LF; > > +#if (NGX_MAIL_SSL) > + b->last = ngx_cpymem(b->last, "Auth-STARTTLS: ", > + sizeof("Auth-STARTTLS: ") - 1); > + *b->last++ = s->starttls ? '1' : '0' ; > + *b->last++ = CR; *b->last++ = LF; > +#endif > + > b->last = ngx_sprintf(b->last, "Auth-Login-Attempt: %ui" CRLF, > s->login_attempt); > I don't think that it's a good idea to pass STARTTLS into auth script. If at all needed, it should be something like a flag "if SSL is used", not an explicit STARTTLS status. From auth script point of view there is no difference if a connection uses SSL on a dedicated port or encryption was negotiated using STARTLS. -- Maxim Dounin http://nginx.org/ From info at phpgangsta.de Thu Mar 6 17:03:25 2014 From: info at phpgangsta.de (Michael Kliewe) Date: Thu, 6 Mar 2014 18:03:25 +0100 Subject: [PATCH] Mail: send starttls flag value to auth script In-Reply-To: <20140306162718.GL34696@mdounin.ru> References: <51fd90f96449c23af007.1394099969@HPC> <20140306162718.GL34696@mdounin.ru> Message-ID: <877FD2F6-57CD-4C14-9F2B-4C9E909C3488@phpgangsta.de> Hi Maxim, On Mar 6, 2014, at 5:27 PM, Maxim Dounin wrote: > Hello! > > On Thu, Mar 06, 2014 at 10:59:29AM +0100, Filipe da Silva wrote: > >> # HG changeset patch >> # User Filipe da Silva >> # Date 1394099468 -3600 >> # Thu Mar 06 10:51:08 2014 +0100 >> # Node ID 51fd90f96449c23af0076a19efbfdb1f88702125 >> # Parent 24df9fa5868957c1fb9a2d1569271e0958327dad >> Mail: send starttls flag value to auth script. >> >> Allow to do logging (if logging takes place in the auth script) and or force >> some users to use STARTTLS while others can use unencrypted connection. >> > > I don't think that it's a good idea to pass STARTTLS into auth > script. If at all needed, it should be something like a flag "if > SSL is used", not an explicit STARTTLS status. From auth script > point of view there is no difference if a connection uses SSL on a > dedicated port or encryption was negotiated using STARTLS. yes, it is needed ;-) You are right, that would also be possible, the auth script then can check which port has been used, and then has the information if it has been STARTTLS or SSL. In our case we want to distinguish between STARTTLS and SSL in the auth script. Both solutions are fine I think, so let's take Maxims ;-) (Sorry Filipe for the extra work) Hope this easy patch gets into nginx then, we need it ;-) Thanks! Michael From pluknet at nginx.com Thu Mar 6 19:17:21 2014 From: pluknet at nginx.com (Sergey Kandaurov) Date: Thu, 06 Mar 2014 19:17:21 +0000 Subject: [nginx] Added connection serial number in logging of left open s... Message-ID: details: http://hg.nginx.org/nginx/rev/e45fa57ef725 branches: changeset: 5601:e45fa57ef725 user: Sergey Kandaurov date: Thu Mar 06 23:15:10 2014 +0400 description: Added connection serial number in logging of left open sockets. diffstat: src/os/unix/ngx_process_cycle.c | 4 ++-- src/os/win32/ngx_process_cycle.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diffs (28 lines): diff -r 957d734362ed -r e45fa57ef725 src/os/unix/ngx_process_cycle.c --- a/src/os/unix/ngx_process_cycle.c Thu Mar 06 18:25:59 2014 +0400 +++ b/src/os/unix/ngx_process_cycle.c Thu Mar 06 23:15:10 2014 +0400 @@ -1046,8 +1046,8 @@ ngx_worker_process_exit(ngx_cycle_t *cyc && !c[i].read->resolver) { ngx_log_error(NGX_LOG_ALERT, cycle->log, 0, - "open socket #%d left in connection %ui", - c[i].fd, i); + "*%uA open socket #%d left in connection %ui", + c[i].number, c[i].fd, i); ngx_debug_quit = 1; } } diff -r 957d734362ed -r e45fa57ef725 src/os/win32/ngx_process_cycle.c --- a/src/os/win32/ngx_process_cycle.c Thu Mar 06 18:25:59 2014 +0400 +++ b/src/os/win32/ngx_process_cycle.c Thu Mar 06 23:15:10 2014 +0400 @@ -881,8 +881,8 @@ ngx_worker_process_exit(ngx_cycle_t *cyc && !c[i].read->resolver) { ngx_log_error(NGX_LOG_ALERT, cycle->log, 0, - "open socket #%d left in connection %ui", - c[i].fd, i); + "*%uA open socket #%d left in connection %ui", + c[i].number, c[i].fd, i); } } } From flygoast at 126.com Fri Mar 7 06:32:43 2014 From: flygoast at 126.com (flygoast) Date: Fri, 7 Mar 2014 14:32:43 +0800 (CST) Subject: [PATCH]Add processing of posted requests in ngx_http_wait_request_handler(). Message-ID: <5ee7524b.aecd.1449b3e6b96.Coremail.flygoast@126.com> # HG changeset patch # User FengGu # Date 1394165926 -28800 # Fri Mar 07 12:18:46 2014 +0800 # Node ID c357bd0684560aa2f90f23bdda5dae9cf7f0b13c # Parent e45fa57ef72590d010c8234cb5c3c6e2ae63e157 Add processing of posted requests in ngx_http_wait_request_handler(). If something like "error_page 400 /foo" is used in a configuration, when the new uri processing failed, the ngx_http_terminate_request(r, -1) would be invoked. The request would not be released resulting in socket leak. diff -r e45fa57ef725 -r c357bd068456 src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c Thu Mar 06 23:15:10 2014 +0400 +++ b/src/http/ngx_http_request.c Fri Mar 07 12:18:46 2014 +0800 @@ -470,6 +470,7 @@ rev->handler = ngx_http_process_request_line; ngx_http_process_request_line(rev); + ngx_http_run_posted_requests(c); } -------------- next part -------------- An HTML attachment was scrubbed... URL: From flevionnois at gmail.com Fri Mar 7 08:40:11 2014 From: flevionnois at gmail.com (Franck Levionnois) Date: Fri, 7 Mar 2014 09:40:11 +0100 Subject: [PATCH] Mail: added support for SSL client certificate In-Reply-To: <5c7ccfc96070fc8b5d77.1392983328@FLEVIONNOIS2.dictao.com> References: <5c7ccfc96070fc8b5d77.1392983328@FLEVIONNOIS2.dictao.com> Message-ID: Hello, I haven't seen any comment on this patch. Is it ok for you ? Kind regards. Franck. 2014-02-21 12:48 GMT+01:00 : > # HG changeset patch > # User Franck Levionnois > # Date 1390577176 -3600 > # Fri Jan 24 16:26:16 2014 +0100 > # Node ID 5c7ccfc96070fc8b5d775643d1e12c4e5a8b438f > # Parent 4dee5ad51e9e5b49085011e8785001e2d6c02b0d > Mail: added support for SSL client certificate. > > Add support for SSL Mutual Authentification like in mail module. > > Added mail configuration directives (like http): > ssl_verify_client, ssl_verify_depth, ssl_client_certificate, > ssl_trusted_certificate, ssl_crl. > > Added http auth headers: > Auth-Certificate, Auth-Verify, Auth-Issuer-DN, Auth-Subject-DN, > Auth-Subject-Serial. > > diff -r 4dee5ad51e9e -r 5c7ccfc96070 src/mail/ngx_mail_auth_http_module.c > --- a/src/mail/ngx_mail_auth_http_module.c Thu Feb 20 13:48:40 2014 > +0400 > +++ b/src/mail/ngx_mail_auth_http_module.c Fri Jan 24 16:26:16 2014 > +0100 > @@ -1143,6 +1143,9 @@ ngx_mail_auth_http_create_request(ngx_ma > size_t len; > ngx_buf_t *b; > ngx_str_t login, passwd; > +#if (NGX_MAIL_SSL) > + ngx_str_t cert, rawcert, verify, subject, issuer, > serial; > +#endif > ngx_mail_core_srv_conf_t *cscf; > > if (ngx_mail_auth_http_escape(pool, &s->login, &login) != NGX_OK) { > @@ -1155,6 +1158,45 @@ ngx_mail_auth_http_create_request(ngx_ma > > cscf = ngx_mail_get_module_srv_conf(s, ngx_mail_core_module); > > +#if (NGX_MAIL_SSL) > + if (s->connection->ssl) { > + if (ngx_ssl_get_client_verify(s->connection, pool, &verify) != > NGX_OK) > + { > + return NULL; > + } > + > + if (ngx_ssl_get_subject_dn(s->connection, pool, > + &subject) != NGX_OK) { > + return NULL; > + } > + > + if (ngx_ssl_get_issuer_dn(s->connection, pool, > + &issuer) != NGX_OK) { > + return NULL; > + } > + > + if (ngx_ssl_get_serial_number(s->connection, pool, > + &serial) != NGX_OK) { > + return NULL; > + } > + > + if (ngx_ssl_get_raw_certificate(s->connection, pool, > + &rawcert) != NGX_OK) { > + return NULL; > + } > + > + if (ngx_mail_auth_http_escape(pool, &rawcert, &cert) != NGX_OK) { > + return NULL; > + } > + } else { > + verify.len = 0; > + issuer.len = 0; > + subject.len = 0; > + serial.len = 0; > + cert.len = 0; > + } > +#endif > + > len = sizeof("GET ") - 1 + ahcf->uri.len + sizeof(" HTTP/1.0" CRLF) - > 1 > + sizeof("Host: ") - 1 + ahcf->host_header.len + sizeof(CRLF) - > 1 > + sizeof("Auth-Method: ") - 1 > @@ -1163,6 +1205,14 @@ ngx_mail_auth_http_create_request(ngx_ma > + sizeof("Auth-User: ") - 1 + login.len + sizeof(CRLF) - 1 > + sizeof("Auth-Pass: ") - 1 + passwd.len + sizeof(CRLF) - 1 > + sizeof("Auth-Salt: ") - 1 + s->salt.len > +#if (NGX_MAIL_SSL) > + + sizeof("Auth-Certificate: ") - 1 + cert.len + sizeof(CRLF) - 1 > + + sizeof("Auth-Verify: ") - 1 + verify.len + sizeof(CRLF) - 1 > + + sizeof("Auth-Issuer-DN: ") - 1 + issuer.len + sizeof(CRLF) - 1 > + + sizeof("Auth-Subject-DN: ") - 1 + subject.len + sizeof(CRLF) > - 1 > + + sizeof("Auth-Subject-Serial: ") - 1 + serial.len > + + sizeof(CRLF) - 1 > +#endif > + sizeof("Auth-Protocol: ") - 1 + cscf->protocol->name.len > + sizeof(CRLF) - 1 > + sizeof("Auth-Login-Attempt: ") - 1 + NGX_INT_T_LEN > @@ -1213,6 +1263,44 @@ ngx_mail_auth_http_create_request(ngx_ma > s->passwd.data = NULL; > } > > +#if (NGX_MAIL_SSL) > + if (cert.len) { > + b->last = ngx_cpymem(b->last, "Auth-Certificate: ", > + sizeof("Auth-Certificate: ") - 1); > + b->last = ngx_copy(b->last, cert.data, cert.len); > + *b->last++ = CR; *b->last++ = LF; > + } > + > + if (verify.len) { > + b->last = ngx_cpymem(b->last, "Auth-Verify: ", > + sizeof("Auth-Verify: ") - 1); > + b->last = ngx_copy(b->last, verify.data, verify.len); > + *b->last++ = CR; *b->last++ = LF; > + } > + > + if (issuer.len) { > + b->last = ngx_cpymem(b->last, "Auth-Issuer-DN: ", > + sizeof("Auth-Issuer-DN: ") - 1); > + b->last = ngx_copy(b->last, issuer.data, issuer.len); > + *b->last++ = CR; *b->last++ = LF; > + } > + > + if (subject.len) { > + b->last = ngx_cpymem(b->last, "Auth-Subject-DN: ", > + sizeof("Auth-Subject-DN: ") - 1); > + b->last = ngx_copy(b->last, subject.data, subject.len); > + *b->last++ = CR; *b->last++ = LF; > + } > + > + if (serial.len) { > + b->last = ngx_cpymem(b->last, "Auth-Subject-Serial: ", > + sizeof("Auth-Subject-Serial: ") - 1); > + b->last = ngx_copy(b->last, serial.data, serial.len); > + *b->last++ = CR; *b->last++ = LF; > + } > + > +#endif > + > b->last = ngx_cpymem(b->last, "Auth-Protocol: ", > sizeof("Auth-Protocol: ") - 1); > b->last = ngx_cpymem(b->last, cscf->protocol->name.data, > diff -r 4dee5ad51e9e -r 5c7ccfc96070 src/mail/ngx_mail_handler.c > --- a/src/mail/ngx_mail_handler.c Thu Feb 20 13:48:40 2014 +0400 > +++ b/src/mail/ngx_mail_handler.c Fri Jan 24 16:26:16 2014 +0100 > @@ -236,11 +236,60 @@ ngx_mail_ssl_handshake_handler(ngx_conne > { > ngx_mail_session_t *s; > ngx_mail_core_srv_conf_t *cscf; > +#if (NGX_MAIL_SSL) > + ngx_mail_ssl_conf_t *sslcf; > +#endif > + > + ngx_log_debug1(NGX_LOG_DEBUG_MAIL, c->log, 0, > + "ngx_mail_ssl_handshake_handler handshaked: %d", > + c->ssl->handshaked); > > if (c->ssl->handshaked) { > > s = c->data; > > +#if (NGX_MAIL_SSL) > + sslcf = ngx_mail_get_module_srv_conf(s, ngx_mail_ssl_module); > + if (sslcf->verify) { > + long rc; > + > + rc = SSL_get_verify_result(c->ssl->connection); > + > + if (rc != X509_V_OK > + && (sslcf->verify != 3 || > !ngx_ssl_verify_error_optional(rc))) > + { > + ngx_log_error(NGX_LOG_INFO, c->log, 0, > + "client SSL certificate verify error: > (%l:%s)", > + rc, X509_verify_cert_error_string(rc)); > + > + ngx_ssl_remove_cached_session(sslcf->ssl.ctx, > + > (SSL_get0_session(c->ssl->connection))); > + > + ngx_mail_close_connection(c); > + return; > + } > + > + if (sslcf->verify == 1) { > + X509 *cert; > + cert = SSL_get_peer_certificate(c->ssl->connection); > + > + if (cert == NULL) { > + ngx_log_error(NGX_LOG_INFO, c->log, 0, > + "client sent no required SSL > certificate"); > + > + ngx_ssl_remove_cached_session(sslcf->ssl.ctx, > + > (SSL_get0_session(c->ssl->connection))); > + > + ngx_mail_close_connection(c); > + return; > + } > + > + X509_free(cert); > + } > + } > + > +#endif > + > if (s->starttls) { > cscf = ngx_mail_get_module_srv_conf(s, ngx_mail_core_module); > > diff -r 4dee5ad51e9e -r 5c7ccfc96070 src/mail/ngx_mail_ssl_module.c > --- a/src/mail/ngx_mail_ssl_module.c Thu Feb 20 13:48:40 2014 +0400 > +++ b/src/mail/ngx_mail_ssl_module.c Fri Jan 24 16:26:16 2014 +0100 > @@ -43,6 +43,14 @@ static ngx_conf_bitmask_t ngx_mail_ssl_ > { ngx_null_string, 0 } > }; > > +static ngx_conf_enum_t ngx_mail_ssl_verify[] = { > + { ngx_string("off"), 0 }, > + { ngx_string("on"), 1 }, > + { ngx_string("optional"), 2 }, > + { ngx_string("optional_no_ca"), 3 }, > + { ngx_null_string, 0 } > +}; > + > > static ngx_command_t ngx_mail_ssl_commands[] = { > > @@ -102,6 +110,34 @@ static ngx_command_t ngx_mail_ssl_comma > offsetof(ngx_mail_ssl_conf_t, ciphers), > NULL }, > > + { ngx_string("ssl_verify_client"), > + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, > + ngx_conf_set_enum_slot, > + NGX_MAIL_SRV_CONF_OFFSET, > + offsetof(ngx_mail_ssl_conf_t, verify), > + &ngx_mail_ssl_verify }, > + > + { ngx_string("ssl_verify_depth"), > + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, > + ngx_conf_set_num_slot, > + NGX_MAIL_SRV_CONF_OFFSET, > + offsetof(ngx_mail_ssl_conf_t, verify_depth), > + NULL }, > + > + { ngx_string("ssl_client_certificate"), > + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, > + ngx_conf_set_str_slot, > + NGX_MAIL_SRV_CONF_OFFSET, > + offsetof(ngx_mail_ssl_conf_t, client_certificate), > + NULL }, > + > + { ngx_string("ssl_trusted_certificate"), > + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, > + ngx_conf_set_str_slot, > + NGX_MAIL_SRV_CONF_OFFSET, > + offsetof(ngx_mail_ssl_conf_t, trusted_certificate), > + NULL }, > + > { ngx_string("ssl_prefer_server_ciphers"), > NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_FLAG, > ngx_conf_set_flag_slot, > @@ -137,6 +173,13 @@ static ngx_command_t ngx_mail_ssl_comma > offsetof(ngx_mail_ssl_conf_t, session_timeout), > NULL }, > > + { ngx_string("ssl_crl"), > + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, > + ngx_conf_set_str_slot, > + NGX_MAIL_SRV_CONF_OFFSET, > + offsetof(ngx_mail_ssl_conf_t, crl), > + NULL }, > + > ngx_null_command > }; > > @@ -189,6 +232,9 @@ ngx_mail_ssl_create_conf(ngx_conf_t *cf) > * scf->certificate_key = { 0, NULL }; > * scf->dhparam = { 0, NULL }; > * scf->ecdh_curve = { 0, NULL }; > + * scf->client_certificate = { 0, NULL }; > + * scf->trusted_certificate = { 0, NULL }; > + * scf->crl = { 0, NULL }; > * scf->ciphers = { 0, NULL }; > * scf->shm_zone = NULL; > */ > @@ -196,6 +242,8 @@ ngx_mail_ssl_create_conf(ngx_conf_t *cf) > scf->enable = NGX_CONF_UNSET; > scf->starttls = NGX_CONF_UNSET_UINT; > scf->prefer_server_ciphers = NGX_CONF_UNSET; > + scf->verify = NGX_CONF_UNSET_UINT; > + scf->verify_depth = NGX_CONF_UNSET_UINT; > scf->builtin_session_cache = NGX_CONF_UNSET; > scf->session_timeout = NGX_CONF_UNSET; > scf->session_tickets = NGX_CONF_UNSET; > @@ -228,11 +276,20 @@ ngx_mail_ssl_merge_conf(ngx_conf_t *cf, > (NGX_CONF_BITMASK_SET|NGX_SSL_SSLv3|NGX_SSL_TLSv1 > |NGX_SSL_TLSv1_1|NGX_SSL_TLSv1_2)); > > + ngx_conf_merge_uint_value(conf->verify, prev->verify, 0); > + ngx_conf_merge_uint_value(conf->verify_depth, prev->verify_depth, 1); > + > ngx_conf_merge_str_value(conf->certificate, prev->certificate, ""); > ngx_conf_merge_str_value(conf->certificate_key, > prev->certificate_key, ""); > > ngx_conf_merge_str_value(conf->dhparam, prev->dhparam, ""); > > + ngx_conf_merge_str_value(conf->client_certificate, > prev->client_certificate, > + ""); > + ngx_conf_merge_str_value(conf->trusted_certificate, > + prev->trusted_certificate, ""); > + ngx_conf_merge_str_value(conf->crl, prev->crl, ""); > + > ngx_conf_merge_str_value(conf->ecdh_curve, prev->ecdh_curve, > NGX_DEFAULT_ECDH_CURVE); > > @@ -318,6 +375,35 @@ ngx_mail_ssl_merge_conf(ngx_conf_t *cf, > return NGX_CONF_ERROR; > } > > + if (conf->verify) { > + > + if (conf->client_certificate.len == 0 && conf->verify != 3) { > + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, > + "no ssl_client_certificate for > ssl_client_verify"); > + return NGX_CONF_ERROR; > + } > + > + if (ngx_ssl_client_certificate(cf, &conf->ssl, > + &conf->client_certificate, > + conf->verify_depth) > + != NGX_OK) > + { > + return NGX_CONF_ERROR; > + } > + } > + > + if (ngx_ssl_trusted_certificate(cf, &conf->ssl, > + &conf->trusted_certificate, > + conf->verify_depth) > + != NGX_OK) > + { > + return NGX_CONF_ERROR; > + } > + > + if (ngx_ssl_crl(cf, &conf->ssl, &conf->crl) != NGX_OK) { > + return NGX_CONF_ERROR; > + } > + > if (conf->prefer_server_ciphers) { > SSL_CTX_set_options(conf->ssl.ctx, > SSL_OP_CIPHER_SERVER_PREFERENCE); > } > diff -r 4dee5ad51e9e -r 5c7ccfc96070 src/mail/ngx_mail_ssl_module.h > --- a/src/mail/ngx_mail_ssl_module.h Thu Feb 20 13:48:40 2014 +0400 > +++ b/src/mail/ngx_mail_ssl_module.h Fri Jan 24 16:26:16 2014 +0100 > @@ -28,6 +28,8 @@ typedef struct { > ngx_uint_t starttls; > ngx_uint_t protocols; > > + ngx_uint_t verify; > + ngx_uint_t verify_depth; > ssize_t builtin_session_cache; > > time_t session_timeout; > @@ -36,6 +38,9 @@ typedef struct { > ngx_str_t certificate_key; > ngx_str_t dhparam; > ngx_str_t ecdh_curve; > + ngx_str_t client_certificate; > + ngx_str_t trusted_certificate; > + ngx_str_t crl; > > ngx_str_t ciphers; > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Fri Mar 7 11:20:37 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 07 Mar 2014 11:20:37 +0000 Subject: [nginx] MIME: added application/xspf+xml MIME type (ticket #479). Message-ID: details: http://hg.nginx.org/nginx/rev/b799b9d62c45 branches: changeset: 5602:b799b9d62c45 user: Filipe da Silva date: Thu Mar 06 10:51:08 2014 +0100 description: MIME: added application/xspf+xml MIME type (ticket #479). diffstat: conf/mime.types | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff --git a/conf/mime.types b/conf/mime.types --- a/conf/mime.types +++ b/conf/mime.types @@ -55,6 +55,7 @@ types { application/x-x509-ca-cert der pem crt; application/x-xpinstall xpi; application/xhtml+xml xhtml; + application/xspf+xml xspf; application/zip zip; application/octet-stream bin exe dll; From mdounin at mdounin.ru Fri Mar 7 11:20:39 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 07 Mar 2014 11:20:39 +0000 Subject: [nginx] Core: fixed wrong indices in listening sockets compare l... Message-ID: details: http://hg.nginx.org/nginx/rev/b2b5b1b74129 branches: changeset: 5603:b2b5b1b74129 user: Maxim Dounin date: Fri Mar 07 15:17:38 2014 +0400 description: Core: fixed wrong indices in listening sockets compare loops. Proper use is "ls[i]" and "nls[n]". Previous code used it wrong in several places, likely due to cut-n-paste errors. diffstat: src/core/ngx_cycle.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diffs (29 lines): diff --git a/src/core/ngx_cycle.c b/src/core/ngx_cycle.c --- a/src/core/ngx_cycle.c +++ b/src/core/ngx_cycle.c @@ -494,14 +494,14 @@ ngx_init_cycle(ngx_cycle_t *old_cycle) } if (ngx_cmp_sockaddr(nls[n].sockaddr, nls[n].socklen, - ls[i].sockaddr, ls[n].socklen, 1) + ls[i].sockaddr, ls[i].socklen, 1) == NGX_OK) { nls[n].fd = ls[i].fd; nls[n].previous = &ls[i]; ls[i].remain = 1; - if (ls[n].backlog != nls[i].backlog) { + if (ls[i].backlog != nls[n].backlog) { nls[n].listen = 1; } @@ -532,7 +532,7 @@ ngx_init_cycle(ngx_cycle_t *old_cycle) #if (NGX_HAVE_DEFERRED_ACCEPT && defined TCP_DEFER_ACCEPT) - if (ls[n].deferred_accept && !nls[n].deferred_accept) { + if (ls[i].deferred_accept && !nls[n].deferred_accept) { nls[n].delete_deferred = 1; } else if (ls[i].deferred_accept != nls[n].deferred_accept) From mdounin at mdounin.ru Fri Mar 7 11:20:59 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 7 Mar 2014 15:20:59 +0400 Subject: [PATCH] MIME: added application/xspf+xml MIME type (ticket #479) In-Reply-To: <24df9fa5868957c1fb9a.1394099886@HPC> References: <24df9fa5868957c1fb9a.1394099886@HPC> Message-ID: <20140307112058.GX34696@mdounin.ru> Hello! On Thu, Mar 06, 2014 at 10:58:06AM +0100, Filipe da Silva wrote: > # HG changeset patch > # User Filipe da Silva > # Date 1394099468 -3600 > # Thu Mar 06 10:51:08 2014 +0100 > # Node ID 24df9fa5868957c1fb9a2d1569271e0958327dad > # Parent 68e41f0cb207212b3cff2023fa1723bb0930e2d3 > MIME: added application/xspf+xml MIME type (ticket #479). > > diff -r 68e41f0cb207 -r 24df9fa58689 conf/mime.types > --- a/conf/mime.types Tue Mar 04 23:40:35 2014 +0400 > +++ b/conf/mime.types Thu Mar 06 10:51:08 2014 +0100 > @@ -55,6 +55,7 @@ types { > application/x-x509-ca-cert der pem crt; > application/x-xpinstall xpi; > application/xhtml+xml xhtml; > + application/xspf+xml xspf; > application/zip zip; > > application/octet-stream bin exe dll; Committed, thanks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Fri Mar 7 11:31:47 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 7 Mar 2014 15:31:47 +0400 Subject: [PATCH] Mail: added support for SSL client certificate In-Reply-To: References: <5c7ccfc96070fc8b5d77.1392983328@FLEVIONNOIS2.dictao.com> Message-ID: <20140307113147.GY34696@mdounin.ru> Hello! On Fri, Mar 07, 2014 at 09:40:11AM +0100, Franck Levionnois wrote: > Hello, > I haven't seen any comment on this patch. Is it ok for you ? Sorry, I haven't yet had a time to look into it in detail. Most problematic part is still auth_http protocol changes - in particular, headers send and names used for them. I tend to think there should be better names, and probably we can safely omit some information as duplicate/unneeded. -- Maxim Dounin http://nginx.org/ From thehuntersays at gmail.com Mon Mar 10 13:46:55 2014 From: thehuntersays at gmail.com (thehuntersays .) Date: Mon, 10 Mar 2014 19:16:55 +0530 Subject: Hi Need help with Throttle Limit. Message-ID: Hi, I use Nginx as front end Engine for handling huge Traffic, we receive around 30k qps and have kept 5 Nginx under ELB (Elastic Load Balancer), each handling 6 k qps. I am using the Nginx Upstream module for connecting with the backend server, my Backend server is able to handle only 300qps, we use Nginx for mainly for throttling the request. But to my surprise on increasing the throttling limit to more than 100 qps, then Nginx is able to handle only around 1000, with throttling rate set to 1500qps, its able to handle only 250. The following is the configuration i am using. worker_processes auto; events { use epoll; multi_accept on; worker_connections 9046; } http { log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log off; error_log /dev/null crit; sendfile on; tcp_nopush on; keepalive_timeout 0 * limit_req_zone $binary_remote_addr zone=jai:100m rate=1500r/s;* * upstream backend* * {* * server 127.0.0.1:5555 ;* * }* server { listen 80 backlog=16384; server_name localhost; location /jai { keepalive_timeout 60; keepalive_requests 10000; * proxy_pass http://backend ;* * proxy_set_header Connection "keep-alive";* * limit_req zone=jai burst=1000 nodelay;* access_log logs/dsp-access.log main buffer=32k; error_log logs/dsp-error.log crit; } Please let me know how to make Nginx handle more request with throttle enable with the backend servers. Thanks in advance, Jaysingh Samuel -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon Mar 10 22:40:48 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 11 Mar 2014 02:40:48 +0400 Subject: Hi Need help with Throttle Limit. In-Reply-To: References: Message-ID: <20140310224048.GH34696@mdounin.ru> Hello! On Mon, Mar 10, 2014 at 07:16:55PM +0530, thehuntersays . wrote: > Hi, > > I use Nginx as front end Engine for handling huge Traffic, we receive > around 30k qps and have kept 5 Nginx under ELB (Elastic Load Balancer), > each handling 6 k qps. > > I am using the Nginx Upstream module for connecting with the backend > server, my Backend server is able to handle only 300qps, we use Nginx for > mainly for throttling the request. > > But to my surprise on increasing the throttling limit to more than 100 qps, > then Nginx is able to handle only around 1000, with throttling rate set to > 1500qps, its able to handle only 250. > > The following is the configuration i am using. [...] > Please let me know how to make Nginx handle more request with throttle > enable with the backend servers. This certainly irrelevant to nginx-devel@ list. Please use nginx@ for such questions. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Tue Mar 11 00:49:05 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 11 Mar 2014 04:49:05 +0400 Subject: [alert] could not add new SSL session to the session cache while SSL handshaking In-Reply-To: References: <20140303174518.GO34696@mdounin.ru> Message-ID: <20140311004905.GM34696@mdounin.ru> Hello! On Sun, Mar 09, 2014 at 12:23:23PM +0100, Lukas Tribus wrote: > Hi Maxim, > > > > You've changed SSL session timeout from 10 minutes to 24 hours, > > and this basically means that sessions will use 144 times more > > space in session cache. On the other hand, cache size wasn't > > changed - so you've run out of space in the cache configured. If > > there is no space in a cache nginx will try to drop one > > non-expired session from the cache, but it may not be enough to > > store a new session (as different sessions may occupy different > > space), resulting in alerts you've quoted. > > I'm trying to understand this better, and read the comments in > ngx_ssl_new_session(). > > Please tell if I got this right. > > > Basically this can only happen if we delete a SSLv2 session from > the cache and but need the space for a SSLv3/TLS session (SSLv2 > session id needs 16 bytes and a SSLv3/TLS session id 32 bytes). > > So if SSLv2 hasn't been enabled via ssl_protocols directive, this > problem will not happen, correct? > > Also this cannot happen on a 64-bit platform, as we use 128 bytes > sized allocations for the session id + ASN1 representation even > with SSLv2. > > So the two workarounds to avoid this problem in production would be: > - disable SSLv2 (which is default anyway and shouldn't be enabled) > - use a 64-bit platform Length of a serialized session may also vary depending on a key size negotiated, various TLS extensions negotiated and so on. Try looking into d2i_SSL_SESSION() source. I also suspect the comment you are referring to is largely outdated, and modern numbers are very different. > > Note well that configuring ssl_buffer_size to 1400 isn't a good > > idea unless you are doing so for your own performance testing. > > See previous discussions for details. > > Sure, but in a production environment we also need to consider > that we hit such limits, even if buffer_size and timeouts are > tuned and state of the art. The ssl_buffer_size directive is completely unrelated to session caching. > An attacker can flood the SSL session cache easily and if he is aware > of such a limitation he could flood it with SSLv2 sessions, basically > disabling the SSL session cache, right? At worst, the cache won't be effective for as many new sessions as many "bad" sessions previously added to it (or rather half of that number, assuming removing two "bad" sessions frees enough space for one new session). And this is basically identical to what can be done by just adding new sessions to the cache, thus forcing expiration of previously added sessions. > Wouldn't it be better: > - drop all expired sessions from cache instead of just one or two? If for each session added two expired sessions are removed, this basically means that there will be no expired sessions - but work done on each particular request is limited. > - try dropping up to 4 non-expired session times before giving up > ? (max allocation is 128, min allocation is 32, max / min = 4 sessions > ? to drop in the worst case) I don't think your calculations are correct, but dropping couple of sessions instead of just one may be beneficial. On the other hand, it has a downside of making it easier to expire previously stored sessions. -- Maxim Dounin http://nginx.org/ From apunch at brandscreen.com Tue Mar 11 03:32:52 2014 From: apunch at brandscreen.com (Andrew Punch) Date: Tue, 11 Mar 2014 14:32:52 +1100 Subject: ngx_reopen_files() assumes null termination Message-ID: Hi, I just noticed that ngx_reopen_files() treats the file name as a null terminated string but there is no guarantee that a ngx_str_t will be null terminated. I noticed in ngx_string.h that the ngx_string() macro sets the length to be exclusive of the null termination character, however the null termination would still be present at s.data[s.len] However ngx_pstrdup() in ngx_string.c only copies the size of s.len - so if there is a null character at s.data[s.len] it will not be copied. Could you please help clarify the correct way to handle the strings? -Andrew -- NOTICE This e-mail and any attachments are confidential and may contain copyright material of Brandscreen or third parties. If you are not the intended recipient of this email you should not read, print, re-transmit, store or act in reliance on this e-mail or any attachments, and should destroy all copies of them. Brandscreen does not guarantee the integrity of any emails or any attached files. The views or opinions expressed are the author's own and may not reflect the views or opinions of Brandscreen. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Tue Mar 11 12:12:35 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 11 Mar 2014 16:12:35 +0400 Subject: ngx_reopen_files() assumes null termination In-Reply-To: References: Message-ID: <20140311121235.GQ34696@mdounin.ru> Hello! On Tue, Mar 11, 2014 at 02:32:52PM +1100, Andrew Punch wrote: > I just noticed that ngx_reopen_files() treats the file name as a null > terminated string but there is no guarantee that a ngx_str_t will be null > terminated. While ngx_str_t strings aren't required to be null-terminated, in some cases terminating null is required, and file names is one of such cases (because file names must be null-terminated for syscalls). There are also some cases where ngx_str_t strings are guaranteed to be null-terminated - e.g., arguments of configuration directives as seen during configuration parsing are null-terminated. Usually there are easy ways to explicitly request null-termination where it may be required but nog guaranteed. E.g., complex values have "zero" flag, which may be set during compilation and results in trailing zero added to values returned at runtime. See auth_basic_user_file implementation which uses it. > I noticed in ngx_string.h that the ngx_string() macro sets the length to be > exclusive of the null termination character, however the null termination > would still be present at s.data[s.len] > > However ngx_pstrdup() in ngx_string.c only copies the size of s.len - so if > there is a null character at s.data[s.len] it will not be copied. > > Could you please help clarify the correct way to handle the strings? Trailing null should be handled explicitly when needed. -- Maxim Dounin http://nginx.org/ From mat999 at gmail.com Thu Mar 13 07:09:43 2014 From: mat999 at gmail.com (SplitIce) Date: Thu, 13 Mar 2014 18:09:43 +1100 Subject: $binary_server_addr Message-ID: Has anyone put any thought into the possibility of a $binary_server_addr variable? Sometimes its quite useful to limit certain events by server_addr instead of the client address (e.g to limit requests made by certain UA's or other events). -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor at sysoev.ru Thu Mar 13 08:10:46 2014 From: igor at sysoev.ru (Igor Sysoev) Date: Thu, 13 Mar 2014 12:10:46 +0400 Subject: $binary_server_addr In-Reply-To: References: Message-ID: <71024075-9496-4B97-BADE-0F5C7191F9AE@sysoev.ru> On Mar 13, 2014, at 11:09 , SplitIce wrote: > Has anyone put any thought into the possibility of a $binary_server_addr variable? > > Sometimes its quite useful to limit certain events by server_addr instead of the client address (e.g to limit requests made by certain UA's or other events). You can use $server_addr. The binary form of $remote_addr saves memory since there is a lot of remote addresses. The number of server addresses is much less so there is no sense to introduce another variable. -- Igor Sysoev http://nginx.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From mat999 at gmail.com Thu Mar 13 08:43:54 2014 From: mat999 at gmail.com (SplitIce) Date: Thu, 13 Mar 2014 19:43:54 +1100 Subject: $binary_server_addr In-Reply-To: <71024075-9496-4B97-BADE-0F5C7191F9AE@sysoev.ru> References: <71024075-9496-4B97-BADE-0F5C7191F9AE@sysoev.ru> Message-ID: I guess our configuration is a but of an edge case pretty much every server serves at-least one /24 (with the intention to raise this to /22 eventually). Anyway thanks for the answer, I am going to go investigate what would be involved in developing a patch for this feature and if it would provide a performance increase in our use case. On Thu, Mar 13, 2014 at 7:10 PM, Igor Sysoev wrote: > On Mar 13, 2014, at 11:09 , SplitIce wrote: > > Has anyone put any thought into the possibility of a $binary_server_addr > variable? > > Sometimes its quite useful to limit certain events by server_addr instead > of the client address (e.g to limit requests made by certain UA's or other > events). > > > You can use $server_addr. The binary form of $remote_addr saves memory > since there is a lot of remote addresses. > The number of server addresses is much less so there is no sense to > introduce another variable. > > > -- > Igor Sysoev > http://nginx.com > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor at sysoev.ru Thu Mar 13 09:08:41 2014 From: igor at sysoev.ru (Igor Sysoev) Date: Thu, 13 Mar 2014 13:08:41 +0400 Subject: $binary_server_addr In-Reply-To: References: <71024075-9496-4B97-BADE-0F5C7191F9AE@sysoev.ru> Message-ID: <74224121-B39C-479B-9D1C-5C712CE17E7D@sysoev.ru> On Mar 13, 2014, at 12:43 , SplitIce wrote: > I guess our configuration is a but of an edge case pretty much every server serves at-least one /24 (with the intention to raise this to /22 eventually). /22, that is, 1024 server addresses. You will save 32K. -- Igor Sysoev http://nginx.com > Anyway thanks for the answer, I am going to go investigate what would be involved in developing a patch for this feature and if it would provide a performance increase in our use case. > > > On Thu, Mar 13, 2014 at 7:10 PM, Igor Sysoev wrote: > On Mar 13, 2014, at 11:09 , SplitIce wrote: > >> Has anyone put any thought into the possibility of a $binary_server_addr variable? >> >> Sometimes its quite useful to limit certain events by server_addr instead of the client address (e.g to limit requests made by certain UA's or other events). > > You can use $server_addr. The binary form of $remote_addr saves memory since there is a lot of remote addresses. > The number of server addresses is much less so there is no sense to introduce another variable. > > > -- > Igor Sysoev > http://nginx.com > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From mat999 at gmail.com Thu Mar 13 09:58:01 2014 From: mat999 at gmail.com (SplitIce) Date: Thu, 13 Mar 2014 20:58:01 +1100 Subject: $binary_server_addr In-Reply-To: <74224121-B39C-479B-9D1C-5C712CE17E7D@sysoev.ru> References: <71024075-9496-4B97-BADE-0F5C7191F9AE@sysoev.ru> <74224121-B39C-479B-9D1C-5C712CE17E7D@sysoev.ru> Message-ID: Thats an interesting figure, certainly worthy of consideration. I am only interested in performance savings, the memory used is negligible as this particular limit takes the brunt of quite a few attacks (insecure Wordpress blogs are quickly becoming the bane of my existence). I am not yet sure if there is room for gains here yet, was mainly curious if thought had been placed in this area (e.g existing patches) for testing prior to a thorough investigation. Thanks, Mathew On Thu, Mar 13, 2014 at 8:08 PM, Igor Sysoev wrote: > On Mar 13, 2014, at 12:43 , SplitIce wrote: > > I guess our configuration is a but of an edge case pretty much every > server serves at-least one /24 (with the intention to raise this to /22 > eventually). > > > /22, that is, 1024 server addresses. You will save 32K. > > > -- > Igor Sysoev > http://nginx.com > > Anyway thanks for the answer, I am going to go investigate what would be > involved in developing a patch for this feature and if it would provide a > performance increase in our use case. > > > On Thu, Mar 13, 2014 at 7:10 PM, Igor Sysoev wrote: > >> On Mar 13, 2014, at 11:09 , SplitIce wrote: >> >> Has anyone put any thought into the possibility of a $binary_server_addr >> variable? >> >> Sometimes its quite useful to limit certain events by server_addr instead >> of the client address (e.g to limit requests made by certain UA's or other >> events). >> >> >> You can use $server_addr. The binary form of $remote_addr saves memory >> since there is a lot of remote addresses. >> The number of server addresses is much less so there is no sense to >> introduce another variable. >> >> >> -- >> Igor Sysoev >> http://nginx.com >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor at sysoev.ru Thu Mar 13 10:25:19 2014 From: igor at sysoev.ru (Igor Sysoev) Date: Thu, 13 Mar 2014 14:25:19 +0400 Subject: $binary_server_addr In-Reply-To: References: <71024075-9496-4B97-BADE-0F5C7191F9AE@sysoev.ru> <74224121-B39C-479B-9D1C-5C712CE17E7D@sysoev.ru> Message-ID: <5AE5D75C-08EC-4FB7-BCAB-E1D1CCD320F7@sysoev.ru> Performance improvement will be negligible, you will not see it in any benchmark. -- Igor Sysoev http://nginx.com On Mar 13, 2014, at 13:58 , SplitIce wrote: > Thats an interesting figure, certainly worthy of consideration. I am only interested in performance savings, the memory used is negligible as this particular limit takes the brunt of quite a few attacks (insecure Wordpress blogs are quickly becoming the bane of my existence). > > I am not yet sure if there is room for gains here yet, was mainly curious if thought had been placed in this area (e.g existing patches) for testing prior to a thorough investigation. > > Thanks, > Mathew > > > On Thu, Mar 13, 2014 at 8:08 PM, Igor Sysoev wrote: > On Mar 13, 2014, at 12:43 , SplitIce wrote: > >> I guess our configuration is a but of an edge case pretty much every server serves at-least one /24 (with the intention to raise this to /22 eventually). > > /22, that is, 1024 server addresses. You will save 32K. > > > -- > Igor Sysoev > http://nginx.com > >> Anyway thanks for the answer, I am going to go investigate what would be involved in developing a patch for this feature and if it would provide a performance increase in our use case. >> >> >> On Thu, Mar 13, 2014 at 7:10 PM, Igor Sysoev wrote: >> On Mar 13, 2014, at 11:09 , SplitIce wrote: >> >>> Has anyone put any thought into the possibility of a $binary_server_addr variable? >>> >>> Sometimes its quite useful to limit certain events by server_addr instead of the client address (e.g to limit requests made by certain UA's or other events). >> >> You can use $server_addr. The binary form of $remote_addr saves memory since there is a lot of remote addresses. >> The number of server addresses is much less so there is no sense to introduce another variable. >> >> >> -- >> Igor Sysoev >> http://nginx.com >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon Mar 17 11:37:48 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 17 Mar 2014 11:37:48 +0000 Subject: [nginx] FastCGI: f->split_parts reset on request start. Message-ID: details: http://hg.nginx.org/nginx/rev/22d485944c20 branches: changeset: 5604:22d485944c20 user: Maxim Dounin date: Mon Mar 17 15:34:36 2014 +0400 description: FastCGI: f->split_parts reset on request start. Additionally, make sure to check for errors from ngx_http_parse_header_line() call after joining saved parts. There shouldn't be any errors, though check may help to catch bugs like missing f->split_parts reset. Reported by Lucas Molas. diffstat: src/http/modules/ngx_http_fastcgi_module.c | 11 +++++++++++ 1 files changed, 11 insertions(+), 0 deletions(-) diffs (28 lines): diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c --- a/src/http/modules/ngx_http_fastcgi_module.c +++ b/src/http/modules/ngx_http_fastcgi_module.c @@ -1208,6 +1208,10 @@ ngx_http_fastcgi_reinit_request(ngx_http f->fastcgi_stdout = 0; f->large_stderr = 0; + if (f->split_parts) { + f->split_parts->nelts = 0; + } + r->state = 0; return NGX_OK; @@ -1488,6 +1492,13 @@ ngx_http_fastcgi_process_header(ngx_http rc = ngx_http_parse_header_line(r, &buf, 1); + if (rc != NGX_OK) { + ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0, + "invalid header after joining " + "FastCGI records"); + return NGX_ERROR; + } + h->key.len = r->header_name_end - r->header_name_start; h->key.data = r->header_name_start; h->key.data[h->key.len] = '\0'; From arut at nginx.com Mon Mar 17 13:43:21 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Mon, 17 Mar 2014 13:43:21 +0000 Subject: [nginx] Added server-side support for PROXY protocol v1 (ticket ... Message-ID: details: http://hg.nginx.org/nginx/rev/3a72b1805c52 branches: changeset: 5605:3a72b1805c52 user: Roman Arutyunyan date: Mon Mar 17 17:41:24 2014 +0400 description: Added server-side support for PROXY protocol v1 (ticket #355). Client address specified in the PROXY protocol header is now saved in the $proxy_protocol_addr variable and can be used in the realip module. This is currently not implemented for mail. diffstat: auto/sources | 6 +- src/core/ngx_connection.h | 2 + src/core/ngx_core.h | 1 + src/core/ngx_proxy_protocol.c | 91 +++++++++++++++++++++++++++++++ src/core/ngx_proxy_protocol.h | 23 +++++++ src/http/modules/ngx_http_realip_module.c | 18 ++++++ src/http/ngx_http.c | 1 + src/http/ngx_http_core_module.c | 5 + src/http/ngx_http_core_module.h | 2 + src/http/ngx_http_request.c | 64 ++++++++++++++++++++- src/http/ngx_http_request.h | 3 +- src/http/ngx_http_spdy.c | 23 +++++++ src/http/ngx_http_variables.c | 19 ++++++ 13 files changed, 252 insertions(+), 6 deletions(-) diffs (truncated from 480 to 300 lines): diff -r 22d485944c20 -r 3a72b1805c52 auto/sources --- a/auto/sources Mon Mar 17 15:34:36 2014 +0400 +++ b/auto/sources Mon Mar 17 17:41:24 2014 +0400 @@ -36,7 +36,8 @@ CORE_DEPS="src/core/nginx.h \ src/core/ngx_conf_file.h \ src/core/ngx_resolver.h \ src/core/ngx_open_file_cache.h \ - src/core/ngx_crypt.h" + src/core/ngx_crypt.h \ + src/core/ngx_proxy_protocol.h" CORE_SRCS="src/core/nginx.c \ @@ -67,7 +68,8 @@ CORE_SRCS="src/core/nginx.c \ src/core/ngx_conf_file.c \ src/core/ngx_resolver.c \ src/core/ngx_open_file_cache.c \ - src/core/ngx_crypt.c" + src/core/ngx_crypt.c \ + src/core/ngx_proxy_protocol.c" REGEX_MODULE=ngx_regex_module diff -r 22d485944c20 -r 3a72b1805c52 src/core/ngx_connection.h --- a/src/core/ngx_connection.h Mon Mar 17 15:34:36 2014 +0400 +++ b/src/core/ngx_connection.h Mon Mar 17 17:41:24 2014 +0400 @@ -139,6 +139,8 @@ struct ngx_connection_s { socklen_t socklen; ngx_str_t addr_text; + ngx_str_t proxy_protocol_addr; + #if (NGX_SSL) ngx_ssl_connection_t *ssl; #endif diff -r 22d485944c20 -r 3a72b1805c52 src/core/ngx_core.h --- a/src/core/ngx_core.h Mon Mar 17 15:34:36 2014 +0400 +++ b/src/core/ngx_core.h Mon Mar 17 17:41:24 2014 +0400 @@ -77,6 +77,7 @@ typedef void (*ngx_connection_handler_pt #include #include #include +#include #define LF (u_char) 10 diff -r 22d485944c20 -r 3a72b1805c52 src/core/ngx_proxy_protocol.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/core/ngx_proxy_protocol.c Mon Mar 17 17:41:24 2014 +0400 @@ -0,0 +1,91 @@ + +/* + * Copyright (C) Roman Arutyunyan + * Copyright (C) Nginx, Inc. + */ + + +#include +#include + + +u_char * +ngx_proxy_protocol_parse(ngx_connection_t *c, u_char *buf, u_char *last) +{ + size_t len; + u_char ch, *p, *addr; + + p = buf; + len = last - buf; + + if (len < 8 || ngx_strncmp(p, "PROXY ", 6) != 0) { + goto invalid; + } + + p += 6; + len -= 6; + + if (len >= 7 && ngx_strncmp(p, "UNKNOWN", 7) == 0) { + ngx_log_debug0(NGX_LOG_DEBUG_CORE, c->log, 0, + "PROXY protocol unknown protocol"); + p += 7; + goto skip; + } + + if (len < 5 || ngx_strncmp(p, "TCP", 3) != 0 + || (p[3] != '4' && p[3] != '6') || p[4] != ' ') + { + goto invalid; + } + + p += 5; + addr = p; + + for ( ;; ) { + if (p == last) { + goto invalid; + } + + ch = *p++; + + if (ch == ' ') { + break; + } + + if (ch != ':' && ch != '.' + && (ch < 'a' || ch > 'f') + && (ch < 'A' || ch > 'F') + && (ch < '0' || ch > '9')) + { + goto invalid; + } + } + + len = p - addr - 1; + c->proxy_protocol_addr.data = ngx_pnalloc(c->pool, len); + + if (c->proxy_protocol_addr.data == NULL) { + return NULL; + } + + ngx_memcpy(c->proxy_protocol_addr.data, addr, len); + c->proxy_protocol_addr.len = len; + + ngx_log_debug1(NGX_LOG_DEBUG_CORE, c->log, 0, + "PROXY protocol address: \"%V\"", &c->proxy_protocol_addr); + +skip: + + for ( /* void */ ; p < last - 1; p++) { + if (p[0] == CR && p[1] == LF) { + return p + 2; + } + } + +invalid: + + ngx_log_error(NGX_LOG_ERR, c->log, 0, + "broken header: \"%*s\"", (size_t) (last - buf), buf); + + return NULL; +} diff -r 22d485944c20 -r 3a72b1805c52 src/core/ngx_proxy_protocol.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/core/ngx_proxy_protocol.h Mon Mar 17 17:41:24 2014 +0400 @@ -0,0 +1,23 @@ + +/* + * Copyright (C) Roman Arutyunyan + * Copyright (C) Nginx, Inc. + */ + + +#ifndef _NGX_PROXY_PROTOCOL_H_INCLUDED_ +#define _NGX_PROXY_PROTOCOL_H_INCLUDED_ + + +#include +#include + + +#define NGX_PROXY_PROTOCOL_MAX_HEADER 107 + + +u_char *ngx_proxy_protocol_parse(ngx_connection_t *c, u_char *buf, + u_char *last); + + +#endif /* _NGX_PROXY_PROTOCOL_H_INCLUDED_ */ diff -r 22d485944c20 -r 3a72b1805c52 src/http/modules/ngx_http_realip_module.c --- a/src/http/modules/ngx_http_realip_module.c Mon Mar 17 15:34:36 2014 +0400 +++ b/src/http/modules/ngx_http_realip_module.c Mon Mar 17 17:41:24 2014 +0400 @@ -13,6 +13,7 @@ #define NGX_HTTP_REALIP_XREALIP 0 #define NGX_HTTP_REALIP_XFWD 1 #define NGX_HTTP_REALIP_HEADER 2 +#define NGX_HTTP_REALIP_PROXY 3 typedef struct { @@ -156,6 +157,18 @@ ngx_http_realip_handler(ngx_http_request break; + case NGX_HTTP_REALIP_PROXY: + + value = &r->connection->proxy_protocol_addr; + + if (value->len == 0) { + return NGX_DECLINED; + } + + xfwd = NULL; + + break; + default: /* NGX_HTTP_REALIP_HEADER */ part = &r->headers_in.headers.part; @@ -343,6 +356,11 @@ ngx_http_realip(ngx_conf_t *cf, ngx_comm return NGX_CONF_OK; } + if (ngx_strcmp(value[1].data, "proxy_protocol") == 0) { + rlcf->type = NGX_HTTP_REALIP_PROXY; + return NGX_CONF_OK; + } + rlcf->type = NGX_HTTP_REALIP_HEADER; rlcf->hash = ngx_hash_strlow(value[1].data, value[1].data, value[1].len); rlcf->header = value[1]; diff -r 22d485944c20 -r 3a72b1805c52 src/http/ngx_http.c --- a/src/http/ngx_http.c Mon Mar 17 15:34:36 2014 +0400 +++ b/src/http/ngx_http.c Mon Mar 17 17:41:24 2014 +0400 @@ -1849,6 +1849,7 @@ ngx_http_add_addrs(ngx_conf_t *cf, ngx_h #if (NGX_HTTP_SPDY) addrs[i].conf.spdy = addr[i].opt.spdy; #endif + addrs[i].conf.proxy_protocol = addr[i].opt.proxy_protocol; if (addr[i].hash.buckets == NULL && (addr[i].wc_head == NULL diff -r 22d485944c20 -r 3a72b1805c52 src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c Mon Mar 17 15:34:36 2014 +0400 +++ b/src/http/ngx_http_core_module.c Mon Mar 17 17:41:24 2014 +0400 @@ -4287,6 +4287,11 @@ ngx_http_core_listen(ngx_conf_t *cf, ngx #endif } + if (ngx_strcmp(value[n].data, "proxy_protocol") == 0) { + lsopt.proxy_protocol = 1; + continue; + } + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid parameter \"%V\"", &value[n]); return NGX_CONF_ERROR; diff -r 22d485944c20 -r 3a72b1805c52 src/http/ngx_http_core_module.h --- a/src/http/ngx_http_core_module.h Mon Mar 17 15:34:36 2014 +0400 +++ b/src/http/ngx_http_core_module.h Mon Mar 17 17:41:24 2014 +0400 @@ -82,6 +82,7 @@ typedef struct { unsigned ipv6only:1; #endif unsigned so_keepalive:2; + unsigned proxy_protocol:1; int backlog; int rcvbuf; @@ -243,6 +244,7 @@ struct ngx_http_addr_conf_s { #if (NGX_HTTP_SPDY) unsigned spdy:1; #endif + unsigned proxy_protocol:1; }; diff -r 22d485944c20 -r 3a72b1805c52 src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c Mon Mar 17 15:34:36 2014 +0400 +++ b/src/http/ngx_http_request.c Mon Mar 17 17:41:24 2014 +0400 @@ -343,6 +343,11 @@ ngx_http_init_connection(ngx_connection_ } #endif + if (hc->addr_conf->proxy_protocol) { + hc->proxy_protocol = 1; + c->log->action = "reading PROXY protocol"; + } + if (rev->ready) { /* the deferred accept(), rtsig, aio, iocp */ @@ -368,6 +373,7 @@ ngx_http_init_connection(ngx_connection_ static void ngx_http_wait_request_handler(ngx_event_t *rev) { + u_char *p; size_t size; ssize_t n; ngx_buf_t *b; @@ -458,6 +464,27 @@ ngx_http_wait_request_handler(ngx_event_ b->last += n; + if (hc->proxy_protocol) { + hc->proxy_protocol = 0; + + p = ngx_proxy_protocol_parse(c, b->pos, b->last); + + if (p == NULL) { + ngx_http_close_connection(c); + return; + } + + b->pos = p; + + if (b->pos == b->last) { + c->log->action = "waiting for request"; + b->pos = b->start; + b->last = b->start; + ngx_post_event(rev, &ngx_posted_events); + return; From flygoast at 126.com Tue Mar 18 01:58:58 2014 From: flygoast at 126.com (flygoast) Date: Tue, 18 Mar 2014 09:58:58 +0800 (CST) Subject: [PATCH]Add processing of posted requests in ngx_http_wait_request_handler(). In-Reply-To: <5ee7524b.aecd.1449b3e6b96.Coremail.flygoast@126.com> References: <5ee7524b.aecd.1449b3e6b96.Coremail.flygoast@126.com> Message-ID: <3249638.d310.144d2e9c23e.Coremail.flygoast@126.com> Is this one worked? At 2014-03-07 14:32:43,flygoast wrote: # HG changeset patch # User FengGu # Date 1394165926 -28800 # Fri Mar 07 12:18:46 2014 +0800 # Node ID c357bd0684560aa2f90f23bdda5dae9cf7f0b13c # Parent e45fa57ef72590d010c8234cb5c3c6e2ae63e157 Add processing of posted requests in ngx_http_wait_request_handler(). If something like "error_page 400 /foo" is used in a configuration, when the new uri processing failed, the ngx_http_terminate_request(r, -1) would be invoked. The request would not be released resulting in socket leak. diff -r e45fa57ef725 -r c357bd068456 src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c Thu Mar 06 23:15:10 2014 +0400 +++ b/src/http/ngx_http_request.c Fri Mar 07 12:18:46 2014 +0800 @@ -470,6 +470,7 @@ rev->handler = ngx_http_process_request_line; ngx_http_process_request_line(rev); + ngx_http_run_posted_requests(c); } -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Tue Mar 18 12:12:30 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 18 Mar 2014 16:12:30 +0400 Subject: [PATCH]Add processing of posted requests in ngx_http_wait_request_handler(). In-Reply-To: <3249638.d310.144d2e9c23e.Coremail.flygoast@126.com> References: <5ee7524b.aecd.1449b3e6b96.Coremail.flygoast@126.com> <3249638.d310.144d2e9c23e.Coremail.flygoast@126.com> Message-ID: <20140318121229.GN34696@mdounin.ru> Hello! On Tue, Mar 18, 2014 at 09:58:58AM +0800, flygoast wrote: > Is this one worked? I haven't had time to look into this yet, sorry. > > > > At 2014-03-07 14:32:43,flygoast wrote: > > # HG changeset patch > # User FengGu > # Date 1394165926 -28800 > # Fri Mar 07 12:18:46 2014 +0800 > # Node ID c357bd0684560aa2f90f23bdda5dae9cf7f0b13c > # Parent e45fa57ef72590d010c8234cb5c3c6e2ae63e157 > Add processing of posted requests in ngx_http_wait_request_handler(). > > > If something like "error_page 400 /foo" is used in a configuration, when the > new uri processing failed, the ngx_http_terminate_request(r, -1) would be > invoked. The request would not be released resulting in socket leak. > > > diff -r e45fa57ef725 -r c357bd068456 src/http/ngx_http_request.c > --- a/src/http/ngx_http_request.c Thu Mar 06 23:15:10 2014 +0400 > +++ b/src/http/ngx_http_request.c Fri Mar 07 12:18:46 2014 +0800 > @@ -470,6 +470,7 @@ > > > rev->handler = ngx_http_process_request_line; > ngx_http_process_request_line(rev); > + ngx_http_run_posted_requests(c); > } > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Maxim Dounin http://nginx.org/ From vbart at nginx.com Tue Mar 18 13:51:26 2014 From: vbart at nginx.com (Valentin V. Bartenev) Date: Tue, 18 Mar 2014 17:51:26 +0400 Subject: [PATCH]Add processing of posted requests in ngx_http_wait_request_handler(). In-Reply-To: <3249638.d310.144d2e9c23e.Coremail.flygoast@126.com> References: <5ee7524b.aecd.1449b3e6b96.Coremail.flygoast@126.com> <3249638.d310.144d2e9c23e.Coremail.flygoast@126.com> Message-ID: <6775959.3IV9U0cdKp@vbart-laptop> On Tuesday 18 March 2014 09:58:58 flygoast wrote: > Is this one worked? > > > At 2014-03-07 14:32:43,flygoast wrote: > > # HG changeset patch > # User FengGu > # Date 1394165926 -28800 > # Fri Mar 07 12:18:46 2014 +0800 > # Node ID c357bd0684560aa2f90f23bdda5dae9cf7f0b13c > # Parent e45fa57ef72590d010c8234cb5c3c6e2ae63e157 > Add processing of posted requests in ngx_http_wait_request_handler(). > > > If something like "error_page 400 /foo" is used in a configuration, when the > new uri processing failed, the ngx_http_terminate_request(r, -1) would be > invoked. The request would not be released resulting in socket leak. > > > diff -r e45fa57ef725 -r c357bd068456 src/http/ngx_http_request.c > --- a/src/http/ngx_http_request.c Thu Mar 06 23:15:10 2014 +0400 > +++ b/src/http/ngx_http_request.c Fri Mar 07 12:18:46 2014 +0800 > @@ -470,6 +470,7 @@ > > > rev->handler = ngx_http_process_request_line; > ngx_http_process_request_line(rev); > + ngx_http_run_posted_requests(c); > } > Seems to me, this is a wrong patch, since ngx_http_wait_request_handler() isn't the only place, where ngx_http_process_request_line() can be invoked. wbr, Valentin V. Bartenev From mdounin at mdounin.ru Tue Mar 18 16:40:32 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 18 Mar 2014 16:40:32 +0000 Subject: [nginx] SPDY: always check size of data to be saved into state b... Message-ID: details: http://hg.nginx.org/nginx/rev/6c7f9fd5fd7e branches: changeset: 5606:6c7f9fd5fd7e user: Maxim Dounin date: Tue Mar 18 17:00:19 2014 +0400 description: SPDY: always check size of data to be saved into state buffer. diffstat: src/http/ngx_http_spdy.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/src/http/ngx_http_spdy.c b/src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c +++ b/src/http/ngx_http_spdy.c @@ -1872,7 +1872,7 @@ static u_char * ngx_http_spdy_state_save(ngx_http_spdy_connection_t *sc, u_char *pos, u_char *end, ngx_http_spdy_handler_pt handler) { -#if (NGX_DEBUG) +#if 1 if (end - pos > NGX_SPDY_STATE_BUFFER_SIZE) { ngx_log_error(NGX_LOG_ALERT, sc->connection->log, 0, "spdy state buffer overflow: " From mdounin at mdounin.ru Tue Mar 18 16:40:33 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 18 Mar 2014 16:40:33 +0000 Subject: [nginx] nginx-1.5.12-RELEASE Message-ID: details: http://hg.nginx.org/nginx/rev/97b47d95e444 branches: changeset: 5607:97b47d95e444 user: Maxim Dounin date: Tue Mar 18 17:08:35 2014 +0400 description: nginx-1.5.12-RELEASE diffstat: docs/xml/nginx/changes.xml | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 46 insertions(+), 0 deletions(-) diffs (56 lines): diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -5,6 +5,52 @@ + + + + +??? ????????? ?????????? ?????????? ??????? ??????? ngx_http_spdy_module +????? ??????????? ???????????? ?????? ? ??????? ????????, +??? ???????????? ????? ????????? ? ?????????? ????????????? ???? +(CVE-2014-0133).
+??????? Lucas Molas ?? Programa STIC, Fundaci?n Dr. Manuel +Sadosky, Buenos Aires, Argentina. +
+ +a heap memory buffer overflow might occur in a worker process +while handling a specially crafted request by ngx_http_spdy_module, +potentially resulting in arbitrary code execution +(CVE-2014-0133).
+Thanks to Lucas Molas, researcher at Programa STIC, Fundaci?n Dr. Manuel +Sadosky, Buenos Aires, Argentina. +
+
+ + + +???????? proxy_protocol ? ?????????? listen ? real_ip_header, +?????????? $proxy_protocol_addr. + + +the "proxy_protocol" parameters of the "listen" and "real_ip_header" directives, +the $proxy_protocol_addr variable. + + + + + +? ????????? fastcgi_next_upstream.
+??????? Lucas Molas. +
+ +in the "fastcgi_next_upstream" directive.
+Thanks to Lucas Molas. +
+
+ +
+ + From mdounin at mdounin.ru Tue Mar 18 16:40:35 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 18 Mar 2014 16:40:35 +0000 Subject: [nginx] release-1.5.12 tag Message-ID: details: http://hg.nginx.org/nginx/rev/659dac82b6f0 branches: changeset: 5608:659dac82b6f0 user: Maxim Dounin date: Tue Mar 18 17:08:35 2014 +0400 description: release-1.5.12 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -367,3 +367,4 @@ eaa76f24975948b0ce8be01838d949122d44ed67 5a1759f33b7fa6270e1617c08d7e655b7b127f26 release-1.5.9 b798fc020e3a84ef68e6c9f47865a319c826d33c release-1.5.10 f995a10d4c7e9a817157a6ce7b753297ad32897e release-1.5.11 +97b47d95e4449cbde976657cf8cbbc118351ffe0 release-1.5.12 From mdounin at mdounin.ru Tue Mar 18 16:40:36 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 18 Mar 2014 16:40:36 +0000 Subject: [nginx] Version bump. Message-ID: details: http://hg.nginx.org/nginx/rev/5ea3bfb4989e branches: stable-1.4 changeset: 5609:5ea3bfb4989e user: Maxim Dounin date: Tue Mar 18 17:12:28 2014 +0400 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff --git a/src/core/nginx.h b/src/core/nginx.h --- a/src/core/nginx.h +++ b/src/core/nginx.h @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1004006 -#define NGINX_VERSION "1.4.6" +#define nginx_version 1004007 +#define NGINX_VERSION "1.4.7" #define NGINX_VER "nginx/" NGINX_VERSION #define NGINX_VAR "NGINX" From mdounin at mdounin.ru Tue Mar 18 16:40:39 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 18 Mar 2014 16:40:39 +0000 Subject: [nginx] FastCGI: f->split_parts reset on request start. Message-ID: details: http://hg.nginx.org/nginx/rev/b246018860a2 branches: stable-1.4 changeset: 5610:b246018860a2 user: Maxim Dounin date: Mon Mar 17 15:34:36 2014 +0400 description: FastCGI: f->split_parts reset on request start. Additionally, make sure to check for errors from ngx_http_parse_header_line() call after joining saved parts. There shouldn't be any errors, though check may help to catch bugs like missing f->split_parts reset. Reported by Lucas Molas. diffstat: src/http/modules/ngx_http_fastcgi_module.c | 11 +++++++++++ 1 files changed, 11 insertions(+), 0 deletions(-) diffs (28 lines): diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c --- a/src/http/modules/ngx_http_fastcgi_module.c +++ b/src/http/modules/ngx_http_fastcgi_module.c @@ -1195,6 +1195,10 @@ ngx_http_fastcgi_reinit_request(ngx_http f->fastcgi_stdout = 0; f->large_stderr = 0; + if (f->split_parts) { + f->split_parts->nelts = 0; + } + r->state = 0; return NGX_OK; @@ -1475,6 +1479,13 @@ ngx_http_fastcgi_process_header(ngx_http rc = ngx_http_parse_header_line(r, &buf, 1); + if (rc != NGX_OK) { + ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0, + "invalid header after joining " + "FastCGI records"); + return NGX_ERROR; + } + h->key.len = r->header_name_end - r->header_name_start; h->key.data = r->header_name_start; h->key.data[h->key.len] = '\0'; From mdounin at mdounin.ru Tue Mar 18 16:40:40 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 18 Mar 2014 16:40:40 +0000 Subject: [nginx] SPDY: always check size of data to be saved into state b... Message-ID: details: http://hg.nginx.org/nginx/rev/e7f7d30196e3 branches: stable-1.4 changeset: 5611:e7f7d30196e3 user: Maxim Dounin date: Tue Mar 18 17:00:19 2014 +0400 description: SPDY: always check size of data to be saved into state buffer. diffstat: src/http/ngx_http_spdy.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/src/http/ngx_http_spdy.c b/src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c +++ b/src/http/ngx_http_spdy.c @@ -1465,7 +1465,7 @@ static u_char * ngx_http_spdy_state_save(ngx_http_spdy_connection_t *sc, u_char *pos, u_char *end, ngx_http_spdy_handler_pt handler) { -#if (NGX_DEBUG) +#if 1 if (end - pos > NGX_SPDY_STATE_BUFFER_SIZE) { ngx_log_error(NGX_LOG_ALERT, sc->connection->log, 0, "spdy state buffer overflow: " From mdounin at mdounin.ru Tue Mar 18 16:40:42 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 18 Mar 2014 16:40:42 +0000 Subject: [nginx] nginx-1.4.7-RELEASE Message-ID: details: http://hg.nginx.org/nginx/rev/636ce03634e7 branches: stable-1.4 changeset: 5612:636ce03634e7 user: Maxim Dounin date: Tue Mar 18 17:17:09 2014 +0400 description: nginx-1.4.7-RELEASE diffstat: docs/xml/nginx/changes.xml | 35 +++++++++++++++++++++++++++++++++++ 1 files changed, 35 insertions(+), 0 deletions(-) diffs (45 lines): diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -5,6 +5,41 @@ + + + + +??? ????????? ?????????? ?????????? ??????? ??????? ngx_http_spdy_module +????? ??????????? ???????????? ?????? ? ??????? ????????, +??? ???????????? ????? ????????? ? ?????????? ????????????? ???? +(CVE-2014-0133).
+??????? Lucas Molas ?? Programa STIC, Fundaci?n Dr. Manuel +Sadosky, Buenos Aires, Argentina. +
+ +a heap memory buffer overflow might occur in a worker process +while handling a specially crafted request by ngx_http_spdy_module, +potentially resulting in arbitrary code execution +(CVE-2014-0133).
+Thanks to Lucas Molas, researcher at Programa STIC, Fundaci?n Dr. Manuel +Sadosky, Buenos Aires, Argentina. +
+
+ + + +? ????????? fastcgi_next_upstream.
+??????? Lucas Molas. +
+ +in the "fastcgi_next_upstream" directive.
+Thanks to Lucas Molas. +
+
+ +
+ + From mdounin at mdounin.ru Tue Mar 18 16:40:43 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 18 Mar 2014 16:40:43 +0000 Subject: [nginx] release-1.4.7 tag Message-ID: details: http://hg.nginx.org/nginx/rev/b39df933f0e6 branches: stable-1.4 changeset: 5613:b39df933f0e6 user: Maxim Dounin date: Tue Mar 18 17:17:09 2014 +0400 description: release-1.4.7 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -361,3 +361,4 @@ 69ffaca7795531e19f3827940cc28dca0b50d0b8 7e9543faf5f0a443ba605d9d483cf4721fae30a5 release-1.4.4 844b2af1d65cbb143e8ecaa1c3ad9968f60d1882 release-1.4.5 e201159f5d4ebe7ddbaf1e9247c843022ec202f3 release-1.4.6 +636ce03634e7489b39a7108f2fed24649df19737 release-1.4.7 From flevionnois at gmail.com Tue Mar 18 17:40:23 2014 From: flevionnois at gmail.com (Franck Levionnois) Date: Tue, 18 Mar 2014 18:40:23 +0100 Subject: [PATCH] Mail: added support for SSL client certificate In-Reply-To: <20140307113147.GY34696@mdounin.ru> References: <5c7ccfc96070fc8b5d77.1392983328@FLEVIONNOIS2.dictao.com> <20140307113147.GY34696@mdounin.ru> Message-ID: Hello, It doesn't seem to exist a standard for this header name. Apache and F5 let the user choose it, but this make the configuration more complicated. I don't think that the name is a problem, because it can be set on the authorization server. If the certificate is transmited, all other informations are duplicated (except Auth-Verify). Forwarding the certificate is the most usefull, because it can be used to make controls on its properties. Kind regards, Franck Levionnois. 2014-03-07 12:31 GMT+01:00 Maxim Dounin : > Hello! > > On Fri, Mar 07, 2014 at 09:40:11AM +0100, Franck Levionnois wrote: > > > Hello, > > I haven't seen any comment on this patch. Is it ok for you ? > > Sorry, I haven't yet had a time to look into it in detail. > > Most problematic part is still auth_http protocol changes - in > particular, headers send and names used for them. I tend to think > there should be better names, and probably we can safely omit some > information as duplicate/unneeded. > > -- > Maxim Dounin > http://nginx.org/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Wed Mar 19 14:37:14 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 19 Mar 2014 14:37:14 +0000 Subject: [nginx] Version bump. Message-ID: details: http://hg.nginx.org/nginx/rev/70607b945c07 branches: changeset: 5614:70607b945c07 user: Maxim Dounin date: Tue Mar 18 22:05:05 2014 +0400 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff --git a/src/core/nginx.h b/src/core/nginx.h --- a/src/core/nginx.h +++ b/src/core/nginx.h @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1005012 -#define NGINX_VERSION "1.5.12" +#define nginx_version 1005013 +#define NGINX_VERSION "1.5.13" #define NGINX_VER "nginx/" NGINX_VERSION #define NGINX_VAR "NGINX" From mdounin at mdounin.ru Wed Mar 19 14:37:16 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 19 Mar 2014 14:37:16 +0000 Subject: [nginx] SPDY: macro used for recv_buffer_size initialization. Message-ID: details: http://hg.nginx.org/nginx/rev/02ee639d18a8 branches: changeset: 5615:02ee639d18a8 user: Maxim Dounin date: Wed Mar 19 12:57:32 2014 +0400 description: SPDY: macro used for recv_buffer_size initialization. diffstat: src/http/ngx_http_spdy_module.c | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diffs (14 lines): diff --git a/src/http/ngx_http_spdy_module.c b/src/http/ngx_http_spdy_module.c --- a/src/http/ngx_http_spdy_module.c +++ b/src/http/ngx_http_spdy_module.c @@ -251,9 +251,7 @@ ngx_http_spdy_init_main_conf(ngx_conf_t { ngx_http_spdy_main_conf_t *smcf = conf; - if (smcf->recv_buffer_size == NGX_CONF_UNSET_SIZE) { - smcf->recv_buffer_size = 256 * 1024; - } + ngx_conf_init_size_value(smcf->recv_buffer_size, 256 * 1024); return NGX_CONF_OK; } From mdounin at mdounin.ru Wed Mar 19 14:37:18 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 19 Mar 2014 14:37:18 +0000 Subject: [nginx] SPDY: improved ngx_http_spdy_state_save() check. Message-ID: details: http://hg.nginx.org/nginx/rev/b93db4e4ff77 branches: changeset: 5616:b93db4e4ff77 user: Maxim Dounin date: Wed Mar 19 12:57:39 2014 +0400 description: SPDY: improved ngx_http_spdy_state_save() check. diffstat: src/http/ngx_http_spdy.c | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diffs (23 lines): diff --git a/src/http/ngx_http_spdy.c b/src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c +++ b/src/http/ngx_http_spdy.c @@ -1872,14 +1872,16 @@ static u_char * ngx_http_spdy_state_save(ngx_http_spdy_connection_t *sc, u_char *pos, u_char *end, ngx_http_spdy_handler_pt handler) { -#if 1 - if (end - pos > NGX_SPDY_STATE_BUFFER_SIZE) { + size_t size; + + size = end - pos; + + if (size > NGX_SPDY_STATE_BUFFER_SIZE) { ngx_log_error(NGX_LOG_ALERT, sc->connection->log, 0, "spdy state buffer overflow: " "%z bytes required", end - pos); return ngx_http_spdy_state_internal_error(sc); } -#endif ngx_memcpy(sc->buffer, pos, NGX_SPDY_STATE_BUFFER_SIZE); From mdounin at mdounin.ru Wed Mar 19 14:37:19 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 19 Mar 2014 14:37:19 +0000 Subject: [nginx] Macros used for initialization in ngx_http_core_init_mai... Message-ID: details: http://hg.nginx.org/nginx/rev/68c6cc832c76 branches: changeset: 5617:68c6cc832c76 user: Maxim Dounin date: Wed Mar 19 12:57:40 2014 +0400 description: Macros used for initialization in ngx_http_core_init_main_conf(). diffstat: src/http/ngx_http_core_module.c | 19 +++++-------------- 1 files changed, 5 insertions(+), 14 deletions(-) diffs (34 lines): diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c +++ b/src/http/ngx_http_core_module.c @@ -3432,25 +3432,16 @@ ngx_http_core_init_main_conf(ngx_conf_t { ngx_http_core_main_conf_t *cmcf = conf; - if (cmcf->server_names_hash_max_size == NGX_CONF_UNSET_UINT) { - cmcf->server_names_hash_max_size = 512; - } - - if (cmcf->server_names_hash_bucket_size == NGX_CONF_UNSET_UINT) { - cmcf->server_names_hash_bucket_size = ngx_cacheline_size; - } + ngx_conf_init_uint_value(cmcf->server_names_hash_max_size, 512); + ngx_conf_init_uint_value(cmcf->server_names_hash_bucket_size, + ngx_cacheline_size); cmcf->server_names_hash_bucket_size = ngx_align(cmcf->server_names_hash_bucket_size, ngx_cacheline_size); - if (cmcf->variables_hash_max_size == NGX_CONF_UNSET_UINT) { - cmcf->variables_hash_max_size = 512; - } - - if (cmcf->variables_hash_bucket_size == NGX_CONF_UNSET_UINT) { - cmcf->variables_hash_bucket_size = 64; - } + ngx_conf_init_uint_value(cmcf->variables_hash_max_size, 512); + ngx_conf_init_uint_value(cmcf->variables_hash_bucket_size, 64); cmcf->variables_hash_bucket_size = ngx_align(cmcf->variables_hash_bucket_size, ngx_cacheline_size); From mdounin at mdounin.ru Wed Mar 19 15:30:54 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 19 Mar 2014 15:30:54 +0000 Subject: [nginx] SPDY: improved ngx_http_spdy_state_save() again. Message-ID: details: http://hg.nginx.org/nginx/rev/c15350f4071c branches: changeset: 5618:c15350f4071c user: Maxim Dounin date: Wed Mar 19 19:30:09 2014 +0400 description: SPDY: improved ngx_http_spdy_state_save() again. diffstat: src/http/ngx_http_spdy.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (19 lines): diff --git a/src/http/ngx_http_spdy.c b/src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c +++ b/src/http/ngx_http_spdy.c @@ -1879,13 +1879,13 @@ ngx_http_spdy_state_save(ngx_http_spdy_c if (size > NGX_SPDY_STATE_BUFFER_SIZE) { ngx_log_error(NGX_LOG_ALERT, sc->connection->log, 0, "spdy state buffer overflow: " - "%z bytes required", end - pos); + "%uz bytes required", size); return ngx_http_spdy_state_internal_error(sc); } ngx_memcpy(sc->buffer, pos, NGX_SPDY_STATE_BUFFER_SIZE); - sc->buffer_used = end - pos; + sc->buffer_used = size; sc->handler = handler; sc->incomplete = 1; From piotr at cloudflare.com Thu Mar 20 01:56:10 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 19 Mar 2014 18:56:10 -0700 Subject: [PATCH] Parse: underscores_in_headers should apply to the first character as well Message-ID: <51d996a6bc65c4ca3c66.1395280570@piotrs-macbook-pro.local> # HG changeset patch # User Piotr Sikora # Date 1395280497 25200 # Wed Mar 19 18:54:57 2014 -0700 # Node ID 51d996a6bc65c4ca3c6621e4ad18235e6aa26abe # Parent c15350f4071c0207cdffbc8bbb3f28e8e57f4d86 Parse: underscores_in_headers should apply to the first character as well. Signed-off-by: Piotr Sikora diff -r c15350f4071c -r 51d996a6bc65 src/http/ngx_http_parse.c --- a/src/http/ngx_http_parse.c Wed Mar 19 19:30:09 2014 +0400 +++ b/src/http/ngx_http_parse.c Wed Mar 19 18:54:57 2014 -0700 @@ -886,6 +886,19 @@ ngx_http_parse_header_line(ngx_http_requ break; } + if (ch == '_') { + if (allow_underscores) { + hash = ngx_hash(hash, ch); + r->lowcase_header[0] = ch; + i = 1; + + } else { + r->invalid_header = 1; + } + + break; + } + if (ch == '\0') { return NGX_HTTP_PARSE_INVALID_HEADER; } From arut at nginx.com Thu Mar 20 12:06:50 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Thu, 20 Mar 2014 12:06:50 +0000 Subject: [nginx] Mp4: moved atom cropping code out of update functions. Message-ID: details: http://hg.nginx.org/nginx/rev/517b5b599e3f branches: changeset: 5619:517b5b599e3f user: Roman Arutyunyan date: Thu Mar 20 16:05:18 2014 +0400 description: Mp4: moved atom cropping code out of update functions. It can now be reused for implementing mp4 end. diffstat: src/http/modules/ngx_http_mp4_module.c | 281 +++++++++++++++++++++----------- 1 files changed, 186 insertions(+), 95 deletions(-) diffs (truncated from 429 to 300 lines): diff -r c15350f4071c -r 517b5b599e3f src/http/modules/ngx_http_mp4_module.c --- a/src/http/modules/ngx_http_mp4_module.c Wed Mar 19 19:30:09 2014 +0400 +++ b/src/http/modules/ngx_http_mp4_module.c Thu Mar 20 16:05:18 2014 +0400 @@ -258,18 +258,26 @@ static ngx_int_t ngx_http_mp4_read_stts_ uint64_t atom_data_size); static ngx_int_t ngx_http_mp4_update_stts_atom(ngx_http_mp4_file_t *mp4, ngx_http_mp4_trak_t *trak); +static ngx_int_t ngx_http_mp4_crop_stts_data(ngx_http_mp4_file_t *mp4, + ngx_http_mp4_trak_t *trak); static ngx_int_t ngx_http_mp4_read_stss_atom(ngx_http_mp4_file_t *mp4, uint64_t atom_data_size); static ngx_int_t ngx_http_mp4_update_stss_atom(ngx_http_mp4_file_t *mp4, ngx_http_mp4_trak_t *trak); +static void ngx_http_mp4_crop_stss_data(ngx_http_mp4_file_t *mp4, + ngx_http_mp4_trak_t *trak); static ngx_int_t ngx_http_mp4_read_ctts_atom(ngx_http_mp4_file_t *mp4, uint64_t atom_data_size); static void ngx_http_mp4_update_ctts_atom(ngx_http_mp4_file_t *mp4, ngx_http_mp4_trak_t *trak); +static void ngx_http_mp4_crop_ctts_data(ngx_http_mp4_file_t *mp4, + ngx_http_mp4_trak_t *trak); static ngx_int_t ngx_http_mp4_read_stsc_atom(ngx_http_mp4_file_t *mp4, uint64_t atom_data_size); static ngx_int_t ngx_http_mp4_update_stsc_atom(ngx_http_mp4_file_t *mp4, ngx_http_mp4_trak_t *trak); +static ngx_int_t ngx_http_mp4_crop_stsc_data(ngx_http_mp4_file_t *mp4, + ngx_http_mp4_trak_t *trak); static ngx_int_t ngx_http_mp4_read_stsz_atom(ngx_http_mp4_file_t *mp4, uint64_t atom_data_size); static ngx_int_t ngx_http_mp4_update_stsz_atom(ngx_http_mp4_file_t *mp4, @@ -1981,13 +1989,9 @@ static ngx_int_t ngx_http_mp4_update_stts_atom(ngx_http_mp4_file_t *mp4, ngx_http_mp4_trak_t *trak) { - size_t atom_size; - uint32_t entries, count, duration; - uint64_t start_time; - ngx_buf_t *atom, *data; - ngx_uint_t start_sample; - ngx_mp4_stts_atom_t *stts_atom; - ngx_mp4_stts_entry_t *entry, *end; + size_t atom_size; + ngx_buf_t *atom, *data; + ngx_mp4_stts_atom_t *stts_atom; /* * mdia.minf.stbl.stts updating requires trak->timescale @@ -2006,12 +2010,40 @@ ngx_http_mp4_update_stts_atom(ngx_http_m return NGX_ERROR; } - entries = trak->time_to_sample_entries; + if (ngx_http_mp4_crop_stts_data(mp4, trak) != NGX_OK) { + return NGX_ERROR; + } + + atom_size = sizeof(ngx_mp4_stts_atom_t) + (data->last - data->pos); + trak->size += atom_size; + + atom = trak->out[NGX_HTTP_MP4_STTS_ATOM].buf; + stts_atom = (ngx_mp4_stts_atom_t *) atom->pos; + ngx_mp4_set_32value(stts_atom->size, atom_size); + ngx_mp4_set_32value(stts_atom->entries, trak->time_to_sample_entries); + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_mp4_crop_stts_data(ngx_http_mp4_file_t *mp4, + ngx_http_mp4_trak_t *trak) +{ + uint32_t count, duration; + uint64_t start_time; + ngx_buf_t *data; + ngx_uint_t start_sample, entries; + ngx_mp4_stts_entry_t *entry, *end; + + data = trak->out[NGX_HTTP_MP4_STTS_DATA].buf; + start_time = (uint64_t) mp4->start * trak->timescale / 1000; ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, "time-to-sample start_time:%uL", start_time); + entries = trak->time_to_sample_entries; start_sample = 0; entry = (ngx_mp4_stts_entry_t *) data->pos; end = (ngx_mp4_stts_entry_t *) data->last; @@ -2047,16 +2079,9 @@ found: ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, "start_sample:%ui, new count:%uD", start_sample, count); + trak->time_to_sample_entries = entries; trak->start_sample = start_sample; - data->pos = (u_char *) entry; - atom_size = sizeof(ngx_mp4_stts_atom_t) + (data->last - data->pos); - trak->size += atom_size; - - atom = trak->out[NGX_HTTP_MP4_STTS_ATOM].buf; - stts_atom = (ngx_mp4_stts_atom_t *) atom->pos; - ngx_mp4_set_32value(stts_atom->size, atom_size); - ngx_mp4_set_32value(stts_atom->entries, entries); return NGX_OK; } @@ -2138,7 +2163,7 @@ ngx_http_mp4_update_stss_atom(ngx_http_m ngx_http_mp4_trak_t *trak) { size_t atom_size; - uint32_t entries, sample, start_sample, *entry, *end; + uint32_t sample, start_sample, *entry, *end; ngx_buf_t *atom, *data; ngx_http_mp4_stss_atom_t *stss_atom; @@ -2157,34 +2182,11 @@ ngx_http_mp4_update_stss_atom(ngx_http_m return NGX_OK; } - /* sync samples starts from 1 */ - start_sample = trak->start_sample + 1; - entries = trak->sync_samples_entries; + ngx_http_mp4_crop_stss_data(mp4, trak); entry = (uint32_t *) data->pos; end = (uint32_t *) data->last; - while (entry < end) { - sample = ngx_mp4_get_32value(entry); - - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "start:%uD, sync:%uD", start_sample, sample); - - if (sample >= start_sample) { - goto found; - } - - entries--; - entry++; - } - - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "start sample is out of mp4 stss atom"); - -found: - - data->pos = (u_char *) entry; - start_sample = trak->start_sample; while (entry < end) { @@ -2201,12 +2203,53 @@ found: stss_atom = (ngx_http_mp4_stss_atom_t *) atom->pos; ngx_mp4_set_32value(stss_atom->size, atom_size); - ngx_mp4_set_32value(stss_atom->entries, entries); + ngx_mp4_set_32value(stss_atom->entries, trak->sync_samples_entries); return NGX_OK; } +static void +ngx_http_mp4_crop_stss_data(ngx_http_mp4_file_t *mp4, + ngx_http_mp4_trak_t *trak) +{ + uint32_t sample, start_sample, *entry, *end; + ngx_buf_t *data; + ngx_uint_t entries; + + data = trak->out[NGX_HTTP_MP4_STSS_DATA].buf; + + /* sync samples starts from 1 */ + start_sample = trak->start_sample + 1; + entries = trak->sync_samples_entries; + + entry = (uint32_t *) data->pos; + end = (uint32_t *) data->last; + + while (entry < end) { + sample = ngx_mp4_get_32value(entry); + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "start:%uD, sync:%uD", start_sample, sample); + + if (sample >= start_sample) { + goto found; + } + + entries--; + entry++; + } + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "start sample is out of mp4 stss atom"); + +found: + + data->pos = (u_char *) entry; + trak->sync_samples_entries = entries; +} + + typedef struct { u_char size[4]; u_char name[4]; @@ -2287,11 +2330,9 @@ static void ngx_http_mp4_update_ctts_atom(ngx_http_mp4_file_t *mp4, ngx_http_mp4_trak_t *trak) { - size_t atom_size; - uint32_t entries, count, start_sample; - ngx_buf_t *atom, *data; - ngx_mp4_ctts_atom_t *ctts_atom; - ngx_mp4_ctts_entry_t *entry, *end; + size_t atom_size; + ngx_buf_t *atom, *data; + ngx_mp4_ctts_atom_t *ctts_atom; /* * mdia.minf.stbl.ctts updating requires trak->start_sample @@ -2308,6 +2349,38 @@ ngx_http_mp4_update_ctts_atom(ngx_http_m return; } + ngx_http_mp4_crop_ctts_data(mp4, trak); + + if (trak->composition_offset_entries == 0) { + trak->out[NGX_HTTP_MP4_CTTS_ATOM].buf = NULL; + trak->out[NGX_HTTP_MP4_CTTS_DATA].buf = NULL; + return; + } + + atom_size = sizeof(ngx_mp4_ctts_atom_t) + (data->last - data->pos); + trak->size += atom_size; + + atom = trak->out[NGX_HTTP_MP4_CTTS_ATOM].buf; + ctts_atom = (ngx_mp4_ctts_atom_t *) atom->pos; + + ngx_mp4_set_32value(ctts_atom->size, atom_size); + ngx_mp4_set_32value(ctts_atom->entries, trak->composition_offset_entries); + + return; +} + + +static void +ngx_http_mp4_crop_ctts_data(ngx_http_mp4_file_t *mp4, + ngx_http_mp4_trak_t *trak) +{ + uint32_t count, start_sample; + ngx_buf_t *data; + ngx_uint_t entries; + ngx_mp4_ctts_entry_t *entry, *end; + + data = trak->out[NGX_HTTP_MP4_CTTS_DATA].buf; + /* sync samples starts from 1 */ start_sample = trak->start_sample + 1; entries = trak->composition_offset_entries; @@ -2332,24 +2405,15 @@ ngx_http_mp4_update_ctts_atom(ngx_http_m entry++; } - trak->out[NGX_HTTP_MP4_CTTS_ATOM].buf = NULL; - trak->out[NGX_HTTP_MP4_CTTS_DATA].buf = NULL; + data->pos = (u_char *) end; + trak->composition_offset_entries = 0; return; found: data->pos = (u_char *) entry; - atom_size = sizeof(ngx_mp4_ctts_atom_t) + (data->last - data->pos); - trak->size += atom_size; - - atom = trak->out[NGX_HTTP_MP4_CTTS_ATOM].buf; - ctts_atom = (ngx_mp4_ctts_atom_t *) atom->pos; - - ngx_mp4_set_32value(ctts_atom->size, atom_size); - ngx_mp4_set_32value(ctts_atom->entries, entries); - - return; + trak->composition_offset_entries = entries; } @@ -2428,11 +2492,10 @@ ngx_http_mp4_update_stsc_atom(ngx_http_m ngx_http_mp4_trak_t *trak) { size_t atom_size; - uint32_t start_sample, entries, chunk, samples, id, - next_chunk, n; - ngx_buf_t *atom, *data, *buf; + uint32_t chunk; + ngx_buf_t *atom, *data; ngx_mp4_stsc_atom_t *stsc_atom; - ngx_mp4_stsc_entry_t *entry, *first, *end; From arut at nginx.com Thu Mar 20 12:06:51 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Thu, 20 Mar 2014 12:06:51 +0000 Subject: [nginx] Mp4: added "end" argument support. Message-ID: details: http://hg.nginx.org/nginx/rev/0a567878254b branches: changeset: 5620:0a567878254b user: Roman Arutyunyan date: Thu Mar 20 16:05:19 2014 +0400 description: Mp4: added "end" argument support. diffstat: src/http/modules/ngx_http_mp4_module.c | 490 ++++++++++++++++++++++++++------ 1 files changed, 389 insertions(+), 101 deletions(-) diffs (truncated from 931 to 300 lines): diff -r 517b5b599e3f -r 0a567878254b src/http/modules/ngx_http_mp4_module.c --- a/src/http/modules/ngx_http_mp4_module.c Thu Mar 20 16:05:18 2014 +0400 +++ b/src/http/modules/ngx_http_mp4_module.c Thu Mar 20 16:05:19 2014 +0400 @@ -27,14 +27,15 @@ #define NGX_HTTP_MP4_CTTS_ATOM 15 #define NGX_HTTP_MP4_CTTS_DATA 16 #define NGX_HTTP_MP4_STSC_ATOM 17 -#define NGX_HTTP_MP4_STSC_CHUNK 18 +#define NGX_HTTP_MP4_STSC_START 18 #define NGX_HTTP_MP4_STSC_DATA 19 -#define NGX_HTTP_MP4_STSZ_ATOM 20 -#define NGX_HTTP_MP4_STSZ_DATA 21 -#define NGX_HTTP_MP4_STCO_ATOM 22 -#define NGX_HTTP_MP4_STCO_DATA 23 -#define NGX_HTTP_MP4_CO64_ATOM 24 -#define NGX_HTTP_MP4_CO64_DATA 25 +#define NGX_HTTP_MP4_STSC_END 20 +#define NGX_HTTP_MP4_STSZ_ATOM 21 +#define NGX_HTTP_MP4_STSZ_DATA 22 +#define NGX_HTTP_MP4_STCO_ATOM 23 +#define NGX_HTTP_MP4_STCO_DATA 24 +#define NGX_HTTP_MP4_CO64_ATOM 25 +#define NGX_HTTP_MP4_CO64_DATA 26 #define NGX_HTTP_MP4_LAST_ATOM NGX_HTTP_MP4_CO64_DATA @@ -62,10 +63,15 @@ typedef struct { uint32_t chunks; ngx_uint_t start_sample; + ngx_uint_t end_sample; ngx_uint_t start_chunk; - ngx_uint_t chunk_samples; - uint64_t chunk_samples_size; + ngx_uint_t end_chunk; + ngx_uint_t start_chunk_samples; + ngx_uint_t end_chunk_samples; + uint64_t start_chunk_samples_size; + uint64_t end_chunk_samples_size; off_t start_offset; + off_t end_offset; size_t tkhd_size; size_t mdhd_size; @@ -95,7 +101,8 @@ typedef struct { ngx_buf_t ctts_atom_buf; ngx_buf_t ctts_data_buf; ngx_buf_t stsc_atom_buf; - ngx_buf_t stsc_chunk_buf; + ngx_buf_t stsc_start_chunk_buf; + ngx_buf_t stsc_end_chunk_buf; ngx_buf_t stsc_data_buf; ngx_buf_t stsz_atom_buf; ngx_buf_t stsz_data_buf; @@ -104,7 +111,8 @@ typedef struct { ngx_buf_t co64_atom_buf; ngx_buf_t co64_data_buf; - ngx_mp4_stsc_entry_t stsc_chunk_entry; + ngx_mp4_stsc_entry_t stsc_start_chunk_entry; + ngx_mp4_stsc_entry_t stsc_end_chunk_entry; } ngx_http_mp4_trak_t; @@ -121,6 +129,7 @@ typedef struct { off_t end; off_t content_length; ngx_uint_t start; + ngx_uint_t length; uint32_t timescale; ngx_http_request_t *request; ngx_array_t trak; @@ -219,7 +228,7 @@ static ngx_int_t ngx_http_mp4_read_moov_ static ngx_int_t ngx_http_mp4_read_mdat_atom(ngx_http_mp4_file_t *mp4, uint64_t atom_data_size); static size_t ngx_http_mp4_update_mdat_atom(ngx_http_mp4_file_t *mp4, - off_t start_offset); + off_t start_offset, off_t end_offset); static ngx_int_t ngx_http_mp4_read_mvhd_atom(ngx_http_mp4_file_t *mp4, uint64_t atom_data_size); static ngx_int_t ngx_http_mp4_read_trak_atom(ngx_http_mp4_file_t *mp4, @@ -259,25 +268,25 @@ static ngx_int_t ngx_http_mp4_read_stts_ static ngx_int_t ngx_http_mp4_update_stts_atom(ngx_http_mp4_file_t *mp4, ngx_http_mp4_trak_t *trak); static ngx_int_t ngx_http_mp4_crop_stts_data(ngx_http_mp4_file_t *mp4, - ngx_http_mp4_trak_t *trak); + ngx_http_mp4_trak_t *trak, ngx_uint_t start); static ngx_int_t ngx_http_mp4_read_stss_atom(ngx_http_mp4_file_t *mp4, uint64_t atom_data_size); static ngx_int_t ngx_http_mp4_update_stss_atom(ngx_http_mp4_file_t *mp4, ngx_http_mp4_trak_t *trak); static void ngx_http_mp4_crop_stss_data(ngx_http_mp4_file_t *mp4, - ngx_http_mp4_trak_t *trak); + ngx_http_mp4_trak_t *trak, ngx_uint_t start); static ngx_int_t ngx_http_mp4_read_ctts_atom(ngx_http_mp4_file_t *mp4, uint64_t atom_data_size); static void ngx_http_mp4_update_ctts_atom(ngx_http_mp4_file_t *mp4, ngx_http_mp4_trak_t *trak); static void ngx_http_mp4_crop_ctts_data(ngx_http_mp4_file_t *mp4, - ngx_http_mp4_trak_t *trak); + ngx_http_mp4_trak_t *trak, ngx_uint_t start); static ngx_int_t ngx_http_mp4_read_stsc_atom(ngx_http_mp4_file_t *mp4, uint64_t atom_data_size); static ngx_int_t ngx_http_mp4_update_stsc_atom(ngx_http_mp4_file_t *mp4, ngx_http_mp4_trak_t *trak); static ngx_int_t ngx_http_mp4_crop_stsc_data(ngx_http_mp4_file_t *mp4, - ngx_http_mp4_trak_t *trak); + ngx_http_mp4_trak_t *trak, ngx_uint_t start); static ngx_int_t ngx_http_mp4_read_stsz_atom(ngx_http_mp4_file_t *mp4, uint64_t atom_data_size); static ngx_int_t ngx_http_mp4_update_stsz_atom(ngx_http_mp4_file_t *mp4, @@ -411,8 +420,8 @@ ngx_http_mp4_handler(ngx_http_request_t { u_char *last; size_t root; - ngx_int_t rc, start; - ngx_uint_t level; + ngx_int_t rc, start, end; + ngx_uint_t level, length; ngx_str_t path, value; ngx_log_t *log; ngx_buf_t *b; @@ -517,6 +526,7 @@ ngx_http_mp4_handler(ngx_http_request_t r->allow_ranges = 1; start = -1; + length = 0; r->headers_out.content_length_n = of.size; mp4 = NULL; b = NULL; @@ -538,6 +548,26 @@ ngx_http_mp4_handler(ngx_http_request_t start = -1; } } + + if (ngx_http_arg(r, (u_char *) "end", 3, &value) == NGX_OK) { + + ngx_set_errno(0); + end = (int) (strtod((char *) value.data, NULL) * 1000); + + if (ngx_errno != 0) { + end = -1; + } + + if (end > 0) { + if (start < 0) { + start = 0; + } + + if (end > start) { + length = end - start; + } + } + } } if (start >= 0) { @@ -553,6 +583,7 @@ ngx_http_mp4_handler(ngx_http_request_t mp4->file.log = r->connection->log; mp4->end = of.size; mp4->start = (ngx_uint_t) start; + mp4->length = length; mp4->request = r; switch (ngx_http_mp4_process(mp4)) { @@ -658,15 +689,15 @@ ngx_http_mp4_handler(ngx_http_request_t static ngx_int_t ngx_http_mp4_process(ngx_http_mp4_file_t *mp4) { - off_t start_offset, adjustment; + off_t start_offset, end_offset, adjustment; ngx_int_t rc; ngx_uint_t i, j; ngx_chain_t **prev; ngx_http_mp4_trak_t *trak; ngx_http_mp4_conf_t *conf; - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "mp4 start:%ui", mp4->start); + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "mp4 start:%ui, length:%ui", mp4->start, mp4->length); conf = ngx_http_get_module_loc_conf(mp4->request, ngx_http_mp4_module); @@ -708,6 +739,7 @@ ngx_http_mp4_process(ngx_http_mp4_file_t } start_offset = mp4->end; + end_offset = 0; trak = mp4->trak.elts; for (i = 0; i < mp4->trak.nelts; i++) { @@ -755,6 +787,10 @@ ngx_http_mp4_process(ngx_http_mp4_file_t start_offset = trak[i].start_offset; } + if (end_offset < trak[i].end_offset) { + end_offset = trak[i].end_offset; + } + *prev = &trak[i].out[NGX_HTTP_MP4_TRAK_ATOM]; prev = &trak[i].out[NGX_HTTP_MP4_TRAK_ATOM].next; @@ -766,6 +802,10 @@ ngx_http_mp4_process(ngx_http_mp4_file_t } } + if (end_offset < start_offset) { + end_offset = start_offset; + } + mp4->moov_size += 8; ngx_mp4_set_32value(mp4->moov_atom_header, mp4->moov_size); @@ -782,7 +822,7 @@ ngx_http_mp4_process(ngx_http_mp4_file_t } adjustment = mp4->ftyp_size + mp4->moov_size - + ngx_http_mp4_update_mdat_atom(mp4, start_offset) + + ngx_http_mp4_update_mdat_atom(mp4, start_offset, end_offset) - start_offset; ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, @@ -1026,7 +1066,7 @@ ngx_http_mp4_read_moov_atom(ngx_http_mp4 no_mdat = (mp4->mdat_atom.buf == NULL); - if (no_mdat && mp4->start == 0) { + if (no_mdat && mp4->start == 0 && mp4->length == 0) { /* * send original file if moov atom resides before * mdat atom and client requests integral file @@ -1125,7 +1165,8 @@ ngx_http_mp4_read_mdat_atom(ngx_http_mp4 static size_t -ngx_http_mp4_update_mdat_atom(ngx_http_mp4_file_t *mp4, off_t start_offset) +ngx_http_mp4_update_mdat_atom(ngx_http_mp4_file_t *mp4, off_t start_offset, + off_t end_offset) { off_t atom_data_size; u_char *atom_header; @@ -1133,8 +1174,9 @@ ngx_http_mp4_update_mdat_atom(ngx_http_m uint64_t atom_size; ngx_buf_t *atom; - atom_data_size = mp4->mdat_data.buf->file_last - start_offset; + atom_data_size = end_offset - start_offset; mp4->mdat_data.buf->file_pos = start_offset; + mp4->mdat_data.buf->file_last = end_offset; ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, "mdat new offset @%O:%O", start_offset, atom_data_size); @@ -1216,7 +1258,7 @@ ngx_http_mp4_read_mvhd_atom(ngx_http_mp4 u_char *atom_header; size_t atom_size; uint32_t timescale; - uint64_t duration, start_time; + uint64_t duration, start_time, length_time; ngx_buf_t *atom; ngx_mp4_mvhd_atom_t *mvhd_atom; ngx_mp4_mvhd64_atom_t *mvhd64_atom; @@ -1270,6 +1312,14 @@ ngx_http_mp4_read_mvhd_atom(ngx_http_mp4 duration -= start_time; + if (mp4->length) { + length_time = (uint64_t) mp4->length * timescale / 1000; + + if (duration > length_time) { + duration = length_time; + } + } + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, "mvhd new duration:%uL, time:%.3fs", duration, (double) duration / timescale); @@ -1415,7 +1465,7 @@ ngx_http_mp4_read_tkhd_atom(ngx_http_mp4 { u_char *atom_header; size_t atom_size; - uint64_t duration, start_time; + uint64_t duration, start_time, length_time; ngx_buf_t *atom; ngx_http_mp4_trak_t *trak; ngx_mp4_tkhd_atom_t *tkhd_atom; @@ -1465,6 +1515,14 @@ ngx_http_mp4_read_tkhd_atom(ngx_http_mp4 duration -= start_time; + if (mp4->length) { + length_time = (uint64_t) mp4->length * mp4->timescale / 1000; + + if (duration > length_time) { + duration = length_time; + } + } + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, "tkhd new duration:%uL, time:%.3fs", From mdounin at mdounin.ru Fri Mar 21 15:37:57 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 21 Mar 2014 15:37:57 +0000 Subject: [nginx] Range filter: single_range flag in request. Message-ID: details: http://hg.nginx.org/nginx/rev/345e4fd4bb64 branches: changeset: 5621:345e4fd4bb64 user: Maxim Dounin date: Fri Mar 21 19:33:21 2014 +0400 description: Range filter: single_range flag in request. If set, it means that response body is going to be in more than one buffer, hence only range requests with a single range should be honored. The flag is now used by mp4 and cacheable upstream responses, thus allowing range requests of mp4 files with start/end, as well as range processing on a first request to a not-yet-cached files with proxy_cache. Notably this makes it possible to play mp4 files (with proxy_cache, or with mp4 module) on iOS devices, as byte-range support is required by Apple. diffstat: src/http/modules/ngx_http_mp4_module.c | 2 +- src/http/modules/ngx_http_range_filter_module.c | 5 ++++- src/http/ngx_http_request.h | 1 + src/http/ngx_http_upstream.c | 7 ++++++- 4 files changed, 12 insertions(+), 3 deletions(-) diffs (62 lines): diff --git a/src/http/modules/ngx_http_mp4_module.c b/src/http/modules/ngx_http_mp4_module.c --- a/src/http/modules/ngx_http_mp4_module.c +++ b/src/http/modules/ngx_http_mp4_module.c @@ -571,7 +571,7 @@ ngx_http_mp4_handler(ngx_http_request_t } if (start >= 0) { - r->allow_ranges = 0; + r->single_range = 1; mp4 = ngx_pcalloc(r->pool, sizeof(ngx_http_mp4_file_t)); if (mp4 == NULL) { diff --git a/src/http/modules/ngx_http_range_filter_module.c b/src/http/modules/ngx_http_range_filter_module.c --- a/src/http/modules/ngx_http_range_filter_module.c +++ b/src/http/modules/ngx_http_range_filter_module.c @@ -148,6 +148,7 @@ ngx_http_range_header_filter(ngx_http_re { time_t if_range_time; ngx_str_t *if_range, *etag; + ngx_uint_t ranges; ngx_http_core_loc_conf_t *clcf; ngx_http_range_filter_ctx_t *ctx; @@ -227,7 +228,9 @@ parse: return NGX_ERROR; } - switch (ngx_http_range_parse(r, ctx, clcf->max_ranges)) { + ranges = r->single_range ? 1 : clcf->max_ranges; + + switch (ngx_http_range_parse(r, ctx, ranges)) { case NGX_OK: ngx_http_set_ctx(r, ctx, ngx_http_range_body_filter_module); diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h --- a/src/http/ngx_http_request.h +++ b/src/http/ngx_http_request.h @@ -528,6 +528,7 @@ struct ngx_http_request_s { unsigned filter_need_in_memory:1; unsigned filter_need_temporary:1; unsigned allow_ranges:1; + unsigned single_range:1; #if (NGX_STAT_STUB) unsigned stat_reading:1; diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -4183,7 +4183,12 @@ ngx_http_upstream_copy_allow_ranges(ngx_ if (r->cached) { r->allow_ranges = 1; return NGX_OK; - + } + + if (r->upstream->cacheable) { + r->allow_ranges = 1; + r->single_range = 1; + return NGX_OK; } #endif From mdounin at mdounin.ru Fri Mar 21 17:12:21 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 21 Mar 2014 21:12:21 +0400 Subject: [PATCH] Parse: underscores_in_headers should apply to the first character as well In-Reply-To: <51d996a6bc65c4ca3c66.1395280570@piotrs-macbook-pro.local> References: <51d996a6bc65c4ca3c66.1395280570@piotrs-macbook-pro.local> Message-ID: <20140321171221.GA34696@mdounin.ru> Hello! On Wed, Mar 19, 2014 at 06:56:10PM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1395280497 25200 > # Wed Mar 19 18:54:57 2014 -0700 > # Node ID 51d996a6bc65c4ca3c6621e4ad18235e6aa26abe > # Parent c15350f4071c0207cdffbc8bbb3f28e8e57f4d86 > Parse: underscores_in_headers should apply to the first character as well. In no particular order: - the "Parse:" prefix isn't something we use, it may be good idea to avoid id; - please keep summary line short to avoid "hg log" wrapping, see http://nginx.org/en/docs/contributing_changes.html; - is it something really needed in practice? > > Signed-off-by: Piotr Sikora > > diff -r c15350f4071c -r 51d996a6bc65 src/http/ngx_http_parse.c > --- a/src/http/ngx_http_parse.c Wed Mar 19 19:30:09 2014 +0400 > +++ b/src/http/ngx_http_parse.c Wed Mar 19 18:54:57 2014 -0700 > @@ -886,6 +886,19 @@ ngx_http_parse_header_line(ngx_http_requ > break; > } > > + if (ch == '_') { > + if (allow_underscores) { > + hash = ngx_hash(hash, ch); Should be hash = ngx_hash(0, ch); instead. > + r->lowcase_header[0] = ch; > + i = 1; > + > + } else { > + r->invalid_header = 1; > + } > + > + break; > + } > + > if (ch == '\0') { > return NGX_HTTP_PARSE_INVALID_HEADER; > } > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Maxim Dounin http://nginx.org/ From pluknet at nginx.com Mon Mar 24 13:59:06 2014 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 24 Mar 2014 13:59:06 +0000 Subject: [nginx] Mp4: skipped empty stss atom table in output. Message-ID: details: http://hg.nginx.org/nginx/rev/a23c35496c2f branches: changeset: 5622:a23c35496c2f user: Sergey Kandaurov date: Mon Mar 24 17:55:10 2014 +0400 description: Mp4: skipped empty stss atom table in output. The atom may have no data entries after cropping. This fixes "zero size buf in output" alerts. diffstat: src/http/modules/ngx_http_mp4_module.c | 25 +++++++++++++++---------- 1 files changed, 15 insertions(+), 10 deletions(-) diffs (35 lines): diff -r 345e4fd4bb64 -r a23c35496c2f src/http/modules/ngx_http_mp4_module.c --- a/src/http/modules/ngx_http_mp4_module.c Fri Mar 21 19:33:21 2014 +0400 +++ b/src/http/modules/ngx_http_mp4_module.c Mon Mar 24 17:55:10 2014 +0400 @@ -2273,16 +2273,21 @@ ngx_http_mp4_update_stss_atom(ngx_http_m ngx_http_mp4_crop_stss_data(mp4, trak, 1); ngx_http_mp4_crop_stss_data(mp4, trak, 0); - entry = (uint32_t *) data->pos; - end = (uint32_t *) data->last; - - start_sample = trak->start_sample; - - while (entry < end) { - sample = ngx_mp4_get_32value(entry); - sample -= start_sample; - ngx_mp4_set_32value(entry, sample); - entry++; + if (trak->sync_samples_entries) { + entry = (uint32_t *) data->pos; + end = (uint32_t *) data->last; + + start_sample = trak->start_sample; + + while (entry < end) { + sample = ngx_mp4_get_32value(entry); + sample -= start_sample; + ngx_mp4_set_32value(entry, sample); + entry++; + } + + } else { + trak->out[NGX_HTTP_MP4_STSS_DATA].buf = NULL; } atom_size = sizeof(ngx_http_mp4_stss_atom_t) + (data->last - data->pos); From piotr at cloudflare.com Mon Mar 24 23:48:29 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Mon, 24 Mar 2014 16:48:29 -0700 Subject: [PATCH] Parse: underscores_in_headers should apply to the first character as well In-Reply-To: <20140321171221.GA34696@mdounin.ru> References: <51d996a6bc65c4ca3c66.1395280570@piotrs-macbook-pro.local> <20140321171221.GA34696@mdounin.ru> Message-ID: Hey Maxim, > - the "Parse:" prefix isn't something we use, it may be good idea > to avoid id; Done. > - please keep summary line short to avoid "hg log" wrapping, see > http://nginx.org/en/docs/contributing_changes.html; Done. > - is it something really needed in practice? Well, I'm not a fan of allowing underscores in the headers in the first place, but since this is already in nginx, I don't see a reason why this directive should allow underscores anywhere in the header name, but not in the first character... And yes, we have customers proxying headers starting with an underscore. > Should be > > hash = ngx_hash(0, ch); > > instead. Good catch, thanks! Best regards, Piotr Sikora # HG changeset patch # User Piotr Sikora # Date 1395704144 25200 # Mon Mar 24 16:35:44 2014 -0700 # Node ID 9b71aa40b65135ed65c5f39c54a743a417ac77e8 # Parent a23c35496c2fc0ba9a34d968c2ca6d1f9374f8a8 Apply underscores_in_headers also to the first character. Signed-off-by: Piotr Sikora diff -r a23c35496c2f -r 9b71aa40b651 src/http/ngx_http_parse.c --- a/src/http/ngx_http_parse.c Mon Mar 24 17:55:10 2014 +0400 +++ b/src/http/ngx_http_parse.c Mon Mar 24 16:35:44 2014 -0700 @@ -886,6 +886,19 @@ ngx_http_parse_header_line(ngx_http_requ break; } + if (ch == '_') { + if (allow_underscores) { + hash = ngx_hash(0, ch); + r->lowcase_header[0] = ch; + i = 1; + + } else { + r->invalid_header = 1; + } + + break; + } + if (ch == '\0') { return NGX_HTTP_PARSE_INVALID_HEADER; } From kondakova at cryptopro.ru Tue Mar 25 12:45:46 2014 From: kondakova at cryptopro.ru (Tatiana Kondakova) Date: Tue, 25 Mar 2014 16:45:46 +0400 Subject: [PATCH 0 of 1] allow to use engine keyform for server private key Message-ID: Hello. I'm a cryptography library developer (http://www.cryptopro.ru/). I want to make our server-side TLS worked with nginx, and we have engine for openssl, which successfully works with openssl utilities. But for security reasons we can not export the private key to a file, so our engine needs something like keyform ENGINE option. This option makes possible to use nginx with our library, with PKCS#11 tokens and with any other engine, which does not support private keys export. From kondakova at cryptopro.ru Tue Mar 25 12:45:47 2014 From: kondakova at cryptopro.ru (Tatiana Kondakova) Date: Tue, 25 Mar 2014 16:45:47 +0400 Subject: [PATCH 1 of 1] add keyform option to SSL config to support loading private key from engine without exporting it to file In-Reply-To: References: Message-ID: <773a8762a7544e77d479.1395751547@build-mozilla-linux64> # HG changeset patch # User Tatiana Kondakova # Date 1395663427 -14400 # Node ID 773a8762a7544e77d4790be8296c592b53314b0e # Parent 345e4fd4bb64f1b3ad48a20b69f62bcd39a443c9 add keyform option to SSL config to support loading private key from engine without exporting it to file diff -r 345e4fd4bb64 -r 773a8762a754 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Fri Mar 21 19:33:21 2014 +0400 +++ b/src/event/ngx_event_openssl.c Mon Mar 24 16:17:07 2014 +0400 @@ -14,6 +14,11 @@ ngx_uint_t engine; /* unsigned engine:1; */ } ngx_openssl_conf_t; +typedef struct pw_cb_data + { + const void *password; + const char *prompt_info; + } PW_CB_DATA; static int ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store); static void ngx_ssl_info_callback(const ngx_ssl_conn_t *ssl_conn, int where, @@ -253,7 +258,7 @@ ngx_int_t ngx_ssl_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, - ngx_str_t *key) + ngx_str_t *key, ngx_str_t *keyform) { BIO *bio; X509 *x509; @@ -340,17 +345,44 @@ BIO_free(bio); - if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { - return NGX_ERROR; + if(keyform->len) { + PW_CB_DATA cb_data; + cb_data.password = NULL; + cb_data.prompt_info = key->data; + + EVP_PKEY *pkey = + ENGINE_load_private_key(ENGINE_by_id(keyform->data), + key->data, + UI_create_method("OpenSSL application" + " user interface"), + &cb_data); + if(!pkey) + { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "ENGINE_load_private_key(\"%s\") failed", key->data); + return NGX_ERROR; + } + + if (SSL_CTX_use_PrivateKey(ssl->ctx, pkey) == 0) + { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_use_PrivateKey_file(\"%s\") failed", key->data); + return NGX_ERROR; + } } - - if (SSL_CTX_use_PrivateKey_file(ssl->ctx, (char *) key->data, - SSL_FILETYPE_PEM) - == 0) - { - ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, - "SSL_CTX_use_PrivateKey_file(\"%s\") failed", key->data); - return NGX_ERROR; + else { + if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { + return NGX_ERROR; + } + + if (SSL_CTX_use_PrivateKey_file(ssl->ctx, + (char *) key->data, SSL_FILETYPE_PEM) + == 0) + { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_use_PrivateKey_file(\"%s\") failed", key->data); + return NGX_ERROR; + } } return NGX_OK; diff -r 345e4fd4bb64 -r 773a8762a754 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Fri Mar 21 19:33:21 2014 +0400 +++ b/src/event/ngx_event_openssl.h Mon Mar 24 16:17:07 2014 +0400 @@ -112,7 +112,7 @@ ngx_int_t ngx_ssl_init(ngx_log_t *log); ngx_int_t ngx_ssl_create(ngx_ssl_t *ssl, ngx_uint_t protocols, void *data); ngx_int_t ngx_ssl_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, - ngx_str_t *cert, ngx_str_t *key); + ngx_str_t *cert, ngx_str_t *key, ngx_str_t *keyform); ngx_int_t ngx_ssl_client_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, ngx_int_t depth); ngx_int_t ngx_ssl_trusted_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, diff -r 345e4fd4bb64 -r 773a8762a754 src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Fri Mar 21 19:33:21 2014 +0400 +++ b/src/http/modules/ngx_http_ssl_module.c Mon Mar 24 16:17:07 2014 +0400 @@ -90,6 +90,13 @@ NGX_HTTP_SRV_CONF_OFFSET, offsetof(ngx_http_ssl_srv_conf_t, certificate_key), NULL }, + + { ngx_string("ssl_keyform_engine"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_SRV_CONF_OFFSET, + offsetof(ngx_http_ssl_srv_conf_t, certificate_keyform), + NULL }, { ngx_string("ssl_dhparam"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_TAKE1, @@ -556,6 +563,8 @@ ngx_conf_merge_str_value(conf->certificate, prev->certificate, ""); ngx_conf_merge_str_value(conf->certificate_key, prev->certificate_key, ""); + ngx_conf_merge_str_value(conf->certificate_keyform, + prev->certificate_keyform, ""); ngx_conf_merge_str_value(conf->dhparam, prev->dhparam, ""); @@ -646,7 +655,7 @@ cln->data = &conf->ssl; if (ngx_ssl_certificate(cf, &conf->ssl, &conf->certificate, - &conf->certificate_key) + &conf->certificate_key, &conf->certificate_keyform) != NGX_OK) { return NGX_CONF_ERROR; diff -r 345e4fd4bb64 -r 773a8762a754 src/http/modules/ngx_http_ssl_module.h --- a/src/http/modules/ngx_http_ssl_module.h Fri Mar 21 19:33:21 2014 +0400 +++ b/src/http/modules/ngx_http_ssl_module.h Mon Mar 24 16:17:07 2014 +0400 @@ -34,6 +34,7 @@ ngx_str_t certificate; ngx_str_t certificate_key; + ngx_str_t certificate_keyform; ngx_str_t dhparam; ngx_str_t ecdh_curve; ngx_str_t client_certificate; From defan at nginx.com Tue Mar 25 15:03:30 2014 From: defan at nginx.com (Andrei Belov) Date: Tue, 25 Mar 2014 15:03:30 +0000 Subject: [nginx] Increased default value of variables_hash_max_size. Message-ID: details: http://hg.nginx.org/nginx/rev/d0e75244a18a branches: changeset: 5623:d0e75244a18a user: Andrei Belov date: Tue Mar 25 18:49:28 2014 +0400 description: Increased default value of variables_hash_max_size. diffstat: src/http/ngx_http_core_module.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r a23c35496c2f -r d0e75244a18a src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c Mon Mar 24 17:55:10 2014 +0400 +++ b/src/http/ngx_http_core_module.c Tue Mar 25 18:49:28 2014 +0400 @@ -3440,7 +3440,7 @@ ngx_http_core_init_main_conf(ngx_conf_t ngx_align(cmcf->server_names_hash_bucket_size, ngx_cacheline_size); - ngx_conf_init_uint_value(cmcf->variables_hash_max_size, 512); + ngx_conf_init_uint_value(cmcf->variables_hash_max_size, 1024); ngx_conf_init_uint_value(cmcf->variables_hash_bucket_size, 64); cmcf->variables_hash_bucket_size = From mdounin at mdounin.ru Tue Mar 25 16:14:51 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 25 Mar 2014 16:14:51 +0000 Subject: [nginx] Apply underscores_in_headers also to the first character. Message-ID: details: http://hg.nginx.org/nginx/rev/bd91f286ee0a branches: changeset: 5624:bd91f286ee0a user: Piotr Sikora date: Mon Mar 24 16:35:44 2014 -0700 description: Apply underscores_in_headers also to the first character. Signed-off-by: Piotr Sikora diffstat: src/http/ngx_http_parse.c | 13 +++++++++++++ 1 files changed, 13 insertions(+), 0 deletions(-) diffs (23 lines): diff --git a/src/http/ngx_http_parse.c b/src/http/ngx_http_parse.c --- a/src/http/ngx_http_parse.c +++ b/src/http/ngx_http_parse.c @@ -886,6 +886,19 @@ ngx_http_parse_header_line(ngx_http_requ break; } + if (ch == '_') { + if (allow_underscores) { + hash = ngx_hash(0, ch); + r->lowcase_header[0] = ch; + i = 1; + + } else { + r->invalid_header = 1; + } + + break; + } + if (ch == '\0') { return NGX_HTTP_PARSE_INVALID_HEADER; } From mdounin at mdounin.ru Tue Mar 25 16:15:33 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 25 Mar 2014 20:15:33 +0400 Subject: [PATCH] Parse: underscores_in_headers should apply to the first character as well In-Reply-To: References: <51d996a6bc65c4ca3c66.1395280570@piotrs-macbook-pro.local> <20140321171221.GA34696@mdounin.ru> Message-ID: <20140325161533.GX34696@mdounin.ru> Hello! On Mon, Mar 24, 2014 at 04:48:29PM -0700, Piotr Sikora wrote: > Hey Maxim, > > > - the "Parse:" prefix isn't something we use, it may be good idea > > to avoid id; > > Done. > > > - please keep summary line short to avoid "hg log" wrapping, see > > http://nginx.org/en/docs/contributing_changes.html; > > Done. > > > - is it something really needed in practice? > > Well, I'm not a fan of allowing underscores in the headers in the > first place, but since this is already in nginx, I don't see a reason > why this directive should allow underscores anywhere in the header > name, but not in the first character... > > And yes, we have customers proxying headers starting with an underscore. [...] Committed, thanks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Tue Mar 25 17:10:25 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 25 Mar 2014 21:10:25 +0400 Subject: [PATCH 0 of 1] allow to use engine keyform for server private key In-Reply-To: References: Message-ID: <20140325171025.GA34696@mdounin.ru> Hello! On Tue, Mar 25, 2014 at 04:45:46PM +0400, Tatiana Kondakova wrote: > Hello. > I'm a cryptography library developer (http://www.cryptopro.ru/). > I want to make our server-side TLS worked with nginx, and we > have engine for openssl, which successfully works with openssl > utilities. But for security reasons we can not export the > private key to a file, so our engine needs something like > keyform ENGINE option. > This option makes possible to use nginx with our library, with > PKCS#11 tokens and with any other engine, which does not support > private keys export. While this functionality looks interesting, the patch certainly needs more work before it will be possible to commit it. In particular, the patch will break compilation with mail module, not even talking about style issues. I also can't say I like the way how it's expected to be configured. There should be a better way to do this, probably some parameter of the ssl_certificate_key directive ("format="? or rather "engine="?) and/or some specific path prefix to load a key from an engine. -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Tue Mar 25 18:24:37 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 25 Mar 2014 11:24:37 -0700 Subject: [PATCH 0 of 1] allow to use engine keyform for server private key In-Reply-To: <20140325171025.GA34696@mdounin.ru> References: <20140325171025.GA34696@mdounin.ru> Message-ID: Hello, > While this functionality looks interesting, the patch certainly > needs more work before it will be possible to commit it. In > particular, the patch will break compilation with mail module, not > even talking about style issues. > > I also can't say I like the way how it's expected to be > configured. There should be a better way to do this, probably > some parameter of the ssl_certificate_key directive ("format="? or > rather "engine="?) and/or some specific path prefix to load a key > from an engine. On top of what Maxim already wrote, I'd like to see the counterpart for the ssl_certificate directive. Also, I prefer the "engine=" parameter. Best regards, Piotr Sikora From mdounin at mdounin.ru Tue Mar 25 18:43:16 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 25 Mar 2014 22:43:16 +0400 Subject: [PATCH 0 of 1] allow to use engine keyform for server private key In-Reply-To: References: <20140325171025.GA34696@mdounin.ru> Message-ID: <20140325184316.GE34696@mdounin.ru> Hello! On Tue, Mar 25, 2014 at 11:24:37AM -0700, Piotr Sikora wrote: > Hello, > > > While this functionality looks interesting, the patch certainly > > needs more work before it will be possible to commit it. In > > particular, the patch will break compilation with mail module, not > > even talking about style issues. > > > > I also can't say I like the way how it's expected to be > > configured. There should be a better way to do this, probably > > some parameter of the ssl_certificate_key directive ("format="? or > > rather "engine="?) and/or some specific path prefix to load a key > > from an engine. > > On top of what Maxim already wrote, I'd like to see the counterpart > for the ssl_certificate directive. I too think it would be good, but I'm not sure it's at all possible. OpenSSL interface seems to allow to load public key from an engine, but not a certificate. I may be wrong though. -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Tue Mar 25 20:11:40 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 25 Mar 2014 13:11:40 -0700 Subject: [PATCH 0 of 1] allow to use engine keyform for server private key In-Reply-To: <20140325184316.GE34696@mdounin.ru> References: <20140325171025.GA34696@mdounin.ru> <20140325184316.GE34696@mdounin.ru> Message-ID: Hey Maxim, > I too think it would be good, but I'm not sure it's at all > possible. OpenSSL interface seems to allow to load public key > from an engine, but not a certificate. I may be wrong though. We could use engine's STORE_METHOD, as it gives us access to: STORE_get_certificate(), STORE_get_private_key(), STORE_get_crl() and STORE_get_arbitrary() and use the old ENGINE_load_private_key() as the fallback in case engine doesn't provide STORE_METHOD. Best regards, Piotr Sikora From wangxiaochen0 at gmail.com Wed Mar 26 04:30:11 2014 From: wangxiaochen0 at gmail.com (Xiaochen Wang) Date: Wed, 26 Mar 2014 12:30:11 +0800 Subject: [PATCH] SPDY: fixed handling of sc->length in ngx_http_spdy_state_read_data() Message-ID: <20140326043011.GA47562@gmail.com> In our production, sometimes, the disk was full. In which case, the requests after the POST request were handled wrongly in one spdy connection. Because the input body (DATA frame) of POST request could not be written to disk, then ngx_http_spdy_state_read_data() tried to skip this DATA frame with wrong sc->length, which broke spdy stream. # HG changeset patch # User Xiaochen Wang # Date 1395807655 -28800 # Node ID c18a0115c6027ad0ad475061d2984d90544d8449 # Parent bd91f286ee0ade98e9c0f8f55bfef54a122adaf2 SPDY: fixed handling of sc->length in ngx_http_spdy_state_read_data() diff -r bd91f286ee0a -r c18a0115c602 src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c Mon Mar 24 16:35:44 2014 -0700 +++ b/src/http/ngx_http_spdy.c Wed Mar 26 12:20:55 2014 +0800 @@ -1519,7 +1519,6 @@ complete = 1; } else { - sc->length -= size; complete = 0; } @@ -1567,6 +1566,7 @@ buf->pos = pos; pos += size; + sc->length -= size; buf->end = pos; buf->last = pos; @@ -1585,6 +1585,7 @@ } else { buf->last = ngx_cpymem(buf->last, pos, size); pos += size; + sc->length -= size; } r->request_length += size; From vbart at nginx.com Wed Mar 26 07:02:29 2014 From: vbart at nginx.com (Valentin V. Bartenev) Date: Wed, 26 Mar 2014 11:02:29 +0400 Subject: [PATCH] SPDY: fixed handling of sc->length in ngx_http_spdy_state_read_data() In-Reply-To: <20140326043011.GA47562@gmail.com> References: <20140326043011.GA47562@gmail.com> Message-ID: <2583480.HkcQmHx6Q1@vbart-laptop> On Wednesday 26 March 2014 12:30:11 Xiaochen Wang wrote: > In our production, sometimes, the disk was full. In which case, the requests > after the POST request were handled wrongly in one spdy connection. > > Because the input body (DATA frame) of POST request could not be written to disk, > then ngx_http_spdy_state_read_data() tried to skip this DATA frame with wrong > sc->length, which broke spdy stream. > > > # HG changeset patch > # User Xiaochen Wang > # Date 1395807655 -28800 > # Node ID c18a0115c6027ad0ad475061d2984d90544d8449 > # Parent bd91f286ee0ade98e9c0f8f55bfef54a122adaf2 > SPDY: fixed handling of sc->length in ngx_http_spdy_state_read_data() > > diff -r bd91f286ee0a -r c18a0115c602 src/http/ngx_http_spdy.c > --- a/src/http/ngx_http_spdy.c Mon Mar 24 16:35:44 2014 -0700 > +++ b/src/http/ngx_http_spdy.c Wed Mar 26 12:20:55 2014 +0800 > @@ -1519,7 +1519,6 @@ > complete = 1; > > } else { > - sc->length -= size; > complete = 0; > } > > @@ -1567,6 +1566,7 @@ > buf->pos = pos; > > pos += size; > + sc->length -= size; > > buf->end = pos; > buf->last = pos; > @@ -1585,6 +1585,7 @@ > } else { > buf->last = ngx_cpymem(buf->last, pos, size); > pos += size; > + sc->length -= size; > } > > r->request_length += size; > Thank you for the report. I'd prefer a slightly smaller patch: diff -r e45fa57ef725 src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c Thu Mar 06 23:15:10 2014 +0400 +++ b/src/http/ngx_http_spdy.c Wed Mar 26 10:55:33 2014 +0400 @@ -1496,7 +1496,6 @@ ngx_http_spdy_state_read_data(ngx_http_s complete = 1; } else { - sc->length -= size; complete = 0; } @@ -1539,6 +1538,8 @@ ngx_http_spdy_state_read_data(ngx_http_s } } + sc->length -= size; + if (tf) { buf->start = pos; buf->pos = pos; -- wbr, Valentin V. Bartenev From wangxiaochen0 at gmail.com Wed Mar 26 07:46:05 2014 From: wangxiaochen0 at gmail.com (Xiaochen Wang) Date: Wed, 26 Mar 2014 15:46:05 +0800 Subject: [PATCH] SPDY: fixed handling of sc->length in ngx_http_spdy_state_read_data() In-Reply-To: <2583480.HkcQmHx6Q1@vbart-laptop> References: <20140326043011.GA47562@gmail.com> <2583480.HkcQmHx6Q1@vbart-laptop> Message-ID: hi, On Wed, Mar 26, 2014 at 3:02 PM, Valentin V. Bartenev wrote: > On Wednesday 26 March 2014 12:30:11 Xiaochen Wang wrote: > > In our production, sometimes, the disk was full. In which case, the > requests > > after the POST request were handled wrongly in one spdy connection. > > > > Because the input body (DATA frame) of POST request could not be written > to disk, > > then ngx_http_spdy_state_read_data() tried to skip this DATA frame with > wrong > > sc->length, which broke spdy stream. > > > > > > # HG changeset patch > > # User Xiaochen Wang > > # Date 1395807655 -28800 > > # Node ID c18a0115c6027ad0ad475061d2984d90544d8449 > > # Parent bd91f286ee0ade98e9c0f8f55bfef54a122adaf2 > > SPDY: fixed handling of sc->length in ngx_http_spdy_state_read_data() > > > > diff -r bd91f286ee0a -r c18a0115c602 src/http/ngx_http_spdy.c > > --- a/src/http/ngx_http_spdy.c Mon Mar 24 16:35:44 2014 -0700 > > +++ b/src/http/ngx_http_spdy.c Wed Mar 26 12:20:55 2014 +0800 > > @@ -1519,7 +1519,6 @@ > > complete = 1; > > > > } else { > > - sc->length -= size; > > complete = 0; > > } > > > > @@ -1567,6 +1566,7 @@ > > buf->pos = pos; > > > > pos += size; > > + sc->length -= size; > > > > buf->end = pos; > > buf->last = pos; > > @@ -1585,6 +1585,7 @@ > > } else { > > buf->last = ngx_cpymem(buf->last, pos, size); > > pos += size; > > + sc->length -= size; > > } > > > > r->request_length += size; > > > > Thank you for the report. I'd prefer a slightly smaller patch: > Agree. > > diff -r e45fa57ef725 src/http/ngx_http_spdy.c > --- a/src/http/ngx_http_spdy.c Thu Mar 06 23:15:10 2014 +0400 > +++ b/src/http/ngx_http_spdy.c Wed Mar 26 10:55:33 2014 +0400 > @@ -1496,7 +1496,6 @@ ngx_http_spdy_state_read_data(ngx_http_s > complete = 1; > > } else { > - sc->length -= size; > complete = 0; > } > > @@ -1539,6 +1538,8 @@ ngx_http_spdy_state_read_data(ngx_http_s > } > } > > + sc->length -= size; > + if (tf) { > buf->start = pos; > buf->pos = pos; > > -- > > wbr, Valentin V. Bartenev > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > Thanks, Xiaochen -------------- next part -------------- An HTML attachment was scrubbed... URL: From vbart at nginx.com Wed Mar 26 14:02:45 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Wed, 26 Mar 2014 14:02:45 +0000 Subject: [nginx] SPDY: detect premature end of frame while start parsing ... Message-ID: details: http://hg.nginx.org/nginx/rev/372cc6e532a6 branches: changeset: 5625:372cc6e532a6 user: Valentin Bartenev date: Wed Mar 26 17:43:39 2014 +0400 description: SPDY: detect premature end of frame while start parsing headers. diffstat: src/http/ngx_http_spdy.c | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-) diffs (19 lines): diff -r bd91f286ee0a -r 372cc6e532a6 src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c Mon Mar 24 16:35:44 2014 -0700 +++ b/src/http/ngx_http_spdy.c Wed Mar 26 17:43:39 2014 +0400 @@ -1049,6 +1049,15 @@ ngx_http_spdy_state_headers(ngx_http_spd if (r->headers_in.headers.part.elts == NULL) { if (buf->last - buf->pos < NGX_SPDY_NV_NUM_SIZE) { + + if (complete) { + ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, + "client sent SYN_STREAM frame " + "with invalid HEADERS block"); + ngx_http_spdy_close_stream(sc->stream, NGX_HTTP_BAD_REQUEST); + return ngx_http_spdy_state_protocol_error(sc); + } + return ngx_http_spdy_state_save(sc, pos, end, ngx_http_spdy_state_headers); } From vbart at nginx.com Wed Mar 26 14:02:47 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Wed, 26 Mar 2014 14:02:47 +0000 Subject: [nginx] SPDY: better detect if headers block has wrong entries c... Message-ID: details: http://hg.nginx.org/nginx/rev/2411d4b5be2c branches: changeset: 5626:2411d4b5be2c user: Valentin Bartenev date: Wed Mar 26 18:01:11 2014 +0400 description: SPDY: better detect if headers block has wrong entries count. Previously, only one case was checked: if there's more data to parse in a r->header_in buffer, but the buffer can be filled to the end by the last parsed entry, so we also need to check that there's no more data to inflate. diffstat: src/http/ngx_http_spdy.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diffs (18 lines): diff -r 372cc6e532a6 -r 2411d4b5be2c src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c Wed Mar 26 17:43:39 2014 +0400 +++ b/src/http/ngx_http_spdy.c Wed Mar 26 18:01:11 2014 +0400 @@ -1196,10 +1196,10 @@ ngx_http_spdy_state_headers(ngx_http_spd } } - if (buf->pos != buf->last) { - /* TODO: improve error message */ - ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "end %ui %p %p", complete, buf->pos, buf->last); + if (buf->pos != buf->last || sc->zstream_in.avail_in) { + ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, + "client sent SYN_STREAM frame " + "with invalid HEADERS block"); ngx_http_spdy_close_stream(sc->stream, NGX_HTTP_BAD_REQUEST); return ngx_http_spdy_state_protocol_error(sc); } From vbart at nginx.com Wed Mar 26 17:01:48 2014 From: vbart at nginx.com (Valentin V. Bartenev) Date: Wed, 26 Mar 2014 21:01:48 +0400 Subject: [PATCH] SPDY: fixed handling of sc->length in ngx_http_spdy_state_read_data() In-Reply-To: <20140326043011.GA47562@gmail.com> References: <20140326043011.GA47562@gmail.com> Message-ID: <2233208.qEzfZ1ABD4@vbart-laptop> On Wednesday 26 March 2014 12:30:11 Xiaochen Wang wrote: > In our production, sometimes, the disk was full. In which case, the requests > after the POST request were handled wrongly in one spdy connection. > > Because the input body (DATA frame) of POST request could not be written to disk, > then ngx_http_spdy_state_read_data() tried to skip this DATA frame with wrong > sc->length, which broke spdy stream. > [..] While I agree that there's a problem with sc->length premature adjustment, but please note that the problem cannot be triggered by the way you described. The "pos" pointer is adjusted as well right before ngx_write_chain_to_temp_file() is called. wbr, Valentin V. Bartenev From wangxiaochen0 at gmail.com Thu Mar 27 01:02:57 2014 From: wangxiaochen0 at gmail.com (Xiaochen Wang) Date: Thu, 27 Mar 2014 09:02:57 +0800 Subject: [PATCH] SPDY: fixed handling of sc->length in ngx_http_spdy_state_read_data() In-Reply-To: <2233208.qEzfZ1ABD4@vbart-laptop> References: <20140326043011.GA47562@gmail.com> <2233208.qEzfZ1ABD4@vbart-laptop> Message-ID: hi On Thu, Mar 27, 2014 at 1:01 AM, Valentin V. Bartenev wrote: > On Wednesday 26 March 2014 12:30:11 Xiaochen Wang wrote: > > In our production, sometimes, the disk was full. In which case, the > requests > > after the POST request were handled wrongly in one spdy connection. > > > > Because the input body (DATA frame) of POST request could not be written > to disk, > > then ngx_http_spdy_state_read_data() tried to skip this DATA frame with > wrong > > sc->length, which broke spdy stream. > > > [..] > > While I agree that there's a problem with sc->length premature adjustment, > but > please note that the problem cannot be triggered by the way you described. > > The "pos" pointer is adjusted as well right before > ngx_write_chain_to_temp_file() > is called. > Yes, the pos pointer is right. But sc->length is not adjusted if it receives a complete DATA frame. Then ngx_http_spdy_state_skip() skips more data. ngx_http_spdy_state_read_data { ... if (size >= sc->length) { size = sc->length; <<< sc->length is not adjusted. complete = 1; } else { sc->length -= size; complete = 0; } ... } > > wbr, Valentin V. Bartenev > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vbart at nginx.com Thu Mar 27 11:08:21 2014 From: vbart at nginx.com (Valentin V. Bartenev) Date: Thu, 27 Mar 2014 15:08:21 +0400 Subject: [PATCH] SPDY: fixed handling of sc->length in ngx_http_spdy_state_read_data() In-Reply-To: References: <20140326043011.GA47562@gmail.com> <2233208.qEzfZ1ABD4@vbart-laptop> Message-ID: <1912887.kFAd2FcvDq@vbart-laptop> On Thursday 27 March 2014 09:02:57 Xiaochen Wang wrote: > hi > > > On Thu, Mar 27, 2014 at 1:01 AM, Valentin V. Bartenev wrote: > > > On Wednesday 26 March 2014 12:30:11 Xiaochen Wang wrote: > > > In our production, sometimes, the disk was full. In which case, the > > requests > > > after the POST request were handled wrongly in one spdy connection. > > > > > > Because the input body (DATA frame) of POST request could not be written > > to disk, > > > then ngx_http_spdy_state_read_data() tried to skip this DATA frame with > > wrong > > > sc->length, which broke spdy stream. > > > > > [..] > > > > While I agree that there's a problem with sc->length premature adjustment, > > but > > please note that the problem cannot be triggered by the way you described. > > > > The "pos" pointer is adjusted as well right before > > ngx_write_chain_to_temp_file() > > is called. > > > > Yes, the pos pointer is right. But sc->length is not adjusted if it > receives a complete DATA frame. > Then ngx_http_spdy_state_skip() skips more data. > > ngx_http_spdy_state_read_data > { > ... > if (size >= sc->length) { > size = sc->length; <<< sc->length is not adjusted. > complete = 1; > > } else { > sc->length -= size; > complete = 0; > } > ... > } > > Ah, ok, I thought about cases when sc->length is adjusted but *pos isn't, but you are right there's also an opposite case. wbr, Valentin V. Bartenev From trygve.vea at redpill-linpro.com Thu Mar 27 15:34:37 2014 From: trygve.vea at redpill-linpro.com (Trygve Vea) Date: Thu, 27 Mar 2014 16:34:37 +0100 Subject: [PATCH] Added so_freebind and so_transparent to the listen directive Message-ID: <13e6a37c2f57443b0d5d.1395934477@miitool> # HG changeset patch # User Trygve Vea # Date 1395933815 -3600 # Thu Mar 27 16:23:35 2014 +0100 # Node ID 13e6a37c2f57443b0d5dd0abce8d9d4ab00e31e3 # Parent 2411d4b5be2ca690a5a00a1d8ad96ff69a00317f Added so_freebind and so_transparent to the listen directive This solves a Linux/IPv6-specific problem. To be able to listen to an IPv6 address that is not yet available on the host, one need to use the IP_FREEBIND and IP_TRANSPARENT socket options. The use case in question is for a failover setup with several service- addresses in a IPv6-only environment. IPv4 has a sysctl available (ip_nonlocal_bind), which is not available for IPv6 - thus making these patches necessary. diff -r 2411d4b5be2c -r 13e6a37c2f57 src/core/ngx_connection.c --- a/src/core/ngx_connection.c Wed Mar 26 18:01:11 2014 +0400 +++ b/src/core/ngx_connection.c Thu Mar 27 16:23:35 2014 +0100 @@ -305,6 +305,14 @@ ngx_open_listening_sockets(ngx_cycle_t *cycle) { int reuseaddr; +#ifdef NGX_LINUX +#ifdef IP_FREEBIND + int so_freebind; +#endif +#ifdef IP_TRANSPARENT + int so_transparent; +#endif +#endif ngx_uint_t i, tries, failed; ngx_err_t err; ngx_log_t *log; @@ -370,6 +378,37 @@ return NGX_ERROR; } +#ifdef NGX_LINUX +#ifdef IP_FREEBIND + if (ls[i].so_freebind) { + so_freebind = (ls[i].so_freebind == 1) ? 1 : 0; + + if (setsockopt(s, SOL_IP, IP_FREEBIND, + (const void *) &so_freebind, sizeof(int)) + == -1) + { + ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_socket_errno, + "setsockopt(IP_FREEBIND, %d) %V failed", + so_freebind, &ls[i].addr_text); + } + } +#endif +#ifdef IP_TRANSPARENT + if (ls[i].so_transparent) { + so_transparent = (ls[i].so_transparent == 1) ? 1 : 0; + + if (setsockopt(s, SOL_IP, IP_TRANSPARENT, + (const void *) &so_transparent, sizeof(int)) + == -1) + { + ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_socket_errno, + "setsockopt(IP_TRANSPARENT, %d) %V failed", + so_transparent, &ls[i].addr_text); + } + } +#endif +#endif + #if (NGX_HAVE_INET6 && defined IPV6_V6ONLY) if (ls[i].sockaddr->sa_family == AF_INET6) { diff -r 2411d4b5be2c -r 13e6a37c2f57 src/core/ngx_connection.h --- a/src/core/ngx_connection.h Wed Mar 26 18:01:11 2014 +0400 +++ b/src/core/ngx_connection.h Thu Mar 27 16:23:35 2014 +0100 @@ -67,6 +67,14 @@ unsigned ipv6only:1; #endif unsigned keepalive:2; +#ifdef NGX_LINUX +#ifdef IP_FREEBIND + unsigned so_freebind:1; +#endif +#ifdef IP_TRANSPARENT + unsigned so_transparent:1; +#endif +#endif #if (NGX_HAVE_DEFERRED_ACCEPT) unsigned deferred_accept:1; diff -r 2411d4b5be2c -r 13e6a37c2f57 src/http/ngx_http.c --- a/src/http/ngx_http.c Wed Mar 26 18:01:11 2014 +0400 +++ b/src/http/ngx_http.c Thu Mar 27 16:23:35 2014 +0100 @@ -1791,6 +1791,14 @@ ls->sndbuf = addr->opt.sndbuf; ls->keepalive = addr->opt.so_keepalive; +#ifdef NGX_LINUX +#ifdef IP_FREEBIND + ls->so_freebind = addr->opt.so_freebind; +#endif +#ifdef IP_TRANSPARENT + ls->so_transparent = addr->opt.so_transparent; +#endif +#endif #if (NGX_HAVE_KEEPALIVE_TUNABLE) ls->keepidle = addr->opt.tcp_keepidle; ls->keepintvl = addr->opt.tcp_keepintvl; diff -r 2411d4b5be2c -r 13e6a37c2f57 src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c Wed Mar 26 18:01:11 2014 +0400 +++ b/src/http/ngx_http_core_module.c Thu Mar 27 16:23:35 2014 +0100 @@ -4283,6 +4283,30 @@ continue; } + if (ngx_strcmp(value[n].data, "so_freebind") == 0) { +#ifdef NGX_LINUX + lsopt.so_freebind = 1; + continue; +#else + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "the \"so_freebind\" parameter requires " + "nginx to be built for Linux"); + return NGX_CONF_ERROR; +#endif + } + + if (ngx_strcmp(value[n].data, "so_transparent") == 0) { +#ifdef NGX_LINUX + lsopt.so_transparent = 1; + continue; +#else + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "the \"so_transparent\" parameter requires " + "nginx to be built for linux"); + return NGX_CONF_ERROR; +#endif + } + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid parameter \"%V\"", &value[n]); return NGX_CONF_ERROR; diff -r 2411d4b5be2c -r 13e6a37c2f57 src/http/ngx_http_core_module.h --- a/src/http/ngx_http_core_module.h Wed Mar 26 18:01:11 2014 +0400 +++ b/src/http/ngx_http_core_module.h Thu Mar 27 16:23:35 2014 +0100 @@ -82,6 +82,14 @@ unsigned ipv6only:1; #endif unsigned so_keepalive:2; +#ifdef NGX_LINUX +#ifdef IP_FREEBIND + unsigned so_freebind:1; +#endif +#ifdef IP_TRANSPARENT + unsigned so_transparent:1; +#endif +#endif unsigned proxy_protocol:1; int backlog; From jimpop at gmail.com Thu Mar 27 16:29:03 2014 From: jimpop at gmail.com (Jim Popovitch) Date: Thu, 27 Mar 2014 12:29:03 -0400 Subject: proxy_pass behavior Message-ID: Hello, Should proxy_pass retry ipv4 if ipv6 fails? Currently (1.5.12), it does not, and there is no way to force proxy_pass to always use ipv4. Specifically, if a proxy_pass host has dual records (A+AAAA), proxy_pass seems to only try the AAAA. I'm considering submitting a patch if there is support for this. Thanks! -Jim P. From mdounin at mdounin.ru Thu Mar 27 16:59:12 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 27 Mar 2014 20:59:12 +0400 Subject: proxy_pass behavior In-Reply-To: References: Message-ID: <20140327165912.GE34696@mdounin.ru> Hello! On Thu, Mar 27, 2014 at 12:29:03PM -0400, Jim Popovitch wrote: > Should proxy_pass retry ipv4 if ipv6 fails? Currently (1.5.12), it > does not, and there is no way to force proxy_pass to always use ipv4. In no particular order: - The proxy_next_upstream is expected to retry by default. - There _is_ a way to force proxy_pass to always use ipv4 - you should configure your system accordingly, and getaddrinfo(AI_ADDRCONFIG) as used by nginx will do the right thing. E.g., on Linux net.ipv6.conf.all.disable_ipv6 can be set to disable use of ipv6. -- Maxim Dounin http://nginx.org/ From jimpop at gmail.com Thu Mar 27 17:12:18 2014 From: jimpop at gmail.com (Jim Popovitch) Date: Thu, 27 Mar 2014 13:12:18 -0400 Subject: proxy_pass behavior In-Reply-To: <20140327165912.GE34696@mdounin.ru> References: <20140327165912.GE34696@mdounin.ru> Message-ID: On Thu, Mar 27, 2014 at 12:59 PM, Maxim Dounin wrote: > Hello! > > On Thu, Mar 27, 2014 at 12:29:03PM -0400, Jim Popovitch wrote: > >> Should proxy_pass retry ipv4 if ipv6 fails? Currently (1.5.12), it >> does not, and there is no way to force proxy_pass to always use ipv4. > > In no particular order: > > - The proxy_next_upstream is expected to retry by default. Yes, that works if the host name has 2 or more A OR 2 or more AAAA records. proxy_next_upstream does not appear to transition to the next protocol (i.e. from ipv4 to ipv6) > - There _is_ a way to force proxy_pass to always use ipv4 - you > should configure your system accordingly, and > getaddrinfo(AI_ADDRCONFIG) as used by nginx will do the right > thing. E.g., on Linux net.ipv6.conf.all.disable_ipv6 can be set > to disable use of ipv6. :-) That's unacceptable!! :-) I don't want people to disable ipv6, I want proxy_pass to use ipv6 and upon failure use ipv4. ;-) -Jim P. From mdounin at mdounin.ru Thu Mar 27 17:13:53 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 27 Mar 2014 21:13:53 +0400 Subject: [PATCH] Added so_freebind and so_transparent to the listen directive In-Reply-To: <13e6a37c2f57443b0d5d.1395934477@miitool> References: <13e6a37c2f57443b0d5d.1395934477@miitool> Message-ID: <20140327171353.GF34696@mdounin.ru> Hello! On Thu, Mar 27, 2014 at 04:34:37PM +0100, Trygve Vea wrote: > # HG changeset patch > # User Trygve Vea > # Date 1395933815 -3600 > # Thu Mar 27 16:23:35 2014 +0100 > # Node ID 13e6a37c2f57443b0d5dd0abce8d9d4ab00e31e3 > # Parent 2411d4b5be2ca690a5a00a1d8ad96ff69a00317f > Added so_freebind and so_transparent to the listen directive > > This solves a Linux/IPv6-specific problem. > > To be able to listen to an IPv6 address that is not yet available on the host, > one need to use the IP_FREEBIND and IP_TRANSPARENT socket options. > > The use case in question is for a failover setup with several service- > addresses in a IPv6-only environment. > > IPv4 has a sysctl available (ip_nonlocal_bind), which is not available for > IPv6 - thus making these patches necessary. Isn't bind on INADDR_ANY/IN6ADDR_ANY works for you? It is expected to work fine and allows to accept connections on all addresses currently available on a host without any non-portable tricks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Thu Mar 27 17:27:33 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 27 Mar 2014 21:27:33 +0400 Subject: proxy_pass behavior In-Reply-To: References: <20140327165912.GE34696@mdounin.ru> Message-ID: <20140327172732.GG34696@mdounin.ru> Hello! On Thu, Mar 27, 2014 at 01:12:18PM -0400, Jim Popovitch wrote: > On Thu, Mar 27, 2014 at 12:59 PM, Maxim Dounin wrote: > > Hello! > > > > On Thu, Mar 27, 2014 at 12:29:03PM -0400, Jim Popovitch wrote: > > > >> Should proxy_pass retry ipv4 if ipv6 fails? Currently (1.5.12), it > >> does not, and there is no way to force proxy_pass to always use ipv4. > > > > In no particular order: > > > > - The proxy_next_upstream is expected to retry by default. > > Yes, that works if the host name has 2 or more A OR 2 or more AAAA > records. proxy_next_upstream does not appear to transition to the > next protocol (i.e. from ipv4 to ipv6) It doesn't care about address family. As long as connect() to one address fails, it will try next one. > > - There _is_ a way to force proxy_pass to always use ipv4 - you > > should configure your system accordingly, and > > getaddrinfo(AI_ADDRCONFIG) as used by nginx will do the right > > thing. E.g., on Linux net.ipv6.conf.all.disable_ipv6 can be set > > to disable use of ipv6. > > :-) That's unacceptable!! :-) I don't want people to disable ipv6, > I want proxy_pass to use ipv6 and upon failure use ipv4. ;-) It was you who claimed that "there is no way to force proxy_pass to always use ipv4". There is. -- Maxim Dounin http://nginx.org/ From jimpop at gmail.com Thu Mar 27 17:50:48 2014 From: jimpop at gmail.com (Jim Popovitch) Date: Thu, 27 Mar 2014 13:50:48 -0400 Subject: proxy_pass behavior In-Reply-To: <20140327172732.GG34696@mdounin.ru> References: <20140327165912.GE34696@mdounin.ru> <20140327172732.GG34696@mdounin.ru> Message-ID: On Thu, Mar 27, 2014 at 1:27 PM, Maxim Dounin wrote: > Hello! > > On Thu, Mar 27, 2014 at 01:12:18PM -0400, Jim Popovitch wrote: > >> On Thu, Mar 27, 2014 at 12:59 PM, Maxim Dounin wrote: >> > Hello! >> > >> > On Thu, Mar 27, 2014 at 12:29:03PM -0400, Jim Popovitch wrote: >> > >> >> Should proxy_pass retry ipv4 if ipv6 fails? Currently (1.5.12), it >> >> does not, and there is no way to force proxy_pass to always use ipv4. >> > >> > In no particular order: >> > >> > - The proxy_next_upstream is expected to retry by default. >> >> Yes, that works if the host name has 2 or more A OR 2 or more AAAA >> records. proxy_next_upstream does not appear to transition to the >> next protocol (i.e. from ipv4 to ipv6) > > It doesn't care about address family. As long as connect() to > one address fails, it will try next one. I will have to do some more tests, but so far 1.5.12 (debian wheezy) does not appear to try the next one if the next one is a different protocol family. >> > - There _is_ a way to force proxy_pass to always use ipv4 - you >> > should configure your system accordingly, and >> > getaddrinfo(AI_ADDRCONFIG) as used by nginx will do the right >> > thing. E.g., on Linux net.ipv6.conf.all.disable_ipv6 can be set >> > to disable use of ipv6. >> >> :-) That's unacceptable!! :-) I don't want people to disable ipv6, >> I want proxy_pass to use ipv6 and upon failure use ipv4. ;-) > > It was you who claimed that "there is no way to force proxy_pass > to always use ipv4". There is. What I was trying to claim was that there is no way to run nginx on ipv4 and ipv6 yet force proxy_pass to use only ipv4. Of course you can proxy bind to a v4 interface, but that doesn't scale with syncing configs to multiple servers. Maybe it's time for a proxy_protocol variable..... -Jim P. From tv at redpill-linpro.com Thu Mar 27 18:14:32 2014 From: tv at redpill-linpro.com (Trygve Vea) Date: Thu, 27 Mar 2014 19:14:32 +0100 (CET) Subject: [PATCH] Added so_freebind and so_transparent to the listen directive In-Reply-To: <20140327171353.GF34696@mdounin.ru> References: <13e6a37c2f57443b0d5d.1395934477@miitool> <20140327171353.GF34696@mdounin.ru> Message-ID: <642114637.11704594.1395944072305.JavaMail.zimbra@redpill-linpro.com> ----- Opprinnelig melding ----- > Hello! Hello! > On Thu, Mar 27, 2014 at 04:34:37PM +0100, Trygve Vea wrote: > > # HG changeset patch > > # User Trygve Vea > > # Date 1395933815 -3600 > > # Thu Mar 27 16:23:35 2014 +0100 > > # Node ID 13e6a37c2f57443b0d5dd0abce8d9d4ab00e31e3 > > # Parent 2411d4b5be2ca690a5a00a1d8ad96ff69a00317f > > Added so_freebind and so_transparent to the listen directive > > > > This solves a Linux/IPv6-specific problem. > > > > To be able to listen to an IPv6 address that is not yet available on the > > host, > > one need to use the IP_FREEBIND and IP_TRANSPARENT socket options. > > > > The use case in question is for a failover setup with several service- > > addresses in a IPv6-only environment. > > > > IPv4 has a sysctl available (ip_nonlocal_bind), which is not available for > > IPv6 - thus making these patches necessary. > > Isn't bind on INADDR_ANY/IN6ADDR_ANY works for you? > > It is expected to work fine and allows to accept connections on > all addresses currently available on a host without any > non-portable tricks. ----- Opprinnelig melding ----- > > IPv4 has a sysctl available (ip_nonlocal_bind), which is not available for > > IPv6 - thus making these patches necessary. > > Isn't bind on INADDR_ANY/IN6ADDR_ANY works for you? > > It is expected to work fine and allows to accept connections on > all addresses currently available on a host without any > non-portable tricks. That would be sufficient for HTTP - and my preferred option, since we can handle routing after the end-user have provided us with the Host-header, and thus know where to send the user. However, with SSL enabled - while we have end users that still do not support SNI (http://en.wikipedia.org/wiki/Server_Name_Indication#Client_side), and using multiple SSL-certificates, for multiple applications - we will need to bind each certificate to its own dedicated service address. From here, we can do routing / forward the connections further down the stack. After I submitted the patch, I noticed that it will probably not build on Linux versions prior to 2.4, so I intend to create a new one addressing that issue tomorrow when I'm back at the office. Are there any issues with the patch that I should take into consideration when making changes? Regards -- Trygve Vea From piotr at cloudflare.com Thu Mar 27 20:38:10 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Thu, 27 Mar 2014 13:38:10 -0700 Subject: [PATCH] Added so_freebind and so_transparent to the listen directive In-Reply-To: <13e6a37c2f57443b0d5d.1395934477@miitool> References: <13e6a37c2f57443b0d5d.1395934477@miitool> Message-ID: Hi Trygve, > Added so_freebind and so_transparent to the listen directive > > This solves a Linux/IPv6-specific problem. > > To be able to listen to an IPv6 address that is not yet available on the host, > one need to use the IP_FREEBIND and IP_TRANSPARENT socket options. > > The use case in question is for a failover setup with several service- > addresses in a IPv6-only environment. > > IPv4 has a sysctl available (ip_nonlocal_bind), which is not available for > IPv6 - thus making these patches necessary. Non-local bind() isn't Linux-specific feature. FreeBSD has IP_BINDANY/IPV6_BINDANY options and OpenBSD has SO_BINDANY option, so it would be good to add support for all of them. Also, requiring user to add both: "so_freebind" and "so_transparent" options to enable single feature doesn't look very user friendly, especially because according to your description, non-local bind() would work just fine with only "so_freebind" for IPv4 addresses, but it would fail for IPv6 addresses... I don't think that the "so_" prefix is necessary, how about just "bindany" or "nonlocal" ("freebind" is just stupid name)? Best regards, Piotr Sikora From tv at redpill-linpro.com Thu Mar 27 20:56:49 2014 From: tv at redpill-linpro.com (Trygve Vea) Date: Thu, 27 Mar 2014 21:56:49 +0100 (CET) Subject: [PATCH] Added so_freebind and so_transparent to the listen directive In-Reply-To: References: <13e6a37c2f57443b0d5d.1395934477@miitool> Message-ID: <204828004.11713294.1395953808999.JavaMail.zimbra@redpill-linpro.com> ----- Opprinnelig melding ----- > Hi Trygve, > > > Added so_freebind and so_transparent to the listen directive > > > > This solves a Linux/IPv6-specific problem. > > > > To be able to listen to an IPv6 address that is not yet available on the > > host, > > one need to use the IP_FREEBIND and IP_TRANSPARENT socket options. > > > > The use case in question is for a failover setup with several service- > > addresses in a IPv6-only environment. > > > > IPv4 has a sysctl available (ip_nonlocal_bind), which is not available for > > IPv6 - thus making these patches necessary. > > Non-local bind() isn't Linux-specific feature. FreeBSD has > IP_BINDANY/IPV6_BINDANY options and OpenBSD has SO_BINDANY option, so > it would be good to add support for all of them. That makes sense. > Also, requiring user to add both: "so_freebind" and "so_transparent" > options to enable single feature doesn't look very user friendly, > especially because according to your description, non-local bind() > would work just fine with only "so_freebind" for IPv4 addresses, but > it would fail for IPv6 addresses... I don't think that the "so_" > prefix is necessary, how about just "bindany" or "nonlocal" > ("freebind" is just stupid name)? The so_-prefix was something I took from the existing so_keepalive option. Are there any conventions I should follow when I name these values? I agree that creating a single option that can handle multiple platforms the same is a good idea. What if I do something like; * Create a configuration variable for the listen-directive named 'nonlocal'. * If IPv6 and Linux >= 2.4, attempt to set the socket options in question. (I'm not sure if I would need to deal with dual stack in a certain way, here - I'll check with one of the IPv6-evangelists at work tomorrow) * If IPv4 and Linux >= 2.4, set the IP_FREEBIND socket option. * Else, throw warning that the parameter will be ignored because running on an unsupported platform. Adding support for more platforms for the configuration option can be easily added by someone with access and knowledge about the platforms. I have actually never used FreeBSD/OpenBSD, so my knowledge is fairly limited. Would that be acceptable? -- Trygve From jimpop at gmail.com Fri Mar 28 00:18:48 2014 From: jimpop at gmail.com (Jim Popovitch) Date: Thu, 27 Mar 2014 20:18:48 -0400 Subject: proxy_pass behavior In-Reply-To: References: <20140327165912.GE34696@mdounin.ru> <20140327172732.GG34696@mdounin.ru> Message-ID: On Thu, Mar 27, 2014 at 1:50 PM, Jim Popovitch wrote: > On Thu, Mar 27, 2014 at 1:27 PM, Maxim Dounin wrote: >> Hello! >> >> On Thu, Mar 27, 2014 at 01:12:18PM -0400, Jim Popovitch wrote: >> >>> On Thu, Mar 27, 2014 at 12:59 PM, Maxim Dounin wrote: >>> > Hello! >>> > >>> > On Thu, Mar 27, 2014 at 12:29:03PM -0400, Jim Popovitch wrote: >>> > >>> >> Should proxy_pass retry ipv4 if ipv6 fails? Currently (1.5.12), it >>> >> does not, and there is no way to force proxy_pass to always use ipv4. >>> > >>> > In no particular order: >>> > >>> > - The proxy_next_upstream is expected to retry by default. >>> >>> Yes, that works if the host name has 2 or more A OR 2 or more AAAA >>> records. proxy_next_upstream does not appear to transition to the >>> next protocol (i.e. from ipv4 to ipv6) >> >> It doesn't care about address family. As long as connect() to >> one address fails, it will try next one. > > I will have to do some more tests, but so far 1.5.12 (debian wheezy) > does not appear to try the next one if the next one is a different > protocol family. So here is what happens: Nginx 1.5.12 on Debian Wheezy 7.4 ~$ host www.acadiau.ca www.acadiau.ca has address 131.162.201.18 www.acadiau.ca has IPv6 address 2620:21:c000:c8:0:aca:d1a:18 ~$ grep proxy_pass /etc/nginx/sites-available/proxy proxy_pass https://www.acadiau.ca; When an IPv6 client connects to the proxy, proxy_pass logs the error as 2014/03/27 21:28:28 [alert] 24862#0: *158336 connect() failed (13: Permission denied) while connecting to upstream, client: 2604:180::82c6:fe9, server: proxy-service.tld, request: "GET / HTTP/1.1", upstream: "https://[2620:21:c000:c8:0:aca:d1a:18]:443/", host: "proxy-service.tld" It will next try the ipv4 addr and succeed, however the reason for the "Permission denied" error appears to be the format of the URL (raw ipv6 addr, not hostname). Why is it using the ipv6 addr instead of the hostname? -Jim P. From tv at redpill-linpro.com Fri Mar 28 09:45:53 2014 From: tv at redpill-linpro.com (Trygve Vea) Date: Fri, 28 Mar 2014 10:45:53 +0100 Subject: [PATCH] Added nonlocal to the listen directive Message-ID: <16eacd8609c8362e9dd7.1395999953@miitool> # HG changeset patch # User Trygve Vea # Date 1395999940 -3600 # Fri Mar 28 10:45:40 2014 +0100 # Node ID 16eacd8609c8362e9dd729c743ed7a869c2993fe # Parent 2411d4b5be2ca690a5a00a1d8ad96ff69a00317f Added nonlocal to the listen directive The nonlocal option is used to set the needed socket options to be able to bind to an address not necessarily owned by the host. This patch currently implements this for Linux >= 2.4 IPv4/IPv6. The problem we solve by doing this, is in an environment where the following conditions are met: * HTTPS with multiple certificates, and a client base that are unable to use SNI - thus having the need to tie specific certificates to specific ip/ports. * Setting the ip_nonlocal_bind-sysctl is not an option (for example for Linux IPv6) * Used in a failover-setup, where the service IP-addresses are moved around by a daemon like linux-ha or keepalived. diff -r 2411d4b5be2c -r 16eacd8609c8 src/core/ngx_connection.c --- a/src/core/ngx_connection.c Wed Mar 26 18:01:11 2014 +0400 +++ b/src/core/ngx_connection.c Fri Mar 28 10:45:40 2014 +0100 @@ -305,6 +305,14 @@ ngx_open_listening_sockets(ngx_cycle_t *cycle) { int reuseaddr; +#ifdef NGX_LINUX +#ifdef IP_FREEBIND + int so_freebind; +#endif +#ifdef IP_TRANSPARENT + int so_transparent; +#endif +#endif ngx_uint_t i, tries, failed; ngx_err_t err; ngx_log_t *log; @@ -370,6 +378,35 @@ return NGX_ERROR; } + if (ls[i].nonlocal) { +#if (defined NGX_LINUX && defined IP_FREEBIND) + so_freebind = 1; + + if (setsockopt(s, SOL_IP, IP_FREEBIND, + (const void *) &so_freebind, sizeof(int)) + == -1) + { + ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_socket_errno, + "setsockopt(IP_FREEBIND, %d) %V failed", + so_freebind, &ls[i].addr_text); + } +#if (NGX_HAVE_INET6 && defined IP_TRANSPARENT) + so_transparent = 1; + + if (setsockopt(s, SOL_IP, IP_TRANSPARENT, + (const void *) &so_transparent, sizeof(int)) + == -1) + { + ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_socket_errno, + "setsockopt(IP_TRANSPARENT, %d) %V failed", + so_transparent, &ls[i].addr_text); + } +#endif + ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_socket_errno, + "nonlocal requires Linux >= 2.4, ignored"); +#endif + } + #if (NGX_HAVE_INET6 && defined IPV6_V6ONLY) if (ls[i].sockaddr->sa_family == AF_INET6) { diff -r 2411d4b5be2c -r 16eacd8609c8 src/core/ngx_connection.h --- a/src/core/ngx_connection.h Wed Mar 26 18:01:11 2014 +0400 +++ b/src/core/ngx_connection.h Fri Mar 28 10:45:40 2014 +0100 @@ -66,6 +66,7 @@ #if (NGX_HAVE_INET6 && defined IPV6_V6ONLY) unsigned ipv6only:1; #endif + unsigned nonlocal:1; unsigned keepalive:2; #if (NGX_HAVE_DEFERRED_ACCEPT) diff -r 2411d4b5be2c -r 16eacd8609c8 src/http/ngx_http.c --- a/src/http/ngx_http.c Wed Mar 26 18:01:11 2014 +0400 +++ b/src/http/ngx_http.c Fri Mar 28 10:45:40 2014 +0100 @@ -1808,6 +1808,7 @@ #if (NGX_HAVE_INET6 && defined IPV6_V6ONLY) ls->ipv6only = addr->opt.ipv6only; #endif + ls->nonlocal = addr->opt.nonlocal; #if (NGX_HAVE_SETFIB) ls->setfib = addr->opt.setfib; diff -r 2411d4b5be2c -r 16eacd8609c8 src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c Wed Mar 26 18:01:11 2014 +0400 +++ b/src/http/ngx_http_core_module.c Fri Mar 28 10:45:40 2014 +0100 @@ -4283,6 +4283,11 @@ continue; } + if (ngx_strcmp(value[n].data, "nonlocal") == 0) { + lsopt.nonlocal = 1; + continue; + } + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid parameter \"%V\"", &value[n]); return NGX_CONF_ERROR; diff -r 2411d4b5be2c -r 16eacd8609c8 src/http/ngx_http_core_module.h --- a/src/http/ngx_http_core_module.h Wed Mar 26 18:01:11 2014 +0400 +++ b/src/http/ngx_http_core_module.h Fri Mar 28 10:45:40 2014 +0100 @@ -81,6 +81,7 @@ #if (NGX_HAVE_INET6 && defined IPV6_V6ONLY) unsigned ipv6only:1; #endif + unsigned nonlocal:1; unsigned so_keepalive:2; unsigned proxy_protocol:1; diff -r 2411d4b5be2c -r 16eacd8609c8 src/mail/ngx_mail.c --- a/src/mail/ngx_mail.c Wed Mar 26 18:01:11 2014 +0400 +++ b/src/mail/ngx_mail.c Fri Mar 28 10:45:40 2014 +0100 @@ -327,6 +327,7 @@ #if (NGX_HAVE_INET6 && defined IPV6_V6ONLY) addr->ipv6only = listen->ipv6only; #endif + addr->nonlocal = listen->nonlocal; return NGX_OK; } diff -r 2411d4b5be2c -r 16eacd8609c8 src/mail/ngx_mail.h --- a/src/mail/ngx_mail.h Wed Mar 26 18:01:11 2014 +0400 +++ b/src/mail/ngx_mail.h Fri Mar 28 10:45:40 2014 +0100 @@ -41,6 +41,7 @@ #if (NGX_HAVE_INET6 && defined IPV6_V6ONLY) unsigned ipv6only:1; #endif + unsigned nonlocal:1; unsigned so_keepalive:2; #if (NGX_HAVE_KEEPALIVE_TUNABLE) int tcp_keepidle; diff -r 2411d4b5be2c -r 16eacd8609c8 src/mail/ngx_mail_core_module.c --- a/src/mail/ngx_mail_core_module.c Wed Mar 26 18:01:11 2014 +0400 +++ b/src/mail/ngx_mail_core_module.c Fri Mar 28 10:45:40 2014 +0100 @@ -456,6 +456,11 @@ #endif } + if (ngx_strcmp(value[i].data, "nonlocal") == 0) { + ls->nonlocal = 1; + continue; + } + if (ngx_strcmp(value[i].data, "ssl") == 0) { #if (NGX_MAIL_SSL) ls->ssl = 1; From tv at redpill-linpro.com Fri Mar 28 09:53:59 2014 From: tv at redpill-linpro.com (Trygve Vea) Date: Fri, 28 Mar 2014 10:53:59 +0100 (CET) Subject: [PATCH] Added so_freebind and so_transparent to the listen directive In-Reply-To: <13e6a37c2f57443b0d5d.1395934477@miitool> References: <13e6a37c2f57443b0d5d.1395934477@miitool> Message-ID: <171097759.21493.1396000439334.JavaMail.zimbra@redpill-linpro.com> ----- Opprinnelig melding ----- > # HG changeset patch > # User Trygve Vea > # Date 1395933815 -3600 > # Thu Mar 27 16:23:35 2014 +0100 > # Node ID 13e6a37c2f57443b0d5dd0abce8d9d4ab00e31e3 > # Parent 2411d4b5be2ca690a5a00a1d8ad96ff69a00317f > Added so_freebind and so_transparent to the listen directive "[PATCH] Added nonlocal to the listen directive" replaces this patch. -- Trygve Vea Redpill Linpro AS From igor at sysoev.ru Fri Mar 28 10:03:07 2014 From: igor at sysoev.ru (Igor Sysoev) Date: Fri, 28 Mar 2014 14:03:07 +0400 Subject: [PATCH] Added so_freebind and so_transparent to the listen directive In-Reply-To: <642114637.11704594.1395944072305.JavaMail.zimbra@redpill-linpro.com> References: <13e6a37c2f57443b0d5d.1395934477@miitool> <20140327171353.GF34696@mdounin.ru> <642114637.11704594.1395944072305.JavaMail.zimbra@redpill-linpro.com> Message-ID: <8D5BD3EB-BA7A-417F-82F7-8902A4D9770A@sysoev.ru> On Mar 27, 2014, at 22:14 , Trygve Vea wrote: > ----- Opprinnelig melding ----- >> Hello! > > Hello! > >> On Thu, Mar 27, 2014 at 04:34:37PM +0100, Trygve Vea wrote: >>> # HG changeset patch >>> # User Trygve Vea >>> # Date 1395933815 -3600 >>> # Thu Mar 27 16:23:35 2014 +0100 >>> # Node ID 13e6a37c2f57443b0d5dd0abce8d9d4ab00e31e3 >>> # Parent 2411d4b5be2ca690a5a00a1d8ad96ff69a00317f >>> Added so_freebind and so_transparent to the listen directive >>> >>> This solves a Linux/IPv6-specific problem. >>> >>> To be able to listen to an IPv6 address that is not yet available on the >>> host, >>> one need to use the IP_FREEBIND and IP_TRANSPARENT socket options. >>> >>> The use case in question is for a failover setup with several service- >>> addresses in a IPv6-only environment. >>> >>> IPv4 has a sysctl available (ip_nonlocal_bind), which is not available for >>> IPv6 - thus making these patches necessary. >> >> Isn't bind on INADDR_ANY/IN6ADDR_ANY works for you? >> >> It is expected to work fine and allows to accept connections on >> all addresses currently available on a host without any >> non-portable tricks. > ----- Opprinnelig melding ----- >>> IPv4 has a sysctl available (ip_nonlocal_bind), which is not available for >>> IPv6 - thus making these patches necessary. >> >> Isn't bind on INADDR_ANY/IN6ADDR_ANY works for you? >> >> It is expected to work fine and allows to accept connections on >> all addresses currently available on a host without any >> non-portable tricks. > > That would be sufficient for HTTP - and my preferred option, since we can handle routing after the end-user have provided us with the Host-header, and thus know where to send the user. > > However, with SSL enabled - while we have end users that still do not support SNI (http://en.wikipedia.org/wiki/Server_Name_Indication#Client_side), and using multiple SSL-certificates, for multiple applications - we will need to bind each certificate to its own dedicated service address. From here, we can do routing / forward the connections further down the stack. This can be handled with following configuration: server { listen *:443 ssl; listen any.non.existent.ip1:443 ssl; ssl_certificate ... ... } server { listen any.non.existent.ip2:443 ssl; ssl_certificate ... ... } nginx will bind only to *:443 and then will call getsockname() to get real local address. -- Igor Sysoev http://nginx.com From kondakova at cryptopro.ru Fri Mar 28 10:26:42 2014 From: kondakova at cryptopro.ru (=?koi8-r?B?68/OxMHLz9fBIPTB1NjRzsEg4c7E0sXF187B?=) Date: Fri, 28 Mar 2014 10:26:42 +0000 Subject: [PATCH 0 of 1] allow to use engine keyform for server private key Message-ID: <3C78B94BFD3E024A9F7164A442F6B8341BD49C76@pegas.cp.ru> Me and my company will be happy to see any method to load private key from engine with any parameter names. What shall I do? Change parameters to "format=" and "engine="? Try to use STORE_get_certificate(), STORE_get_private_key() before using ENGINE_load_private_key()? Do you have any instruction which can help me to check the patch? To do not break compilation with mail module again etc. Tatiana Kondakova mailto: kondakova at cryptopro.ru From tv at redpill-linpro.com Fri Mar 28 10:45:45 2014 From: tv at redpill-linpro.com (Trygve Vea) Date: Fri, 28 Mar 2014 11:45:45 +0100 (CET) Subject: [PATCH] Added so_freebind and so_transparent to the listen directive In-Reply-To: <8D5BD3EB-BA7A-417F-82F7-8902A4D9770A@sysoev.ru> References: <13e6a37c2f57443b0d5d.1395934477@miitool> <20140327171353.GF34696@mdounin.ru> <642114637.11704594.1395944072305.JavaMail.zimbra@redpill-linpro.com> <8D5BD3EB-BA7A-417F-82F7-8902A4D9770A@sysoev.ru> Message-ID: <404367197.25609.1396003545067.JavaMail.zimbra@redpill-linpro.com> ----- Opprinnelig melding ----- > On Mar 27, 2014, at 22:14 , Trygve Vea wrote: > > > ----- Opprinnelig melding ----- > >> Hello! > > > > Hello! > > > >> On Thu, Mar 27, 2014 at 04:34:37PM +0100, Trygve Vea wrote: > >>> # HG changeset patch > >>> # User Trygve Vea > >>> # Date 1395933815 -3600 > >>> # Thu Mar 27 16:23:35 2014 +0100 > >>> # Node ID 13e6a37c2f57443b0d5dd0abce8d9d4ab00e31e3 > >>> # Parent 2411d4b5be2ca690a5a00a1d8ad96ff69a00317f > >>> Added so_freebind and so_transparent to the listen directive > >>> > >>> This solves a Linux/IPv6-specific problem. > >>> > >>> To be able to listen to an IPv6 address that is not yet available on the > >>> host, > >>> one need to use the IP_FREEBIND and IP_TRANSPARENT socket options. > >>> > >>> The use case in question is for a failover setup with several service- > >>> addresses in a IPv6-only environment. > >>> > >>> IPv4 has a sysctl available (ip_nonlocal_bind), which is not available > >>> for > >>> IPv6 - thus making these patches necessary. > >> > >> Isn't bind on INADDR_ANY/IN6ADDR_ANY works for you? > >> > >> It is expected to work fine and allows to accept connections on > >> all addresses currently available on a host without any > >> non-portable tricks. > > ----- Opprinnelig melding ----- > >>> IPv4 has a sysctl available (ip_nonlocal_bind), which is not available > >>> for > >>> IPv6 - thus making these patches necessary. > >> > >> Isn't bind on INADDR_ANY/IN6ADDR_ANY works for you? > >> > >> It is expected to work fine and allows to accept connections on > >> all addresses currently available on a host without any > >> non-portable tricks. > > > > That would be sufficient for HTTP - and my preferred option, since we can > > handle routing after the end-user have provided us with the Host-header, > > and thus know where to send the user. > > > > However, with SSL enabled - while we have end users that still do not > > support SNI > > (http://en.wikipedia.org/wiki/Server_Name_Indication#Client_side), and > > using multiple SSL-certificates, for multiple applications - we will need > > to bind each certificate to its own dedicated service address. From here, > > we can do routing / forward the connections further down the stack. > > This can be handled with following configuration: > > server { > listen *:443 ssl; > listen any.non.existent.ip1:443 ssl; > ssl_certificate ... > ... > } > > > server { > listen any.non.existent.ip2:443 ssl; > ssl_certificate ... > ... > } > > nginx will bind only to *:443 and then will call getsockname() to get real > local address. Hm, are you sure? I haven't been able to succeed - as this was what I was initially attempting to do. server { listen *:443 ssl; listen [2a02:c0:209::F1]:443 ssl; nginx: [emerg] bind() to [2a02:c0:209::f1]:443 failed (99: Cannot assign requested address) Are there any additional requirements to make this work? -- Trygve Vea Redpill Linpro AS From igor at sysoev.ru Fri Mar 28 10:54:47 2014 From: igor at sysoev.ru (Igor Sysoev) Date: Fri, 28 Mar 2014 14:54:47 +0400 Subject: [PATCH] Added so_freebind and so_transparent to the listen directive In-Reply-To: <404367197.25609.1396003545067.JavaMail.zimbra@redpill-linpro.com> References: <13e6a37c2f57443b0d5d.1395934477@miitool> <20140327171353.GF34696@mdounin.ru> <642114637.11704594.1395944072305.JavaMail.zimbra@redpill-linpro.com> <8D5BD3EB-BA7A-417F-82F7-8902A4D9770A@sysoev.ru> <404367197.25609.1396003545067.JavaMail.zimbra@redpill-linpro.com> Message-ID: <3998938C-F68E-4FD4-A652-0188CAE72AEB@sysoev.ru> On Mar 28, 2014, at 14:45 , Trygve Vea wrote: > ----- Opprinnelig melding ----- >> On Mar 27, 2014, at 22:14 , Trygve Vea wrote: >> >>> ----- Opprinnelig melding ----- >>>> Hello! >>> >>> Hello! >>> >>>> On Thu, Mar 27, 2014 at 04:34:37PM +0100, Trygve Vea wrote: >>>>> # HG changeset patch >>>>> # User Trygve Vea >>>>> # Date 1395933815 -3600 >>>>> # Thu Mar 27 16:23:35 2014 +0100 >>>>> # Node ID 13e6a37c2f57443b0d5dd0abce8d9d4ab00e31e3 >>>>> # Parent 2411d4b5be2ca690a5a00a1d8ad96ff69a00317f >>>>> Added so_freebind and so_transparent to the listen directive >>>>> >>>>> This solves a Linux/IPv6-specific problem. >>>>> >>>>> To be able to listen to an IPv6 address that is not yet available on the >>>>> host, >>>>> one need to use the IP_FREEBIND and IP_TRANSPARENT socket options. >>>>> >>>>> The use case in question is for a failover setup with several service- >>>>> addresses in a IPv6-only environment. >>>>> >>>>> IPv4 has a sysctl available (ip_nonlocal_bind), which is not available >>>>> for >>>>> IPv6 - thus making these patches necessary. >>>> >>>> Isn't bind on INADDR_ANY/IN6ADDR_ANY works for you? >>>> >>>> It is expected to work fine and allows to accept connections on >>>> all addresses currently available on a host without any >>>> non-portable tricks. >>> ----- Opprinnelig melding ----- >>>>> IPv4 has a sysctl available (ip_nonlocal_bind), which is not available >>>>> for >>>>> IPv6 - thus making these patches necessary. >>>> >>>> Isn't bind on INADDR_ANY/IN6ADDR_ANY works for you? >>>> >>>> It is expected to work fine and allows to accept connections on >>>> all addresses currently available on a host without any >>>> non-portable tricks. >>> >>> That would be sufficient for HTTP - and my preferred option, since we can >>> handle routing after the end-user have provided us with the Host-header, >>> and thus know where to send the user. >>> >>> However, with SSL enabled - while we have end users that still do not >>> support SNI >>> (http://en.wikipedia.org/wiki/Server_Name_Indication#Client_side), and >>> using multiple SSL-certificates, for multiple applications - we will need >>> to bind each certificate to its own dedicated service address. From here, >>> we can do routing / forward the connections further down the stack. >> >> This can be handled with following configuration: >> >> server { >> listen *:443 ssl; >> listen any.non.existent.ip1:443 ssl; >> ssl_certificate ... >> ... >> } >> >> >> server { >> listen any.non.existent.ip2:443 ssl; >> ssl_certificate ... >> ... >> } >> >> nginx will bind only to *:443 and then will call getsockname() to get real >> local address. > > Hm, are you sure? > > I haven't been able to succeed - as this was what I was initially attempting to do. > > server { > listen *:443 ssl; > > listen [2a02:c0:209::F1]:443 ssl; > > > nginx: [emerg] bind() to [2a02:c0:209::f1]:443 failed (99: Cannot assign requested address) > > > Are there any additional requirements to make this work? You need to add also: listen [::]:443 ssl; -- Igor Sysoev http://nginx.com From mdounin at mdounin.ru Fri Mar 28 12:20:29 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 28 Mar 2014 16:20:29 +0400 Subject: proxy_pass behavior In-Reply-To: References: <20140327165912.GE34696@mdounin.ru> <20140327172732.GG34696@mdounin.ru> Message-ID: <20140328122029.GK34696@mdounin.ru> Hello! On Thu, Mar 27, 2014 at 08:18:48PM -0400, Jim Popovitch wrote: > On Thu, Mar 27, 2014 at 1:50 PM, Jim Popovitch wrote: > > On Thu, Mar 27, 2014 at 1:27 PM, Maxim Dounin wrote: > >> Hello! > >> > >> On Thu, Mar 27, 2014 at 01:12:18PM -0400, Jim Popovitch wrote: > >> > >>> On Thu, Mar 27, 2014 at 12:59 PM, Maxim Dounin wrote: > >>> > Hello! > >>> > > >>> > On Thu, Mar 27, 2014 at 12:29:03PM -0400, Jim Popovitch wrote: > >>> > > >>> >> Should proxy_pass retry ipv4 if ipv6 fails? Currently (1.5.12), it > >>> >> does not, and there is no way to force proxy_pass to always use ipv4. > >>> > > >>> > In no particular order: > >>> > > >>> > - The proxy_next_upstream is expected to retry by default. > >>> > >>> Yes, that works if the host name has 2 or more A OR 2 or more AAAA > >>> records. proxy_next_upstream does not appear to transition to the > >>> next protocol (i.e. from ipv4 to ipv6) > >> > >> It doesn't care about address family. As long as connect() to > >> one address fails, it will try next one. > > > > I will have to do some more tests, but so far 1.5.12 (debian wheezy) > > does not appear to try the next one if the next one is a different > > protocol family. > > So here is what happens: > > Nginx 1.5.12 on Debian Wheezy 7.4 > > ~$ host www.acadiau.ca > www.acadiau.ca has address 131.162.201.18 > www.acadiau.ca has IPv6 address 2620:21:c000:c8:0:aca:d1a:18 > > ~$ grep proxy_pass /etc/nginx/sites-available/proxy > proxy_pass https://www.acadiau.ca; > > When an IPv6 client connects to the proxy, proxy_pass logs the error as > > 2014/03/27 21:28:28 [alert] 24862#0: *158336 connect() failed (13: > Permission denied) while connecting to upstream, client: > 2604:180::82c6:fe9, server: proxy-service.tld, request: "GET / > HTTP/1.1", upstream: "https://[2620:21:c000:c8:0:aca:d1a:18]:443/", > host: "proxy-service.tld" > > It will next try the ipv4 addr and succeed, however the reason for the So the original claim that it "does not appear to try the next one" is withdrawn, right? > "Permission denied" error appears to be the format of the URL (raw > ipv6 addr, not hostname). Why is it using the ipv6 addr instead of > the hostname? No, it's an error from connect() syscall, returned by your OS for some reason (likely because there is no route available, or due to a firewall rule). The address is one of the upstream server nginx uses at the moment - it's is vital for logging if there are more than one address. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Fri Mar 28 12:25:28 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 28 Mar 2014 16:25:28 +0400 Subject: [PATCH] Added nonlocal to the listen directive In-Reply-To: <16eacd8609c8362e9dd7.1395999953@miitool> References: <16eacd8609c8362e9dd7.1395999953@miitool> Message-ID: <20140328122528.GL34696@mdounin.ru> Hello! On Fri, Mar 28, 2014 at 10:45:53AM +0100, Trygve Vea wrote: > # HG changeset patch > # User Trygve Vea > # Date 1395999940 -3600 > # Fri Mar 28 10:45:40 2014 +0100 > # Node ID 16eacd8609c8362e9dd729c743ed7a869c2993fe > # Parent 2411d4b5be2ca690a5a00a1d8ad96ff69a00317f > Added nonlocal to the listen directive > > The nonlocal option is used to set the needed socket options to be able to bind > to an address not necessarily owned by the host. > > This patch currently implements this for Linux >= 2.4 IPv4/IPv6. > > The problem we solve by doing this, is in an environment where the following > conditions are met: > > * HTTPS with multiple certificates, and a client base that are unable to use > SNI - thus having the need to tie specific certificates to specific ip/ports. > * Setting the ip_nonlocal_bind-sysctl is not an option (for example for Linux > IPv6) > * Used in a failover-setup, where the service IP-addresses are moved around by > a daemon like linux-ha or keepalived. As already explained, the patch is not needed for the use case claimed. Just a bind on INADDR_ANY/IN6ADDR_ANY will do the trick. -- Maxim Dounin http://nginx.org/ From tv at redpill-linpro.com Fri Mar 28 12:29:18 2014 From: tv at redpill-linpro.com (Trygve Vea) Date: Fri, 28 Mar 2014 13:29:18 +0100 (CET) Subject: [PATCH] Added so_freebind and so_transparent to the listen directive In-Reply-To: <3998938C-F68E-4FD4-A652-0188CAE72AEB@sysoev.ru> References: <13e6a37c2f57443b0d5d.1395934477@miitool> <20140327171353.GF34696@mdounin.ru> <642114637.11704594.1395944072305.JavaMail.zimbra@redpill-linpro.com> <8D5BD3EB-BA7A-417F-82F7-8902A4D9770A@sysoev.ru> <404367197.25609.1396003545067.JavaMail.zimbra@redpill-linpro.com> <3998938C-F68E-4FD4-A652-0188CAE72AEB@sysoev.ru> Message-ID: <262738728.34876.1396009758458.JavaMail.zimbra@redpill-linpro.com> ----- Opprinnelig melding ----- > On Mar 28, 2014, at 14:45 , Trygve Vea wrote: > > > ----- Opprinnelig melding ----- > >> On Mar 27, 2014, at 22:14 , Trygve Vea wrote: > >> > >>> ----- Opprinnelig melding ----- > >>>> Hello! > >>> > >>> Hello! > >>> > >>>> On Thu, Mar 27, 2014 at 04:34:37PM +0100, Trygve Vea wrote: > >>>>> # HG changeset patch > >>>>> # User Trygve Vea > >>>>> # Date 1395933815 -3600 > >>>>> # Thu Mar 27 16:23:35 2014 +0100 > >>>>> # Node ID 13e6a37c2f57443b0d5dd0abce8d9d4ab00e31e3 > >>>>> # Parent 2411d4b5be2ca690a5a00a1d8ad96ff69a00317f > >>>>> Added so_freebind and so_transparent to the listen directive > >>>>> > >>>>> This solves a Linux/IPv6-specific problem. > >>>>> > >>>>> To be able to listen to an IPv6 address that is not yet available on > >>>>> the > >>>>> host, > >>>>> one need to use the IP_FREEBIND and IP_TRANSPARENT socket options. > >>>>> > >>>>> The use case in question is for a failover setup with several service- > >>>>> addresses in a IPv6-only environment. > >>>>> > >>>>> IPv4 has a sysctl available (ip_nonlocal_bind), which is not available > >>>>> for > >>>>> IPv6 - thus making these patches necessary. > >>>> > >>>> Isn't bind on INADDR_ANY/IN6ADDR_ANY works for you? > >>>> > >>>> It is expected to work fine and allows to accept connections on > >>>> all addresses currently available on a host without any > >>>> non-portable tricks. > >>> ----- Opprinnelig melding ----- > >>>>> IPv4 has a sysctl available (ip_nonlocal_bind), which is not available > >>>>> for > >>>>> IPv6 - thus making these patches necessary. > >>>> > >>>> Isn't bind on INADDR_ANY/IN6ADDR_ANY works for you? > >>>> > >>>> It is expected to work fine and allows to accept connections on > >>>> all addresses currently available on a host without any > >>>> non-portable tricks. > >>> > >>> That would be sufficient for HTTP - and my preferred option, since we can > >>> handle routing after the end-user have provided us with the Host-header, > >>> and thus know where to send the user. > >>> > >>> However, with SSL enabled - while we have end users that still do not > >>> support SNI > >>> (http://en.wikipedia.org/wiki/Server_Name_Indication#Client_side), and > >>> using multiple SSL-certificates, for multiple applications - we will need > >>> to bind each certificate to its own dedicated service address. From > >>> here, > >>> we can do routing / forward the connections further down the stack. > >> > >> This can be handled with following configuration: > >> > >> server { > >> listen *:443 ssl; > >> listen any.non.existent.ip1:443 ssl; > >> ssl_certificate ... > >> ... > >> } > >> > >> > >> server { > >> listen any.non.existent.ip2:443 ssl; > >> ssl_certificate ... > >> ... > >> } > >> > >> nginx will bind only to *:443 and then will call getsockname() to get real > >> local address. > > > > Hm, are you sure? > > > > I haven't been able to succeed - as this was what I was initially > > attempting to do. > > > > server { > > listen *:443 ssl; > > > > listen [2a02:c0:209::F1]:443 ssl; > > > > > > nginx: [emerg] bind() to [2a02:c0:209::f1]:443 failed (99: Cannot assign > > requested address) > > > > > > Are there any additional requirements to make this work? > > You need to add also: > listen [::]:443 ssl; Ah, thank you. This seems to do the trick. I spoke with a colleague about this, and he pointed out that this does not cover the use case where you do NOT want to listen on *:443 (which the patch does cover). However, that is not something I need. Feel free to both ignore and merge the patch; if you would like to merge it with some modifications, feel free to tell me what to change - and I'd be happy to make them. Regards -- Trygve Vea Redpill Linpro AS From mdounin at mdounin.ru Fri Mar 28 13:09:02 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 28 Mar 2014 17:09:02 +0400 Subject: [PATCH 0 of 1] allow to use engine keyform for server private key In-Reply-To: <3C78B94BFD3E024A9F7164A442F6B8341BD49C76@pegas.cp.ru> References: <3C78B94BFD3E024A9F7164A442F6B8341BD49C76@pegas.cp.ru> Message-ID: <20140328130901.GN34696@mdounin.ru> Hello! On Fri, Mar 28, 2014 at 10:26:42AM +0000, ????????? ??????? ????????? wrote: > Me and my company will be happy to see any method to load > private key from engine with any parameter names. > What shall I do? > Change parameters to "format=" and "engine="? Try to use > STORE_get_certificate(), STORE_get_private_key() before using > ENGINE_load_private_key()? > Do you have any instruction which can help me to check the > patch? To do not break compilation with mail module again etc. Some basic hints can be found here: http://nginx.org/en/docs/contributing_changes.html -- Maxim Dounin http://nginx.org/ From vbart at nginx.com Fri Mar 28 16:05:55 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Fri, 28 Mar 2014 16:05:55 +0000 Subject: [nginx] SPDY: fixed the DATA frame length handling in case of so... Message-ID: details: http://hg.nginx.org/nginx/rev/d74889fbf06d branches: changeset: 5627:d74889fbf06d user: Valentin Bartenev date: Fri Mar 28 20:05:07 2014 +0400 description: SPDY: fixed the DATA frame length handling in case of some errors. There are a few cases in ngx_http_spdy_state_read_data() related to error handling when ngx_http_spdy_state_skip() might be called with an inconsistent state between *pos and sc->length, that leads to violation of frame layout parsing and resuted in corruption of spdy connection. Based on a patch by Xiaochen Wang. diffstat: src/http/ngx_http_spdy.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (20 lines): diff -r 2411d4b5be2c -r d74889fbf06d src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c Wed Mar 26 18:01:11 2014 +0400 +++ b/src/http/ngx_http_spdy.c Fri Mar 28 20:05:07 2014 +0400 @@ -1528,7 +1528,6 @@ ngx_http_spdy_state_read_data(ngx_http_s complete = 1; } else { - sc->length -= size; complete = 0; } @@ -1571,6 +1570,8 @@ ngx_http_spdy_state_read_data(ngx_http_s } } + sc->length -= size; + if (tf) { buf->start = pos; buf->pos = pos; From vbart at nginx.com Fri Mar 28 16:11:03 2014 From: vbart at nginx.com (Valentin V. Bartenev) Date: Fri, 28 Mar 2014 20:11:03 +0400 Subject: [PATCH] SPDY: fixed handling of sc->length in ngx_http_spdy_state_read_data() In-Reply-To: References: <20140326043011.GA47562@gmail.com> <2583480.HkcQmHx6Q1@vbart-laptop> Message-ID: <1457575.XFD8nbofvo@vbart-laptop> On Wednesday 26 March 2014 15:46:05 Xiaochen Wang wrote: > hi, > > On Wed, Mar 26, 2014 at 3:02 PM, Valentin V. Bartenev wrote: > > > On Wednesday 26 March 2014 12:30:11 Xiaochen Wang wrote: > > > In our production, sometimes, the disk was full. In which case, the > > > requests after the POST request were handled wrongly in one spdy > > > connection. > > > > > > Because the input body (DATA frame) of POST request could not be written > > > to disk, then ngx_http_spdy_state_read_data() tried to skip this DATA > > > frame with wrong sc->length, which broke spdy stream. > > > > > > > > > # HG changeset patch > > > # User Xiaochen Wang > > > # Date 1395807655 -28800 > > > # Node ID c18a0115c6027ad0ad475061d2984d90544d8449 > > > # Parent bd91f286ee0ade98e9c0f8f55bfef54a122adaf2 > > > SPDY: fixed handling of sc->length in ngx_http_spdy_state_read_data() > > > > > > diff -r bd91f286ee0a -r c18a0115c602 src/http/ngx_http_spdy.c > > > --- a/src/http/ngx_http_spdy.c Mon Mar 24 16:35:44 2014 -0700 > > > +++ b/src/http/ngx_http_spdy.c Wed Mar 26 12:20:55 2014 +0800 > > > @@ -1519,7 +1519,6 @@ > > > complete = 1; > > > > > > } else { > > > - sc->length -= size; > > > complete = 0; > > > } > > > > > > @@ -1567,6 +1566,7 @@ > > > buf->pos = pos; > > > > > > pos += size; > > > + sc->length -= size; > > > > > > buf->end = pos; > > > buf->last = pos; > > > @@ -1585,6 +1585,7 @@ > > > } else { > > > buf->last = ngx_cpymem(buf->last, pos, size); > > > pos += size; > > > + sc->length -= size; > > > } > > > > > > r->request_length += size; > > > > > > > Thank you for the report. I'd prefer a slightly smaller patch: > > > Agree. It has been committed: http://hg.nginx.org/nginx/rev/d74889fbf06d Thanks. wbr, Valentin V. Bartenev From vbart at nginx.com Fri Mar 28 16:27:23 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Fri, 28 Mar 2014 16:27:23 +0000 Subject: [nginx] SPDY: detect premature closing of stream. Message-ID: details: http://hg.nginx.org/nginx/rev/a24f88eff684 branches: changeset: 5628:a24f88eff684 user: Valentin Bartenev date: Fri Mar 28 20:22:57 2014 +0400 description: SPDY: detect premature closing of stream. The SPDY/3.1 specification requires that the server must respond with a 400 "Bad request" error if the sum of the data frame payload lengths does not equal the size of the Content-Length header. This also fixes "zero size buf in output" alert, that might be triggered if client sends a greater than zero Content-Length header and closes stream using the FIN flag with an empty request body. diffstat: src/http/ngx_http_spdy.c | 17 +++++++++++++---- 1 files changed, 13 insertions(+), 4 deletions(-) diffs (34 lines): diff -r d74889fbf06d -r a24f88eff684 src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c Fri Mar 28 20:05:07 2014 +0400 +++ b/src/http/ngx_http_spdy.c Fri Mar 28 20:22:57 2014 +0400 @@ -1609,6 +1609,19 @@ ngx_http_spdy_state_read_data(ngx_http_s stream->in_closed = 1; + if (r->headers_in.content_length_n < 0) { + r->headers_in.content_length_n = rb->rest; + + } else if (r->headers_in.content_length_n != rb->rest) { + ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, + "client prematurely closed stream: " + "%O of %O bytes of request body received", + rb->rest, r->headers_in.content_length_n); + + stream->skip_data = NGX_SPDY_DATA_ERROR; + goto error; + } + if (tf) { ngx_memzero(buf, sizeof(ngx_buf_t)); @@ -1619,10 +1632,6 @@ ngx_http_spdy_state_read_data(ngx_http_s rb->buf = NULL; } - if (r->headers_in.content_length_n < 0) { - r->headers_in.content_length_n = rb->rest; - } - if (rb->post_handler) { r->read_event_handler = ngx_http_block_reading; rb->post_handler(r); From jimpop at gmail.com Fri Mar 28 17:56:59 2014 From: jimpop at gmail.com (Jim Popovitch) Date: Fri, 28 Mar 2014 13:56:59 -0400 Subject: proxy_pass behavior In-Reply-To: <20140328122029.GK34696@mdounin.ru> References: <20140327165912.GE34696@mdounin.ru> <20140327172732.GG34696@mdounin.ru> <20140328122029.GK34696@mdounin.ru> Message-ID: On Fri, Mar 28, 2014 at 8:20 AM, Maxim Dounin wrote: > Hello! > > On Thu, Mar 27, 2014 at 08:18:48PM -0400, Jim Popovitch wrote: > >> On Thu, Mar 27, 2014 at 1:50 PM, Jim Popovitch wrote: >> > On Thu, Mar 27, 2014 at 1:27 PM, Maxim Dounin wrote: >> >> Hello! >> >> >> >> On Thu, Mar 27, 2014 at 01:12:18PM -0400, Jim Popovitch wrote: >> >> >> >>> On Thu, Mar 27, 2014 at 12:59 PM, Maxim Dounin wrote: >> >>> > Hello! >> >>> > >> >>> > On Thu, Mar 27, 2014 at 12:29:03PM -0400, Jim Popovitch wrote: >> >>> > >> >>> >> Should proxy_pass retry ipv4 if ipv6 fails? Currently (1.5.12), it >> >>> >> does not, and there is no way to force proxy_pass to always use ipv4. >> >>> > >> >>> > In no particular order: >> >>> > >> >>> > - The proxy_next_upstream is expected to retry by default. >> >>> >> >>> Yes, that works if the host name has 2 or more A OR 2 or more AAAA >> >>> records. proxy_next_upstream does not appear to transition to the >> >>> next protocol (i.e. from ipv4 to ipv6) >> >> >> >> It doesn't care about address family. As long as connect() to >> >> one address fails, it will try next one. >> > >> > I will have to do some more tests, but so far 1.5.12 (debian wheezy) >> > does not appear to try the next one if the next one is a different >> > protocol family. >> >> So here is what happens: >> >> Nginx 1.5.12 on Debian Wheezy 7.4 >> >> ~$ host www.acadiau.ca >> www.acadiau.ca has address 131.162.201.18 >> www.acadiau.ca has IPv6 address 2620:21:c000:c8:0:aca:d1a:18 >> >> ~$ grep proxy_pass /etc/nginx/sites-available/proxy >> proxy_pass https://www.acadiau.ca; >> >> When an IPv6 client connects to the proxy, proxy_pass logs the error as >> >> 2014/03/27 21:28:28 [alert] 24862#0: *158336 connect() failed (13: >> Permission denied) while connecting to upstream, client: >> 2604:180::82c6:fe9, server: proxy-service.tld, request: "GET / >> HTTP/1.1", upstream: "https://[2620:21:c000:c8:0:aca:d1a:18]:443/", >> host: "proxy-service.tld" >> >> It will next try the ipv4 addr and succeed, however the reason for the > > So the original claim that it "does not appear to try the next > one" is withdrawn, right? 99% of it is, I did see it fail once. >> "Permission denied" error appears to be the format of the URL (raw >> ipv6 addr, not hostname). Why is it using the ipv6 addr instead of >> the hostname? > > No, it's an error from connect() syscall, returned by your OS for > some reason (likely because there is no route available, or due to > a firewall rule). The address is one of the upstream server nginx > uses at the moment - it's is vital for logging if there are more > than one address. I agree it's vital for logging so that you can see which address failed. It's just not clear that the destination server was presented with the hostname (SNI) because the "Permission denied" message seems to idicate connect() was successful but the resource was denied (due to permissions). But I can live with bad OS level error messages as long as the behavior is accurate. Thank you Maxim, -Jim P. From mdounin at mdounin.ru Fri Mar 28 18:36:18 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 28 Mar 2014 22:36:18 +0400 Subject: proxy_pass behavior In-Reply-To: References: <20140327165912.GE34696@mdounin.ru> <20140327172732.GG34696@mdounin.ru> <20140328122029.GK34696@mdounin.ru> Message-ID: <20140328183618.GX34696@mdounin.ru> Hello! On Fri, Mar 28, 2014 at 01:56:59PM -0400, Jim Popovitch wrote: > On Fri, Mar 28, 2014 at 8:20 AM, Maxim Dounin wrote: > > Hello! > > > > On Thu, Mar 27, 2014 at 08:18:48PM -0400, Jim Popovitch wrote: > > > >> On Thu, Mar 27, 2014 at 1:50 PM, Jim Popovitch wrote: > >> > On Thu, Mar 27, 2014 at 1:27 PM, Maxim Dounin wrote: > >> >> Hello! > >> >> > >> >> On Thu, Mar 27, 2014 at 01:12:18PM -0400, Jim Popovitch wrote: > >> >> > >> >>> On Thu, Mar 27, 2014 at 12:59 PM, Maxim Dounin wrote: > >> >>> > Hello! > >> >>> > > >> >>> > On Thu, Mar 27, 2014 at 12:29:03PM -0400, Jim Popovitch wrote: > >> >>> > > >> >>> >> Should proxy_pass retry ipv4 if ipv6 fails? Currently (1.5.12), it > >> >>> >> does not, and there is no way to force proxy_pass to always use ipv4. > >> >>> > > >> >>> > In no particular order: > >> >>> > > >> >>> > - The proxy_next_upstream is expected to retry by default. > >> >>> > >> >>> Yes, that works if the host name has 2 or more A OR 2 or more AAAA > >> >>> records. proxy_next_upstream does not appear to transition to the > >> >>> next protocol (i.e. from ipv4 to ipv6) > >> >> > >> >> It doesn't care about address family. As long as connect() to > >> >> one address fails, it will try next one. > >> > > >> > I will have to do some more tests, but so far 1.5.12 (debian wheezy) > >> > does not appear to try the next one if the next one is a different > >> > protocol family. > >> > >> So here is what happens: > >> > >> Nginx 1.5.12 on Debian Wheezy 7.4 > >> > >> ~$ host www.acadiau.ca > >> www.acadiau.ca has address 131.162.201.18 > >> www.acadiau.ca has IPv6 address 2620:21:c000:c8:0:aca:d1a:18 > >> > >> ~$ grep proxy_pass /etc/nginx/sites-available/proxy > >> proxy_pass https://www.acadiau.ca; > >> > >> When an IPv6 client connects to the proxy, proxy_pass logs the error as > >> > >> 2014/03/27 21:28:28 [alert] 24862#0: *158336 connect() failed (13: > >> Permission denied) while connecting to upstream, client: > >> 2604:180::82c6:fe9, server: proxy-service.tld, request: "GET / > >> HTTP/1.1", upstream: "https://[2620:21:c000:c8:0:aca:d1a:18]:443/", > >> host: "proxy-service.tld" > >> > >> It will next try the ipv4 addr and succeed, however the reason for the > > > > So the original claim that it "does not appear to try the next > > one" is withdrawn, right? > > 99% of it is, I did see it fail once. > > >> "Permission denied" error appears to be the format of the URL (raw > >> ipv6 addr, not hostname). Why is it using the ipv6 addr instead of > >> the hostname? > > > > No, it's an error from connect() syscall, returned by your OS for > > some reason (likely because there is no route available, or due to > > a firewall rule). The address is one of the upstream server nginx > > uses at the moment - it's is vital for logging if there are more > > than one address. > > I agree it's vital for logging so that you can see which address > failed. It's just not clear that the destination server was presented > with the hostname (SNI) because the "Permission denied" message seems > to idicate connect() was successful but the resource was denied (due > to permissions). But I can live with bad OS level error messages as > long as the behavior is accurate. The message in question indicates that connect() syscall failed (note "connect() failed"), and errno was set to EPERM (note "13: Permission denied" in brackets). As previosly said, EPERM was seen to be returend from connect() due to either firewall rules in place that deny the connection, or due to lack of route to the destination address. -- Maxim Dounin http://nginx.org/ From jimpop at gmail.com Fri Mar 28 18:41:15 2014 From: jimpop at gmail.com (Jim Popovitch) Date: Fri, 28 Mar 2014 14:41:15 -0400 Subject: proxy_pass behavior In-Reply-To: <20140328183618.GX34696@mdounin.ru> References: <20140327165912.GE34696@mdounin.ru> <20140327172732.GG34696@mdounin.ru> <20140328122029.GK34696@mdounin.ru> <20140328183618.GX34696@mdounin.ru> Message-ID: On Fri, Mar 28, 2014 at 2:36 PM, Maxim Dounin wrote: > Hello! > > On Fri, Mar 28, 2014 at 01:56:59PM -0400, Jim Popovitch wrote: > >> On Fri, Mar 28, 2014 at 8:20 AM, Maxim Dounin wrote: >> > Hello! >> > >> > On Thu, Mar 27, 2014 at 08:18:48PM -0400, Jim Popovitch wrote: >> > >> >> On Thu, Mar 27, 2014 at 1:50 PM, Jim Popovitch wrote: >> >> > On Thu, Mar 27, 2014 at 1:27 PM, Maxim Dounin wrote: >> >> >> Hello! >> >> >> >> >> >> On Thu, Mar 27, 2014 at 01:12:18PM -0400, Jim Popovitch wrote: >> >> >> >> >> >>> On Thu, Mar 27, 2014 at 12:59 PM, Maxim Dounin wrote: >> >> >>> > Hello! >> >> >>> > >> >> >>> > On Thu, Mar 27, 2014 at 12:29:03PM -0400, Jim Popovitch wrote: >> >> >>> > >> >> >>> >> Should proxy_pass retry ipv4 if ipv6 fails? Currently (1.5.12), it >> >> >>> >> does not, and there is no way to force proxy_pass to always use ipv4. >> >> >>> > >> >> >>> > In no particular order: >> >> >>> > >> >> >>> > - The proxy_next_upstream is expected to retry by default. >> >> >>> >> >> >>> Yes, that works if the host name has 2 or more A OR 2 or more AAAA >> >> >>> records. proxy_next_upstream does not appear to transition to the >> >> >>> next protocol (i.e. from ipv4 to ipv6) >> >> >> >> >> >> It doesn't care about address family. As long as connect() to >> >> >> one address fails, it will try next one. >> >> > >> >> > I will have to do some more tests, but so far 1.5.12 (debian wheezy) >> >> > does not appear to try the next one if the next one is a different >> >> > protocol family. >> >> >> >> So here is what happens: >> >> >> >> Nginx 1.5.12 on Debian Wheezy 7.4 >> >> >> >> ~$ host www.acadiau.ca >> >> www.acadiau.ca has address 131.162.201.18 >> >> www.acadiau.ca has IPv6 address 2620:21:c000:c8:0:aca:d1a:18 >> >> >> >> ~$ grep proxy_pass /etc/nginx/sites-available/proxy >> >> proxy_pass https://www.acadiau.ca; >> >> >> >> When an IPv6 client connects to the proxy, proxy_pass logs the error as >> >> >> >> 2014/03/27 21:28:28 [alert] 24862#0: *158336 connect() failed (13: >> >> Permission denied) while connecting to upstream, client: >> >> 2604:180::82c6:fe9, server: proxy-service.tld, request: "GET / >> >> HTTP/1.1", upstream: "https://[2620:21:c000:c8:0:aca:d1a:18]:443/", >> >> host: "proxy-service.tld" >> >> >> >> It will next try the ipv4 addr and succeed, however the reason for the >> > >> > So the original claim that it "does not appear to try the next >> > one" is withdrawn, right? >> >> 99% of it is, I did see it fail once. >> >> >> "Permission denied" error appears to be the format of the URL (raw >> >> ipv6 addr, not hostname). Why is it using the ipv6 addr instead of >> >> the hostname? >> > >> > No, it's an error from connect() syscall, returned by your OS for >> > some reason (likely because there is no route available, or due to >> > a firewall rule). The address is one of the upstream server nginx >> > uses at the moment - it's is vital for logging if there are more >> > than one address. >> >> I agree it's vital for logging so that you can see which address >> failed. It's just not clear that the destination server was presented >> with the hostname (SNI) because the "Permission denied" message seems >> to idicate connect() was successful but the resource was denied (due >> to permissions). But I can live with bad OS level error messages as >> long as the behavior is accurate. > > The message in question indicates that connect() syscall failed > (note "connect() failed"), and errno was set to EPERM (note "13: > Permission denied" in brackets). > > As previosly said, EPERM was seen to be returend from connect() > due to either firewall rules in place that deny the connection, or > due to lack of route to the destination address. Yes. That is my understanding. It is an OS level error message from the connect() call, not an nginx generated message. I'm still going to try and submit a patch for proxy_protocol in order to force proxies to use a certain protocol rather than allowing any. ;-) -Jim P. From sfan5 at live.de Sun Mar 30 15:54:43 2014 From: sfan5 at live.de (sfan5) Date: Sun, 30 Mar 2014 17:54:43 +0200 Subject: [PATCH] HTTP: Add 'autoindex_max_length' directive Message-ID: # HG changeset patch # User sfan5 # Date 1396194835 -7200 # Sun Mar 30 17:53:55 2014 +0200 # Node ID 35328d85f9b629e3593b1f862af61af9bd743b3f # Parent a24f88eff684889a203b04e8be70ce0e3899dc30 HTTP: Add 'autoindex_max_length' directive The 'autoindex_max_length' directive controls the length after which a filename is cut-off by the http_autoindex module. diff -r a24f88eff684 -r 35328d85f9b6 src/http/modules/ngx_http_autoindex_module.c --- a/src/http/modules/ngx_http_autoindex_module.c Fri Mar 28 20:22:57 2014 +0400 +++ b/src/http/modules/ngx_http_autoindex_module.c Sun Mar 30 17:53:55 2014 +0200 @@ -40,13 +40,12 @@ ngx_flag_t enable; ngx_flag_t localtime; ngx_flag_t exact_size; + ngx_uint_t max_length; } ngx_http_autoindex_loc_conf_t; #define NGX_HTTP_AUTOINDEX_PREALLOCATE 50 -#define NGX_HTTP_AUTOINDEX_NAME_LEN 50 - static int ngx_libc_cdecl ngx_http_autoindex_cmp_entries(const void *one, const void *two); @@ -81,6 +80,13 @@ offsetof(ngx_http_autoindex_loc_conf_t, exact_size), NULL }, + { ngx_string("autoindex_max_length"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_num_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_autoindex_loc_conf_t, max_length), + NULL }, + ngx_null_command }; @@ -380,7 +386,7 @@ + sizeof("\">") - 1 + entry[i].name.len - entry[i].utf_len + entry[i].escape_html - + NGX_HTTP_AUTOINDEX_NAME_LEN + sizeof(">") - 2 + + alcf->max_length + sizeof(">") - 2 + sizeof("") - 1 + sizeof(" 28-Sep-1970 12:00 ") - 1 + 20 /* the file size */ @@ -442,11 +448,11 @@ len = entry[i].utf_len; if (entry[i].name.len != len) { - if (len > NGX_HTTP_AUTOINDEX_NAME_LEN) { - char_len = NGX_HTTP_AUTOINDEX_NAME_LEN - 3 + 1; + if (len > alcf->max_length) { + char_len = alcf->max_length - 3 + 1; } else { - char_len = NGX_HTTP_AUTOINDEX_NAME_LEN + 1; + char_len = alcf->max_length + 1; } last = b->last; @@ -462,8 +468,8 @@ } else { if (entry[i].escape_html) { - if (len > NGX_HTTP_AUTOINDEX_NAME_LEN) { - char_len = NGX_HTTP_AUTOINDEX_NAME_LEN - 3; + if (len > alcf->max_length) { + char_len = alcf->max_length - 3; } else { char_len = len; @@ -475,25 +481,25 @@ } else { b->last = ngx_cpystrn(b->last, entry[i].name.data, - NGX_HTTP_AUTOINDEX_NAME_LEN + 1); + alcf->max_length + 1); last = b->last - 3; } } - if (len > NGX_HTTP_AUTOINDEX_NAME_LEN) { + if (len > alcf->max_length) { b->last = ngx_cpymem(last, "..>", sizeof("..>") - 1); } else { - if (entry[i].dir && NGX_HTTP_AUTOINDEX_NAME_LEN - len > 0) { + if (entry[i].dir && alcf->max_length - len > 0) { *b->last++ = '/'; len++; } b->last = ngx_cpymem(b->last, "", sizeof("") - 1); - if (NGX_HTTP_AUTOINDEX_NAME_LEN - len > 0) { - ngx_memset(b->last, ' ', NGX_HTTP_AUTOINDEX_NAME_LEN - len); - b->last += NGX_HTTP_AUTOINDEX_NAME_LEN - len; + if (alcf->max_length - len > 0) { + ngx_memset(b->last, ' ', alcf->max_length - len); + b->last += alcf->max_length - len; } } @@ -667,6 +673,7 @@ conf->enable = NGX_CONF_UNSET; conf->localtime = NGX_CONF_UNSET; conf->exact_size = NGX_CONF_UNSET; + conf->max_length = NGX_CONF_UNSET_UINT; return conf; } @@ -681,6 +688,7 @@ ngx_conf_merge_value(conf->enable, prev->enable, 0); ngx_conf_merge_value(conf->localtime, prev->localtime, 0); ngx_conf_merge_value(conf->exact_size, prev->exact_size, 1); + ngx_conf_merge_uint_value(conf->max_length, prev->max_length, 50); return NGX_CONF_OK; } From mdounin at mdounin.ru Mon Mar 31 15:12:07 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 31 Mar 2014 19:12:07 +0400 Subject: [PATCH] HTTP: Add 'autoindex_max_length' directive In-Reply-To: References: Message-ID: <20140331151206.GP34696@mdounin.ru> Hello! On Sun, Mar 30, 2014 at 05:54:43PM +0200, sfan5 wrote: > # HG changeset patch > # User sfan5 > # Date 1396194835 -7200 > # Sun Mar 30 17:53:55 2014 +0200 > # Node ID 35328d85f9b629e3593b1f862af61af9bd743b3f > # Parent a24f88eff684889a203b04e8be70ce0e3899dc30 > HTTP: Add 'autoindex_max_length' directive > > The 'autoindex_max_length' directive controls the length after which a > filename is cut-off by the http_autoindex module. > > diff -r a24f88eff684 -r 35328d85f9b6 > src/http/modules/ngx_http_autoindex_module.c > --- a/src/http/modules/ngx_http_autoindex_module.c Fri Mar 28 20:22:57 > 2014 +0400 > +++ b/src/http/modules/ngx_http_autoindex_module.c Sun Mar 30 17:53:55 > 2014 +0200 > @@ -40,13 +40,12 @@ > ngx_flag_t enable; > ngx_flag_t localtime; > ngx_flag_t exact_size; > + ngx_uint_t max_length; > } ngx_http_autoindex_loc_conf_t; > > > #define NGX_HTTP_AUTOINDEX_PREALLOCATE 50 > > -#define NGX_HTTP_AUTOINDEX_NAME_LEN 50 > - You may attach this patch to the ticket #9 (http://trac.nginx.org/nginx/ticket/9), but it's highly unlikely to be committed. Overral I would prefer to make it possible to generate xml and/or json out of autoindex, to allow arbitrary presentation processing either using xslt filter or with client-side code. Adding presentation-level options like this one looks like a wrong way to go. -- Maxim Dounin http://nginx.org/ From arut at nginx.com Mon Mar 31 15:54:20 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Mon, 31 Mar 2014 15:54:20 +0000 Subject: [nginx] Mp4: fixed seeking to a track end. Message-ID: details: http://hg.nginx.org/nginx/rev/c7bbc5fcf27c branches: changeset: 5629:c7bbc5fcf27c user: Roman Arutyunyan date: Mon Mar 31 19:52:17 2014 +0400 description: Mp4: fixed seeking to a track end. When "start" value is equal to a track duration the request fails with "time is out mp4 stts" like it did before track duration check was added. Now such tracks are considered short and skipped. diffstat: src/http/modules/ngx_http_mp4_module.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (21 lines): diff -r a24f88eff684 -r c7bbc5fcf27c src/http/modules/ngx_http_mp4_module.c --- a/src/http/modules/ngx_http_mp4_module.c Fri Mar 28 20:22:57 2014 +0400 +++ b/src/http/modules/ngx_http_mp4_module.c Mon Mar 31 19:52:17 2014 +0400 @@ -1507,7 +1507,7 @@ ngx_http_mp4_read_tkhd_atom(ngx_http_mp4 start_time = (uint64_t) mp4->start * mp4->timescale / 1000; - if (duration < start_time) { + if (duration <= start_time) { ngx_log_debug0(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, "tkhd duration is less than start time"); return NGX_DECLINED; @@ -1668,7 +1668,7 @@ ngx_http_mp4_read_mdhd_atom(ngx_http_mp4 start_time = (uint64_t) mp4->start * timescale / 1000; - if (duration < start_time) { + if (duration <= start_time) { ngx_log_debug0(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, "mdhd duration is less than start time"); return NGX_DECLINED; From arut at nginx.com Mon Mar 31 16:10:48 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Mon, 31 Mar 2014 16:10:48 +0000 Subject: [nginx] Mp4: improved logging after adding "end" support. Message-ID: details: http://hg.nginx.org/nginx/rev/02584b335727 branches: changeset: 5630:02584b335727 user: Roman Arutyunyan date: Mon Mar 31 20:05:53 2014 +0400 description: Mp4: improved logging after adding "end" support. Despite introducing start and end crop operations existing log messages still mostly refer only to start. Logging is improved to match both cases. New debug logging is added to track entry count in atoms after cropping. Two format type mismatches are fixed as well. diffstat: src/http/modules/ngx_http_mp4_module.c | 98 ++++++++++++++++++++++++--------- 1 files changed, 71 insertions(+), 27 deletions(-) diffs (291 lines): diff -r c7bbc5fcf27c -r 02584b335727 src/http/modules/ngx_http_mp4_module.c --- a/src/http/modules/ngx_http_mp4_module.c Mon Mar 31 19:52:17 2014 +0400 +++ b/src/http/modules/ngx_http_mp4_module.c Mon Mar 31 20:05:53 2014 +0400 @@ -2084,6 +2084,9 @@ ngx_http_mp4_update_stts_atom(ngx_http_m return NGX_ERROR; } + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "time-to-sample entries:%uD", trak->time_to_sample_entries); + atom_size = sizeof(ngx_mp4_stts_atom_t) + (data->last - data->pos); trak->size += atom_size; @@ -2109,9 +2112,15 @@ ngx_http_mp4_crop_stts_data(ngx_http_mp4 if (start) { start_sec = mp4->start; + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "mp4 stts crop start_time:%ui", start_sec); + } else if (mp4->length) { start_sec = mp4->length; + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "mp4 stts crop end_time:%ui", start_sec); + } else { return NGX_OK; } @@ -2120,9 +2129,6 @@ ngx_http_mp4_crop_stts_data(ngx_http_mp4 start_time = (uint64_t) start_sec * trak->timescale / 1000; - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "time-to-sample start_time:%uL", start_time); - entries = trak->time_to_sample_entries; start_sample = 0; entry = (ngx_mp4_stts_entry_t *) data->pos; @@ -2132,8 +2138,9 @@ ngx_http_mp4_crop_stts_data(ngx_http_mp4 count = ngx_mp4_get_32value(entry->count); duration = ngx_mp4_get_32value(entry->duration); - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "count:%uD, duration:%uD", count, duration); + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "time:%uL, count:%uD, duration:%uD", + start_time, count, duration); if (start_time < (uint64_t) count * duration) { start_sample += (ngx_uint_t) (start_time / duration); @@ -2148,27 +2155,32 @@ ngx_http_mp4_crop_stts_data(ngx_http_mp4 } ngx_log_error(NGX_LOG_ERR, mp4->file.log, 0, - "start time is out mp4 stts samples in \"%s\"", - mp4->file.name.data); + "%s time is out mp4 stts samples in \"%s\"", + start ? "start" : "end", mp4->file.name.data); return NGX_ERROR; found: - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "start_sample:%ui, new count:%uD", start_sample, count); - if (start) { ngx_mp4_set_32value(entry->count, count - rest); data->pos = (u_char *) entry; trak->time_to_sample_entries = entries; trak->start_sample = start_sample; + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "start_sample:%ui, new count:%uD", + trak->start_sample, count - rest); + } else { ngx_mp4_set_32value(entry->count, rest); data->last = (u_char *) (entry + 1); trak->time_to_sample_entries -= entries - 1; trak->end_sample = trak->start_sample + start_sample; + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "end_sample:%ui, new count:%uD", + trak->end_sample, rest); } return NGX_OK; @@ -2273,6 +2285,9 @@ ngx_http_mp4_update_stss_atom(ngx_http_m ngx_http_mp4_crop_stss_data(mp4, trak, 1); ngx_http_mp4_crop_stss_data(mp4, trak, 0); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "sync sample entries:%uD", trak->sync_samples_entries); + if (trak->sync_samples_entries) { entry = (uint32_t *) data->pos; end = (uint32_t *) data->last; @@ -2316,9 +2331,15 @@ ngx_http_mp4_crop_stss_data(ngx_http_mp4 if (start) { start_sample = trak->start_sample + 1; + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "mp4 stss crop start_sample:%uD", start_sample); + } else if (mp4->length) { start_sample = trak->end_sample + 1; + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "mp4 stss crop end_sample:%uD", start_sample); + } else { return; } @@ -2332,8 +2353,8 @@ ngx_http_mp4_crop_stss_data(ngx_http_mp4 while (entry < end) { sample = ngx_mp4_get_32value(entry); - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "start:%uD, sync:%uD", start_sample, sample); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "sync:%uD", sample); if (sample >= start_sample) { goto found; @@ -2344,7 +2365,7 @@ ngx_http_mp4_crop_stss_data(ngx_http_mp4 } ngx_log_debug0(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "start sample is out of mp4 stss atom"); + "sample is out of mp4 stss atom"); found: @@ -2461,6 +2482,10 @@ ngx_http_mp4_update_ctts_atom(ngx_http_m ngx_http_mp4_crop_ctts_data(mp4, trak, 1); ngx_http_mp4_crop_ctts_data(mp4, trak, 0); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "composition offset entries:%uD", + trak->composition_offset_entries); + if (trak->composition_offset_entries == 0) { trak->out[NGX_HTTP_MP4_CTTS_ATOM].buf = NULL; trak->out[NGX_HTTP_MP4_CTTS_DATA].buf = NULL; @@ -2494,9 +2519,15 @@ ngx_http_mp4_crop_ctts_data(ngx_http_mp4 if (start) { start_sample = trak->start_sample + 1; + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "mp4 ctts crop start_sample:%uD", start_sample); + } else if (mp4->length) { start_sample = trak->end_sample - trak->start_sample + 1; + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "mp4 ctts crop end_sample:%uD", start_sample); + } else { return; } @@ -2511,7 +2542,7 @@ ngx_http_mp4_crop_ctts_data(ngx_http_mp4 count = ngx_mp4_get_32value(entry->count); ngx_log_debug3(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "start:%uD, count:%uD, offset:%uD", + "sample:%uD, count:%uD, offset:%uD", start_sample, count, ngx_mp4_get_32value(entry->offset)); if (start_sample <= count) { @@ -2659,6 +2690,10 @@ ngx_http_mp4_update_stsc_atom(ngx_http_m return NGX_ERROR; } + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "sample-to-chunk entries:%uD", + trak->sample_to_chunk_entries); + entry = (ngx_mp4_stsc_entry_t *) data->pos; end = (ngx_mp4_stsc_entry_t *) data->last; @@ -2699,8 +2734,12 @@ ngx_http_mp4_crop_stsc_data(ngx_http_mp4 if (start) { start_sample = (uint32_t) trak->start_sample; + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "mp4 stsc crop start_sample:%uD", start_sample); + } else if (mp4->length) { start_sample = (uint32_t) (trak->end_sample - trak->start_sample); + samples = 0; data = trak->out[NGX_HTTP_MP4_STSC_START].buf; @@ -2714,12 +2753,13 @@ ngx_http_mp4_crop_stsc_data(ngx_http_mp4 ngx_mp4_set_32value(entry->samples, samples); } - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "mp4 stsc using %uD start samples", samples); - start_sample -= samples; } + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "mp4 stsc crop end_sample:%uD, ext_samples:%uD", + start_sample, samples); + } else { return NGX_OK; } @@ -2740,7 +2780,7 @@ ngx_http_mp4_crop_stsc_data(ngx_http_mp4 next_chunk = ngx_mp4_get_32value(entry->chunk); ngx_log_debug5(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "start_sample:%uD, chunk:%uD, chunks:%uD, " + "sample:%uD, chunk:%uD, chunks:%uD, " "samples:%uD, id:%uD", start_sample, chunk, next_chunk - chunk, samples, id); @@ -2763,15 +2803,15 @@ ngx_http_mp4_crop_stsc_data(ngx_http_mp4 next_chunk = trak->chunks + 1; ngx_log_debug4(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "start_sample:%uD, chunk:%uD, chunks:%uD, samples:%uD", + "sample:%uD, chunk:%uD, chunks:%uD, samples:%uD", start_sample, chunk, next_chunk - chunk, samples); n = (next_chunk - chunk) * samples; if (start_sample > n) { ngx_log_error(NGX_LOG_ERR, mp4->file.log, 0, - "start time is out mp4 stsc chunks in \"%s\"", - mp4->file.name.data); + "%s time is out mp4 stsc chunks in \"%s\"", + start ? "start" : "end", mp4->file.name.data); return NGX_ERROR; } @@ -2791,10 +2831,6 @@ found: target_chunk += start_sample / samples; chunk_samples = start_sample % samples; - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "start chunk:%ui, samples:%uD", - target_chunk, chunk_samples); - if (start) { data->pos = (u_char *) entry; @@ -2806,6 +2842,10 @@ found: samples -= chunk_samples; + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "start_chunk:%ui, start_chunk_samples:%ui", + trak->start_chunk, trak->start_chunk_samples); + } else { if (start_sample) { data->last = (u_char *) (entry + 1); @@ -2828,6 +2868,10 @@ found: samples = chunk_samples; next_chunk = chunk + 1; + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, + "end_chunk:%ui, end_chunk_samples:%ui", + trak->end_chunk, trak->end_chunk_samples); } if (chunk_samples && next_chunk - target_chunk == 2) { @@ -3145,7 +3189,7 @@ ngx_http_mp4_update_stco_atom(ngx_http_m ngx_mp4_set_32value(data->pos, trak->start_offset); ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "start chunk offset:%uD", trak->start_offset); + "start chunk offset:%O", trak->start_offset); if (mp4->length) { @@ -3329,7 +3373,7 @@ ngx_http_mp4_update_co64_atom(ngx_http_m ngx_mp4_set_64value(data->pos, trak->start_offset); ngx_log_debug1(NGX_LOG_DEBUG_HTTP, mp4->file.log, 0, - "start chunk offset:%uL", trak->start_offset); + "start chunk offset:%O", trak->start_offset); if (mp4->length) { From mdounin at mdounin.ru Mon Mar 31 17:39:59 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 31 Mar 2014 17:39:59 +0000 Subject: [nginx] Added #ifndef around NGX_HAVE_CASELESS_FILESYSTEM define. Message-ID: details: http://hg.nginx.org/nginx/rev/f1dfe7ff4511 branches: changeset: 5631:f1dfe7ff4511 user: Maxim Dounin date: Mon Mar 31 21:38:25 2014 +0400 description: Added #ifndef around NGX_HAVE_CASELESS_FILESYSTEM define. This brings Cygwin compilation in line with other case-insensitive systems (notably win32 and OS X) where one can force case sensitivity using -DNGX_HAVE_CASELESS_FILESYSTEM=0. diffstat: src/os/unix/ngx_files.h | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (13 lines): diff --git a/src/os/unix/ngx_files.h b/src/os/unix/ngx_files.h --- a/src/os/unix/ngx_files.h +++ b/src/os/unix/ngx_files.h @@ -53,7 +53,9 @@ typedef struct { #ifdef __CYGWIN__ +#ifndef NGX_HAVE_CASELESS_FILESYSTEM #define NGX_HAVE_CASELESS_FILESYSTEM 1 +#endif #define ngx_open_file(name, mode, create, access) \ open((const char *) name, mode|create|O_BINARY, access) From mdounin at mdounin.ru Mon Mar 31 17:40:00 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 31 Mar 2014 17:40:00 +0000 Subject: [nginx] Mail: fixed ngx_mail_send() (ticket #519). Message-ID: details: http://hg.nginx.org/nginx/rev/b42e7c790b81 branches: changeset: 5632:b42e7c790b81 user: Maxim Dounin date: Mon Mar 31 21:38:27 2014 +0400 description: Mail: fixed ngx_mail_send() (ticket #519). diffstat: src/mail/ngx_mail_handler.c | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diffs (26 lines): diff --git a/src/mail/ngx_mail_handler.c b/src/mail/ngx_mail_handler.c --- a/src/mail/ngx_mail_handler.c +++ b/src/mail/ngx_mail_handler.c @@ -559,8 +559,13 @@ ngx_mail_send(ngx_event_t *wev) n = c->send(c, s->out.data, s->out.len); if (n > 0) { + s->out.data += n; s->out.len -= n; + if (s->out.len != 0) { + goto again; + } + if (wev->timer_set) { ngx_del_timer(wev); } @@ -584,6 +589,8 @@ ngx_mail_send(ngx_event_t *wev) /* n == NGX_AGAIN */ +again: + cscf = ngx_mail_get_module_srv_conf(s, ngx_mail_core_module); ngx_add_timer(c->write, cscf->timeout); From mdounin at mdounin.ru Mon Mar 31 17:40:02 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 31 Mar 2014 17:40:02 +0000 Subject: [nginx] Win32: fixed cpu hog by cache manager on exit (ticket #5... Message-ID: details: http://hg.nginx.org/nginx/rev/b74f1106f920 branches: changeset: 5633:b74f1106f920 user: Maxim Dounin date: Mon Mar 31 21:38:28 2014 +0400 description: Win32: fixed cpu hog by cache manager on exit (ticket #514). The "ngx_quit" may be reset by the worker thread before it's seen by a ngx_cache_manager_thread(), resulting in an infinite loop. Make sure to test ngx_exiting as well. diffstat: src/os/win32/ngx_process_cycle.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diffs (30 lines): diff --git a/src/os/win32/ngx_process_cycle.c b/src/os/win32/ngx_process_cycle.c --- a/src/os/win32/ngx_process_cycle.c +++ b/src/os/win32/ngx_process_cycle.c @@ -926,7 +926,7 @@ ngx_cache_manager_thread(void *data) * ev == WAIT_ABANDONED_0 + 1 */ - if (ngx_terminate || ngx_quit) { + if (ngx_terminate || ngx_quit || ngx_exiting) { ngx_log_error(NGX_LOG_NOTICE, cycle->log, 0, "exiting"); return 0; } @@ -936,7 +936,7 @@ ngx_cache_manager_thread(void *data) for ( ;; ) { - if (ngx_terminate || ngx_quit) { + if (ngx_terminate || ngx_quit || ngx_exiting) { ngx_log_error(NGX_LOG_NOTICE, cycle->log, 0, "exiting"); break; } @@ -1005,7 +1005,7 @@ ngx_cache_loader_thread(void *data) path = cycle->paths.elts; for (i = 0; i < cycle->paths.nelts; i++) { - if (ngx_terminate || ngx_quit) { + if (ngx_terminate || ngx_quit || ngx_exiting) { break; } From mdounin at mdounin.ru Mon Mar 31 17:40:04 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 31 Mar 2014 17:40:04 +0000 Subject: [nginx] Core: slab log_nomem flag. Message-ID: details: http://hg.nginx.org/nginx/rev/5024d29354f1 branches: changeset: 5634:5024d29354f1 user: Maxim Dounin date: Mon Mar 31 21:38:30 2014 +0400 description: Core: slab log_nomem flag. The flag allows to suppress "ngx_slab_alloc() failed: no memory" messages from a slab allocator, e.g., if an LRU expiration is used by a consumer and allocation failures aren't fatal. The flag is now used in the SSL session cache code, and in the limit_req module. diffstat: src/core/ngx_slab.c | 6 +++++- src/core/ngx_slab.h | 2 ++ src/event/ngx_event_openssl.c | 4 +++- src/http/modules/ngx_http_limit_req_module.c | 4 ++++ 4 files changed, 14 insertions(+), 2 deletions(-) diffs (77 lines): diff --git a/src/core/ngx_slab.c b/src/core/ngx_slab.c --- a/src/core/ngx_slab.c +++ b/src/core/ngx_slab.c @@ -129,6 +129,7 @@ ngx_slab_init(ngx_slab_pool_t *pool) pool->pages->slab = pages; } + pool->log_nomem = 1; pool->log_ctx = &pool->zero; pool->zero = '\0'; } @@ -658,7 +659,10 @@ ngx_slab_alloc_pages(ngx_slab_pool_t *po } } - ngx_slab_error(pool, NGX_LOG_CRIT, "ngx_slab_alloc() failed: no memory"); + if (pool->log_nomem) { + ngx_slab_error(pool, NGX_LOG_CRIT, + "ngx_slab_alloc() failed: no memory"); + } return NULL; } diff --git a/src/core/ngx_slab.h b/src/core/ngx_slab.h --- a/src/core/ngx_slab.h +++ b/src/core/ngx_slab.h @@ -39,6 +39,8 @@ typedef struct { u_char *log_ctx; u_char zero; + unsigned log_nomem:1; + void *data; void *addr; } ngx_slab_pool_t; diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -1834,6 +1834,8 @@ ngx_ssl_session_cache_init(ngx_shm_zone_ ngx_sprintf(shpool->log_ctx, " in SSL session shared cache \"%V\"%Z", &shm_zone->shm.name); + shpool->log_nomem = 0; + return NGX_OK; } @@ -1986,7 +1988,7 @@ failed: ngx_shmtx_unlock(&shpool->mutex); ngx_log_error(NGX_LOG_ALERT, c->log, 0, - "could not add new SSL session to the session cache"); + "could not allocate new session%s", shpool->log_ctx); return 0; } diff --git a/src/http/modules/ngx_http_limit_req_module.c b/src/http/modules/ngx_http_limit_req_module.c --- a/src/http/modules/ngx_http_limit_req_module.c +++ b/src/http/modules/ngx_http_limit_req_module.c @@ -451,6 +451,8 @@ ngx_http_limit_req_lookup(ngx_http_limit node = ngx_slab_alloc_locked(ctx->shpool, size); if (node == NULL) { + ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, + "could not allocate node%s", ctx->shpool->log_ctx); return NGX_ERROR; } } @@ -674,6 +676,8 @@ ngx_http_limit_req_init_zone(ngx_shm_zon ngx_sprintf(ctx->shpool->log_ctx, " in limit_req zone \"%V\"%Z", &shm_zone->shm.name); + ctx->shpool->log_nomem = 0; + return NGX_OK; } From nipunn at dropbox.com Mon Mar 31 23:26:36 2014 From: nipunn at dropbox.com (Nipunn Koorapati) Date: Mon, 31 Mar 2014 16:26:36 -0700 Subject: SSL Session Caching - Memcached Message-ID: Hi nginx-devel, I'm new to the world of nginx development. I'm looking at patching nginx to allow for configurable session caching via memcached. Goal is to be able to session cache across boxes within a cluster when there's no guarantee that a client will connect to the same box each time. I've seen this previous work from mpalmer: https://github.com/mpalmer/nginx/compare/0.8.x...memcache-ssl-0.8 However it is 3 years old. I'm going to look at patching it onto the mainline and auditing the code to see if it still makes sense. A few questions. 1) Is there anything I should look out for / be careful of? 2) Is there any desire for this feature on mainline? If so, any suggestions on how to request feedback. Via patches to the mailing list? Thanks! --Nipunn -------------- next part -------------- An HTML attachment was scrubbed... URL: